Using Full-Text Search to improve Dynamics CRM search experience

Consider the following scenario: You need to perform email searches in your CRM and your search needs to look at the content (body) of your email messages. When you start having a lot of emails in your CRM (500k+ in my case), searches become extremely slow and you need to start to look at indexes or other solutions to make the search faster and keep your users happy. This post describes a solution that leverage a new CRM feature in CRM 2015 Update 0.1 (On Premise) : Full-text search.

Enabling Search on email body

This is the easy part. If you are having this issue, you have probably done this already. To enable the ability to search on the email body, you need to add the “Description” field as a “Find Column” in the Quick Find view of the Email entity, as shown in the screenshot below. After completing that step, save the view and publish the entity to make the change available in the system.

image

Searching through emails

Now that we’ve configured the search column, we can do a quick search to get our emails by using the search box on the activity list view view or by using the “Global Search” on the top right of the top navigation bar, next to the “Quick Create” button. Note that emails are not enabled in the “Global Search” by default so you’d have to add them (not required).

If you have a few emails in your system, results are going to come back quickly. In the system that I tested against, I had about 700.000 emails in CRM so the search result took on average just over 30 seconds to return results (and I consider this good, because I’m running on SSD drives). That being said, in real life, 30 seconds or more of waiting time is enough to make users think that the system is frozen, or to just stop trusting in the application, especially if they rely heavily on this type of search.

Making search return results faster

If you are using CRM 2015 On Premise, you are in luck. In CRM 2015 Update 0.1, the product team introduced the ability to enable full-text search for Quick Finds queries. To enable it, go do Settings –> General –> Set up Quick Find. There you’ll see the option to “Enable full-text search for Quick Find.

image

Greater details about the feature can be found here and you can also take a look at this video on the Microsoft Dynamics YouTube Channel.

Important points to keep into consideration before enabling full-text search:

  • Full-text indexing for Quick Find uses SQL Server full-text indexing
  • It can take up to 24 hours for the system to enable or disable full-text search, or add and remove find columns (maintenance job has to run on the server and make the adjustments specified in the CRM configuration)
  • As a bonus, you should also activate the Org Settings “EnableQuickFindOptimization” and “EnableRetrieveMultipleOptimization” in order to get the best results. Thanks to Jean-François Cantin for that indication. If you don’t know how to update the Org Settings, here is a reference.

After waiting long enough time for the emails to be indexed, the email searches started to return results within 5 seconds. Drastic improvement. The same solution was implemented into the client’s database and it seems to have made a huge improvement. 

Have fun.

Advertisements

SSRS: How to open CRM Forms from Custom Reports

Just thought I would share how to add a URL in your SSRS reports to open a related CRM record. I spend a little bit of time figuring this out so here it is. I will skip the CRM/SSRS reporting 101 so if you are not familiar with building reports for CRM, you may want to familiarize yourself with basic concepts before you dive into this.

  1. Add a parameter to your report and make sure it prompts for the URL:

    This will return an URL that looks like this: http://YourOrganizationURL/CRMReports/viewer/drillopen.aspx?

  2. Now you can add the URL as an action when some of your elements are clicked. In my example, I have a few columns that when clicking on the value, will open a case, its owner or its customer.

    To do that, I select the column in the table that contains the data and head to the “Action” table; Select “Go to URL” as the action and then click on the function button to setup the URL.

  3. It’s time to use the URL parameter that was defined at step 1 that looks like this http://YourOrganizationURL/CRMReports/viewer/drillopen.aspx. To create a valid URL from here, we have 2 options that I know of… We can
  • Play around with VB to manipulate the String value and remove the part after the Organization URL highlighted in yellow above; from there we build the record URL as per Microsoft Dynamics CRM URL Patterns. It shouldn’t be complicated but I have not tried it.
  • Use a CRM\SSRS report built-in functionality that allows to simple pass the ID and logical name of the record we want to open to the URL received in parameter. It is able redirect to the appropriate form magically.

Obviously, I went with the 2nd option. You expression will look like this: Parameters!CRM_URL.Value & “?ID={” & Fields!incidentid.Value & “}&LogicalName=incident”)

Parameters!CRM_URL.Value : This is the parameter created in step 1
Fields!incidentid.Value: This is the ID of the record you want to navigate to
LogicalName=incident : This is the logical name of the record you want to valigate to


And just as a proof (although this really doesn’t prove anything!) J


And that’s it! It’s magical. Key takeaway here is the fact that if you ask CRM for the URL as a parameter to your report, you get a URL and you then have the flexibility to play around with it and create links to any page you need. The other takeaway is the functionality built in the report to open a record’s form by simple receiving the ID and logical name as parameters. Hope this saves some time for someone one day.

Microsoft Dynamics CRM Data Archiving Solutions

As Dynamics CRM consultants, one of the questions that we often get from clients is “how do we archive CRM data when it’s no longer needed”? I’ve seen companies that had to keep all of their data for a certain amount of years for legal reasons. In any case, if you are using Dynamics CRM as a service management tool for a busy call center for example, chances are you are creating a LOT of records at a very fast pace. Imagine having to retain all of the data you created in the past 5 to 10 years on the same system. The size of the database can grow to become extremely large, making every single query a long one and impacting directly the users experience and their trust in the system.

Before I go into details about the available options that I can talk to, here is a quick recap of what Microsoft officially said a while ago about archiving Dynamics CRM data.

  1. Archiving implies moving records from one location, storing those records for possible retrieval in another location, and deleting the archived records from the original location.” Given the complexity of the relationships in CRM, archiving as defined in the previous sentence can be very risky and must imply extensive thought, design and knowledge of the CRM application and database.
  2. An arching solution should not be implemented with the goal of improving system performance.
  3. Instead of deleting records, it is recommended to deactivate them when they are no longer required.
  4. The Dynamics CRM application can support very large databases, it is important to make sure that the hardware and database are configured in order to get optimal performance before looking at Archiving.

Here are some possible approaches that can be used to archive your Microsoft Dynamics CRM data. Before going down that path, make sure this is something you really need, don’t just build an archiving solution because you want to improve your system performance. Do your homework first!

Archiving Method How it work Pros Cons
1 – Create a copy of the database It requires a strong DBA with knowledge of the CRM database to make a copy of the DB, copy all the records that need to be archived to the new DB and finally delete them from their original location. Sounds easy for a DB expert I assume (I’m not!)
  • It’s one of the less expensive solution as it only requires a strong DB developer with knowledge of CRM and an available SQL Server database
  • Not supported nor recommended by Microsoft
  • A DBA with knowledge of the Dynamics CRM database is always required to retrieve data from the archive database
2 – Create a new instance of Dynamics CRM
  • Create new organization or new CRM installation and install the same solution as the one in production
  • Create a mechanism to copy records from the production environment to the new archiving organization and delete them from their original location. This can be done with :
    • A custom .NET application using the CRM SDK
    • Scribe, Microsoft’s CRM 2011 Instance Adapter or other third party providers routines
    • Database scripting (unsupported)
  • The archived records are easy to retrieve (same UI as prod)
  • Access to these records can be controlled by the Dynamics CRM security model
  • Can be done in a supported way
  • Expensive solution that may requires additional hardware, server and/or user licenses for CRM or SQL Server
  • Company ends up maintaining more than one CRM organizations in the long run
3 – Deactivated unwanted records and tweak database Indexes
This is easily the number 1 choice from Microsoft’s perspective. To me, this should be one of the drivers in the CRM application design. The lifecycle of all records that are created (except for master data) should always be to:

  • create record (with active state obviously)
  • do work
  • deactivate record after the job is complete (set state to inactive)

In many systems that I have seen, some records just remain active forever, even though they are old and on longer required in the active records view.

Also, it is important to create SQL indexes to enable a faster retrieval of active records.

 

  • This method does not involve the deleting on any record from the CRM database which is what MS recommends
  • It forces the application to be designed with best practice for records lifecycle management
I could write down that the down side is that records remain in the CRM database but again, you are not archiving simply because you want to reduce the size of your database…
4 – Create and store custom archive reports This is an interesting one. I’ve personally recommended using this solution to capture the history of successful workflows execution prior to deleting them from the system.
The idea is to create custom reports containing information from the records that are no longer needed in the system. Once the reports have been created, they can be stored in a document management system or sent by email. After that, then the records can be deleted from the CRM database.
There are many ways to create reports in CRM so I won’t spend time discussion that topic here. A simple Google (or should I say Bing searchJ) will give you plenty of starting points.
  • Lots of options for reports creation and execution (3rd party software, SSRS, scheduling jobs, custom workflow activities etc.)
  • Can be done in a supported way
  • What used to be CRM records are now lines of text in different reports
  • Records must be saved in a managed location (example: doc management system with redundancy instead of external hard drive for the system admin…)
  • Access to the reports must be controlled
  • Ability to search for archived information can be limited and/or very painful in flat files
  • Building the reports can be expensive :
    • Report design can take time
    • Development cost depends on the tool used (SSRS, 3rd party…)
5 – Create a custom Archive Database
This is very similar to option 2. The difference would be instead of creating a new instance of CRM, you can create a custom database where you would only keep the information that you need. This database would be populated by a custom .NET app, third party data migration tool routine or database scripting.
Optionally, a custom UI can be developed to enable administrators to search through the custom database.
  • Tailored archiving solution, enables for more granularity and focus on retaining only the information that is required
  • If done well and if CRM solution(s) doesn’t evolve too much over time, it’s a one effort and maintenance should be easier than maintaining 2 CRM instances
  • Can be done in a supported way
  • Expensive to build (requires deep analysis and development time)
  • Custom database must be maintained
  • Security is no longer controlled by the Dynamics CRM application

I personally like solution 5 better but it’s only because I’m a developer and I’d enjoy having to design the new database and the archiving tool. Of course, every company has its own preferences and resources with different set of skills. This should all be taken into consideration prior to deciding what solution to use. These are only the ones that I came up with, there are certainly other options out there.

Hope this helps!