Customizing your CRM platform the “right way”

Over the course of my consulting experiences, I have learned to work with different platforms and products “the right way.” This means that when you are using a product (for example Microsoft Dynamics CRM), you are agreeing to use the product in the ways specified by the makers of the product. It’s like when you buy a car, there are parts that go with it and if you need to change them, you need to get them from the manufacturer. When you start modifying the car with custom parts, you lose your manufacturer’s warranty. Same thing with a phone, you jailbreak it, you don’t get support anymore. The reason behind that is very simple. When you make changes that are not prescribed by the maker of a product, it’s possible that you altered the way the product works (i.e. you may have broken it or it may break in the future), therefore the manufacturer can no longer support it because it doesn’t know if it still works as designed.

When it comes to software and especially in the case of CRM, it is very important not to break that contract. There are a few things to consider if you do and they are very well documented:

  • Unsupported changes are dangerous by nature, you don’t know what the impact is on things that you are not supposed to have control over
  • They will probably break in the future (after a patch or upgrade)
  • They can bring tons of other issues (performance, testability, bugs…)

If you are buying a product that is integrated with Dynamics CRM (ISV solution), always ask this question to the vendor: “Is you product and/or integration built in a supported way by following all Microsoft customization guidelines and APIs”? See the table of what should be your reaction based on the answer below

Answer My2cents What to do
No That’s just bad. It will either stop working at some point or it will break something. Run away!
I don’t know If the vendor doesn’t know, it means he/she is not really aware of what the impact of unsupported customizations are. That sends a wrong message and represents a serious risk. Run away or hire a technology expert to validate that all is supported.
Yes Perfecto! The vendor knows what it means and understand the importance of supported customizations. It’s a go! To protect yourself, make sure it’s mentioned in your purchase contract.

As consultant and community contributor, I keep seeing a lot of web application and ISV providers that integrate with different CRM or other platforms by simply checking out how to do stuff online and customization the platform in unsupported ways to achieve their goal and sell their product. Keep in mind that working with a platform requires the due diligence of working with experts who know how to customize it. Taking shortcuts can seem like a great idea to quickly go to market but consequences can be bad and the result is often hiring experts to figure out what is wrong when they should have been brought in from the beginning.

It’s OK to use unsupported customization as long as you are aware of the risks and are willing to live with them.

You can read more on unsupported customization for MS Dynamics CRM on the links below.

http://dynamicsuniversity.com/blog/customizing-microsoft-dynamics-crm-2011-supported-or-not-supported
http://gustafwesterlund.blogspot.ca/2012/06/unsupported-customizations.html
http://gonzaloruizcrm.blogspot.ca/2011/08/risks-of-unsupported-customizations-in.html

Cheers

Error Logging Options in CRM

Logging is an often frustrating topic when it comes to Dynamics CRM. How do we manage error logging or even debug and verbose/information logging? The answer from Microsoft is simple and standard: use the ITracingService to write logging information. If your requirements are a little more complex, say you need to track your code execution on a very granular level even if no error occurs, then the ITracingService gives you very little options …

  1. In CRM Online, you cannot activate and review the CRM log files (it’s only available with on premise installations)
  2. In CRM Online, you can view the details that are related to the job execution (works for Asynchronous plugins and custom WF); For synchronous plugins, you only see the logs if an error is thrown by downloading the log file – there is no permanent record of the log
  3. If you enable Tracing in CRM (on premise), there is plenty of system written logs in the CRM trace that makes the log file difficult to read and to find your application specific logs, even if you are using the very nice CRM Log Viewer from Stunnware

If you want to have a custom logging module for CRM code, there are a few things to take into consideration:

  • What kind of information do you need to log?
  • Where do you want/need to write it?
  • How and when do you need to access it?
  • Do you need to control the logging level at a given time? (write error messages only, verbose, info etc.)
  • Who will write the logs? Does that user have permission to write in your chosen location?

Here are a couple of options available to create logs from your CRM code. It is up to the development/architecture team to decide what they want to use that fits their need.

Custom External Logging

You can create logging utilities to write logs in locations other than CRM. For example, you could write logs in the Windows Event Viewer, in a text file or even a custom database. There are also third party tools that help you do that as well (NLog, Log4Net…) if you don’t want to start from scratch. The problem here is that if you are writing logs from plugins or workflow activities that are executed as CRM users, chances are these users won’t have permission to write in your designated location. One option is to simply give access to “Everyone” to these locations but this usually goes against most Enterprises’ IT policies. There are probably ways to set up your log writers to impersonate an admin or user with permissions but I have personally never done it.

Pros:

  • Tailored logging engine built for your needs and in a location of your choice

Cons:

  • Doesn’t work for CRM Online (or not easily)
  • Requires system users to have permission to write the logs in the selected location

Custom CRM Entity

Another way to do this is to create one (or many) custom entity to capture your logs. Your CRM custom code can just create new rows in the table(s) to log the required information/error.

Pros:

  • Works for all CRM Installation;
  • Security is simple to manage;
  • Logging information is available directly in CRM;
  • Custom entity can be customized to capture specific logging fields

Cons:

  • Depending on the amount of logging required, your log table(s) may grow really quickly. Think about having a mechanism to clear all unnecessary or outdated logs on a regular basis
  • If you are writing logs in a failing transaction, the entire transaction will be rolled back including creation of logging records. Check out a workaround for this issue here.

These are just a few thoughts on logging and as always in technology, there are probably plenty of other things we can do around logging in CRM. Thought this could be helpful. Feel free to reach out if you want more details or guidance on the topic.

Rule Based Forms Strategies for CRM 2011

CRM 2011 introduces the out of the Role Based UI (RBUI or RBF for Role Based Forms). It is a very nice feature and frankly, it was about time they introduced it. Obviously, it is limited because we can only display a form based on CRM user’s security roles. A very common request is to display a specific form based on field value(s). Example could be showing different opportunity forms based on the stage. Here I am sharing a couple of ways to achieve this.

  • Client Side Solution: The idea is to have a Jscript method that executes when the form is loaded. It reads the ‘decision’ field(s) value(s) and navigates to the appropriate form using the Xrm.Page.ui.formSelector item navigate()
    method. A very clear and simple example of that is available here.
    • Pros
      • It is 100% supported
      • Work on all types of installations (CRM Online, On Premise, ADFS)
    • Cons
      • Introduces a visible “refresh” after the form is loaded when a redirection is required
      • Selecting another form manually is no longer possible (you are always redirected to a specific form)
  • Server Side Solution: This is a little bit more complicated to set up. We did it by using a custom IIS rewrite provider. It means writing some code that
    • Reads the record ID and type name or type code from the URL
    • Loads the record from the CRM DB
    • Reads the “decision” field(s) value(s) and redirects to the appropriate form if required

    These steps sound pretty simple but there are a few things to take into consideration. In order to load the record from the CRM DB, you need to have CRM context information to connect to CRM which you don’t necessarily have from the IIS rewrite custom provider (it is executed from the GAC). In addition to that, you need to make changes to the CRM web.config file to add a rewrite section pointing to your custom rewrite provider. You can find more information on how to set up the custom rewrite provider here. If you want a concrete example, you can comment here or contact me directly (no code sample today!).

    • Pros
      • No visible “refresh”, the redirection is made directly on the server
    • Cons
      • It is complicated to setup and requires an unsupported step (modifying CRM web.config)
      • Only works for On Premise installation
      • Selecting another form manually is no longer possible (you are always redirected to a specific form)

In both cases, remember that you can hard-code the IDs of the forms in your script or C# code because they remain the same as you go from one environment to another (i.e. forms are transported in CRM solutions with their ID) so there is no additional work required for this. Given the choice, I would always go for solution one (client side script) even if it introduces a visible refresh at times. Unfortunately, sometimes that refresh is not acceptable from a client’s perspective so you have to be creative and come up with other approaches like the IIS URL rewrite custom provider.

Cheers!

Retrieving Business Closure dates from Dynamics CRM

As developers, we are often asked to write plugins to calculate durations. A common example would be that the business wants to calculate how long an Opportunity was opened for. From a business perspective, it makes more sense to count only the business days, which is easy to do in C# (there are plenty of algorithms out there to filter out the weekends from a time interval). It makes even more sense to also exclude the holidays which are defined in the Business Closures calendar in CRM.

It is a nice feature and it integrates well with the out of the box Service module in the sense that you can configure it to prevent users from scheduling service activities on holidays.

From a development perspective however, it’s not that easy to deal with the Business Closures calendar for a few different reasons:

  • There is no clear documentation how to deal with them programmatically
  • In the Business Closures Calendar, we know they are stored in the related list of Calendar Rules, but the calendar rule entity doesn’t support Retrieve Multiple queries which makes the dates uneasy to retrieve
  • You can only retrieve the business closures by getting the business closure calendar, the latter supports Retrieve Multiple

You may get confused with the ExpandCalendarRequest that is out there. It is used to retrieve the calendar rules from a user’s calendar. The idea being to retrieve a set of time blocks with appointment information for a specific user (these are also CalendarRules). If you try to retrieve the rules for the business closures calendar, you get a rather obscure error: “Invalid time zone code”.

After spending some time on that route, I reverted back to the original recommendation by many on forums and blogs and I managed to make it work with the code below. The idea is to retrieve the business closure calendar ID of the organization and then return its list of CalendarRule. From there, it’s easy to use to result to do what you need.

private IEnumerable<Entity> GetBusinessClosureCalendarRules(IExecutionContext context, IOrganizationService service)
{
    // Get Organization Business Closure Calendar Id
    Entity org = service.Retrieve("organization", context.OrganizationId, new Microsoft.Xrm.Sdk.Query.ColumnSet("businessclosurecalendarid"));

    QueryExpression q = new QueryExpression("calendar");
    q.ColumnSet = new ColumnSet(true);
    q.Criteria = new FilterExpression();
    q.Criteria.AddCondition(new ConditionExpression("calendarid", ConditionOperator.Equal, org["businessclosurecalendarid"].ToString()));
    Entity businessClosureCalendar = service.RetrieveMultiple(q).Entities[0];
    if (businessClosureCalendar != null)
    {
        return businessClosureCalendar.GetAttributeValue<EntityCollection>("calendarrules").Entities;
    }
    return null;
}

With all this in place, you now know how to retrieve your business closures dates. My next step is to see if it is possible to restrict dates on the calendar rules that are retrieved. In the code above, the calendar is retrieved with all its business closure dates regardless of the year. If your CRM has been running a few years, it’s possible you have a lot of records in there and you may want to optimize your code by doing some fine tuning (get business closure dates from a specific date range). I’ll update this post after I get it to work if it ever happens J.

Making Cross Domain AJAX calls from CRM Form JScript

One of the cool things about JScript in CRM 2011 is that you can now use the jQuery $.ajax method to call, most notably the CRM REST endpoint. Now, one of the things that is sometimes required in customers’ solution is the ability to call other REST endpoints from a CRM form. An example of that use case would be to have a button that calls a web service in order to perform some sort of operation (e.g. modify your current record, or just send data over another systems for synchronization etc.) in a synchronous and interactive way. Another example would be to automatically fill fields on a form with information coming from another system (retrieve them via web service and set the values on the form). There are two main issues with doing that :

  • First, some browsers will simply not allow jQuery AJAX requests to run if they are made to a domain different than that of the running web application. The reason behind is that not all browsers can create XMLHttpRequest objects that have a withCredentials property. That is used to specify that we want to include the user credentials in a cross domain request. To always have that working, jQuery has the solution for us. Use the following statement in the code before making the AJAX call: jQuery.support.cors = true;
  • The other more complicated issue is that most browsers simply won’t allow JavaScript to make a cross domain request for security reasons. In most contexts, this sounds normal and reasonable.

Here are a few workarounds if you are facing the second issue.

  1. Try to transfer your logic to the server side. Can you do what you need by using a dialog or a workflow with a custom activity (server side code)? Can your logic be performed on a synchronous plugin (server side code)? If one of these solutions is acceptable, then this is what you should do.Applies to: CRM Online, On Premise, ADFS / IFD
  2. Use IIS URL Rewrite rules on your server. To illustrate, let’s say your CRM server URL is http://MyCrmServer/MyOrg and your external REST endpoint is http://RestEndpoint/RestService?params. You can create a URL rewrite rule that will transform all requests coming in as http://MyCrmServer/RESTENDPOINT/RestService?params to http://RestEndpoint/RestService?params. In other words, whenever IIS sees the string MyCrmServer/RESTENDPOINT in a URL it replaces it with RestEndpoint. It is simple to configure with Regular Expressions.

    This is a good solution because it handles the request on the server side. From a Jscript perspective, it is as if you are querying the CRM server and the request is transformed once it reaches the server. You can read more about IIS URL Rewrite here. The downside of this approach is that every single page that is opened from the CRM web app triggers a URL rewrite rule evaluation. Microsoft says the algorithm are optimized for performance and I have never experienced visible delays due to using the rewriter. Also worth noting that obviously, this is not supported for CRM Online since we don’t have access to Microsoft’s CRM Servers.

    Applies to: CRM On Premise, ADFS / IFD (if provider gives access to CRM Server)

  3. Enable Cross Domain browsing on the browser. In IE, this is blocked by default. You can change the settings by going to Internet Options Security and Custom Level Settings. In most case, this solution is not acceptable for an enterprise’s perspective for security reasons.


    Applies to: CRM On Premise, ADFS / IFD, (CRM Online? Not confirmed but it should work)

I personally think making requests to a cross domain web site via JavaScript should not be done if there are acceptable other options simply because it adds overhead somewhere based on the solution you go with. (e.g. if you go with solution 2 on premises, you have to make sure all your CRM servers have rewrite rules create and enable etc.).

[*** Update | 13/06/2014 ***] If you are using CRM 2013, see this article on how to use Action processes to get around this challenge.

Hope this helps!

Microsoft Dynamics CRM Data Archiving Solutions

As Dynamics CRM consultants, one of the questions that we often get from clients is “how do we archive CRM data when it’s no longer needed”? I’ve seen companies that had to keep all of their data for a certain amount of years for legal reasons. In any case, if you are using Dynamics CRM as a service management tool for a busy call center for example, chances are you are creating a LOT of records at a very fast pace. Imagine having to retain all of the data you created in the past 5 to 10 years on the same system. The size of the database can grow to become extremely large, making every single query a long one and impacting directly the users experience and their trust in the system.

Before I go into details about the available options that I can talk to, here is a quick recap of what Microsoft officially said a while ago about archiving Dynamics CRM data.

  1. Archiving implies moving records from one location, storing those records for possible retrieval in another location, and deleting the archived records from the original location.” Given the complexity of the relationships in CRM, archiving as defined in the previous sentence can be very risky and must imply extensive thought, design and knowledge of the CRM application and database.
  2. An arching solution should not be implemented with the goal of improving system performance.
  3. Instead of deleting records, it is recommended to deactivate them when they are no longer required.
  4. The Dynamics CRM application can support very large databases, it is important to make sure that the hardware and database are configured in order to get optimal performance before looking at Archiving.

Here are some possible approaches that can be used to archive your Microsoft Dynamics CRM data. Before going down that path, make sure this is something you really need, don’t just build an archiving solution because you want to improve your system performance. Do your homework first!

Archiving Method How it work Pros Cons
1 – Create a copy of the database It requires a strong DBA with knowledge of the CRM database to make a copy of the DB, copy all the records that need to be archived to the new DB and finally delete them from their original location. Sounds easy for a DB expert I assume (I’m not!)
  • It’s one of the less expensive solution as it only requires a strong DB developer with knowledge of CRM and an available SQL Server database
  • Not supported nor recommended by Microsoft
  • A DBA with knowledge of the Dynamics CRM database is always required to retrieve data from the archive database
2 – Create a new instance of Dynamics CRM
  • Create new organization or new CRM installation and install the same solution as the one in production
  • Create a mechanism to copy records from the production environment to the new archiving organization and delete them from their original location. This can be done with :
    • A custom .NET application using the CRM SDK
    • Scribe, Microsoft’s CRM 2011 Instance Adapter or other third party providers routines
    • Database scripting (unsupported)
  • The archived records are easy to retrieve (same UI as prod)
  • Access to these records can be controlled by the Dynamics CRM security model
  • Can be done in a supported way
  • Expensive solution that may requires additional hardware, server and/or user licenses for CRM or SQL Server
  • Company ends up maintaining more than one CRM organizations in the long run
3 – Deactivated unwanted records and tweak database Indexes
This is easily the number 1 choice from Microsoft’s perspective. To me, this should be one of the drivers in the CRM application design. The lifecycle of all records that are created (except for master data) should always be to:

  • create record (with active state obviously)
  • do work
  • deactivate record after the job is complete (set state to inactive)

In many systems that I have seen, some records just remain active forever, even though they are old and on longer required in the active records view.

Also, it is important to create SQL indexes to enable a faster retrieval of active records.

 

  • This method does not involve the deleting on any record from the CRM database which is what MS recommends
  • It forces the application to be designed with best practice for records lifecycle management
I could write down that the down side is that records remain in the CRM database but again, you are not archiving simply because you want to reduce the size of your database…
4 – Create and store custom archive reports This is an interesting one. I’ve personally recommended using this solution to capture the history of successful workflows execution prior to deleting them from the system.
The idea is to create custom reports containing information from the records that are no longer needed in the system. Once the reports have been created, they can be stored in a document management system or sent by email. After that, then the records can be deleted from the CRM database.
There are many ways to create reports in CRM so I won’t spend time discussion that topic here. A simple Google (or should I say Bing searchJ) will give you plenty of starting points.
  • Lots of options for reports creation and execution (3rd party software, SSRS, scheduling jobs, custom workflow activities etc.)
  • Can be done in a supported way
  • What used to be CRM records are now lines of text in different reports
  • Records must be saved in a managed location (example: doc management system with redundancy instead of external hard drive for the system admin…)
  • Access to the reports must be controlled
  • Ability to search for archived information can be limited and/or very painful in flat files
  • Building the reports can be expensive :
    • Report design can take time
    • Development cost depends on the tool used (SSRS, 3rd party…)
5 – Create a custom Archive Database
This is very similar to option 2. The difference would be instead of creating a new instance of CRM, you can create a custom database where you would only keep the information that you need. This database would be populated by a custom .NET app, third party data migration tool routine or database scripting.
Optionally, a custom UI can be developed to enable administrators to search through the custom database.
  • Tailored archiving solution, enables for more granularity and focus on retaining only the information that is required
  • If done well and if CRM solution(s) doesn’t evolve too much over time, it’s a one effort and maintenance should be easier than maintaining 2 CRM instances
  • Can be done in a supported way
  • Expensive to build (requires deep analysis and development time)
  • Custom database must be maintained
  • Security is no longer controlled by the Dynamics CRM application

I personally like solution 5 better but it’s only because I’m a developer and I’d enjoy having to design the new database and the archiving tool. Of course, every company has its own preferences and resources with different set of skills. This should all be taken into consideration prior to deciding what solution to use. These are only the ones that I came up with, there are certainly other options out there.

Hope this helps!