Plugin on Retrieve and Retrieve Multiple – How bad is it?

I have managed to be in the Dynamics CRM/365 world for over 7 years without having to write a single plugin on Retrieve and Retrieve Multiple. The recommendation that I give is to stay away from those. The reason is simple, it sounds horrible from a performance standpoint, and even people from Microsoft have recommended against it in many scenarios. Faced with an issue recently where we had to really consider it, I did some research and testing to try to measure the impact of such plugins on system performance. This article provides some background as to why we recommend against these types of plugins, and it also provides some of our finding after we tested for performance.

Why are Plugins on Retrieve Multiple scary?

When looking at the event execution pipeline for Dynamics CRM/365, we need to consider that there are a lot of steps involved as part of every CRM transactions. To do anything, we need to go through the CRM web service APIs which will start the chain of events in the pipeline (pre-validate, pre-event, core action/database access and then post-event).


This means that in general, it is a good practice to build everything for optimized performance to give your user base a good experience. It’s not like having a custom database where you can create store procedures easily, add triggers and so on, taking advantage of the SQL Server features and infrastructure.

Now, back to Retrieve and Retrieve Multiple plugins.

Retrieve Multiple: Think about it this way, you retrieve a list of 200 accounts, your plugin on Retrieve Multiple fires once and gives you the list of accounts being returned to the screen with all columns being retrieved. For each of these row, you do some type of operation. It doesn’t too sound good, does it?

Retrieve: You double click on an account from a list view. As the account columns are being retrieved to display the account form on the screen, you plugin on Retrieve fires and gives you the account object. At that point you can modify the content of the columns being retured as required before they are returned to the screen for the user. This really doesn’t sound too bad.

Some Findings on the impact on Performance

To provide some context into what we were trying to do, I wrote about multi-language lookup in a previous blog post. One of the solution that we have considered for one of our client is to use a plugin on Retrieve and Retrieve Multiple in order to change the value of the lookup primary fields in order to display a value in the user’s current language. The method we used is similar to what Aileen Gusni does here and almost identical to what Scott Durow does here.

We store a Region in a custom Entity. Accounts have a lookup that indicates its region.

Scenario 1:

The Region’s Primary Field contains a concatenation of the English and French region names with a relatively safe separator (we use “|”). When you load the list view of accounts, we have a plugin on Retrieve Multiple that looks at the columns being returned. If the Region column is returned, we retrieve the user’s language, we split the name of the region with the separator (the name is available in the entity reference) and we replace its value in the target object by the region name in the user’s language. The plugin on Retrieve does the same operation on the single role being retrieved.

The average execution time for the Retrieve Multiple plugin when loading 150 rows was 15.43 milliseconds so 0.01543 seconds.
The average execution time for the Retrieve plugin to load one rows was too little for the system to return a value (we got 0 milliseconds every time).

Scenario 2:

The Region entity has English Name and French Name attributes. When you load the list view of accounts, we have a plugin on retrieve multiple that looks at the columns being returned. If the Region column is returned, we retrieve the user’s language, we then retrieve the English or French name from the Region entity and we replace its value in the target object by the region name in the user’s language. The plugin on Retrieve does the same operation on the single role being retrieved.

The average execution time for the Retrieve Multiple plugin to load 150 rows was 1003.94 milliseconds so 1.00394 seconds.
The average execution time for the Retrieve plugin to load one rows was 16.02 milliseconds so 0.01602 seconds.

What should you read into this?

While these numbers don’t look too crazy at all, especially in the first scenario, there are a lot of factors to take into consideration that are not really showing here and that will vary in almost any scenario.

  • What is the infrastructure you are running on? The faster your severs and networks, the better the performance will be.
  • What you do in these plugins matters a great deal. You should avoid or limit the number of read/writes to the database during the execution of those plugins.
  • Our tests were made with a low level of users in the system, it is critical to scale up and see what these numbers look like at peak time of your system.

With all this said, I still recommend against it. Use with a great deal of caution! If/when possible, use calculated fields instead of writing plugins on these messages. This will also keep you away from limitation such as this one.

Hope this helps!

Advertisements

CRM Plugin Code Structure – By functionality or by event?

When we work on CRM plugins, there is a key decision at the beginning of the development cycle when it comes to structuring your plugin code. There are two main ways of writing your plugins :  by events or by functionality. In this post, I will describe the two approaches and discuss some of their pros and cons.

Plugins can be described as handlers for events fired by Microsoft Dynamics CRM (source). There can be multiple handlers for single events.  Let’s use an example for the purpose of this post.

For the event, we’ll use the creation of a case (pre operation). We will consider that when  the case is created, there are two actions that need to be performed : generate and assign a unique case number and validate that required fields have authorized values in them.

Plugin structured based on Events

For the plugin structure based on event, it means we’ll have a single plugin registered on the Case Pre create event, and inside the plugin class we’ll have two business logic blocks of code to execute (one to generate the unique number, the validate the required fields).

SNAGHTML4e81484     SNAGHTML4e74f7f

Plugin structured based on Functionality

In this case, we’ll have two plugin classes registered on the same event (again, one plugin for unique number generation and another to validate the required fields) – Sorry, no screen shot here but you get the idea Smile.

Pros and Cons

Functionality Based Plugins

  • Pros
    • Ease to see what plugins does what when looking at Plugin Registration tool (or similar tools) because of more descriptive feature name in plugin name
    • Ability to disable specific features for various scenario (e.g. disable system integration or auto calculation plugins during data migration)
    • Unit Tests can be written against a plugin class to test one functionality
    • Provides the ability to easily write generic plugins and register on multiple events by any power user
  • Cons
    • Large amount of plugin classes
    • Large amount of registered plugin steps; can become big and hard to read/maintain

Event Based Plugins

  • Pros
    • Plugin registration is straight forward and easy to maintain
    • Ease to easily the operations performed at each event when looking at the code
    • Plugin classes are standardized : always tied to an event and single point of entry for all developer to see actions executed on the event
    • Unit tests can be written against specific Business logic methods and/or classes (even independently from the plugin execution context depending on how they are written)
  • Cons
    • Must use alternate mechanism to disable specific features if needed (e.g. configuration data element)
    • Possible timeout if executing multiple large business logic blocs (mind you, if you have a timeout in a plugin – which happens after two minutes, you probably have a design issue)
    • General  dependency to having a developer around
      • Generic business logic code blocks can be written but a developer is required to insert the business logic on additional events
      • To view the all operations performed for an event, a developer needs to look at the code

So which model should you use?

I have used both models heavily on various occasions. For me personally, the decision on which one to use comes down to a project’s needs and a coding style preference. As an example, if you know you’ll have a lot of generic features and want to have reusable plugins, you are better off using the functionality based model. The same applies if you know there are processes that require to disable specific functionality on a regular basic. On the other hand, if you project is relatively small and includes simple business logic on a few events, the event based approach can be a good choice. Feel free to add feedback in the comment section if you see other pros and cons for each of these models or want to share experience in the area.

Creating readable unique identifiers for your CRM records

A common requirement in CRM implementations is the need to have readable identification numbers auto-generated for custom attributes. There are various ways of achieving this. This article provides a few options to do that.  

Let’s use one scenario as a context. Let’s say we are building a Grants Management application and we are tracking scholarship applications in Dynamics CRM. We have an application entity which contains an Application Id field. For business reasons, we need to have a unique identifier to refer to the applications. What are out options? For starters, let’s look at some of the elements to consider when looking for an auto-numbering solution in Dynamics CRM?

  • Are you using an entity that is auto-number enabled? (e.g. Case, Quotes, Orders etc.)
  • Do you need the numbers to be sequential?
  • Do you have potential race conditions? (i.e. can the event assigning the auto-number be triggered by multiple processes simultaneously?)
  • Do the auto-numbers need to be human readable?
  • What is the format required? (numeric only, alpha-numeric, letters only, prefix, suffix etc.)
  • Can you and/or do you want to write code to generate these numbers or do you want an simple configurable solution?

Generate an unique number based on the current Date Time

Of course, this required a plugin on an event that will generate the application Id. From a technical standpoint, there are a few algorithms out there that will give you an indication as to how this can be done. The full date time and the “tick” should be use to generate the number to ensure uniqueness. But even with that, there is a very very very small chance of having race conditions depending on your application and on your (bad) luck. Also, this will not allow your to use sequential numbers.

Create a record of an auto-numbering CRM entity

If you are *not* using one of the CRM auto-numbering enabled entities (for example, Campaign), this solution is have a plugin (or a Workflow) on your triggering event that will create a Campaign record. This campaign will automatically be assigned a unique number based on your CRM configuration (image below). You can re-use that number as the unique ID for your record and delete the Campaign record that was created. While this is a simple method that can be implemented without coding, the numbers are not really sequential, “garbage” records are created at a cost (create/delete transactions time, what happens if you want/need to use that entity in the future?..). To be honest, it feels like a quick-and-dirty way of doing things, but it works.

The advantage here is that because the auto-numbers are platform generated, so you do not have to worry about race conditions in theory.


Use a model based on CRM Synchronous Workflow + Custom Auto-Number entity

I will not go too much into details about this one. There are lots of articles on the topic. A few “how-to” articles for reference:

Bottom line, this is a great solution if you cannot write any code and don’t want to spend any money for something that should be standard in the platform (yes, I’m ranting, can’t help it). That being said, it you have race conditions, I’m sure you can have a few duplicated numbers with this solution.

Using a 3rd Party Tool

There are plenty of options out there. Some are more flexible than others. Some explicitly guarantee uniqueness, some don’t really mention it. Some are free, some are not.

These are just a few, a research will allow you find more of these and decide the one that fits your needs the best.

Build your own solution

At this point, you can do it anyway you want.

  • If you have access to an external database, a CRM plugin that create a record in the database with an identity column will guarantee to always have unique and sequential numbers. This comes at a cost obviously (external database dependency, speed)
  • There is also a smart way of doing robust auto-numbering suggested by Ken Heiman of Green Beacon here. Good read, smart idea.
  • Some enterprises already have Auto-Numbering web services, you can simply consume that web service from a CRM plugin as well (same cost as database, dependency, speed)
  • I have also seen the idea of using the Guid as a unique identifier, by copying the value into a text field with a plugin. I don’t think it’s readable, but definitely unique.

 

It had been too long. Happy CRM’ing ! 

How to remove dependencies to CRM Developer Toolkit

Today I had to work on a solution that I hadn’t touch in a while. It was built with Visual Studio 2012 using the CRM Developer Toolkit. Since I made the decision not to use it, I needed to find a way to use the solution with Visual Studio 2015 without having to recreate one.

It is pretty simple.

If you have a “Package Project” that includes Web Resources, here are the steps you should take to get rid of it:

  • Create a new project (I usually use a library, empty project).
  • Drag the web resource from the CRM Package project to your newly created project
  • Delete the CRM Package project from your solution
  • To make sure the new project does not built (it doesn’t need to, it’s only JavaScript files), you can remove it from the build actions in the solution’s properties as shown below.

image

For the Plugin or Custom Workflow activities project types, it is also simple enough. If you open the solution in VS2013 or VS2015, you will see that the project is flagged as not compatible.

image

To fix this, a small operation is required. Find the location of the project on the computer and open the corresponding project file (.csproj) and open it with a text editor.

image

Once open, find the “ProjectTypeGuids” node under the Project –> PropertyGroup node and delete it.

image

Close and re-open the Solution in Visual Studio 2013 or 2015. In my case, it didn’t work right away. I was hit with an error when trying to open the project files “No exports were found that match the constraint: ContractName”:

image

To fix this, simply remove the ComponentModelCache folder content from the folder %AppData%\..\Local\Microsoft\VisualStudio\11.0\ComponentModelCache

Again, close the solution and re-open it, It should work like charm! Life is good.

Cheers

Managing CRM User Roles Programmatically

You often hear CRM users and admin complain about having to manage the system users’ roles too often. It can be a pain if you have a lot of movement in the company, for example people going from one business unit to another, high turnover rate or fast growing company.

I thought I would share a design idea for making user management a little bit easier for your organization. It consists of a few steps, and it requires some coding skills.

  • Define your business roles and match each of them to a set of CRM Security Roles.
Business Role CRM Security Roles
Sales Manager SalesPeron; Sales Manager
IT Manager System Customizer
  • On the “User” entity, create an Option Set field representing the business roles capture in step 1

Now the fun begins.

  • Create an XML file to map the business role option set values to a set to CRM Security Role names (see XML format in picture below)
  • Upload the XML file to CRM as a web resource
  • Switching over to the plugin side. You need to build a plugin that runs on update of a User record and that that does the following operations:
    • Read the “Business Role” field value on the User record
    • Retrieve the web resource configuration XML file based on its name or ID
    • Parse the XML to get the CRM roles corresponding to the business role selected
    • For each CRM Security Role found, assign it to the user
    • This is what my plugin code looks like:
public override void Execute(IExecutionContext context, IOrganizationService service, ITracingService tracingService)
{
 if (context.MessageName != "Update") return;

if (context.InputParameters.Contains("Target") && context.InputParameters["Target"] is Entity)
 {
 Entity target = (Entity)context.InputParameters["Target"];
 if (target.Attributes.Contains("sa_userrole") && target.Attributes["sa_userrole"] != null)
 {
 SystemUser user = target.ToEntity<SystemUser>();

// Get Security Roles from Configuration
 WebResource wr = GetWebResourceByName(WebResourceName, service);
 string xmlContent = UnicodeEncoding.UTF8.GetString(Convert.FromBase64String(wr.Content));

XmlDocument config = new XmlDocument();
 config.LoadXml(xmlContent);

XmlNodeList crmRoles = config.SelectNodes("/SystemRoles/SystemRole[@ID=" + user.sa_UserRole.Value + "]/Role");
 foreach (XmlNode role in crmRoles)
 {
 Role crmRole = FindSecurityRole(user, role.InnerText, service);
 AssignSecurityRole(crmRole, user, service);
 }
 }
 }
}

/// <summary>
/// Retrieves a CRM Web resource by name
/// </summary>
private WebResource GetWebResourceByName(string webResourceName, IOrganizationService service)
{
 QueryByAttribute requestWebResource = new QueryByAttribute
 {
 EntityName = WebResource.EntityLogicalName,
 ColumnSet = new ColumnSet(true),
 };
 requestWebResource.Attributes.AddRange("name");
 requestWebResource.Values.AddRange(webResourceName);

EntityCollection webResourceCollection = service.RetrieveMultiple(requestWebResource);
 return (WebResource)webResourceCollection.Entities.FirstOrDefault();
}

/// <summary>
/// Find a Security Role by name and business unit
/// </summary>
private Role FindSecurityRole(SystemUser user, string roleName, IOrganizationService service)
{

if (user.BusinessUnitId == null || user.BusinessUnitId.Id == Guid.Empty)
 {
 SystemUser retrieveUser = service.Retrieve(user.LogicalName, user.Id, new ColumnSet("businessunitid")).ToEntity<SystemUser>();
 user.BusinessUnitId = retrieveUser.BusinessUnitId;
 }

// Retrieve a role from CRM.
 QueryExpression query = new QueryExpression
 {
 EntityName = Role.EntityLogicalName,
 ColumnSet = new ColumnSet("roleid"),
 Criteria = new FilterExpression
 {
 Conditions =
 {
 new ConditionExpression
 {
 AttributeName = "name",
 Operator = ConditionOperator.Equal,
 Values = {roleName}
 },

 new ConditionExpression
 {
 AttributeName = "businessunitid",
 Operator = ConditionOperator.Equal,
 Values = { user.BusinessUnitId.Id }
 }
 }
 }
 };

return service.RetrieveMultiple(query).Entities.Cast<Role>().FirstOrDefault();
}

/// <summary>
/// Assign Security Role to a CRM User
/// </summary>
private void AssignSecurityRole(Role securityRole, SystemUser user, IOrganizationService service)
{
 service.Associate("systemuser",
 user.Id,
 new Relationship("systemuserroles_association"),
 new EntityReferenceCollection() { securityRole.ToEntityReference() });
}

At this point, all you have to do is change that field on your users’ form and the plugin will handle assigning the security roles automatically. You may want to implement a way to handle the security roles assigned to users before the business role is changed. If you are planning to use this design or something similar, here are some key takeaways:

  • The same logic cannot be processed at the creation of the “User” record. That is because before being able to assign security roles to a user, the system needs additional information that gets created after the user record is created.
  • The plugin must run as an administrator or a user who has enough privileges to create and assign security roles to a user
  • The configuration is stored in a XML web resource but it could be stored anywhere (CRM entity, external database, flat file etc.)
  • Also note that Security Roles are copied to each business unit, so when looking a Security Role in CRM based on its name, it is important to search in the same business unit as the user’s
  • CRM does not allow to edit multiple user records at the same time

Have fun with this!

Retrieving Business Closure dates from Dynamics CRM

As developers, we are often asked to write plugins to calculate durations. A common example would be that the business wants to calculate how long an Opportunity was opened for. From a business perspective, it makes more sense to count only the business days, which is easy to do in C# (there are plenty of algorithms out there to filter out the weekends from a time interval). It makes even more sense to also exclude the holidays which are defined in the Business Closures calendar in CRM.

It is a nice feature and it integrates well with the out of the box Service module in the sense that you can configure it to prevent users from scheduling service activities on holidays.

From a development perspective however, it’s not that easy to deal with the Business Closures calendar for a few different reasons:

  • There is no clear documentation how to deal with them programmatically
  • In the Business Closures Calendar, we know they are stored in the related list of Calendar Rules, but the calendar rule entity doesn’t support Retrieve Multiple queries which makes the dates uneasy to retrieve
  • You can only retrieve the business closures by getting the business closure calendar, the latter supports Retrieve Multiple

You may get confused with the ExpandCalendarRequest that is out there. It is used to retrieve the calendar rules from a user’s calendar. The idea being to retrieve a set of time blocks with appointment information for a specific user (these are also CalendarRules). If you try to retrieve the rules for the business closures calendar, you get a rather obscure error: “Invalid time zone code”.

After spending some time on that route, I reverted back to the original recommendation by many on forums and blogs and I managed to make it work with the code below. The idea is to retrieve the business closure calendar ID of the organization and then return its list of CalendarRule. From there, it’s easy to use to result to do what you need.

private IEnumerable<Entity> GetBusinessClosureCalendarRules(IExecutionContext context, IOrganizationService service)
{
    // Get Organization Business Closure Calendar Id
    Entity org = service.Retrieve("organization", context.OrganizationId, new Microsoft.Xrm.Sdk.Query.ColumnSet("businessclosurecalendarid"));

    QueryExpression q = new QueryExpression("calendar");
    q.ColumnSet = new ColumnSet(true);
    q.Criteria = new FilterExpression();
    q.Criteria.AddCondition(new ConditionExpression("calendarid", ConditionOperator.Equal, org["businessclosurecalendarid"].ToString()));
    Entity businessClosureCalendar = service.RetrieveMultiple(q).Entities[0];
    if (businessClosureCalendar != null)
    {
        return businessClosureCalendar.GetAttributeValue<EntityCollection>("calendarrules").Entities;
    }
    return null;
}

With all this in place, you now know how to retrieve your business closures dates. My next step is to see if it is possible to restrict dates on the calendar rules that are retrieved. In the code above, the calendar is retrieved with all its business closure dates regardless of the year. If your CRM has been running a few years, it’s possible you have a lot of records in there and you may want to optimize your code by doing some fine tuning (get business closure dates from a specific date range). I’ll update this post after I get it to work if it ever happens J.