Plugin on Retrieve and Retrieve Multiple – How bad is it?

I have managed to be in the Dynamics CRM/365 world for over 7 years without having to write a single plugin on Retrieve and Retrieve Multiple. The recommendation that I give is to stay away from those. The reason is simple, it sounds horrible from a performance standpoint, and even people from Microsoft have recommended against it in many scenarios. Faced with an issue recently where we had to really consider it, I did some research and testing to try to measure the impact of such plugins on system performance. This article provides some background as to why we recommend against these types of plugins, and it also provides some of our finding after we tested for performance.

Why are Plugins on Retrieve Multiple scary?

When looking at the event execution pipeline for Dynamics CRM/365, we need to consider that there are a lot of steps involved as part of every CRM transactions. To do anything, we need to go through the CRM web service APIs which will start the chain of events in the pipeline (pre-validate, pre-event, core action/database access and then post-event).


This means that in general, it is a good practice to build everything for optimized performance to give your user base a good experience. It’s not like having a custom database where you can create store procedures easily, add triggers and so on, taking advantage of the SQL Server features and infrastructure.

Now, back to Retrieve and Retrieve Multiple plugins.

Retrieve Multiple: Think about it this way, you retrieve a list of 200 accounts, your plugin on Retrieve Multiple fires once and gives you the list of accounts being returned to the screen with all columns being retrieved. For each of these row, you do some type of operation. It doesn’t too sound good, does it?

Retrieve: You double click on an account from a list view. As the account columns are being retrieved to display the account form on the screen, you plugin on Retrieve fires and gives you the account object. At that point you can modify the content of the columns being retured as required before they are returned to the screen for the user. This really doesn’t sound too bad.

Some Findings on the impact on Performance

To provide some context into what we were trying to do, I wrote about multi-language lookup in a previous blog post. One of the solution that we have considered for one of our client is to use a plugin on Retrieve and Retrieve Multiple in order to change the value of the lookup primary fields in order to display a value in the user’s current language. The method we used is similar to what Aileen Gusni does here and almost identical to what Scott Durow does here.

We store a Region in a custom Entity. Accounts have a lookup that indicates its region.

Scenario 1:

The Region’s Primary Field contains a concatenation of the English and French region names with a relatively safe separator (we use “|”). When you load the list view of accounts, we have a plugin on Retrieve Multiple that looks at the columns being returned. If the Region column is returned, we retrieve the user’s language, we split the name of the region with the separator (the name is available in the entity reference) and we replace its value in the target object by the region name in the user’s language. The plugin on Retrieve does the same operation on the single role being retrieved.

The average execution time for the Retrieve Multiple plugin when loading 150 rows was 15.43 milliseconds so 0.01543 seconds.
The average execution time for the Retrieve plugin to load one rows was too little for the system to return a value (we got 0 milliseconds every time).

Scenario 2:

The Region entity has English Name and French Name attributes. When you load the list view of accounts, we have a plugin on retrieve multiple that looks at the columns being returned. If the Region column is returned, we retrieve the user’s language, we then retrieve the English or French name from the Region entity and we replace its value in the target object by the region name in the user’s language. The plugin on Retrieve does the same operation on the single role being retrieved.

The average execution time for the Retrieve Multiple plugin to load 150 rows was 1003.94 milliseconds so 1.00394 seconds.
The average execution time for the Retrieve plugin to load one rows was 16.02 milliseconds so 0.01602 seconds.

What should you read into this?

While these numbers don’t look too crazy at all, especially in the first scenario, there are a lot of factors to take into consideration that are not really showing here and that will vary in almost any scenario.

  • What is the infrastructure you are running on? The faster your severs and networks, the better the performance will be.
  • What you do in these plugins matters a great deal. You should avoid or limit the number of read/writes to the database during the execution of those plugins.
  • Our tests were made with a low level of users in the system, it is critical to scale up and see what these numbers look like at peak time of your system.

With all this said, I still recommend against it. Use with a great deal of caution! If/when possible, use calculated fields instead of writing plugins on these messages. This will also keep you away from limitation such as this one.

Hope this helps!

Setting up your Dynamics CRM Development & Source Control Environment

Every time I meet people trying to get CRM projects going, I get plenty of questions about how to setup environments and source control properly. Unfortunately, there is no easy answer. However I thought I would share a model that I’ve used on many occasions and that works well with a combination of a defined methodology, appropriate tools, training, commitment and discipline from the project team members. It is mostly relevant for large projects with multiple developers.

The proposed setup is based on the flowchart below, and it is inspired by fellow MVP Gonzalo Ruiz’s article on the same topic.

 

Development machines (CRM DEV)

The development machines can be a developer’s laptop or work station with CRM development tools + a dedicated CRM organization for the developer (CRM Online or on premise). It can also be a full server (or VM) with CRM installed and dedicated to a single developer. These organizations are setup with a master configuration solution (more on this later) and smaller transport solutions to bring required components for development.

Developers use the various development tools to write code and to debug (attach to CRM services, attach to IIS, using Profiler, browsers debugging tools or using tracing).

When development items are completed, developers build temporary “transport solutions” containing the components they worked on and they push them to the Master Configuration organization. Developers must be careful and not make schema changes on their own development organizations. As the code is deployed to the master configuration, it should also be archived in the source control.

Master Configuration CRM Organization

There is only one Master Configuration environment. It contains the schema configuration and all schema changes should be made there. There is usually one or multiple “Master” solutions sitting in this organization based on how you decide to structure your deployments. You may want to have multiple solutions for various types of customizations (e.g. one for schema, one for plugins, one for web resources etc.), or based on functionality grouping.

When developers set up their organizations, they can pull the entire master solutions from the master configuration organization as unmanaged and install them in their development. They can also create temporary “transport solutions” to pull only the customizations they need.

A process (manual or automated) should be put in place to archive the master solutions on a regular basis (daily is recommended). Archiving could be saving the CRM solution files in source control, or even creating snapshots of the server and keep them around for a reasonable period of time.

Integration / Test CRM Organization

This organization receives the master solutions and is used to perform tests. It can be combined with the master configuration environment. The choice to go with a managed or unmanaged solution when going from master configuration to integration/UAT/Prod environment is a business and technical decision. It is a separate discussion that we could write an entire blog post about J.

UAT / PRE PROD / PROD

These environments receive the master solution(s) after they have been tested and validated on the Integration/Test CRM organization.

Wrap Up

Putting this type of environment in place is not easy to do. It takes a lot of process implementation, tools and automation build and more important, training and discipline from the project team members. When working on small project (small number of developer, low amount of customizations and code), going with this approach is overkill. It applies best for big projects when you have multiple developers and heavy customization requirements.

 

This was fun. Hope it helps!

Dynamics CRM Archiving Solutions Revisited

I wrote about Archiving solutions for Microsoft Dynamics CRM about 2 years ago. I still get a lot of questions from the article and from customers in the field about what they should do when it comes to archiving their Dynamics CRM data. While there is no obvious answer, my generic approach is to make sure my clients understand that they shouldn’t try to build an archiving solution before their system has become slower. They should see it more as an opportunity to perform a thorough review of their implementation and isolate the bottleneck(s) and pain point(s) and see if those can be fixed.

A few months ago, I gave a presentation on the topic that you can see below. I go from the reasons to archive data to recommendations. Hope this helps!

Dynamics CRM Solution Architecture

Hello readers!

I just completed a series of very animated presentations on Solution Architecture for Dynamics CRM in a few different cities in the east of Canada. I thought I would share the slide deck.

As a summary, I complied a few thoughts about the CRM Solution Architect’s role and typical design consideration. Enjoy the content and feel free to reach out to me with questions and comments.

“Only Secure Content is displayed” warning message when viewing CRM forms with IFRAME

After configuring an existing CRM solution to display some external content inside of IFRAMEs, we had to move the CRM web site from port 80 to SSL. After completing the configuration, we ran into a constant warning message in the browsers:

  • Internet Explorer: “Only Secure Content is displayed” with a button to allow all the content to be loaded
  • Chrome: “This page includes script from unauthenticated sources” with a link to load the unsafe script

The warning is actually indicating an obvious point: when you are running a web site configured for HTTPS/SSL, it makes sense from a security perspective to not display frames or other content that is not secured. The resolution is extremely simple, just move your custom content to a secured web site. The warnings will go away and the content of the frame will be displayed.

On the flip side, you can display secured content even if your CRM website is not configured for HTTPS.

Common CRM Solution Package Components

As a CRM Solution Architect, one thing that I often try to do is keep the complete solution package with a minimal amount of components in order to simplify the deployment process. In an ideal world, we want to limit a deployment to a solution deployment. In reality, this never happens on complex implementations. The purpose of this article is to point out some of common elements that are typically deployed as part of a CRM solution deployment.

  • CRM Solution: This is the .ZIP file extracted from your development CRM server that contains the system configuration and customizations. As part of a deployment, you typically always have one or multiple CRM solutions to apply to the target environment
    • Steps: Import CRM solution(s) in your target environment
    • Can be automated: Yes, use CRM SDK
  • CSV Files for data import: There is often a need to load master data on the system. For example, if you have entities for “Country” and “State”, you may want to populate them with data before people start using the system. This is easily done with a CRM data import wizard.
    • Steps: Import CSV or XML files in your target environment
    • Can be automated: Yes, use CRM SDK
  • Run Workflow(s) against a specific set of record: This is a technique that is often used to fix data issues. Examples of use: change record ownership, populate new fields created as part of the current release, delete records (data clean-up).
    • Steps: Define the query that will return the records you need to run your workflow against, find them (Advanced find or FetchXml) and execute the workflow against all records
    • Can be automated: Yes, use CRM SDK
  • Perform data load from external source: Loading data is usually done by external system (SSIS, Scribe, custom integration etc.) that have connectors to CRM. As part of a CRM deployment, you’ll often have to run data load/update tasks after configuring your integration service to point to your production server. This is used to do data migration or to enable on-going integration between multiple information systems.
    • Steps:
      • Deploy integration components and/or configure them to execute on your target environment (e.g. install SSIS packages, configure Scribe etc.);
      • Turn on integration jobs
    • Can be automated: Depends on the integration tools used
  • Deploy custom web services and/or web sites: To provide extended integration with CRM, we often need to create our own web services that expose custom operations involving CRM data. We also frequently see the need to build custom web applications outside of CRM to display complex and/or integrated data within CRM screens or to create external portals. These web services and web apps become part of the release cycle of your CRM solution.
    • Steps: Package web services and web sites; deploy them on target IIS server(s) and configure CRM endpoint(s) in Configuration files as needed
    • Can be automated: Yes, depending on how web apps is built and packaged

When delivering enterprise solution deployment packages, one best practice is to always plan to automate the deployment of all components that be can be automated. It shortens the solution package deployment process, limits the possibilities of human errors and helps make deployment administrators happier which results in them having a better relationship with the solution and product (I speak from experience J).

Customizing your CRM platform the “right way”

Over the course of my consulting experiences, I have learned to work with different platforms and products “the right way.” This means that when you are using a product (for example Microsoft Dynamics CRM), you are agreeing to use the product in the ways specified by the makers of the product. It’s like when you buy a car, there are parts that go with it and if you need to change them, you need to get them from the manufacturer. When you start modifying the car with custom parts, you lose your manufacturer’s warranty. Same thing with a phone, you jailbreak it, you don’t get support anymore. The reason behind that is very simple. When you make changes that are not prescribed by the maker of a product, it’s possible that you altered the way the product works (i.e. you may have broken it or it may break in the future), therefore the manufacturer can no longer support it because it doesn’t know if it still works as designed.

When it comes to software and especially in the case of CRM, it is very important not to break that contract. There are a few things to consider if you do and they are very well documented:

  • Unsupported changes are dangerous by nature, you don’t know what the impact is on things that you are not supposed to have control over
  • They will probably break in the future (after a patch or upgrade)
  • They can bring tons of other issues (performance, testability, bugs…)

If you are buying a product that is integrated with Dynamics CRM (ISV solution), always ask this question to the vendor: “Is you product and/or integration built in a supported way by following all Microsoft customization guidelines and APIs”? See the table of what should be your reaction based on the answer below

Answer My2cents What to do
No That’s just bad. It will either stop working at some point or it will break something. Run away!
I don’t know If the vendor doesn’t know, it means he/she is not really aware of what the impact of unsupported customizations are. That sends a wrong message and represents a serious risk. Run away or hire a technology expert to validate that all is supported.
Yes Perfecto! The vendor knows what it means and understand the importance of supported customizations. It’s a go! To protect yourself, make sure it’s mentioned in your purchase contract.

As consultant and community contributor, I keep seeing a lot of web application and ISV providers that integrate with different CRM or other platforms by simply checking out how to do stuff online and customization the platform in unsupported ways to achieve their goal and sell their product. Keep in mind that working with a platform requires the due diligence of working with experts who know how to customize it. Taking shortcuts can seem like a great idea to quickly go to market but consequences can be bad and the result is often hiring experts to figure out what is wrong when they should have been brought in from the beginning.

It’s OK to use unsupported customization as long as you are aware of the risks and are willing to live with them.

You can read more on unsupported customization for MS Dynamics CRM on the links below.

http://dynamicsuniversity.com/blog/customizing-microsoft-dynamics-crm-2011-supported-or-not-supported
http://gustafwesterlund.blogspot.ca/2012/06/unsupported-customizations.html
http://gonzaloruizcrm.blogspot.ca/2011/08/risks-of-unsupported-customizations-in.html

Cheers