Custom User Interface in CRM – HTML5/JavaScript or ASP.NET Web application?

Lots of times on CRM projects, we are required to build and display custom screens inside of Dynamics CRM. There are a lot of factors that come into play in the choice how we build these custom user interfaces. It is a very key decision to make because it can sets the standard technologies and tools that you use for custom screens on project (among other things). I thought I’d share some key points to consider in your decision making process to determine what technologies and tools to use.

The need for custom user interface

First, it is important to know and understand when it is required to build custom UI as part of your CRM solution implementation. As a wrote a few years ago, we sometimes find ourselves in situations where we can come up with a complex data model and we feel like the out of the box UI capabilities won’t be user friendly enough to drive user adoption. Situations like that create the need to build more user friendly interfaces to simply make people’s lives easier when they start using the CRM/XRM application. Some examples are

  • Custom calendar view for timekeeping
  • Tree view to display a complex hierarchy
  • Interactive Bing or Google maps views of CRM data
  • Editable grids
  • Complex grids with aggregates, grouping functionalities
  • Integrated views to other systems

And the list goes on. The needs are very different depending on the nature of the enterprise looking to implement a CRM/XRM solution.

Most common technology choices

From experience, I have seen a lot of custom screens built using the following tools/technologies:

  • Silverlight
  • ASP.NET / ASP.NET MVC
  • HTML/HTML5 + JavaScript + CSS

The screens are usually displayed in web resources or in an IFRAMES inside CRM on forms or as standalone modules (open from a button, display in sitemap area). I will not cover Silverlight since Microsoft announced its end of life and continued support for a few additional years, plus the world has started to move on.

ASP.NET / ASP.NET MVC

  • Pros
    • Flexible server side business logic
    • Easily integration with external systems (web services, databases etc.)
    • Can be very quick to develop depending on the requirements, expertise and accelerators (advanced controls provided by Telerik, Infragistics or others can speed up development time significantly)
    • Develop with a familiar programming language (C# + Visual Studio) – this of course depends on your development background
  • Cons
    • Requires the deployment and maintenance of a hosted web application
    • You have to deal with the weight of ASP.NET (ViewState, Postbacks etc.)
    • Not a great fit with CRM Online and cloud offerings in general (cloud deployment, cross origin web sites, integrations etc)
    • Required additional configuration in CRM + JavaScript to set the IFRAME’s sources (URL)
    • Effort for developing custom web applications is often under-evaluated

HMTL(5)/JavaScript

  • Pros
    • Usually, a much lighter solution and if done properly, custom UI components can be very responsive and provide great user experience
    • All components are in CRM Web Resources (html pages, JavaScript files, images…)
    • Custom screens are CRM Solution aware and there is no need for deploying and maintaining an additional web site on a web server
    • A definite plus for cloud based Dynamics CRM implementations
    • Can be very quick to develop depending on the requirements, expertise and accelerators (Kendo UI controls or others can speed up development time significantly; other frameworks like Knockout JS, Bootstrap can also be useful)
    • Leverage trending technologies and tools
    • Use any IDE you want (no obligation to use Visual Studio, even if you should J)
  • Cons
    • Possibly additional complexity if you need to display information from an external system
    • If not structured properly, JavaScript code can become massive and hard to navigate/maintain
    • For non-web developers, might be a steep learning curve
    • Need to establish good project structure and standards (e.g. use TypeScript or Script Sharp)

What else to consider?

There are other factors to take into account, some being specific to each organization. A few examples:

  • What is the expertise or your resource pool?
  • Do we have people to build and maintain what we deploy?
  • Are we in CRM Online or planning to move to CRM online?
  • Are we, as a company, trying to get into HTML/JavaScript more?

This is good food for thoughts for starters. If you want more guidance, feel free to reach out to us.

Advertisements

Using Full-Text Search to improve Dynamics CRM search experience

Consider the following scenario: You need to perform email searches in your CRM and your search needs to look at the content (body) of your email messages. When you start having a lot of emails in your CRM (500k+ in my case), searches become extremely slow and you need to start to look at indexes or other solutions to make the search faster and keep your users happy. This post describes a solution that leverage a new CRM feature in CRM 2015 Update 0.1 (On Premise) : Full-text search.

Enabling Search on email body

This is the easy part. If you are having this issue, you have probably done this already. To enable the ability to search on the email body, you need to add the “Description” field as a “Find Column” in the Quick Find view of the Email entity, as shown in the screenshot below. After completing that step, save the view and publish the entity to make the change available in the system.

image

Searching through emails

Now that we’ve configured the search column, we can do a quick search to get our emails by using the search box on the activity list view view or by using the “Global Search” on the top right of the top navigation bar, next to the “Quick Create” button. Note that emails are not enabled in the “Global Search” by default so you’d have to add them (not required).

If you have a few emails in your system, results are going to come back quickly. In the system that I tested against, I had about 700.000 emails in CRM so the search result took on average just over 30 seconds to return results (and I consider this good, because I’m running on SSD drives). That being said, in real life, 30 seconds or more of waiting time is enough to make users think that the system is frozen, or to just stop trusting in the application, especially if they rely heavily on this type of search.

Making search return results faster

If you are using CRM 2015 On Premise, you are in luck. In CRM 2015 Update 0.1, the product team introduced the ability to enable full-text search for Quick Finds queries. To enable it, go do Settings –> General –> Set up Quick Find. There you’ll see the option to “Enable full-text search for Quick Find.

image

Greater details about the feature can be found here and you can also take a look at this video on the Microsoft Dynamics YouTube Channel.

Important points to keep into consideration before enabling full-text search:

  • Full-text indexing for Quick Find uses SQL Server full-text indexing
  • It can take up to 24 hours for the system to enable or disable full-text search, or add and remove find columns (maintenance job has to run on the server and make the adjustments specified in the CRM configuration)
  • As a bonus, you should also activate the Org Settings “EnableQuickFindOptimization” and “EnableRetrieveMultipleOptimization” in order to get the best results. Thanks to Jean-François Cantin for that indication. If you don’t know how to update the Org Settings, here is a reference.

After waiting long enough time for the emails to be indexed, the email searches started to return results within 5 seconds. Drastic improvement. The same solution was implemented into the client’s database and it seems to have made a huge improvement. 

Have fun.

Leverage the power of Power BI with Dynamics CRM data

Yesterday I attended a presentation from felow MVP Sophie Marchand about Power BI. I had previous exposure to Power BI when it became generally available for free as part of the Office 365 suite. The content of the presentation was around multiple areas in Power BI, so I thought I’d revisit some of them and put some thought into how we could leverage the power of the Business Intelligence platform to manipulate and play with Dynamics CRM data.

Connecting Dynamics CRM Online to Power BI (web)

This area is well documented. When you create or log on to you Power BI account, you can get Dynamics CRM Online data by going to the “Get Data” section, selecting “Services” and finding the Microsoft Dynamics CRM content pack.

image

Once you click on Connect, you will need to provide your Dynamics CRM Online OData Service URL (make sure you don’t leave a trailing slash in the end).

image

Once this process is completed, Power BI will connect to your Dynamics CRM Online Organization, get a set of data and build a Sales Manager Dashboard. That dashboard is pre-configured as part of the native integration between CRM Online and Power BI. Keep in mind that you can also connect to CRM Online from Power BI for Desktop. The way the data is structured is a bit different and you don’t get the Sales Manager Dashboard by default. That being said, it bring the data in using the OData Service and gives you the ability to create your own visualizations and reports. Let’s take it a step further and see what options you have if you need to manipulate data from custom entities and create visuals, dashboards that are useful for your business.

Custom Dynamics CRM data sources for Power BI

If you want to leverage Power BI, which you should in many ways, there are other ways to connect it to your CRM data. Here are some of the options that you have.

Extract Dynamics CRM in Excel Spreadsheets and Import the files to Power BI

Power BI (web)

image

Power BI for Desktop

image

For CRM On-Premise organizations, using Power BI for Desktop, you can simply connect to your CRM Database start creating for dimensions, cubes and reports from there. If you need more information on how to get started with Power BI for Desktop, make sure you check out Microsoft’s article here.

image

Worth noting, you can export your Power BI for Desktop content and upload it to the Power BI web platform. This gives you the ability to view the dashboards online, question it differently and also share it with people in your organization. In the Dynamics CRM and Microsoft ecosystem, it has never been easier to build reports and intelligence around your operational data. Enjoy!

Dealing with record level access restriction with Dynamics CRM

CRM works extremely well with giving users access to records based on a different set of security configuration tools : cumulative security roles with a “most permissive win” mechanism, business units, team memberships, access teams, hierarchy and so on. One thing it does not do well is restricting access to specific record. Imagine the following scenario:

  • Company ABC uses Microsoft Dynamics CRM to manage its cases
  • An employee leaves company XYZ to join ABC
  • After joining ABC, the new employee is given access to Dynamics CRM and has access to ongoing and past cases, say in his/her region
  • Company ABC has open cases against firm XYZ. The employee should not be able to see those cases because it represents a conflict of interest (e.g. he/she might give insights to his/her old friends at XYZ)

The following image provides an illustration of what we are trying to accomplish.

image

If you have worked with Dynamics CRM and know its security model, you can already read between the lines here. There are no easy ways to make this happen. This article provides a possible solution for this requirement based on a recent experience on a large CRM implementation.

Ruling out Team Membership & Ownership

In our scenario, members of a team or a business unit get access to all records  owned by their team or business unit. If we follow basic team/BU record ownership, the problem in this case is that once a record needs to be isolated from one individual member of the team, one of two things needs to happen:

  1. The record owner has to be changed (it can no longer be the team everyone is a member of) and the record needs to be shared with users who can still see it, but not with the individual that is getting the restriction applied OR
  2. The individual who is getting a restriction applied has to be removed from the team and all the records owned by his/her team need to be shared with him/her except the one he/she is restricted to see.  This technique implies heavy maintenance since the user will need to access new cases created for the team he/she used to be a member of.

Sharing in both cases can be done via the record share functionality, access team or any other mechanism that suits the business need. It should also be automated if there is a large number of records to share. You also need to save the information about the user and its restriction on a separate entity to be able to backtrack and understand why the records/ownership and so on have been modified.

Why we didn’t like the “Sharing” functionality

Using the out of the box sharing of record can cause problems in the long run. The PrincipalObjectAccess (POA) table gets bigger and bigger, causes performance issues and has to be cleaned up on a regular basis. In addition to that, the more access is controlled by sharing, the more you run the risk of having performance issues because of the complexity overhead in system lookups when it needs to display a record or list of records to users.

A solution using “Access Teams”

We decided to go with a design centered around Access Teams. The “Case” entity gets an “Access Team Template.” When a user is created and added to his/her group (team or business unit), we start a process to insert the user in the Access Team of each case related to his/her business unit. In order to restrict access to a specific case for a user, we can simply remove the user from the case’s Access Team.  For tracking purposes, we created a “Security Exception” entity that is created and has a relationship with a case and a user and an additional attribute that indicate the type and reason for the “Restriction”. To apply the access restriction on a case for a user (i.e. remove user from access team), we create a Security Exception record. We link the target case and user, and specify the restriction type. After the Security Exception record is created, a plugin is fired to perform validation and to remove the user from the case’s access team.

One of the reasons with went with that model was that we didn’t want to have a base security model that gets “broken” every time there a single person with restricted access, by changing record ownership or removing user from the team he/she is supposed to be a member of. With access teams, the record ownership remains the same, and users also remain on their team at all times which makes everybody happy and maintenance easier. The downside is that when a user is created, we have to look for all the cases in his/her business unit and manually (with code) add the user to each access team. Same when a case is created, users member of the case’s business unit have to be added to the case’s access team manually (also with code). That is the comprise we had to make.

As far as performance goes, the end results remain to be seen in the long run. Our system will have between 2000 and 3000 users when we complete our rollout. It will deal with over 1 million cases across multiple teams. Preliminary performance tests results look promising. I will repost if we get into any issue as it relates to performance.

It’s been a while! This feels great Smile  – Happy CRM’ing !

Synchronize CRM Online Data with Skyvia

I just discovered an interesting tool, thought I’d share a quick review.

Skyvia is a cloud data integration solution. It has connectors to a few data sources and is able to move data from one cloud based source to another based on configuration and schedule. Oh I forgot to mention, it’s FREE J

As a test, I configured it to connect to my Dynamics CRM Online sandbox organization and my Zoho CRM organization:

Once my connections are created and validated, I create and design a package to transfer contacts from Dynamics CRM Online to Zoho CRM:

As you can see on the screenshot above, Skyvia gives you the ability to create synchronization tasks (1) and also to schedule the execution your packages (2). The package can also be ran manually.

The tool can be very handy especially if you have simple synchronization needs between CRM and other systems. The supported data sources are Dynamics CRM Online, MySQL, PostgreSQL, SQL Server, Salesforce.com, Zoho CRM, Sugar CRM. Obviously such a tool does not replace the need for more complete and mature data integration products such as Scribe and SSIS connectors, but it provides an easy way to achieve simple data integration between multiple cloud based systems. Works perfectly for me demos! And it’s free (at least for now).

Dynamics CRM and Address Validation

Last year I was involved in a couple of CRM projects for which we had heavy requirements for address validation. The main scenarios that we encountered are the following:

  • All existing addresses must be validated and/or corrected (existing records)
  • New addresses saved in the system should be validated and/or corrected (synchronous validation)
  • When an address is entered, there should be an auto-complete functionality to help users enter the correct address

That was content for great research. First I want to share some facts about what you usually get with address validation solutions and then I will talk about the different solutions we used for the scenarios mentioned above.

About Address Validation Tools

An address validation solution is a tool that helps make sure an address is valid according to a standard (post service, government, other). It does so by taking an address as input, parsing it, comparing it to addresses in a proprietary database using match algorithms and returning a corrected/validated when it is found. There are plenty of address validation tools out there. Most of them have some common points.

First, they are sold as web services (cloud based or on premise) or as local APIs (.NET Libraries). If you are going with a local installation, it usually includes one or more address databases along with a web service. That means you will require little space on a server to install your address validation tool. Most vendors also offer a batch address validation/correction module.

Second, most of them function the same way: you send an address as an input and you get the validated/corrected address as an output. When an address is corrected, you get the changes that have been made (e.g. postal code changed, street type added etc.). In the same logic, when an address is not validated, you also get a validation status and sometimes a reason why the address is not validated. The batch processes usually provide a way to upload an address spreadsheet and get an output spreadsheet with the validation results.

Lastly, the pricing models are similar. You get can buy a certain amount of transactions (monthly, yearly or other). The price will vary based on the set of tools selected.

Obviously, there are other things to take into account. Some vendors only offer validation for specific countries and regions, some provide more than just address validation, for example address standardization, geocoding, multi-language support and more.

  • Correcting Existing Addresses

There are many ways to tackle this. The easier way is to use the validator’s batch processing module. When using Dynamics CRM, you can easily export your address using the “Export to re-import” functionality as shown here. Once you have your spreadsheet, you can easily import it to your address validation tool and get the corrected rows back into your CRM. There are two issues with this approach:

  • There is the 10000 records limit when you export data to Excel spreadsheets. If you have more rows to correct, then you need to consider an alternative solution
  • When an address is sent for validation, you may want to indicate a validation status in CRM which you can’t do by simply reimporting the corrected rows

In our case, we decided to write an application that reads all the rows for which we need to validate an address. For each row, a web service call is made to the address validation service provider, once we get the result, we store the corrected address as well as the correction details. When an address is not corrected, we store the details (if available) and we send the record through a manual processing for someone to perform the validation manually.

  • Address must be validated when saved to CRM

If you are adding this feature when building your system (before there is any address in your CRM), then you should never need the logic described in the point above.

The design here is very simple:

  1. Create a plugin on the address save or update (could be a custom entity or OOB such as Account or Contact)
  2. The plugin calls the address validation web service
  3. The plugin handles the business logic that is executed post validation (e.g. overwrite input address or creates a manual validation activity/task linked to the address being processed

Keep in mind that in point b, we call an external web service. This design works well for on premise installations. For CRM Online, you may not be able to reach those external web services if there are not hosted in Azure.

  • Address Auto-Complete Feature

For this client request, my colleague had experience with a CRM Add-On that does just that. It is called PostCodeAnywhere. It is a light weight solution for CRM (it only contains a few web resources). Here are a few screenshots.

It is worth noting that they support a lot of different countries so it makes it a very handy tool if you work for a company that stores international addresses. You can configure it to work on OOB address fields as well as custom fields.

I will also mention that using this tool is great to help system users entering correct addresses. However, there is no way to know (from an auditing perspective) that an address was entered using the add-on. Anyone could overwrite it manually.

A few tools

Here are some of the tools that we tried, used and/or tested for address validation:

  • Service Object : Great overall experience. The product is very good and the support and sales team are very responsive, efficient and flexible. They offer validation web services for US and Canada only.
  • Melissa Data : The product list is impressive (on prem, hosted web services, SSIS integrated tools etc.). They offer a global address validation web service.
  • nCode : The product has a lot of search algorithms for address validation. It comes as an on premise web service installation. It only support Canadian addresses.

The Google MAP and Bing APIs are also worth taking a look at.

Have fun!

Dynamics CRM Archiving Solutions Revisited

I wrote about Archiving solutions for Microsoft Dynamics CRM about 2 years ago. I still get a lot of questions from the article and from customers in the field about what they should do when it comes to archiving their Dynamics CRM data. While there is no obvious answer, my generic approach is to make sure my clients understand that they shouldn’t try to build an archiving solution before their system has become slower. They should see it more as an opportunity to perform a thorough review of their implementation and isolate the bottleneck(s) and pain point(s) and see if those can be fixed.

A few months ago, I gave a presentation on the topic that you can see below. I go from the reasons to archive data to recommendations. Hope this helps!