Microsoft Dynamics CRM UK Reseller: MSCRM

Data Migration Part I

Sat, 26 Feb 2005 12:33:18 GMT

Getting your data into your CRM system is one of those problems people don't think about when planning their CRM project either because they are concentrating on the features and how they are going to use their new CRM system or because they just do not realise what is involved.

Data Migration can be tricky for a number of reasons, such as:

multiple data sources which need to be merged which can lead to difficulty matching and end up with duplicate data
existing systems do not have export or full export functions so getting the data can be difficult
the structure of the data in the existing systems does not match the CRM target structure
individual fields on existing systems do not have equivalent fields on CRM e.g. you may have a free text field on the legacy system but are planning a picklist for the dame data on CRM
the quality of the data - a lot of legacy systems do not enforce data integrity which makes the migration process difficult
misuse of data fields - users will use fields for different purposes and it is not uncommon to find "codings" embedded in fields especially where the legacy system is hard to customise

There issues mean that data migration can take 30-40% of your CRM project budget and it you haven't allowed for it then you will miss your implementation date and having a system without data is far from useful.

Data Migration can be hard if you just dive in without planning and working what you need to do. By following a well defined process you are more likely to success in getting your data into CRM ready for go-live. There is a generic method for data migration with three stages known as ETL which breaks the data migration down into separate problems

Extract - getting the data out of the legacy systems
Transform - this is the conversion of the data from its legacy structures into the structure CRM expects
Load - this is the actual process of importing the data into CRM

[You can add data cleansing into the above but I find that cleansing tends to be performed in one or all stages depending on the situation]

In the Extract stage you need to get the data in a neutral format e.g., Excel, Text file or Access. How you do this depends on the legacy system. Some have export functions but beware that some of these exports do not export all the data you need e.g. in ACT! the export only covers groups and contacts not activities and opportunities. So you may need an alternative approach and which may require a third party utility all of which adds to the costs.

In the Transform stage you need to bring together all the data sources and create data that "looks like" CRM i.e. Accounts, Contacts, Activities, Opportunities, Orders etc. Generally this requires processing and logic and either a specific program needs to be developed or a tool used.

In the Load stage you need to use CRM's functions to import the data into the CRM database.

Microsoft CRM provides four methods for importing data:

Within CRM you can import Leads and Contacts from Excel - this is of limited use in data migration as often there is more data needed. This function is more aimed at importing lists of people you have either collected at trade shows or have purchased from a commercial list
Within CRM you can import Contacts from Outlook - this is useful where the only data source is Outlook but again you will not be able to import tasks and will have to manually create Accounts as these are not created from the contacts.
SDK - Microsoft provides an API for creating (and updating) data in CRM. You could use this to create the records in CRM using software you
Data Migration Framework - a set of tools and documentation provided for importing data into CRM

What we have migrated into CRM using the above approach:

ACT!
Goldmine
Sage Line 50
Bespoke solutions based on SQL, Superbase, Access, Filemaker and Excel

Part II of this document will go into the migration process in more detail, tell you what tools we use in each of the stages and explain the features of Microsoft CRM Data Migration Framework.

Mappings

Fri, 18 Feb 2005 00:41:20 GMT

In the Schema Manager in the Deployment Manager as well as the entities and attributes there are mappings which control the which attribute values are copied from one record type to another when creating a new instance of a record e.g. when creating a contact from within an account certain values are automatically copied from the account to the contact such as the account address.

This is one reason why creating a record from within its parent is a good idea - it saves you re-entering data you already have.

Why then are the mappings so sparse? There are many obvious fields not mapped e.g. fax. The fax number for a contact is almost always the same as its parent account so why is this mapping not included.

Anyway I spent past day adding in all the attributes we nearly always end up adding for each customer and building up the mappings so we now have a base customisation we can use at each new customer saving us time and the customer money

We have also gone through and standardised the picklist values and made them and field names UK centric. We have also removed fields such as tickersymbol and pager which are not in general use in the UK.

So my advice is take a look at how you are using MSCRM and see if you are having to re-enter data already in CRM and see if adding a mapping will increase usability and save your users time.

Finding Records in CRM

Tue, 15 Feb 2005 23:45:11 GMT

In Microsoft CRM there are many ways to find records.

There is the name find available in all list views from which all records which start with the letters entered are found. This tends to work on name fields e.g. Account Name, Quote Name, Product Name etc. It is not possible to change which field is searched on. It tends to be restricted to Active records although I not quite sure if this is true for all record types.

There are the jump letters at the bottom of the list view e.g. clicking B shows all records beginning with B. This just seems to be the same a entering B in the name find above.

There is the advanced find which is powerful but requires a lot of training and if you only need to use it occasionally it can be daunting.

The problem comes when you want to find a record quickly using information businesses use all the time such as Account Number, Purchase Order number, Invoice Number, Case Number etc.

The only options are to:

use the Advanced Find which is too cumbersome for such a simple query
make sure everyone fills in the name fields in a consistent way (not the easiest thing to enforce)

Neither of these are that useful but fortunately Microsoft provides an SDK which exposes all objects and methods in .NET. With a bit of knowledge of .NET it is actually quite straightforward to develop small functions for MSCRM.

It only took me 2-3 hours today to create a menu option in CRM which allows a user to quickly open the record just using the number for that record.

This is one of the reasons I like MSCRM compared to other CRM products - the ease and speed in extending the product for different customer requirements. So even if there is something missing Microsoft provides the means to fill that gap. One of the reasons I am disappointed in the delay of CRM 2005 is that Microsoft were giving us even more tools to extend the product to meet more demanding and varied customer requirements. We can still do the same things just some extensions will require more coding and hence risk and cost until CRM 2005 is released. The questions then are, can you the customer gain benefit and use CRM without these additional functions or are you willing pay for this work?