Integrating your data across applications can be a challenge, particularly when crossing organizational and application boundaries.  A lot of the control that IT organizations take for granted is lost when going outside of the organization.  Below, I will outline a number of challenges you're likely to encounter in an integration effort and provide some guidenace to help you work through them.

In a recent Salesforce.com project, the biggest issue I faced was integrating Salesforce.com with SQL Server.  I thought it would be easy with tools like Pervasive, CastIron, and Scribe.  It even seemed doable with tools such as Salesforce.com's Dataloader.  The fact is, it's not always that easy.  There are lots of things that need to be considered to make an integration effort successful.

The first big hurdle in any integration is planning.  It's extremely important to know the data of the systems you are planning to integrate.  I highly recommend putting together an Entity Relationship Diagram (ERD) by System and Database with Tables, Views, and Stored Procedures.  Make sure you list the keys that are available and delineate any of the objects on the diagram that can be changed or modified to support integration.  This is very important as you move forward with planning because it gives you a visual road map of where your existing assets lie.

The next big step after the data is modeled is to figure out what data will be stored in the CRM system and which system will be the system of record.  This will require an ERD of your CRM solution.  In complex scenarios, the system of record could change along with the status of a record in your CRM.  For integration purposes you must know which system is the system of record for what data at all times.  Otherwise, you're likely to face some real challenges.

One thing that's very important to note is how important it is to get started early on integration.  This is not something you should wait to do until after you've created all your objects and laid out all your fields.  This effort should begin before you even pick your CRM platform.  Why? If you company is like many other's I've seen, you're likely to have a lot of duplicatation, poorly organized, denormalized, and in some cases sparse data to deal with.  This takes time to clean-up.  If you're dealing with this, it takes lots of planning to figure out what data you'll bring into CRM and how much you should tie together.  It's never a good idea to start out with bad data in your system and the more effort you put into cleaning things up, the better your ROI later.  If you're doing a migration, then you'll probably want to clean this data up in the source system prior to migration and integration.

Yes, I did just mention sparse data.  What I'm talking about here is putting plans into place early in your integration to populate missing data.  There are some great data providers out there that can augment your lead and contact data.  One of these that I've worked with in the past is Melissa Data. They provide deduplication, data augmentation, address verification, and a number of other services.  Another you might consider if you're a B2B company is Dunn & Bradstreet. 

Really the sky is the limit and it's important to be creative and do due diligence in discovering what'a available.  This can be a one time effort, or you can get creative and integrate with one of these data vendors to provide these services in real time as you add data to your system, or to add data to your systems for you.  Clearly, this is another integration data point that needs to be considered.

Now that you know your data, what data you're planning to incorporate, and where it's to be stored, it's time to choose your data integration platform.  This is a hard decision to make and it's one that shouldn't be taken lightly.  The tools can be costly and difficult to manage depending on the complexity of your environment, or they can be easy to use.  You'll need to consider speed of message delivery and processing, workflow, and the level of complexity of your specific integrations.

Also, don't overlook the possibility that you can realize significant gains by investing in a solid integration tool in other areas of your business.  As SAAS has taken off, these tools have gotten better and better in integrating many different end points. Some of these tools have built in workflow capabilities and can be used to overcome limitations in your existing systems.

So, now that you know your data, planned your integration, installed your tools it's time to get down to business.  Your CRM system should have all the fields and objects built and you should be ready to start hooking things up.  You should also have most if not all of your clean-up work completed.

It's a good idea to build out a complete end to end development/test environment that models your final production system.  This can get expensive, but it's a necessity to have an environment that you can test in because you won't want to corrupt your user acceptance test data, or other data in the system while your configuration folks are trying to build workflow and validation rules.  With Virtual Machines this sort of thing is getting a lot easier, but it can still be complex if you've got a lot of systems involved.  You can leverage a sandbox in Salesforce.com for your development and test environments.

So, it's time to begin work on your integrations.  In general, if you're doing this yourself, you'll want to have training completed and some experience working with the tools well in advance of your delivery schedule.  Again, the more complex your environment, the more important it is to have worked through the scenarios that you'll be faced with in your integration.

I'm not going to go into all of the issues you can run into in building integrations, but I will say that among these you are likely to work through one of more of the following challenges while your integrations are being built:  1) data relation issues, 2) corrupted data, 3) inadequate code coverage testing, 4) challenges identifying the specific workflow step that's failing, 5) connectivity issues, 6) overwritten data, 7) performance issues, 8) issues cleaning up undelivered messages, 9) over engineering, and a host of others.  Hopefully your integration team will have done the proper planning and testing so that these issues don't crop up in your production environment.  One thing I can't emphasize enough is that the right tool and the right team can go a long way in reducing your exposure to these issues both in development and in production.

So, now the integration is built and tested. The final stage is deployment.  Again, the more complex the integration, the more important the planning for deployment.  It's important to plan the deployment  even before development begins because there are often data dependencies and relationships that need to be considered.  The order in which your integrations are turned on can impact the quality of your data.

Data integration is an ongoing process, whereas data migration happens once.  It's important to decide how much data you will migrate and how much will be integrated.  Also, if you do clean-up in your migration effort, but don't build it into your integration, or fail to consider the relationships in your data, you could end up with a real mess on your hands.

Another consideration I haven't mentioned yet is reporting.  When you move to a CRM system its important to consider how reporting works and where your data resides.  Salesforce.com reports on data within Salesforce, but if you need data from other systems for reporting you will need to plan to bring this into Salesforce and create any supporting objects, or consider another reporting solution and how you will get data from Salesforce into those reports.

One last point I want to make.  If you bring in a consultant to do your CRM implementation, be sure you have someone who has a clear understanding of the challenges of working with your data.  They will be able to discuss tradeoffs of your design in Salesforce.com with your integration and reporting needs.  Consultants with just experience in building systems will lack this expertise and may give you a system that works, but limits your ability to realize your full potential.

After all, at the end of the day is all about providing the right data at the right time in the right place to the right people. Right?