In-Depth
Assessing Data Migration Readiness, Part 1: The Right Way to Begin
The power of a migration readiness assessment strategy framework is that it enables you to see what you are getting into with a data migration before you actually begin.
by Arvind Parthasarathi
Sooner or later, your organization will be involved in a major data migration project, either because of a system upgrade, a new application implementation, or perhaps a merger, acquisition or divestiture/spin-off. It is well understood that data migrations are difficult and risky undertakings. Analyst research has shown that 80 percent of migration projects fail to come in on time or on budget, while a third of them fail altogether. This can be disastrous, as data migration is often a component of an overarching initiative, such as a modernized financial system or a consolidated CRM system. If the migration overruns or fails, the larger project will overrun or fail as well.
The main reason so many data migrations veer off course is that project requirements are not fully understood at the start. For example, there are numerous potential issues with source data, but they are not going to be visible unless you have up-to-date documentation and are intimately familiar with the content, structure, quality, and relationships in the source data. There are just as many potential pitfalls inherent in repurposing, or mapping, that data for use in the target system. Moreover, the deliverables and outputs of data migration projects are different from all other components of the larger initiative. Where application implementation is business-process-focused, data migration requires specialized skill sets such as data analysis and data modeling. It is rare for a company to have people who possess all these skills and experiences.
Use an MRA to Begin on the Right Foot
As with any major and risky undertaking, it is highly valuable to appraise your migration readiness and develop the optimal strategy. This is the role of a Migration Readiness Assessment (MRA), a pre-migration strategy framework that provides leverageable insights into the true scope and complexity of migration projects in order to substantially mitigate the inherent risks.
Data migration moves data from a system you have and don’t know much about, to a new system that hasn’t been built yet. It is similar to moving a household into a new dwelling. You don’t necessarily know where you are going to put all your belongings (a lot of which are legacies you have “inherited” and don’t fully understand), so you do an inventory and floor plan in order to visualize up front where all your furniture and boxes have to go. An MRA helps you create this inventory and floor plan for your migration, and lets you gain experience by moving one or two “boxes,” or data elements. The result is a better understanding of the entire move – the narrow doorways, the steep stairs – and a knowledge of the tools and techniques you will need to accomplish your project on time and on budget.
The first step in an MRA is to bring in one or more data migration specialists with a background in data analysis and data modeling. Pair them with people inside your organization who best know your source and target systems. The specialists are your “professional movers” – you leverage their expertise rather than buying your own moving van, dollies, and back braces. The people you pair them with are your subject matter experts, who will assist the outside team in accurately analyzing the sources and mapping to the target.
You should engage these specialists and commence an MRA as soon as you know that a large migration project is pending. This can occur before the target system is chosen or in place because, unlike the migration itself, an MRA is not dependent on the target. An MRA can set its sights on moving to an idealized target.The important thing is start early, even as the strategy for the overarching initiative or application project is being set out. This enables you to feed the knowledge gained from the MRA into the overall project plan, instead of being a drag on that plan’s development. Going back to the mover analogy, an MRA puts you in a position to get an informed movers’ quote and timeline, based on all the variables of migrating from your old house to the new one.
A Two-week Process
An MRA comprises six steps, from initiation to data mapping to final report. The duration is typically one to two weeks, start to finish. Here are the steps to follow.
Project initiation: An MRA begins with a joint workshop, involving both inside and outside resources, in order to scope out the MRA exercise, assign roles and commence the first round of activities.
The first goal is to identify the data sources, discuss the target system, and define the source data entities to be moved. In a migration project of 200-300 data elements, it is not uncommon to move 2-3 data elements in an MRA exercise. Remember to choose representative data elements—in a practice household move you’re probably not going to pick your heaviest sofa or your lightest jewelry.
The second goal is to identify the in-house stakeholders who will assist in the MRA and the subsequent migration.
The third and final goal is to determine one or two critical data elements that you will move as part of the MRA.
Data acquisition: Flat-file data, metadata, and documentation are made available for each source entity identified in the initiation stage.
Data and metadata validation: The supplied data and metadata of the source systems is validated using data profiling techniques to gain a complete picture of the content, structure, semantics, quality, and relationships in the supplied source data. This, in turn, enables an understanding of the data risks in the elements you’ve chosen to move as part of the MRA.
Entity mapping: Source entities are analyzed to determine structural relationships between the entities and the targets. One goal is to determine the critical structural differences between source and target entities—a potential cause of project slippage if not resolved correctly. Transformation requirements are identified (including the need for aggregation, joining, de-duplication, and other transformations) and reveal risks.
Attribute mapping: Once the entities have been mapped, the process dives one level deeper into the attributes of source and target entities to establish the mapping and transformation rules that will ensure source data will meet the target system’s requirements.
Final report: Using the knowledge gained from the previous steps, the migration specialist(s) can determine and report on the data and mapping risks involved in the migration, and supply a mapping diagram and rules for the chosen data elements. From understanding what it takes to move these one or two elements, a reasonable estimate will emerge of the efforts and resources required for the entire migration project.
Benefits
The power of the MRA framework is that it enables you to see what you are getting into with respect to a data migration project before you begin. Without the MRA framework, you are flying blind. With an MRA framework in place, you gain a clear picture of the efforts that will be involved, along with knowledge of the potential problem areas and how to deal with them. In addition, you gain the opportunity for your subject-matter experts to work closely with migration specialists before the actual migration commences to elevate comfort levels and establish a team environment.
You also are better equipped to establish credibility within the overarching initiative, as you are able to make informed projections as to the data migration’s impact on the overall project’s timeline and budget.
Finally, you have a running start on the migration itself in the form of mapping diagrams for the chosen critical data elements. This important first step, coupled with the knowledge you’ve gained from the MRA, can put you miles ahead on your migration project.
Next week we will discuss how to leverage the results of an MRA as you develop migration strategies and staffing plans, establish rules for data conversion and system cutover, and move further to mitigate project risk.
Arvind Parthasarathi is senior director of solutions at Informatica, a leading provider of data integration software. He has a wide experience managing data migration projects at some of the world's leading ERP and supply chain implementations. aparthasarathi@informatica.com