Worth Waiting For
Deploying business intelligence projects in short iterations lets your enterprise adjust quickly to changes in the business climate.
We all have times when we put off tasks until the last minute. It's called procrastination, and it's generally considered a habit to avoid. But there are situations in life—and in data warehousing—in which it's actually smart to wait until the last minute. I call that healthy procrastination.
Plenty of successful manufacturing operations put off manufacturing goods until immediately before they're needed. It's called "just in time" (JIT) manufacturing and has been considered a best practice for decades. Through JIT, a manufacturing operation can directly increase its profitability. When a factory postpones all manufacturing operations until immediately before finished goods are needed, a backward chain reaction occurs throughout the manufacturing processes. Through procrastination, inventory levels drop. When inventory levels fall off, the cost of manufacturing operations are reduced in many ways, ranging from lower storage costs to reduced cost of inventory. Procrastination, in short, saves money.
You can achieve similar benefits by putting things off in your business intelligence environment.
Good Procrastination
Healthy procrastination in data warehousing allows your business to adapt to change faster and to deploy capabilities at lower costs. Through planned delays, you can reduce the impact of change. In business intelligence, as in most parts of today's enterprises, the ability to successfully manage change is a major indicator of business success.
The global economy has slowed down significantly, and many businesses have shifted their goals from increasing revenue and marketshare to cutting costs. Your customers' expectations continue to change at a rapid pace—but it's harder for you to acquire the capital needed to meet those expectations. The economy continues to globalize, trade barriers continue to drop and businesses scramble to take advantage of new business opportunities and challenges. These external business drivers create new information needs that are addressed by business intelligence capabilities. The primary benefit of data warehousing procrastination is faster responses to all this change and less rework.
Iterative Project Management
It's usually easy for businesses to choose what opportunities to pursue and what supporting activities to take up. The hard part is deciding what not to do. Iterative business intelligence projects imply procrastination—that is, making decisions about what not to do. Iterative projects focus on delivering concrete, measurable business value in short (90- to 150-day) projects, with each new project building on the deliverables of past projects. Iterations beyond projects that are currently underway can represent decisions to wait. That gives you the opportunity to adjust for change later.
When you pause and take time to evaluate activities and subject areas beyond current iterations, you can have your project teams postpone activities as much as possible. That, in turn, reduces the impact of change. After all, every hour that a data warehousing project team spends on tasks that support business rules and data elements outside the scope of the current project creates assets that might change before you need to use them.
As I mentioned earlier, the last six to 12 months have been dominated by concerns about the first global economic downturn in nearly a decade. Around the end of 1999, business goals were still dominated by the need for enhanced revenue and competitive advantage. Data warehousing projects appropriately reflected these goals. Just 18 months later, business leaders have begun focusing on cost reduction instead.
Here's where procrastination paid off: Data warehousing programs with short iterations were able to respond to changing business strategies faster than businesses with long data warehousing project cycles. A 12-month project with goals set before the economic downturn isn't likely to support current business needs. When the economy strengthens, the same long projects will continue to support cost-cutting strategies when the business has already shifted back to a revenue and competitive-advantage focus.
By deploying business intelligence capabilities in short, iterative projects, data warehousing programs are actively putting off decisions about what to do a few months down the road. This kind of delay lets your enterprise adjust to change more quickly—and gain a higher return on its business intelligence investments.
Data Model Procrastination
In the 1980s, many large enterprises engaged in massive projects to create enterprise data models. Though the goals were valid and, theoretically at least, the projects were valuable, most failed. In fact, some failed so miserably that the mere mention of "enterprise data models" can send shivers down the spines of seasoned IT professionals even today. The goal was noble—create an enterprisewide data model resource to provide a single, consistent logical view of business rules about data. Once created, all new applications and databases would conform to, or be semantically consistent with, the enterprise data model. The resulting massive data models were created by both reverse engineering of existing information systems and forward engineering from stated business requirements and the information needs of business leaders and analysts.
So why did those projects fail so miserably? They lacked healthy procrastination.
The lofty goal of creating a single enterprise data model in isolation from ongoing projects wasn't met because of a failure to put off data modeling activities until just before they were needed. It wasn't the ideal that was flawed; rather, it was the methodology. Specifically, the projects lacked iteration and modeling data within the scope of existing projects that bring business value.
In today's data warehousing best practices, data models are built iteratively as a part of projects under the oversight of data warehousing programs. The only data modeled in a specific project is within the scope of that project, but within the framework of the data model assets of the data warehouse program. By managing data models from a data warehousing program perspective as a shared resource enhanced by each successive project, the enterprise can gradually develop enterprise data model resources that are relevant and valuable. These data models aren't shelfware; they're critical representations of business rules about data from an enterprisewide perspective.
Shifting Programs to Projects
As demonstrated by managing data model resources from a data warehousing program perspective and building the data models in the scope of specific projects, one form of procrastination is shifting tasks from the program level into specific projects. For example, one of the resources that a data warehousing program provides to each new project is the suitability of operational systems as a source of business data. This information is accumulated in two ways. First, at a data warehousing program level, source systems analysis can assess relative value of operational systems as sources of data for the data warehouse. This is sometimes done as part of data warehousing assessments when data warehousing programs are initiated. Second, the source systems are assessed in individual projects as they become relevant. This second approach is an example of healthy data warehousing procrastination.
If operational systems are assessed as sources for data warehousing months in advance, things may change before the results of the assessment are useful. New systems are implemented. Operational systems may experience significant change in business rules and data models. In fact, assessing source systems in advance of the current iteration may be a complete waste of time and money if external business drivers or business goals shift the information needs of the business into completely different subject areas. Ideally, source systems are assessed at the beginning of each project and based on the scope of the project. If operational systems have been used, or assessed as sources in previous projects, the data warehousing program provides this information to the project team to review and update.
Data models and source system analysis are both prime examples of assets that are managed by the data warehousing program, but can be procrastinated so that the actual work is done in specific projects on a "just in time" basis.
Unhealthy Procrastination
Not all procrastination is good. Certain decisions need to be made far in advance. A decision to pursue a federated data mart architecture or an enterprise data warehouse with dependent data marts architecture has long-term implications and should be made early in the evolution of the data warehousing program. Because database technology decisions in data warehouses and staging areas can be costly to reverse, you should make them early as well. ETL tool strategies, including a decision not to use a tool, are best made for years in the future. The same can be said of metadata management strategies and tools.
Learn when to put off tasks and when to plan long-term strategies. You'll benefit from both.