Revitalizing the Aging Data Warehouse
If your data warehouse is more than a few years old, chances are it needs a revitalization to align with evolving business needs.
Data warehousing is a relatively young technical innovation—Bill Inmon's original definition dates back less than a decade. Still, we already use the term "second-generation data warehousing" and hear discussions about "next-generation business intelligence (BI)." BI changes the focus from the technology (the data warehouse) to the business impact (information access and delivery). If your company is like most large organizations, you probably have something called a data warehouse—a repository of integrated data.
Data warehousessource of pride for early adoptersare now cause for concern. Best practices have evolved while these early warehouses remained static. In many cases, technology has outgrown the warehousing architecture.
Sometimes the temptation to scrap and replace the warehouse is strong. That mature data warehouse, however, represents a significant investment and is likely to have unrealized potential. Revitalization is a process of recognizing the potential and turning it into reality. Any data warehouse with correct business rules, sound data structures or good technical infrastructure has some valuable potential.
If your data warehouse suffers from outdated technology, obsolete practices or business misalignment, it may be a candidate for the two-step process of revitalization: assessment and renewal.
Assess Where You Are
The data warehouse assessment is a project unto itself. Just as data warehouses are complex and challenging, so too is the process of assessing themand the challenge is compounded by time constraints. Assessments, by their nature, should be rapid. Six weeks is typically upper management's outer limit of patience for completion. Yet, much data must be collected and evaluated. Data gathering and evaluation is necessary to identify strengths to be leveraged, problems to be corrected and risks to be mitigated.
Data warehouse assessment works best when broken down into a set of smaller, more manageable activities. An effective way to diminish the warehouse assessment problem is to group the tasks into five distinct areas:
- Business needs assessment evaluates the degree to which the data warehouse is aligned with current business goals and needs. If your warehouse is more than a few years old, it may be well matched to past needs but not designed for today.
- Information architecture assessment includes analysis of logical data structures, which focuses on their completeness, documentation and ability to respond to known business information needs. Also include an analysis of data quality, data sources and data acquisition procedures.
- Technical architecture assessment looks at hardware, software and network infrastructure, and examines physical database designs. Technical examination identifies risks or constraints with regard to accessibility, performance, maintenance, scalability, data distribution, disaster recovery and sizing.
- Organizational assessment examines the organizational structure with respect to warehousing roles and responsibilities. IT and the business community have distinct roles and responsibilities for technical support, business support, configuration management, continuing requirements definition, and growth and innovation.
- Project assessment reviews the ongoing need for the data warehouse to evolve and how to achieve that evolution. This review measures the degree to which the warehouse is unfolding as a continuing series of development projects, rather than as a maintenance problem.
Plan Where You're Going
Assessment analyzes the differences between your current warehouse and where you want it to be. Renewal planning focuses on problems and needs, leading to solutions. Begin the planning process by identifying an initial list of potential solutions and actions, and then refine and consolidate that list to a set of practical, prioritized actions based upon:
- Contributions to meeting business needs
- Ability to address highest impact architectural problems
- Logical groupings of functionality
- Organizational capability to implement
- Estimates of time and cost to complete.
Group the list of solutions based on business and technical affinity, and package them as a set of incremental projects. Plan and execute each as a new development project to add, adapt or extend warehouse functionality. Each project moves you further along the path to revitalizing a mature data warehouse. This series of development projects is an essential step to putting new life into an old warehouse. The data warehouse that stops developing isn't finishedit's simply stalled.
David Wells is an enterprise systems manager at the University of Washington, the founder and principal consultant of Infocentric,
and a fellow of The Data Warehousing Institute (TDWI)