In-Depth
What Businesses Must Do to Improve Data Accuracy
Don't blame IT if you have bad data. Individual lines of business also contribute to data accuracy woes. They're also in the best position to help push for change.
Justly or not, IT tends to take the rap for data accuracy issues, in spite of the fact that IT itself -- or, more precisely, data management (DM) practitioners -- are never entirely to blame. Individual lines of business also contribute to data accuracy woes.
Part of the process of digging out -- of improving the accuracy, quality, or consistency of data -- is to recognize as much. That's according to a new report from industry veteran Wayne Eckerson, director of research and services with TDWI. Eckerson isn't playing a blame game, either; instead, he urges improved collaboration between business leaders and DM pros.
"[T]he business needs to recognize that data quality and consistency are a joint responsibility," writes Eckerson, in "Who Ensures Clean, Consistent Data?", a new monograph published by TDWI. "Top executives need to view data as a valuable corporate asset, assign senior managers to oversee data governance initiatives, make subject matter experts available to assess and resolve data issues, and partner with the IT department to implement its policies and standards regarding data. In short, the business establishes policies and standards for enterprise data and the IT department implements them."
Much of what Eckerson writes seems like so much common sense.
The rub, of course, is that -- almost a decade after the dot-com implosion catapulted business leaders back into the saddle (after they'd been temporarily unseated by IT wunderkinds) -- business and IT continue both to talk past one another and to effectively work at opposite removes.
This is inconceivable, as Eckerson points out -- and it isn't entirely (or even mostly) IT's fault. He ascribes the problem to a failure of understanding (and, to a degree, of imagination) on the part of business leaders. "[M]any business executives don't understand the high costs of bad data. Or if they are aware of the problems, they don't know what steps to take to resolve them," he writes.
Even though DM practitioners do recognize the many and varied costs of bad or inaccurate, they're often powerless to do anything about it. "IT departments feel helpless to resolve the issue without strong executive sponsorship and funding," Eckerson explains. Moreover, what was true two decades ago is still true, to a surprising degree, today: IT and the line-of-business still don't speak the same language. They communicate, to be sure, but they do so using a kind of kludgey argot that doesn't help them to understand one another.
"[T]here is a communications barrier between the two groups. Business folks often feel that IT people speak a different language [e.g., "IT speak" vs. "business speak"], and IT developers often find that business executives and experts won't devote the time needed to help them understand the business content of the data."
The two most common "bad data" issues involve either defective data or inconsistent data. In the case of the former, Eckerson suggests, the best way of fixing it is to nip at the bud. "This requires the business to invest in systems that validate and fix data at the source -- at the point when it's entered into the system or moved between systems via an application interface," he explains. Fixing existing bad data is a trickier proposition, requiring as it does both tooling (e.g., data profiling and data quality technologies) and substantive, ongoing involvement on the part of the business.
No matter how you look at it, Eckerson says, is that minimizing bad data -- or eliminating it entirely -- involves an ongoing commitment from the line-of-business. This is particularly the case for the second bad data category: inconsistent data. Not only do inconsistent data issues affect all organizations to some degree, they're also far more difficult to stamp out.
One reason for this, Eckerson notes, is that inconsistent data is all-too-frequently the product of political jockeying between and among different business domains (what is often described as "petty fiefdom-ing"), to say nothing of regional or divisional posturing. The likelihood -- the inevitability -- of intra-departmental, sectional, or factional resistance underscores the importance of a strong top-down push to address data accuracy issues, he indicates.
"It's clear … that business must take an active role in both the creation of data quality standards and in working with IT to improve the quality of data," Eckerson writes. This is a more significant commitment than it might at first appear. In spite of the line-of-business's wishes, data accuracy is not a set-it-and-forget-it kind of consummation. In other words, Eckerson urges, business leaders can't afford to take their eyes off of the "good data" ball.
"Typically, after a business initiative or application has been implemented, companies don't pay much attention to the quality and consistency of their data until a crisis or opportunity occurs," he points out, conceding that (in the absence of a strong top-down push) "grassroots" collaboration between business and IT is by no means valueless. For example, Eckerson says, business and IT leaders can work to promote collaboration between business subject matter experts and developers.
At the same time, he concludes, there's really no substitute for a top-down mandate, which demonstrates an irrefragable commitment -- as both strategy and policy -- to a "good data" retrofit.
"Senior executives must take responsibility for managing data as a corporate asset and work together with IT to spearhead initiatives to deliver high-quality, consistent data," he concludes. "In doing so, future initiatives will be more successful by avoiding the costs, delays, and embarrassment frequently associated with bad data."