In-Depth

Metadata Management: An Imperative for Your Enterprise

A new report delves into the importance, use, and management of metadata and how it's become a critical component for your enterprise.

• By Stephen Swoyer
• 09/22/2010

Metadata is the glue that binds a business together, writes Philip Russom, senior manager with TDWI Research, in a new Checklist Report from TDWI Research.

Why, then, Russom wonders, is the management of metadata too often an afterthought in so many shops?

One explanation, he suggests, is that the bottom-line benefits of managing one's metadata aren't always clearly understood. Russom is the author of Cost Justification for Metadata Management, a new report that aims to help organizations calculate the benefits of metadata management to determine ROI.

It's a hard problem, Russom suggests, precisely because of the way metadata is treated. "[M]etadata -- as managed by most organizations today -- has been kept invisible to all but data specialists and a few business people. Up to this point, it has not been the actual deliverable the way that solutions for data integration, data quality, master data management, and operational applications are," he argues.

"[T]he assistance provided by a quality metadata management solution is masked because it is embedded within a larger solution, or it has been a degree or two of separation from the business processes it enables."

That's why it can be difficult to reliably assess the impact of a proposed metadata management effort, Russom explains. He champions a multi-step process in which a shop first comes to terms with the why and the what-for of managing its metadata.

For one thing, metadata is a component of (and can be said to enable) most intelligent business processes, applications, and (of course) data models.

To the extent that almost all day-to-day production and commercial activities are automated (or have been information-enabled), they likewise depend on the existence of reliable, consistent metadata -- either as a means to define information for consumption by internal resources or (as is the case with EDI) as a means to reliably exchange information between and among businesses.

In addition, he argues, effective metadata management can pay dividends in other management disciplines (including both operations management and performance management) and can likewise improve the efficacy of business intelligence and data governance initiatives. In fact, Russom asserts, metadata management is an essential best practice for any data-driven business process or technical implementation.

From another perspective -- that of the data management (DM) professional -- metadata management is associated with a litany of concrete benefits. To the degree that metadata is poorly managed, it likewise has concrete costs.

According to a May, 2010 survey conducted by TDWI Research, DM pros associate good metadata management with a number of desirable outcomes or activities, including "better data sharing across an enterprise" (cited by almost 80 percent of respondents), "more compliant use of data" (almost three-fifths of respondents), "faster response to change" (just over half of respondents), and "reduced data-related risk" (almost half of respondents).

Although DM pros regard improved data sharing as metadata management's biggest benefit, they likewise cite limited data sharing as the most obvious symptom of poor metadata management. "Sharing data across diverse business units and their information systems is key to business integration," according to Russom. "[M]etadata greatly facilitates broad data sharing, which in turn contributes to the success of important business initiatives and processes."

Two of metadata management's four most frequently cited benefits involve risk or governance. This isn't an accident, Russom says.

"For example, when business rules are embedded in metadata, they help control data access and use, which yields better compliance. When metadata describes data structures clearly, the documentation reduces the risk of users selecting the wrong data for a report, portal, data set, or other application," he notes.

There are tangible dollars-and-cents drivers, too. Just under half (46 percent) of respondents say effective metadata management is strongly associated with "more effective data management projects." In addition, almost two-fifths (39 percent) of DM pros believe that effective metadata management can lead to "reduced costs for the business."

Cost, it seems, is key: respondents associate "smoother operations for the business" or "increased revenue for the business" with improved metadata management. In the latter case, Russom explains, DM pros have told TDWI Research that good metadata can assist with "speeding up changes to applications and data that are required for new product development and rollout, careful targeting for sales and marketing campaigns, and on-boarding new partner companies."

Such activities tend to lift revenue, Russom reports. In addition, solid metadata management can improve both the speed and the accuracy of these efforts.

Russom's report provides considerable details, particularly for those who like hard data about dollars and cents. For example, he derives a formula shops can use to calculate how much -- in hard payroll dollars -- effective metadata management can save them. This stems from an operating assumption (endorsed by TDWI Research) that payroll is one of the biggest expenses that a shop must shoulder. In this respect, Russom estimates, using a dedicated metadata management tool can:

• Eliminate almost $104,000 per year (on average) in payroll costs associated with the collection and sourcing of metadata (a task typically performed by at least three full-time workers, according to TDWI survey data)
• Populate a repository that's used in reused in multiple projects throughout the year
• Free technical staff and boost developer efficiency, permitting shops to fund and complete more projects

There's a trickle-down effect at work here, too, according to Russom: just as the investment in a metadata management tool can have an impact that immediately redounds to the benefit of front-line technical staff, it can likewise have an impact (call it a shockwave impact) that trickles down throughout an organization, boosting the work of business analysts, data stewards, data governors, report designers, and other data-driven professionals.

These benefits are similarly calculable. "Alter the formula to include the kinds of people in your organization and plug in numbers that represent their work metrics," he suggests. "Your alteration of the formula should produce a useful indication of potential savings from reusable metadata."

Buy Versus Buy

That leaves the thorny matter of ROI. It's called "return on investment" for a reason, of course: metadata management isn't (or shouldn't be) a tactical or homegrown enterprise. Metadata management doesn't involve "managing" metadata in product-specific silos, after all. Best results -- reusable, predictable, ROI-able results -- come from investing in a central metadata repository, Russom suggests. "[T]he reuse of collected and documented metadata depends very much on the use of a metadata repository. Furthermore, the repository must be centralized, in the sense that there's one repository or a small collection of integrated or federated repositories," he argues.

A number of large vendors -- including application giants IBM Corp., Oracle Corp., and SAP AG -- market metadata management solutions. Others -- such as Informatica Corp. and SAS Institute Inc. subsidiary Dataflux -- have data management-centric pedigrees and market metadata management tools as part of their information integration stacks. Most metadata management offerings, in fact, prescribe the use of a centralized metadata repository.

It's centralizing metadata that's sometimes the hardest change for organizations to digest, however. After all, Russom points out, centralization involves the un-siloing of data and places a premium on cooperation between IT and the lines of business, as well as entails people or process (to say nothing of data ownership) changes. "For some organizations, centralizing metadata management is a big change, because for years they've managed metadata in silos, typically with an isolated metadata solution per application, database, or data management tool," he points out.

There's at least one other respect in which metadata management can be an especially thorny issue. Consider a related field -- that of master data management (MDM) -- where industry veteran Jill Dyché famously raised the specter of a "dirty little secret" that's becoming harder and harder to suppress.

"Once you invest in an MDM technology and on-board a system or two, you're pretty much on the hook. It becomes foundational, not only from an IT perspective -- as it continues to link data from heterogeneous systems -- but from a business-enablement perspective," she argued. "MDM is the new ERP. It's really, really hard to de-commission. Way harder than replacing a BI tool or re-platforming a data warehouse. The vendor that owns MDM can become the de-facto owner of its customers' enterprise data."

That's one reason shops might opt to eschew a packaged metadata management offering from a non-specialty vendor in favor of a best-of-breed metadata management tool, Russom suggests.

"For some organizations, successfully centralizing metadata may require the acquisition of an independent metadata repository that's vendor-agnostic and suited to broad enterprise use, managing metadata for many IT systems from a shared, central source," he writes.

Enter Rochade, by Allen Systems Group Inc. (better known as ASG Software), which ASG likes to postion as … a vendor-agnostic metadata management repository. (Full disclosure: ASG sponsored Russom's report.) Unlike most players, ASG doesn't market its own metadata discovery or data integration technology: instead, it uses a best-of-breed tool (Saphir) from another metadata management specialist (Sillwood Technology Inc.) to populate Rochade.

Russom outlines a number of other metadata management benefits, such as broad application support and improved reporting. In the latter case, he notes, metadata management can pay other dividends, too: in the event of a regulatory audit, for example, a reliable financial reporting bolstered by a credible audit-trail could potentially save an organization millions of dollars.

There's another important reason to bite the metadata management bullet: it's almost inevitable. Not only is the use of metadata is growing, but the capabilities of metadata repositories have evolved to support a host of non-traditional activities. With so many different applications and stakeholders, metadata should and must be managed in a centralized repository.

"[T]he trend toward extending metadata's uses will continue as organizations go deeper" into activities such as governance (which typically prescribes rules for accessing and using data), compliance (which always requires a digital audit trail), and MDM, Russom asserts.

"Nowadays, [metadata repositories] also store and provide broad access to application routines, project management documents, master and reference data, data quality metrics, reports germane to development, and so on. When centralized, the repository is a conduit through which developers, managers, and other people collaborate. To facilitate collaboration, some repositories support browser-based access, security for multiple user roles, annotations for managed objects, and discussion threads."