In-Depth
Is One Version of the Truth Still Achievable?
The Aberdeen Group finds that despite failed attempts to see just one version of the truth, the goal is achievable.
By David Hatch
Managers are questioning their level of trust in corporate data: Do the reports, charts, and analytic tools in use today represent the truth? During August and September 2008, Aberdeen Group surveyed over 200 professionals from 155 companies, and interviewed a diverse range of senior executives and operational management professionals working in different industries and geographies. Aberdeen's research study -- "One Version of the Truth 2.0: Are Your Decisions Based on Reality?" -- uncovers what Best-in-Class companies (those that have achieved the top 20 percent of performance results across a wide range of metrics) are doing to alleviate "many versions of the truth" that exist within their organizations.
The study reveals the strategies, actions, and technologies that are providing the highest degree of success toward achieving "one version of the truth." The report is available for free (short registration is required) through November 2008 ($399 thereafter) at Aberdeen Group's site (see http://www.aberdeen.com/summary/report/benchmark/5298-RA-decisions-based-reality.asp).
To understand the value of achieving a single version of the truth, it is important to tie initiatives to the achievement of bottom-line company performance improvement. Aberdeen used four performance metrics to identify Best-in-Class (perform in the top 20 percent across all four metrics) companies:
- Customer satisfaction: percent year-over-year change in customer satisfaction (past 24 months)
- Employee productivity: percent year-over-year improvement in employee time spent looking for correct information or correcting erroneous information (past 24 months)
- Overtime incurred: percent year-over-year change in the amount of overtime incurred (past 24 months)
- Litigation costs: percent year-over-year change in the cost of litigation to the organization (past 24 months)
The desire to obtain a single version of the truth is not new, but many companies are still planning or have just begun to deal with the issue as it relates to their business intelligence implementations. One of the first characteristics that Best-in-Class companies reveal is that they have been aware of the problem and have started to address it. Eighty-six percent of Best-in-Class companies have begun to take action, while over one-third of all other respondents have yet to start. However, the length of time spent on the problem is not the only factor determining Best-in-Class status.
Survey results show that the firms enjoying Best-in-Class performance shared several common characteristics:
- Best-in-Class companies are more mature than others in the length of time they have been applying resources toward obtaining a single version of the truth; 59 percent of Best-in-Class companies have more than two years of experience compared to 47 percent of Industry Average companies and 37 percent of Laggards
- Best-in-Class companies are more than twice as likely as all others to form policies about data access for end users
- Best-in-Class companies are 48 percent more likely than Industry Average companies to have implemented a data stewardship program (a formalized program focused on end-to-end data quality and ownership) in their organizations
To understand the problem, companies must identify the levels of data management, delivery and consumption that are the root causes for multiple versions of information to exist within their organizations. Respondents were asked to choose the top 2 root-causes of multiple versions of the truth in their organizations. Figure 1 reveals the top-five stages of data management from which multiple versions of the truth emanate.
The majority of problems arise at the data source and integration levels, which explains why master data management (MDM) and data warehousing technologies and services are at the forefront of technology investment today. Almost half of respondents identified end users as the culprits. Despite efforts to clean and integrate information, many organizations are still struggling with the analytical and reporting skill sets necessary to present information correctly.
Strategic action is being inhibited by three major factors that challenge business executives, line-of-business management, front-line end users, and IT staff alike. Fifty-three percent said the top inhibitor involved required data that is difficult to access and integrate from source systems. One third (33%) identified a lack of top management commitment to projects) while just one one-fourth of respondents (26%) said lack of IT resources/bandwidth was the top inhibitor.
As companies continue to focus on changing their culture, organizational "buy-in" needs to come from the top. However, one-third of respondents report that top management commitment to projects is difficult to attain. Likewise, as companies seek to obtain more timely information, they are finding that the required data sources are difficult to access and integrate. Third, but not to be overlooked, is the lack of IT resources and bandwidth of existing resources that can be applied to solve these problems.
Best-in-Class companies are making the solution to these barriers a corporate initiative. The establishment of a formal committee or working group to identify data sources and access policies (based on the end-user information requirements and security process management described above) is a logical first step. Establishing true "ownership" of the data and its uses, access, and flow is important to determine. This is a critical aspect of data stewardship and a formalized approach to data management. Best-in-Class companies are implementing this approach at more than twice the rate of all other respondents (Figure 2). The formation of a committee or working group also denotes another important factor -- "buy-in" from senior management.
Figure 2 also illustrates the importance of "data stewardship." More than just "ownership" of data sources, stewardship denotes a responsibility on the part of a steward to ensure that data is managed, cleansed, stored, and delivered according to the policies, guidelines, and governance specifications as defined by a formal committee. Best-in-Class companies are currently utilizing a stewardship program at significantly higher rates than other respondents.
Figure 2 - Best-in-Class Organizational Management Capabilities
Source: Aberdeen Group, Sept. 2008
Click to see full image
There are many technologies, and therefore many technological skills and fluencies to learn when it comes to obtaining a single version of the truth. Many of these technology and service enablers are discussed below. Before these can be described, it is important to spearate "technology management" from the actual technologies themselves.
The top inhibitor blocking efforts to obtain a single version of the truth is the inability to access and integrate data from source systems. Therefore, this is exactly the technology management capability that is most critical to obtain. Best-in-Class companies are 37 percent more likely to have established this capability than other respondents, on average.
Source systems are rapidly changing as indicated by the data point above.
Internal data is still at the heart of the matter, but companies are seeking and accessing data sources that exist externally, often in incompatible formats, or more often, unstructured formats. This is resulting in larger data volumes that must be manipulated and processed so true integration occurs. Best-in-Class companies, therefore, are becoming more adept at handling new and non-formatted data.Dealing with disparate data sources involves many data management tools and technologies (Figure 3).
Best-in-Class companies are only slightly more likely to currently utilize data acquisition and integration tools along with data discovery, profiling, and mapping tools. However, when it comes to tools dealing with master and metadata management, along with data cleansing/quality enablement, Best-in-Class companies are investing significantly more. (Aberdeen Group has produced extensive research on MDM, including Winning Master Data Management Strategies for 2008-2009 -- http://www.aberdeen.com/summary/report/benchmark/4634-RA-master-data-management.asp).
Survey respondents we interviewed cautioned that once a data management strategy and enabling technologies have been applied, there are still many potential pitfalls and stumbling blocks. Business executives who wish to analyze data also find that they must apply business rules and calculations to the data to derive the answers to questions and the summaries and metrics needed to manage the business. All too often, this is accomplished in a spreadsheet environment. In fact, 69 percent of respondents report that spreadsheets are their primary method for conducting data analysis. Best-in-Class companies, however, are also very likely to utilize BI software platforms for providing analytics capabilities.
The determining factors between the use of spreadsheets and what is best accomplished within BI software environments depends upon the nature of the data and its intended goal. If data is "transient" in nature -- if it is used for one-time analysis by a user or group of users within a single point in time -- a spreadsheet may be well-suited for sharing and acting upon information. If the data is non-transient -- if it is data that changes during the analysis and reporting timeframe and is used multiple times by many users and groups for multiple decisions -- spreadsheets are most likely a poor vehicle for sharing and initiating actions and decisions. Best-in-Class organizations are more capable of replacing "gut-feel" decisions with "fact-based" decisions and improving management visibility to critical business information.
As reported in Aberdeen's March 2008 Benchmark Report, Data Management for Business Intelligence (see http://www.aberdeen.com/summary/report/benchmark/4654-RA-data-management-business-intelligence.asp), automation of report generation is a critical Best-in-Class enabler. Through automation, errors in data handling and report formatting can be avoided. This reduces a common entry point for erroneous information and also can reduce report creation costs. The level of difficulty associated with mastering all of these technologies is not lost on Best-in-Class companies. Outsourcing is rapidly becoming a necessary step to fill IT skill-set gaps. In fact, Best-in-Class companies have invested in IT consulting services at a rate 21 percent greater than all other respondents.
Adoption of the technologies involved varies, and is not universal. All respondents reported some level of adoption for the technology enablers studied in this survey, and most notably, IT consulting services seem to be the most prevalently used by all businesses, Best-in-Class, Industry Average, and Laggards alike. Due to the existing heavy investment in IT consulting services, they are the least likely enabler to be adopted in the next 12+ months. Data discovery/profiling/mapping tools, MDM tools, and dashboard/scorecarding tools are reported as being the most planned-for technology investments by survey respondents.
Whether a company is trying to move its performance from Laggard to Industry Average, or Industry Average to Best-in-Class, the following actions will help spur the necessary performance improvements:
- Start with end-user information requirements. Laggard companies are slightly less than half as likely as Best-in-Class organizations to formally poll end users about their information requirements. This leads to failed BI projects and can force end users to seek their own means of obtaining necessary business information. This alone can be a root cause of multiple versions of existing information..
- Build a working group or committee. Best-in-Class companies have realized that to gain management buy-in (from top management as well as functional business unit managers) it is necessary to form a committee or working group. Obtaining a single version of the truth requires that companies first stop processes and practices that lead to multiple versions. Start with a single data set and a single department or operational area, then move to a cross-departmental committee that involves members from many disciplines, including IT and functional business units. Best-in-Class companies use this approach more than twice as often of Laggards.
- Focus on understanding data relationships to the end application. Obtaining a single version of the truth might not entail applying data management, analytics and information delivery technologies to all data all of the time. A deep understanding of the relationships between source system data and the requirements of the end application can result in the discovery of efficiencies and opportunities for automation that can actually simplify the process. This requires judgment and management involvement based on the context of how data relates to business activity. Laggards are 1.2 times less likely to take this route versus Best-in-Class companies.
- Establish a formal data stewardship role. Data stewardship is more than "ownership" of data. Companies that expand the role to encompass the responsibilities related to data cleanliness, integrity, and adherence to security and access are more successful. Best-in-Class companies are nearly twice as likely to have established this role as Industry Average companies.
- Apply integration techniques to all data types from internal and external sources. Best-in-Class companies are 37 percent more likely to apply integration techniques to both internal and external data. Even if the initial focus is on internally managed data, it is important to establish data integration policies and standards. As external data becomes an increasingly critical necessity, be aware of increasing data volumes and the strain this may have on existing systems.
- Security is not a job. While establishing security policies governing data access is a Best-in-Class strength, the disparity between Industry Average and Laggard companies is not nearly as great as compared to other capabilities. Despite the high level of adoption, interviews with Best-in-Class respondents suggest that many organizations place the security and data access responsibilities in the hands of a single person. Aberdeen research has found that an organizational approach to data access policy has a profound effect on performance. Often, this involves setting up a "Center of Excellence." For more information about the value of establishing a Business Intelligence Center of Excellence, read Aberdeen's July 2007 benchmark study, Delivering Actionable Information to the Enterprise (see http://www.aberdeen.com/summary/report/benchmark/4278-RA-it-bi-skills.asp).
- Develop and manage milestones. The ability to obtain a single version of the truth is elusive, and perhaps even more so if project milestones and related key performance metrics are not defined and managed. Although Best-in-Class companies are far more likely than other respondents to take this approach, more than two-thirds are not currently applying performance management today. Aberdeen's Operational KPIs and Performance Management report indicates that one of the first steps to achieving operational performance objectives (such as this data management exercise exemplifies) is to establish a formal process for clearly defining related KPIs.
The Bottom Line
Aberdeen's research has revealed that there are several factors to consider when dealing with multiple versions of the truth within business intelligence implementations, including:
- End-user trust in data within BI applications: data integration and quality issues must be resolved if end users are to trust the information they are accessing
- The absence of a common naming process for data elements: the term used for "customer" may be "account" or "client" or "cust" and a "sale" may be confused with "order" or "purchase"
- End-user skills: even the best data may be misinterpreted if users do not have the BI skills necessary to create the reports, calculate the measures that drive the business, and interpret the results
- Management of data and governance of analytics: different areas and groups within the business may have differing methods for calculating key performance metrics, such as joins between tables; data hierarchies to support relationships between attributes; and derived measures and attributes
- Security and access: security procedures and capabilities not only address the need to re-strict access to confidential information but can also help simplify users' experience; e.g., regional sales managers only see sales for their region, etc.
- - -
David Hatch is vice president and principal analyst of Aberdeen Group's business intelligence practice, where he benchmarks user organizations' BI strategies, actions, and planned technology investments. His research focuses on the collection, assembly, and delivery of information throughout the enterprise. Dave holds a BA in Communications from the University of Massachusetts.