Q&A: BI Trends Point Up Despite Down Economy

With the economy showing few signs of rebounding any time soon, what should business intelligence buyers do?

With the economy showing few signs of rebounding any time soon, what should business intelligence buyers do? Closer financial scrutiny of technology decisions is virtually guaranteed in today’s markets, so how can IT emphasize business results and the competitive potential that business intelligence software can offer? In addition, what BI and data warehousing trends will continue to grow, and how can BI tools better address the needs of true business users?

To find out, we turned to Vickie Farrell, strategic marketing manager for HP’s BI Solutions business unit. Ms. Farrell has spent over 20 years marketing database, data management, and data warehousing products at a variety of companies. Farrell speaks frequently and has written numerous articles about implementation success and trends in data warehousing.

BI This Week: How is the global financial crisis affecting business intelligence? What are some of the forces shaping customers’ selection and use of BI right now?

Vickie Farrell: The weak economy is taking its toll on the IT industry. In February, IDC’s estimate of worldwide IT spending growth for the year was 0.5 percent (IDC, Worldwide Black Book Q4 2008, February 2009). Even so, analysts see BI as a relative bright spot, with a projected growth rate between 2 and 10 percent.

As HP noted in its recent white paper, Top 10 Trends in Business Intelligence for 2009 (http://h20195.www2.hp.com/V2/GetPDF.aspx/4AA1-8346ENA.pdf), there is awareness that conducting business in an economy that has no precedent makes fact-based decision making more important than ever. However, because of the economy, organizations have become stricter about requiring that BI investment be tightly linked to specific business objectives with quantified business value. We are seeing requirements for shorter payback periods and “self-funding” BI.

Although financial scrutiny is a requirement, BI buyers who emphasize business results and the competitive potential of exploiting technology focus less on limiting spending and more on optimizing the investment they are making.

One approach that has become popular for BI, in order to meet users’ demands while mitigating risk -- and one that economic pressures may accelerate -- is the use of managed, hosted, and outsourced services. An extended weak economy is likely to result in mergers and acquisitions in several industries, which will accelerate the need to integrate disparate data to get a single view of the business.

BI clearly continues to increase in importance, as evidenced by the spending patterns and predictions you’ve cited, as well as others. What BI trends will continue to grow despite the economy?

Data warehouses in place today were built for strategic decisions, the sweet spot of traditional BI. Timeframes allowed analysts to manually cleanse and reconcile data from multiple sources. Many organizations want to increase their intelligence by giving more employees access to analytic tools and applying them to operational decisions. They’ve come to realize that the limitations of first generation BI systems are not just their inability to handle large volumes of data and users but their lack of data integration rigor, including data cleansing, master data management, and metadata management.

As a result, there is an increased focus on data governance and data integration as a means to establish a foundation for enterprisewide advanced analytics. Cost control pressures increase the priority as organizations realize that the “people” commitment for custom-coded and semi-manual data integration is no longer reasonable.

In a 2008 HP white paper examining top trends in BI, we noted the small number of well-formed governance committees and processes, predicting that companies would begin to move from theory to practice in 2008.

Early results of a 2009 HP Business Intelligence solutions survey about BI initiatives shows that a third of respondents do, indeed, have a formal enterprisewide data governance process in place, with another third to a half planning to implement one in the next 12 months.

Analytics seem to be increasingly necessary in order for companies to compete effectively, yet analytic tools are complex and can be costly. What’s the solution?

Organizations are realizing the need for an overall enterprise information management (EIM) strategy in order to leverage data as a corporate asset, to apply advanced analytics that will help them achieve a discipline of fact-based decision-making, and eliminate the wastefulness of different teams using different tools with little consistency and lots of overlap and redundancy. Data integration is a critical first step to an overall EIM strategy. Inconsistent meanings create barriers to reliable analytics. As the boundaries between CRM, ERP, and product lifecycle management application domains erode, there is a growing need to create an enterprisewide information strategy to ensure semantic consistency for all users, applications and services.

As BI tools filter down into the enterprise and into the hands of more business users, are they becoming any less complex?

Improved data quality, rigorous data governance by business users themselves, automated data classification, enterprisewide metadata management providing semantic consistency of all relevant data -- all of this will serve to make the application of analytics less complex and more self-service for business users.

The traditional approach to analytics has been to hire modelers with Ph.D.s who spend three months developing a model to do customer segmentation, for example, resulting in up to a few dozen or perhaps a few hundred models a year. Capturing the sophistication in tools that can be used by business analysts enables the development and use of not hundreds but thousands of models, with a much shorter development time. This approach makes it possible for someone who doesn’t know what a neural network is to use one. We are seeing analytic tools move in this direction.

How are companies addressing the gap between the ability of operational applications to deliver operational reporting and the data warehouse to deliver operational analytics?

In the 2009 HP survey on BI solutions that I cited earlier, a top initiative is operational reporting. Initially, organizations expected ERP systems to provide this. When they couldn’t -- because of backlogs and overload -- users turned to the data warehouse as a platform optimized for strategic analysis. Traditional BI satisfies most strategic reporting and analysis, but not real-time operational reporting with its associated needs for high-volume real-time data updates, high availability needs, and high throughput rate of operational queries.

We are seeing a desire for convergence of the data warehouse, operational data store (ODS), and operational systems. Organizations are attempting to architect a data warehouse that can balance the needs of strategic analysis with the priorities of real-time operational reporting and analysis, ideally collapsing the ODS into the data warehouse without increasing management complexity, and still meet all service-level agreements (SLAs).

How is HP addressing this emerging convergence of the data warehouse, ODS, and operational system?

HP designed the HP Neoview data warehouse platform specifically for that purpose: to meet the performance requirements of strategic analysis, including predictable and consistent response times, while meeting the high-volume data update and query performance requirements of an operational environment, without compromising one for the other. HP Neoview’s adaptive segmentation approach dynamically reallocates system resources and parallelism to optimize overall performance and to meet a wide range of SLAs.

Notable are HP Neoview customers’ ability to continuously ingest data at a high volume and maintain high throughput of short-running tactical queries at peak loads, without disrupting long-running reports and queries or requiring special management overhead.

Must Read Articles