In-Depth
Business Intelligence: BI/DW in the Millennium
The business intelligence and data warehousing market has entered the early majority stage of its development; organizations are now recognizing the strategic importance of these technologies in conducting their business. Many companies view these enabling technologies as key to maintaining their overall competitiveness and increasing their revenue and profitability.
Our research indicates that many organizations are beginning to significantly increase the size of their data warehouse, while others are just implementing their first data warehouse project. Over the last two months, we conducted five primary market research surveys that include more than 1,200 interviews of BI/DW implementers, including: DBS III, ETL, CRM, SQL Server, ERP and data warehousing.
Future business intelligence initiatives will facilitate a broader range of functionality and deliver information to a mainstream audience within and outside of the organization, via the Internet. Many of these initiatives include CRM and customer knowledge management (CKM)-oriented functions, in addition to the data warehousing of ERP information. This Web-enabled data will be used extensively in facilitating business-to-business and business-to-consumer e-commerce in the Internet economy of the new millennium.
This month, the discussion focuses on BI/DW market growth, and we present some results from our recently completed ETL survey. Our forecast is derived from primary market research and econometric modeling, so don’t be surprised to see significantly larger numbers.
ETL Survey Results
Access to and integration of disparate data sources is one of the most difficult processes of any data warehousing initiative. According to our survey, organizations are willing to spend more than one third of their overall data warehousing budget on ETL tools. The ETL process must easily facilitate heterogeneous database access and ensure high-quality data upon which strategic organizational initiatives are based. Data modeling and profiling was a feature high on the wish list of most survey respondents. Not surprisingly, this primary research determined that the largest volume of data that was moved on a daily basis was financial data for sales and revenue analysis. The second largest volume of data moved on a weekly or biweekly basis was customer knowledge management-related, reflecting the organizational importance of CRM.
The impact of Microsoft is just beginning to be felt in the ETL space, as only a small percentage of the sample base is currently using Data Transformation Services (DTS). In our view, Microsoft’s Data Warehousing Architecture Framework, which includes DTS and OLAP services with SQL server, will significantly decrease the overall cost of data warehousing. Interestingly, survey respondents also indicated that cost was a significant reason for not purchasing ETL tools in general.
BI/DW Market Growth
We expect the worldwide and North American markets for BI/DW to grow at an AAGR of 43 percent between 1998-2003 to more than $148 billion. This forecast is consistent with the findings of the database solutions series of studies that we have conducted over the past few years.
The BI/DW market forecast consists of five major segements or categories, which we refer to as product types:
• Systems – the hardware, the operating system and associated management tools.
• Storage – storage hardware and storage management software.
• Packaged Software – DBMSs; analysis tools, such as query, reporting, OLAP, data mining, etc.; backend or ETL tools for data extraction, cleaning and transformation; and applications packages (often called analytical applications).
• Services – business consulting, implementation, technical support, user training and education, outsourcing of solutions and system integration.
• In-house expenditures – what survey respondents indicated they will spend internally (including the salaries of data warehouse administrators, etc.).
About the Author: Peter J. Auditore is the Vice President of the Business Intelligence and Data Warehousing Program, responsible for syndicated research services at World Research Inc. (San Jose, Calif.).