In-Depth

The Future of BI: A Business Intelligence and Data Warehousing Market Overview

• 02/01/2000

The enabling technologies of business intelligence and data warehousing (BI/DW) are playing a vital role in facilitating strategic corporate and organizational initiatives, such as customer relationship management and e-commerce, worldwide. Evolving CRM application areas, such as customer knowledge management (CKM), are viewed as key technology solutions for not only understanding customers, but for maintaining corporate competitiveness and increasing revenue.

One of the fastest growth areas in the BI/DW market is the data warehousing of ERP data, and Web-enabling ERP information to facilitate business-to-business, and business-to-consumer e-commerce. BI/DW technologies will also play an important role in the evolution of knowledge management platforms and what are now being called enterprise information portals (EIP). Although there is currently little data, and/or primary research to support the existence of an EIP market (the majority of assumptions in this article are based on DBS III, a worldwide survey of data warehouse implementers that was completed during the fall of 1999), the concept of an enterprise portal with access to multiple data sources and information is sound.

The business factors driving data warehousing and business intelligence projects include:

• Organizational access to financial information
• Data warehousing of ERP information
• Customer knowledge management
• Customer relationship marketing
• Data warehousing human resource and other organizational information
• Organizational e-commerce initiatives
• Business-to-business
• Business-to-consumer
• Business-to-employee
• Business-to-supplier

The overall cost of data warehousing and enterprise software licensing are key issues for organizations that are providing mainstream user access to the data warehouse/data mart. These are also important issues for organizations that are beginning to transition from client/server to distributed computing architectures that enable concurrent use and Internet access.

As with any early majority market, in the coming years many large enterprises will attempt to standardize on a single BI/DW vendor in order to reduce up-front cost and overall maintenance and administration. Small- to mid-sized organizations will seek affordable and tightly integrated BI/DW solutions that deliver immediate ROI, such as Microsoft's SQL server platform, and/or Linux with IBM's DB2 or Oracle. If Microsoft delivers a stable, reliable operating system in Windows 2000, the SQL server platform will provide an affordable and easy-to-implement business intelligence and data warehousing solution that will radically change the landscape of the BI/DW market.

The combination of inexpensive Intel hardware, along with built-in OLAP and data transformation services, and a highly functional Excel front end and pivot table services, will be a very attractive, easy-to-implement BI/DW solution for many organizations.

If Microsoft fails to deliver a stable OS in Windows 2000, we predict that Linux (which is already 64 bit), with the support of the major DBMS vendors, Oracle and IBM, could become an important platform for the DBMS and data warehousing. On the contrary, it is not clear if Linux will have the severe impact; Microsoft and UNIX vendors may, indeed, bear the brunt of this disruptive technology.

With Y2K issues behind us, demand for business intelligence solutions, data enabling and data warehousing technologies will continue to drive the market to what we estimate will be approximately $150 billion in services, software, systems and in-house spending by the year 2003. According to the DBS III survey, organizational expenditures for data warehousing initiatives are expected to increase by 290 percent over the next three years. And approximately 25 percent of BI/DW expenditures will be in-house followed by:

• Systems, 23%
• Storage, 23%
• Packaged software, 16%
• Outside services, 16%

Overall, a significant opportunity exists for VARs, systems integrators and BI solutions vendors in assisting small- to medium-sized organizations whom have indicated that they are implementing BI/DW solutions on Microsoft SQL server.

Organizations are Web-enabling the data warehouse, and 87 percent of the respondents indicated that their DW was designed for Internet/intranet access. Nearly all members of the organization will become consumers of business intelligence information over the next three years as organizations provide access to increasing numbers of:

• Knowledge workers
• Mainstream employees
• Customers
• Suppliers
• Prospects
• Internet surfers
• Employees

New technologies will emerge, eliminating the need to follow the traditional path of implementing data warehouses. Line of business management will place demands on IT for virtual data warehousing and immediate access to operational data with OLAP integration.

A significant number of respondents in our survey indicated a desire to data mine or directly analyze operational data, and would like OLAP integrated with the operational database.

Demand for solutions, and not technology, will continue to drive vendors to enhance the features and functionality of their offerings. Analytical applications will require customization for specific vertical market segments, thereby driving up the barrier of entry into market segments for vendors.

In our survey, more than 50 percent of the respondents indicated they would purchase a packaged BI solution. And the primary motivators for buying packaged BI solutions are faster deployment and lower cost.

Pre-packaged applications that meet these requirements will require significant vendor investment for each vertical market segment. Overall consolidation of the software sector will continue as vendors attempt to provide customers with end-to-end business intelligence solutions. IBM, Microsoft and Oracle are the dominant DBMS players in large enterprises, and also play a significant role in small- to medium-sized businesses, along with Microsoft SQL server.

Microsoft SQL server is beginning to play a significant role in small- to medium-size business and also at the department level of large enterprises. DBMS represent the largest segment of software expenditures, (35 percent) followed by analytical tools, analytical applications and ETL tools. Organizations will develop ERP data warehouses with in-house resources or look to ERP application vendors for solutions.

The systems and hardware market is also set for significant growth, with UNIX and Windows 2000 leading the pack. In fact, as of this writing, the hottest data warehousing platform in the market is the Sun Starfire. UNIX will also become an important platform for hosting front-end business intelligence tools for large enterprise deployments. Many customers are currently hitting the wall with NT systems that can't provide enterprise-strength scalability, and are waiting for UNIX.

For these users, the enterprise is moving away from the mainframe as a platform for new business intelligence applications and data warehouses. UNIX is now an important platform for large enterprise data warehouse servers and is dominated by four vendors: IBM, Compaq, Sun and HP. The storage subsystem component of the data warehousing market is expected to grow significantly in the next five years.

The services category is driven primarily by the lack of skilled in-house talent, which we expect will persist well beyond the year 2000, therefore the accompanying forecast may be too conservative. The largest service categories are system integration and implementation services, which include design, architecture and software development.

The fastest growing category, however, is outsourcing of solutions. (As distinguished from outsourcing of IT operations, which is not included in the numbers.) This is a new forecast category for Survey.com and reflects the trend toward the use of ASPs (Application Service Providers). In conclusion, Web enabling of the data warehouse and preparing the infrastructure to facilitate business-to-business e-commerce, will place a heavy strain on IT resources and open the vault for systems integrators.

The BI/DW market is set for enormous growth as organizations begin their strategic top-down initiatives to drive up revenue, facilitate e-commerce and maintain their competitive status within vertical market segments.

In the years ahead, organizations will be continuously challenged to meet the informational demands of their customers, business partners and employees. Web-enabling the data warehouse and providing access to mainstream users within the organization, as well as business partners and customers, presents a tremendous business opportunity to BI/DW vendors.

This trend also represents an important point in the evolution of business intelligence and data warehousing; the market is now well past the early adopter stage and is entering the early majority stage. Organizations are now demanding business intelligence solutions with enterprise scalability and functionality that address issues, such as cost of ownership and administration.

About the Author: Peter J. Auditore is the Vice President of the Business Intelligence and Data Warehousing Program, responsible for syndicated research services at World Research Inc. (San Jose, Calif.).