Special Report: Business Intelligence Overview -- Data Warehousing and a Lot More
What if every MIDRANGE Systems article was specifically written to your exact AS/400 specifications? What if we knew the hardware configuration, memory capacity, DASD storage, and software applications for each and every reader -- note that's reader, not subscriber -- to our publication? And what if we knew exactly which articles you read, who you pass the magazine to and what they read, as well as how many business trips you expect to make in 1999? That, in a nutshell, is the kind of customer detail your end users are looking for from you when it comes to providing business intelligence (BI) to your organization.
"Business intelligence is the process of turning data into knowledge and knowledge into action for business gain. It encompasses all types of data -- hierarchical, relational, text, spatial, audio, video, etc." Says Wayne Eckerson, Vice President of Technology Services, TDWI, "Where a data warehouse focuses on structured data stored neatly in relational tables, business intelligence encompasses the broad range of data and information that business people use to make decisions and act."
Ken Holec, President and CEO, ShowCase Corporation points out, "Business Intelligence is not a new concept. Almost every customer has clamored to find ways to have access to their data. All the users know the data's out there. They just want to be able to get it in the form that they need it."
The BI Continuum
Holec describes BI as a continuum with three types of users. The first level of BI is data access. IS departments offer users simple report writing tools attached to AS/400 databases. This empowers power users and technical users of the system to create reports for the rest of the users in company.
The second level is where data is turned into information. Here IT or power users build some data definitions and data structures that make it easy for the others to create their own queries or reports.
And the third level of Holec's continuum is intelligence. He says, "This is where your people are performing 'speed-of-thought' types of analyses using large volumes of data."
If your OS/400 is V4R3, you're in a significantly better position to provide this kind of usable data than with any previous release.
Encoded Vector Indexes
"First and foremost, Release 3 introduced a brand new technology -- Encoded Vector Indexes," explains Mark Wulf, IBM's Business Intelligence Segment Manager for AS/400 Partners in Development. Encoded Vector Indexes (EVIs) are, "A new indexing technology specifically designed for the business intelligence environment," says Wulf. "It allows us to process ad hoc or randomly generated queries much more efficiently."
You know traditional indexing works well when you know what queries are going to be asked. You optimize the indexing for those queries, and are fairly certain answers will be efficiently generated. The strain on your system occurs when a query is entered that isn't accounted for. That's when response time can change from milliseconds or minutes into hours.
For example, applications originally designed to handle requests for daily and monthly information are going to tax a system severely if an end user wants to look up that data weekly.
"What the encoded vector index allows is more complete coverage from your design without the extra storage and processing time it would have taken with a traditional approach. Most importantly, encoded vector indexes give you a way to rapidly combine indexes so that system can create the perfect index on the fly," says Wulf.
"This is why data warehousing is taking off ," says Holec. "Online analytical processing (OLAP) analyzes databases, anticipates and pre-calculates what end users will look at, then calculates the total. In the days before OLAP you would have to write query after query. OLAP takes seconds to drill down and find problems or opportunities. A lot of our customers need analysis on 1000's of products in 100's of stores serving 100,000 customers."
Other V4R3 capabilities that make BI solutions more accessible include hierarchical storage management for more cost effective data storage and the introduction of new hardware. "Larger main memory, larger DASD, and 40 gigabytes of main memory on a 64 bit machine allows us to do things that can not be done on any other system," says Mark Wulf.
Ben Barnes, General Manager, Global Business Intelligence Solutions, IBM, adds, "Over the past few years, we've increased business intelligence performance by 400%. We've increased chip function. And we've increased query response by 100 to 200%." Barnes believes, "A lot of the operational efficiencies have already been reached. But customers can't afford to neglect the time and energy it takes to rethink their processes. Utilization of business intelligence solutions changes the way you do business. The first thing people want to do is build a relationship view. Then, from that view, they determine how to utilize their products and services."
Business Drivers
Barnes says there are three basic business drivers for business intelligence, "First you want to focus on reducing costs and continue to deliver the same level of service." Some examples he cites are improved product distribution as well as handling fraud and abuse in banking, healthcare and at the retail cash register. "The second area is profitability. Who are the profitable customers? What are the most profitable products? And three is relationship marketing or database marketing. Here you use the database build to analyze who are profitable customers, how do I retain them, cross-sell and up-sell them, promote customer loyalty, and analyze attrition to win back customers."
Mitchell's of Westport has used an AS/400 since 1989. With 150 employees and about $40 million in sales of custom-tailored men's clothing last year, they've grown their data warehousing and mining activities into full-blown BI. Jack Mitchell, CEO declares, "We have data down to the SKU level. Jim Preston, the retiring CEO and Chairman of Avon is one of our customers, and we can give you all kinds of client information and intelligence. What suits he bought, what designers he likes and doesn't like, when was the last time he bought topcoat. We have a list of family members and birthdays. We know he likes to buy international suits for travel… It enables us to tailor everything we do for every customer -- even our 27 tailors and seamstresses are online."
Mitchell adds, "Our sales associates love it because they can go the extra mile. We have one associate who has only 212 customers, and she writes over $1.3 million in business. Everything is personalized. Basically it's an interactive system on the screen." The company started developing its BI in 1992, with a client accumulation program that targeted 286 clients who spent over $5000. They now have over 1800 clients on line. "I truly believe that the AS/400 gives us the ability to do this," says Mitchell.
"Business drivers are the single most valuable resources companies have to gain a competitive advantage in marketplace," notes Eckerson. "It's information and the ability to harness that information in the right place, at the right time, and in the right direction that results in the practical application of knowledge."
Caveats
Eckerson continues, "What's happening is that the AS/400 is proving to be a very powerful platform for delivering integrated BI solutions." But there are pitfalls IS managers need to be very aware of.
"They have to adopt the right methodology approach to creating data warehouses, otherwise they could end up spending wads of money with nothing to show for it," he says. "There is a paradigm shift that catches many IT folks unprepared. Transaction systems are fairly stable and don't change much once you roll them out. Data warehousing is an organic system that constantly changes once you roll it out, to meet the changing needs of business."
Eckerson has seen a number of instances where IS managers have seen the value before business people do, but he says there's a big danger in spearheading the charge. "What IT needs to do is scout around the organization for a department or division that is feeling pain and a manager anxious to do something about it." He says they'll see the benefit and ask you to develop a prototype. You want them to spearhead the initiative and push things through the inevitable snafus, obstacles and political barriers.
Eckerson warns, BI is a process not a technology. It requires long term commitment. It requires enterprise architecture that is incrementally implemented with tight collaboration among business and IS people who can communicate in a common language.
What's Ahead
"We expect that over the next three to five years that the majority of access to [business intelligence] data will be done through a browser on the web," says Holec. Another trend he is already seeing is a move from enterprise resource planning (ERP) to the next stage. "The business intelligence boom is a follow-on to the ERP boom. When companies put in new business applications they look to leverage it. A lot of our sales are going to be those companies who added ERP and now want to move to business intelligence." He also expects applications to combine various "canned" solutions with the flexibility to customize the app's in-house to meet a wide variety of customer needs.
One user-customizable ERP application is IntellipriseÔ, an integrated supply chain management package from American Software. Product literature says Intelliprise handles planning, scheduling and inter-department collaboration. It lets users develop and change rules that guide operations management and measurement on their own. And the system continuously monitors performance compared to benchmarks.
In broader terms, Eckerson envisions, "Technology will either store all critical business information into a relational knowledge-base, using advanced database technologies from Oracle, IBM, Informix, etc. Or, we will integrate various engines to create a giant publish-and-subscribe architecture that can dish up any combination of information to users or applications based on predefined preferences or business processes. I suspect both types of architectures will gain momentum."
In the immediate future expect IBM to support BI via: universal database support for the AS/400, including non-relational data; SQL imbedded directly into JAVA applications; extending BI information onto the web; business partners providing JAVA "beans" that can run inside the Lotus Notes environment; main memory up to half a terabyte to help clear up BI's biggest bottleneck -- getting data from DASD to memory; new substrates and insulators; as well as tools to monitor database usage.