Business Intelligence Booming on AS/400

The AS/400 has been one of the leading transactional business computing systems for the past 10 years. So with all that transactional business data, it’s only natural that it’s now coming into its own as a business intelligence server.

Business intelligence is loosely defined as the access and analysis of information that resides in databases throughout an enterprise. Many AS/400 shops are involved in business intelligence at some level, if only just using a report writer or query tool that resides on an NT server and drives against a transactional DB2/400 database.

But the most dynamic area of business intelligence is data warehousing, specifically native data warehousing on the AS/400. It’s a segment considered by IBM and analysts alike to be growing and one in which application choices are increasing as vendors from other platforms port their data warehousing solutions over to the AS/400.

"As IBM continues to invest in it and support it, more vendors are coming to the platform and providing more application choice," says Liz Shahnam, senior research analyst at the Stamford, Conn.-based Meta Group. Shahnam says the Meta Group predicted the trend towards AS/400 data warehousing last year.

Mark Wulf, business intelligence manager at IBM AS/400 Partners in Development agrees that the data warehousing application choice on the AS/400 is increasing. "Certainly, a number of top industry tool providers have ported over to the AS/400 for data mart type applications, whether its Dimensional Insight or Venture SystemSource bringing Pilot and Pilot applications, or applications that run on [Arbor] Essbase on other servers," he says.

But it wasn’t always that way. The AS/400 was originally perceived as strictly a transactional system that couldn’t handle data warehousing, according to Shahnam. "The AS/400, coming from the transactional world, was perceived as being not robust enough for data warehousing and not having enough application choices," she says.

Chicken-and-Egg Scenario

The AS/400 found itself in a typical chicken-and-egg scenario, Shahnam adds. Vendors didn’t want to develop data warehousing applications for the AS/400 because they didn’t think it would make a good data warehousing server. Meanwhile, customers didn’t take the AS/400 seriously as a data warehousing server if it didn’t have enough applications.

What changed? Shahnam says large AS/400 users who wanted to build data warehouses on their AS/400s from their vast amounts of OLTP data helped drive the market. The November 1994 release of V3R1 of OS/400 with its support for symmetric multi-processing resolved any technological issues the AS/400 might have had for data warehousing.

DB2 Symmetric Multiprocessing for OS/400 expanded on the parallel capabilities of DB2/400, optimizing it for data warehousing and decision support processing. A single database operation – most typically a query – could run on multiple processors at the same time, achieving exponential reductions in query time depending on how many processors were used.

"There’s been a pretty constant focus [on business intelligence] ever since V3R1," Wulf explains. "We’ve tried to focus on business intelligence with our larger system announcements. Even if people are still small, they want to know what the high-end capabilities are when they’re making their buying decision."

While data warehousing is an all-encompassing term, data marts – smaller, departmentalized data warehouses – are what most AS/400 data warehousing vendors specialize in building.

As Wulf notes, Dimensional Insight (Burlington, Mass.) and Venture SystemSource (Ridgeland, Miss.) are two of the newest vendors in the AS/400 data mart space, both coming from the Unix/NT world. SystemSource has ported Pilot Software’s OLAP engine to the AS/400.

Dimensional Insight introduced its DI-Atlantis product to the AS/400 early this year, and brought more than half a dozen native AS/400 data marts spread over 20 sites into production in its first six months, according to company officials.

"We focus on customers with ‘thin IS,’ meaning that the resources devoted to IS are relatively small compared to the size of the company," says Dan Jablonski, product manager for DI-Atlantis. "Thin IS companies of all sizes do exist, and quite often they are AS/400 shops."

He says DI’s primary industry focus is wholesale distribution. He also adds that DI-Atlantis has a "dimcount" feature, which specifies the number of unique values of any dimension per unique value of any other dimension. This feature is particularly useful in wholesale distribution.

"For distributors, this makes it easy to look at the number of customers sold per salesperson, or the number of brands sold by customer. Lost business and sold/unsold reports are a breeze, and they are the heart of the wholesale distribution firm’s reporting needs," Jablonski says.

For him, porting DI-Atlantis to the AS/400 was "only natural" since it’s a turnkey solution that works out of the box without extensive development, the kind of solution that AS/400 customers bought their AS/400s for.

"We believe the market for data warehousing products on the AS/400 is opening up rapidly," he says. "The industry as a whole is in the process of crossing [business author and lecturer] Geoffrey Moore’s chasm, and the mainstream of American business is about to get heavily involved in data warehousing. The AS/400 is right at the heart of the mainstream."

The First AS/400 Data Mart

While Dimensional Insight is a newcomer to AS/400 data warehousing, Silvon Software (Westmont, Ill.) has been at it for a while. Silvon, which derives its name from Silverlake, the original code name for the AS/400, takes credit for building the first AS/400 data mart in 1990. It built a data mart for agriculture product manufacturer Sandoz Crop Protection (now a part of Novartis) for sales analysis purposes and marketed the software it used for the project as SalesTracker.

"There were no proven tools in the marketplace in those days for sales analysis," says John Hughes, senior VP of sales and marketing at Silvon. "Business management tools vendors didn’t focus on that, they just focused on building order entry and MRP systems.

"Companies didn’t know how the product was sold or what went into it. There was so much data missing in the classic reporting environment. Companies wanted a multi-dimensional view of key business transactions and they couldn’t get it with conventional databases so we built an integrated database and pieced together their data so they could drill in and get quick reports and analysis."

SalesTracker has since evolved to DataTracker, which builds what Silvon calls Application Data Marts. The company has more than 900 AS/400 customers. Hughes says the way these companies use the technology has changed since the early days. "In the past, we just targeted the sales analysis people. Today, every key business manager touches DataTracker. A larger part of the company is involved. That’s a really big change," he notes.

Hughes says DataTracker is designed for "industrial supply chain companies," such as those in manufacturing, distribution and retail, with annual revenues from $50 million up to the Fortune 500. While the company has also developed Windows NT data marts for the past three years, more than 90 percent of its customer base is still on the AS/400.

Included among those is what Hughes says is the largest AS/400 data mart at auto parts retailer Parts America. The data mart is 310 GB, supporting 100 users and covering 640 retail stores nationwide.

"It’s hard to outgrow one of those things. It’s a scaleable, single subject environment. And you can put other logical data marts inside of one physical one. We’ve yet to run into anybody who owns our second-generation product who feels like they’ve outgrown it," he explains.

The next frontier for Silvon is delivering analytical applications on top of its data marts in such areas as balanced scorecard, forecasting and planning, Hughes says. Silvon, which was founded in 1987 as a change management software vendor, recently sold that business and is now solely focused on data warehousing solutions.

Hughes adds that he expects more customers to take advantage of data warehousing on the AS/400 since the price point of AS/400 hardware has come down. "I don’t know what the market saturation level is, but I think it’s a fairly substantial portion that’s still untapped. In the AS/400 space, I’d say the market is 25 percent penetrated. There’s a good 75 percent left."

Olympic Amis Ltd., traces its roots even farther back than Silvon. The Dublin, Ireland-based company developed a System/34 modeling tool called Insight more than 20 years ago.

"Insight was a revolutionary product in the System/34 world," explains John Schoenherr, the Chicago-based director of North American operations at Olympic Amis. "For the first time, S/34 users could draw data from their transactional systems and build models of their business. Insight provided the ability to create budgets and compare the numbers online, and had a report writer that was used to develop management reports."

By the late 1980s, Olympic Amis released its Amis product, the forerunner of today’s Amis V8 data warehousing solution. Schoenherr claims Amis as the first multi-dimensional database on the AS/400.

In 1995 the company released its first complete data warehousing solution with the addition of Quester, a relational data store manager. Amis sits on top of the data store and is fully integrated with it to allow users to analyze their data in a multi-dimensional format and drill down into the transactional data in the data store on demand. This complete product is called Amis V8.

Schoenherr says Amis V8 goes beyond the original Insight product by allowing customers to analyze their information down to the product and customer level rather than just generating static accounting reports. "I think the major change in the information technology business has been the need by our customers to work with detail data intimately," he says.

Today, Olympic Amis has about 550 users on Amis, about 50 of which use the complete Amis V8 suite. Customers tend to be mid-size, like most AS/400 customers. Schoenherr says Amis V8 can be applied to any industry, though the company has focused on the insurance industry, developing a service business to help insurance companies improve their reporting applications. He says adds focus will sharpen later this year when Olympic Amis rolls out an insurance-specific data warehouse application. It plans to follow with data warehousing applications for the banking and manufacturing industries.

That focus is designed to take advantage of the tremendous growth Schoenherr says he foresees in AS/400 data warehousing. He believes all companies now using general ledger reporting applications will eventually replace them with data warehouse applications.

Entering the Mainstream

"We have passed the innovators and the early adopters phase of the market and are about to enter the mainstream market of buyers. To take advantage of this market, we need a product that’s better defined. That is why we are developing our insurance application," Schoenherr explains.

ShowCase Corp. is another veteran player in AS/400 business intelligence. The Rochester, Minn.-based company was founded in 1989 and delivered a Windows-based DB2/400 query tool called Vista early in 1991. "It was really the first tool to show off the AS/400 in a graphical way," says Ken Holec, president and CEO of ShowCase Corp.

Over the next five years, the company continued to release new versions of Vista and other products for AS/400 data access and even dabbled in Microsoft SQL Server with the since discontinued Data Distributor product.

"We spent a long time thinking about what we wanted to be when we grew up," says Holec. "We decided to focus exclusively on the AS/400 and focus on all the components of bringing data warehousing to the AS/400."

The company was effectively reborn in 1996, he adds, when it licensed the Arbor Essbase OLAP server, porting it from Unix and NT to the AS/400. The ShowCase Strategy 1.0 suite would follow a month later in September of that year, giving ShowCase an end-to-end native data warehousing solution on the AS/400.

"Arbor Essbase changed the company," says Holec. "Vista was used by IT people and power users, now Strategy was being used by hundreds of users at a single site and they weren’t technical people, they were business people – store managers and merchandisers."

Strategy, which came out in a 2.0 version earlier this year, now boasts about 200 customers of the nearly 3,000 customers ShowCase has company-wide, Holec says. About 70 percent of that customer base are supply chain organizations.

"We cover several industries – manufacturing, distribution, retail, consumer products companies. We have customers in just about every sector – banking, insurance, service companies, hospitals," he says.

Strategy is an all-in-one solution that can build enterprise-wide data warehouses as well as data marts.

"In AS/400 shops, people want to buy solutions. There’s a heavy emphasis on putting the technology together into one package," Holec points out.

Like Silvon and Olympic Amis, ShowCase is also moving into analytic applications, extending Strategy through partnerships with companies such as Dimension Data and Walker Interactive (both financial management and budgeting), Total Solutions Group (Domino Integration) and Fiserve (banking and risk management).

Holec says the AS/400 data warehousing market is "massive" and estimates that the appetite for AS/400 data warehousing is about $1 billion. "There aren’t enough sales and services people and IT staff to get everyone aboard who says they want to do data warehousing," he observes.

Perhaps the most advanced data warehousing technology in the AS/400 world today is Coglin Mill’s Rodin. The Australian-based company has its North American headquarters at IBM’s Rochester, Minn. AS/400 development center and has worked closely with IBM at its Rochester Teraplex Center, stretching the limits of AS/400 data warehousing.

Coglin Mill specializes in building "industrial strength" enterprise-wide data warehouses on the AS/400 and claims to be the only vendor that can do that. A source at IBM backs up the claim.

"If you have a big need for transformation and cleansing or are doing a lot of high volume transactional stuff, you need Rodin," says the IBM source.

Rodin has about 50 customers of all sizes. "True data warehousing function and need is not limited to a certain size of customer," says J.J. Morrow, Coglin Mill’s VP of sales and marketing. "Hardware technology and software function now can deliver true enterprise warehouses running native on the AS/400 that are more than competitive from a cost perspective. The true secret is for a customer to do the right level of definition up front and have the right level of executive commitment to avoid project slow downs."

Rodin doesn’t specialize in any particular industry either. "We serve all industries that are interested in shifting from myopic product-only focus to strategic and completely new customer-driven business objectives," he says.

"Those who compete with an enterprise solution rather than a departmental solution will win out in the marketplace of their customers."

While Rodin is the premier choice for cleansing, building and management of data warehouses, it does not include a query tool. Coglin Mill partners with Venture SystemSource to integrate SystemSource’s Velocity decision support suite with Rodin.

Powered by Pilot Software’s OLAP engine, which SystemSource ported to the AS/400, Velocity provides access to information on business indicators, trends, and financial performance. The suite delivers answers on demand, allowing executives, managers, and analysts to interactively access and analyze information about their markets, products and customers from Rodin.

Wulf says all of these data warehousing solutions can bring value to AS/400 shops. "Each vendor is concentrated on performance and scalability. There’s more emphasis on making it much easier to implement and install the applications so that the customers can see the value of data warehousing in a short amount of time with a small staff."

"That falls under the AS/400’s value proposition in the marketplace," he concludes.