Simplifying Integration
Organizations want IT departments to do more than deploy an application that works. They want interoperability among all applications. They especially want interoperability when they roll out large packaged solutions from companies such as SAP, PeopleSoft, Oracle Financials and Baan.
But interoperability doesn't come cheap. In a report on the application integration challenge, Forrester Research (www.forrester.com), notes that process-based integration for supply chain management and customer management applications can run from $5 million to $8 million. High cost combined with the technical difficulties of integration is fueling the growth of both a professional services market and a tools market aimed at making integration easier.
Interoperability, however, needn't be a multimillion-dollar endeavor. Depending upon the amount of integration and the platforms involved, interoperability can be as simple and inexpensive as accessing SQL data captured in a packaged application using a graphical desktop tool and some data access middleware. For example, IT can use packaged data access solutions such as HeadStart Solutions from Cognos (www.cognos.com) or SnapPacks from Information Builders (www.informationbuilders.com). Both offer preconfigured access to leading package applications from SAP (www.sap.com), Baan (www.baan.com), and PeopleSoft (www.peoplesoft.com). On the high end, interoperability can be as complicated and expensive as trying to bidirectionally pass transactions in real-time between different packaged applications running on different platforms.
Published GartnerGroup studies suggest that 40 percent of IT work today is focused on integration. In fact, at many corporate IT organizations application integration has become the primary form of application development. Rather than build custom applications, many organizations purchase packaged applications and integrate them, maybe adding some enhancements and customizations in the process. "The packaged application is the starting point," says Larry Ferrere, vice president, Vastera Inc. (www.vastera.com), a packaged application vendor.
IT shops have numerous options when it comes to integration with packaged applications. Which option an organization chooses "is a question of tradeoffs," says Nick Manha, vice president of professional services at the Taylor Group (www.taylornet.com), a firm specializing in integrating packaged applications in the Windows NT environment.
The biggest tradeoff is how many of the business rules to apply, Manha explains. This may vary from validating that a phone number has the correct amount of digits to enforcing complex transaction rules. The more that has to be done with the data or the transaction, the more business rules will have to be applied and the more difficult integration will become.
Once an organization decides what to do and how many business rules have to be applied, it can then review its integration options. These options range from using the integration tools built into the packaged application to enterprise integration applications that claim shrink-wrapped integration with popular packaged applications. In between are integration approaches based on read-only data access, asynchronous messaging, scripting, application servers, and object interoperability. Middleware is behind most of these approaches in one form or another. These approaches also require different programming skills.
To start, an organization can choose any integration tools provided by or supported by the initial packaged application. Products from Great Plains Software Inc. (www.gps.com), for example, are "built upon the Dexterity 4GL," Manha says. While he advises against trying to modify source code, programmers can use 4GL to create add-on modules that smooth integration or deliver enhanced functionality.
Popular midrange ERP packages from QAD (www.qad.com) and Symix Systems Inc. (www.symix.com) are built on the Progress Software’s (www.progress.com) database and the Progress 4GL. As with Great Plains, IT developers using the Progress 4GL can integrate other applications with these packages. Progress runs on Windows NT. SAP, on the other hand, uses ABAP, a proprietary programming language. Integration via ABAP programming requires highly skilled ABAP programmers.
Increasingly, ERP vendors are creating application programming interfaces (APIs) to facilitate integration without touching underlying code. SAP, for example, boasts over 1,000 business APIs (BAPIs) in its latest release. But even with APIs or BAPIs, integration is not simple. Significant programming is required at both ends of the integration. Frequently, IT organizations look to use middleware tools or packaged adapters in conjunction with ERP vendor APIs.
Data Access
If an organization wants to access data in a packaged application, it can turn to SQL access tools. Using Information Builder's SnapPacks or Cognos' HeadStarts in conjunction with Cognos’ Impromptu and PowerPlay tools, for example, the IT group can quickly set up ad hoc query capabilities that allow end users to access information within an SAP or another large, complex packaged application. These tools provide knowledge of the packaged application and its data structure, which allows programmers, and even users without expertise in the packaged applications, to quickly and easily set up queries and get meaningful results.
The next level of data access is extracting data from a packaged solution and moving it to a staging area -- a data mart or data warehouse where it can be combined with other organizational data for detailed analysis, OLAP or data mining. SAP takes this approach with its Business Information Warehouse. A number of vendors, including Information Builders, offer tools that automate the creation of extensive data warehouses involving data from a packaged application and other applications.
Organizations, however, want to get beyond accessing data for decision support purposes only. They want real-time exchange of data and transactions between an ERP solution and other applications. In effect, they want to call from one application to another and pass control back and forth. This is a greater challenge.
At the simplest level, organizations can try scripting. Scripting languages like Tcl can connect almost anything to anything else, says John Ousterhout CEO of Scriptics Corp. (www.scriptics.com). Tcl scripts are interpreted, string-oriented code that can hook into APIs published by ERP vendors. But the programmer must know all the necessary mappings for the applications being integrated.
But low-level scripting and programming are too slow and difficult for the kind of tight, deep integration many organizations require. To achieve this level, companies must turn to asynchronous messaging middleware, object middleware, off-the-shelf application integration adapters, and the Web.
Messaging
Asynchronous messaging is a communications method for delivering information between discrete systems. Messaging tools, such as IBM Corp.'s MQSeries, offer guaranteed delivery -- the message is guaranteed to arrive once and only once. Most higher-level middleware, object brokers and integration applications incorporate messaging.
While messaging is effective for navigating dissimilar systems, it does not produce tightly coupled integration. Messaging systems are oblivious to the content of the message and don't do anything with a message other than drop it at its destination. "Message passing doesn't solve the integration problem. It solves a communications problem," says Reuven Koblick, deputy director at Mitsubishi Electric Research Lab (www.mitsubishi.com). Messaging must be combined with other middleware or programming. IBM, for example, had to bolster its MQSeries with MQ Integrator, which understands SAP's IDOCs, a file exchange format, and can provide some integration.
Many vendors combine messaging and other communications middleware with preconfigured adapters or connectors for packaged applications. These adapters take advantage of published APIs to offer a more transparent integration with the packaged application. Active Software (www.activesoftware.com), Oberon Software (www.oberon.com) and Crossworlds Software Inc. (www.crossworlds.com) offer adapters in conjunction with other tools, such as rules engines or workflow capabilities, to provide nearly seamless integration.
These tools, however, can be expensive. They also lock you into a set of proprietary adapters are maintained by the middleware vendor, says Dave Kelly, vice president at Hurwitz Group (www.hurwitz.com), a research and consulting firm. When packaged applications are upgraded, the middleware companies have to scramble to upgrade their adapters. This has the potential to overwhelm a small or young company.
Object-based components may offer a practical approach to integration for organizations committed to component-based applications. Most packaged applications are adopting component-based architectures, although their component models tend to be proprietary rather than based on widely accepted standards, such as COM, CORBA or enterprise Java Beans (EJB). The idea here is that through the use of standard components and component/object broker middleware, organizations can integrate applications at both the function logic and data levels. The components can call into each other and pass the results of processes and data between applications at the component level.
For the component-based approach to work, however, organizations need to implement bridges to cross the different object models. COM-CORBA bridges already exist. Visual Edge Software Ltd. (www.visualedge.com) has a CORBA to SAP bridge that also supports EJB via IIOP and CORBA, reports Ken Neff, Visual Edge's director of professional services. The company will soon offer a bridge to PeopleSoft applications. It will introduce bridges to other packaged applications as demand warrants.
Finally, organizations are turning to the Internet as a way to integrate packaged applications. The Web provides the communications and access infrastructure. Middleware takes the form of Web and application servers. Java servlets and COM, EJB, and CORBA objects provide services. Some additional custom programming will likely be required.
Clearly the Web is not a simple solution either. ERP packages are beginning to provide Web hooks, but the packaged application may not be able to absorb Web traffic volumes and should be buffered through an intervening staging area, advises Mike Price, chief technology officer at Alta Software (www.altasoft.com).
There is no ideal way to integrate applications. Each choice -- even a decision to outsource to an integration services firm -- involves balancing cost, time, programming skill, functionality, maintainability and transparency. Emerging technologies such as XML promise to simplify some aspects of integration, but there is no simple solution. As long as ERP packages remain complex, proprietary solutions, integration will remain a challenge.