Working Smart With Application Integration

• 08/16/1999

Work smart, not hard, and avoid the nightmare of EAI spaghetti code and management woes.

These days, one of the hottest concepts in IT is Enterprise Resource Planning (ERP). Part of its allure is that it promises that implementing an ERP package will solve any and all enterprise integration problems. Once the ERP package is up and running, all of those old applications will go away, and life will be wonderful-goes the thinking. Unfortunately, this is a myth.

One of the primary reasons for implementing an ERP package is the tight integration between application components that ERP packages enable. As long as the ERP package is the only enterprise application being used, that's fine. However, while there may be IT shops where an ERP package can replace all of the applications that are currently being used, this is typically the exception, not the rule. There are many cases where a single enterprise must run not only multiple ERP packages, but "legacy" applications, as well. In this case, the benefits for installing ERP in the first place are negated, and problems that didn't even exist previously can result.

Problems often arise because ERP packages don't always make it possible to decommission the custom applications that have been written over the past thirty years. If the ERP system doesn't provide all of the functionality that the "legacy" application does, you've got to keep the "legacy" application in operation.

So now we have a world where "legacy" applications sit next to multiple ERP systems, and if you don't do anything special, none of the applications-ERP or otherwise-can talk to each other. In other words, you are no better off, from an application integration standpoint, than you were before you started your ERP implementation project.

This is exactly what prompted the rise of a new area of application development, Enterprise Application Integration (EAI). At its most basic level, EAI allows different applications, custom or packaged, to speak to each other to streamline business processes. Although this inter-application communication is relatively easy to build for applications that run on the same system, some applications that need to communicate may not run on the same physical system, or even the same operating system. Obviously this means that some method of machine-to-machine and OS-to-OS communication must be established.

This dialog between different applications running on different operating systems can be accomplished in many ways, but essentially breaks down into two broad categories: Functional Integration and Data Integration.

Functional integration enables an action or event in one application to cause an action in another application. An example of functional integration would be an entry in a human resource application indicating that an employee has received a raise, triggering an entry in the payroll application to make the raise appear in the employee's next paycheck.

Data integration allows data from different applications, possibly on different physical platforms, to be seen and accessed as one logical view. Data integration can also encompass the automatic replication of data between applications.

For the purposes of this discussion, the focus is on functional integration. To start with, we'll take a look at the infrastructure required to support functional integration.

The Plumbing
A common protocol or API (Application Programming Interface) must be defined to enable the integration of disparate applications. Applications can be integrated using several different methods, the most common being message-oriented middleware (MOM) and publish-and-subscribe systems. Message-oriented middleware is software that establishes a simple API for inter-application communication through direct connections, or message queues. Publish-and-subscribe systems allow an application to "publish" messages. Other applications can then "subscribe" to these various message types. Both of these methods can be used to enable inter-application communication, although MOM has been around longer and is used more frequently.

MOM was developed in the early '90s to enable inter-application communication, with the emphasis being communication between applications that were being built from the ground up, not on communication between existing custom, or packaged, applications. As time passed, enterprises that used MOM came to the conclusion that it was equally well suited to being used as the infrastructure to tie all of their applications together.

Unfortunately, little thought was given to the connections between the applications and how to efficiently manage the flow of messages. What resulted from poorly thought out integration projects was spaghetti code that had to be updated every time a new connection between applications needed to be made.

As EAI evolved, it became clear that it was not efficient to integrate applications by brute force using MOM for point-to-point connections between applications. Clearly, some method that built on the solid base of MOM that would allow multiple application connections without creating spaghetti code, was needed.

Work Hard or Work Smart?
We have all heard the adage "It is better to work smart than to work hard" and this adage applies especially well to the subject of EAI. Your first instinct will be to work hard, not smart, but you should resist the temptation to plow ahead, and think long and hard about an architecture for all of your enterprise integration needs.

Assuming you have resisted the urge to create a module for each application-to-application connection, what can be done to efficiently manage all of these connections?

The solution to this problem can be found in the now ubiquitous Internet infrastructure. Most people who surf the Web have no idea how their requests for web pages get from the PC to the web site and back. Those who have worked with internetworks know the answer lies with routers, and this is exactly how the EAI spaghetti code problem is being solved.

On an internetwork, requests do not flow directly from system to system, in a logical sense, unless the destination system is directly connected to the same network as the source system. In any internetwork, a system knows how to address any other system on the local network as well as how to access the address of a router that knows where to send the packets destined for non-local networks. All of these routers have routing tables that are used to route a packet through successive routers until the packet reaches its final destination. The elegant part of this solution is that each system only needs to know where to send packets destined for non-local networks.

The exact same concept can be applied to EAI, and has been, with the creation of message routers. Message routers work in exactly the same manner as internetwork routers, except they pass inter-application messages, instead of network packets. Because the message router only maintains a table of the addresses of directly connected message routers, the complexity of EAI projects that use message routers is drastically reduced.

It is important to understand how quickly the complexity of an application integration project can grow without the use of message routers. You can apply the formula for determining the number of unique pairs of connections in a connected graph, n!/2(n-2)!, to this problem. For example, to connect three applications together you would have to program only three connections, but doubling the number of applications that need to be connected to six, increases the number of connections to 15. One can see how adding even a few additional applications can make this a programming and management nightmare.

Integration Architecture
It definitely pays in the long run to carefully plan how your applications will be integrated, now, and in the future. When this planning is done, you will have an "Integration Architecture" to guide your development efforts.

There are many things to consider when developing this architecture. WinterGreen Research, a Lexington, Mass. based market research firm that concentrates on EAI outlines the following as design concerns for developing an integration architecture: flexibility; change; encapsulation; scalability; distributed computing; focus on business problems; productivity; integration of resources; management of systems.

Chief among the preceding concerns is a focus on business problems. An architecture that is "neat" from a technological perspective, but does not solve the original business problems is not the answer. This is what gave rise to the need for an EAI solution in the first place.

Another important factor to note is that EAI projects, by their very nature, run in distributed computing environments. Therefore, when designing an integration architecture, remember that a new application could be added that doesn't run on current hardware or software, or could be connected in a way that doesn't exist today. Obviously, you can not anticipate everything that might happen in the future, but careful thought at this stage will save great pain later on. Going hand in hand with this is the concern for flexibility. Any flexibility that is built into the architecture will pay for itself many times over later in the implementation.

Finally, you must not forget to develop an architecture that can be managed easily. Remember that there will be many pieces to the EAI puzzle filled in later and that somehow you must manage all of these pieces when the project goes live.

The Hub of the Universe
Once you have designed your integration architecture, it's time to start implementation. Since every enterprise's integration architecture will be different, we can only speak about the most common pieces that are likely to make up an EAI implementation.

The central part of most EAI projects will be the "integration broker" or message broker. According to the Gartner Group, a San Jose, CA based consulting firm, the most effective approach to application integration involves the use of an integration broker. Their research shows that the use of an integration broker can reduce the cost of an EAI project by one third in the initial stages of a project. Later on, when changes to an application can have a domino effect on many interfaces, the use of an integration broker can reduce costs by two-thirds, claims Gartner.

When shopping for an integration broker, there are a few things to you should look for according to Gartner:

• Scalability
• Team development capabilities including version control, source code management, and check-in/check-out
• Ability to handle batch and real-time applications, as well as, client/server and mainframe applications.
• Low to moderate learning curve which reduces the personnel costs of the application integration project.

Further, Gartner indicates that an integration broker vendor should have strong service and support capabilities, as well as a solid reputation. As integration projects are continuing processes, the reputation of the vendor-and its ability to support the software in the long run-are of paramount importance.

A Framework for EAI Success
Although the message router is the central part of any EAI project, there are other pieces that fit into the puzzle to form the complete EAI solution. Rules engines, application adapters, message formatters, and transformation engines round out the EAI technology puzzle.

The above pieces fit together with the message router and message transport to form a framework for application integration. "No one has it all," says Susan Eustis, Principal of WinterGreen Research, referring to EAI frameworks. Therefore, it is important that any vendor that provides a framework for application integration be flexible enough to play with other vendors in a best-of-breed EAI solution.

If you believe WinterGreen Research, which reports that the overall market for EAI products will grow from almost $4 billion in 1999 to nearly $13 billion by 2003, you're likely to be involved in some sort of application integration project in the near future. So, if you haven't already done so, now is a good time to familiarize yourself with the options available. For quick reference, we've included a list of company's providing EAI products and services along with this article. For more information about the specific EAI products and vendors followed closely by Wintergreen Research, check out their Web site.