In-Depth

Keeping in the Middle of Things: Using Middleware to Integrate External Resources

• By David S. Linthicum
• 11/01/2000

Enterprise Application Integration (EAI) is the solution to the unanticipated outcome of generations of development undertaken without a central vision or strategy. Systems have been built that have served a single purpose for a single set of users without sufficient thought to integrating these systems into larger systems and multiple applications.

While the technology has aged, the value of the applications to your enterprise likely remains fresh. Indeed, that "ancient" technology has probably remained critical to the workings of your enterprise.

Unfortunately, many of these business-critical systems are difficult to adapt to allow them to communicate and share information with other more advanced systems. While there always exists the option of replacing these older systems, the cost of doing so is generally prohibitive.

If EAI articulates the problem, then traditional middleware has sought to articulate the solution. Traditional middleware addresses the EAI problem in a limited manner.

The primary limitation is that middleware that uses message queuing or remote procedure calls (RPCs) only provides point-to-point solutions -- linkage between system A and system B. Unfortunately, any attempt to link additional systems quickly becomes a complex tangle of middleware links. Worse still, traditional middleware demands significant alterations to the source and target systems, embedding the middleware layer into the application or data store.

An additional complication to this scenario is that IT managers must perform integration projects inside fluid environments using rapidly advancing technology. In seeking to integrate links, the manager may also encounter additional problems, such as a variety of legacy systems that contain mission-critical applications; several packaged applications with both proprietary and open frameworks; a hodgepodge of hardware and operating system platforms; a medley of communication protocols and networking equipment; and geographically dispersed applications and databases. In addition to these structural limitations, the economics of traditional middleware has placed EAI out of reach for most IT organizations.

Given these significant limitations, it follows that EAI represents a very different method of application integration than using traditional middleware. EAI provides a set of integration-level application semantics, and creates a common way for both business processes and data to speak to one another across applications. More importantly, we approach this old problem with a new set of technologies designed specifically for EAI.

Keeping this in mind, we can focus on the differences between traditional approaches and the vision of EAI:

•EAI focuses on the integration of both business-level processes and data, whereas the traditional middleware approach is data-oriented.

• EAI includes the notion of reuse, as well as distribution of business processes and data.

• EAI allows users who understand very little about the details of the applications to integrate the applications.

First, middleware is not a magic bullet. What it offers is, quite simply, a mechanism that allows one entity (application or database) to communicate with another entity, or entities. In other words, middleware is any type of software that facilitates communications between two or more software systems. That’s it.

If middleware is nothing more than just another tool, why use it? The answer is as simple as middleware itself: It is the only solution that allows applications to communicate with one another. Middleware is able to hide the complexities of the source and target systems, freeing developers from focusing on low-level APIs and network protocols, allowing them to concentrate on sharing information.

Because of middleware’s importance in the sharing of information, its importance to the EAI solution is growing more and more evident. Although it was a tool for moving information between systems within a single enterprise, we now look to middleware to allow us to move information between multiple enterprises. This new demand on middleware presents vendors with a significant challenge, since middleware products were conceived for, and built exclusively for, intra-enterprise integration.

Middleware can work in a point-to-point, as well as many-to-many (including one-to-many) configuration. Each has its advantages and disadvantages.

Point-to-point middleware uses a simple pipe to allow one application to link to one other application -- application A links to application B. When application A seeks to communicate with application B, it simply "shouts down the pipe" by using a procedure call or message.

When compared to other types of middleware, point-to-point middleware is limited by its inability to properly bind more than two applications. It also lacks any facility for middle-tier processing, such as the ability to house application logic, or to change messages as they flow through the pipe.

The idea behind the point-to-point model should make it clear that it does not represent an effective EAI solution, because most problem domains require linking many applications. Perhaps, more to the point, in order to share information in this scenario, applications must be linked, and information must be brokered through a shared, centralized server. In other words, sharing information requires a message broker or transactional middleware.

As its name implies, many-to-many middleware links many applications to many other applications. This capability makes it the best option for EAI. Being the best option makes it the obvious trend in middleware. In addition to this capacity to link many to many, it is the most powerful logical middleware model, because it provides both flexibility and applicability to the EAI problem domain.

While the advantage of the point-to-point model is its simplicity, the disadvantage of the model is its complexity.

Asynchronous middleware moves information between one, or many applications in an asynchronous mode -- that is, the middleware software is able to decouple itself from the source or target applications. The applications are not dependent on other connected applications for processing. The process that allows this to occur has the application(s) placing a message in a queue and then going about its business, waiting for the response at some later time from the other application(s).

The primary advantage of the asynchronous model is that the middleware does not block the application for processing. Moreover, because the middleware is decoupled from the application, the application can always continue processing, regardless of the state of the other applications.

In contrast, synchronous middleware is tightly coupled to applications. In turn, the applications are dependent on the middleware to process one or more function calls at a remote application. As a result, the calling application must halt processing in order to wait for the remote application to respond. We call this a "blocking" type of middleware.

The disadvantage of the synchronous model is the coupling of the application to the middleware and the remote application. Because the application is dependent on the middleware, problems with middleware -- such as network or remote server problems -- stop the application from processing. In addition, synchronous middleware eats up bandwidth, because several calls must be made across the network in order to support a synchronous function call. This disadvantage, and its implications, makes it clear that the asynchronous model is the better EAI solution.

The evolution of middleware is changing many of the identifying features that had once made categorizing middleware a straightforward task. However, even as we acknowledge the difficulty in categorizing middleware, we note that several types of middleware continue to solve particular types of problems. For the purposes of our EAI discussion, we will describe RPCs, MOM, distributed objects, database-oriented middleware, transactional middleware (including TP monitors and application servers) and message brokers.

RPCs. RPCs are the oldest type of middleware. They are also the easiest to understand and use. They provide developers with the ability to invoke a function within one program, and have that function execute within another program on a remote machine. To the developer, the function is executing locally. The fact that it is actually being carried out on a remote computer is hidden.

RPCs are synchronous. In order to carry out a remote procedure call, they stop the execution of the program. This quality is what defines RPCs as "blocking middleware." They also require more bandwidth than other types of middleware products because carrying out a remote procedure call requires so much "overhead."

Message-Oriented. MOM was created to address some of the shortcomings of RPCs by the use of messaging. Traditional MOM is queuing software, using messages, byte-sized units of information that move between applications, as a mechanism to move information from point to point.

Because MOM uses the notion of messages to communicate between applications, direct coupling with the middleware mechanism and the application is not required. MOM products rely on an asynchronous paradigm.

The asynchronous model allows the application to function independently -- that is, to continue processing after making a middleware service request. The message is dispatched to a queue manager, which ascertains that the message is delivered to its final destination. Messages returning to the calling application are handled when the calling application finds the time.

Unlike the synchronous paradigm, the asynchronous paradigm does not block the application from processing. Although this model is more complex than the synchronous model, it is more convenient for developers and users. In addition, MOM can ensure delivery of a message from one application to the next for several sophisticated mechanisms, such as message persistence.

Distributed Objects. Distributed objects are classified as middleware, because they facilitate interapplication communications. However, they are also mechanisms for application development, providing enabling technology for the enterprise, or trading communitywide method sharing. In fact, distributed objects are really small application programs that utilize standard interfaces and protocols to communicate with one another.

Database-Oriented. Database-oriented middleware is any middleware that facilitates communications with a database, whether from an application, or between databases. Developers typically use database-oriented middleware as a mechanism to extract information from either local or remote databases.

Transaction-Oriented. Transactional middleware, such as TP monitors, and application servers, coordinates information movement and method sharing between many different resources. However, although the transactional paradigm this middleware employs provides an excellent mechanism for method sharing, it is not as effective at simple information sharing -- the primary goal of EAI.

TP monitors (and application servers) are based on the concept of a transaction -- a unit of work with a beginning and an end. The reasoning here is that if the application logic is encapsulated within a transaction, then the transaction either completes, or is rolled back completely. If the transaction has been updating remote resources, such as databases and queues, then they too will be rolled back if a problem occurs.

Message Brokers. Message brokers represent the nirvana of EAI-enabled middleware. (At least, their potential does.) Message brokers can facilitate information movement between two or more resources (source or target applications), and can account for differences in application semantics and platforms. As such, they are a perfect match for EAI.

Message brokers can also join many applications by using common rules and routing engines. They can transform the schema and content of the information as it flows between various applications and databases. The importance of message brokers is a function of their place within the application integration problem domain. In general, message brokers are not an application development-enabling technology. Instead, they are a technology that allows many applications to communicate with one another -- without any application necessarily understanding anything about the other applications it is sharing information with.