In-Depth

EAI Middleware: Heterogeneous Computing Comes of Age

• 06/01/2000

Middleware is about 10 years old as a defined software market segment. Communication and database middleware products have been offered for some time now as a means to drive complexity out of projects where you need to integrate heterogeneous software and data resources. RPCs, MOM, queuing, naming, security and federating APIs for transport services or superset APIs for SQL-capable databases are all well known and widely deployed now.

Transaction managers, ORBs and screen scrapers of various sorts are used by nearly every organization. Yet, year after year, estimates have been that "next year" would be the big breakout year for middleware. It didn’t happen. Middleware has remained a mystery, elusively defined as "just about everything in software that could not otherwise be defined in concrete terms."

Suddenly, a new class of software emerged: Enterprise Application Integration (EAI) software. The move from older middleware to newer middleware is a result of the adoption of enterprise application ERP and CRM products. With adoption of those packages came the need to make them talk to each other. Packages, unlike in-house developed applications, are harder to integrate because you didn’t write the code. MOM-based middleware vendors came into the market with "adapters," based on technical and business relationships established with the package vendors. These products are often called Message Brokers or Integration Brokers. The initial objective of these middleware products was to provide for interoperability among commercial application products, and secondarily to include in-house developed applications.

Shortly after the initial interest in package integration came the Web commerce wave. New applications, needing to pull together information from applications and databases in the back office for front office Web users, added requirements for user interfaces, transactionality and realtime scalability. This produced a second breed of EAI software, called application servers, for the practice of "Web EAI."

Today, all business is e-business. The Internet means fundamental change. Supply chains are reconstituting at regular, though unpredictable, intervals. Old market channels are slowly disappearing and new marketplaces are being born. Organizations must act on initiative to survive, and be nimble enough to react in short timeframes to new market conditions, and under trying circumstances. The business models of the past are fading rapidly, but nobody really knows yet the fundamentals for the new economy. It’s under these somewhat chaotic conditions that organizations are turning to business architectures that can be independent of technical implementations. The goals are to maintain adaptable and unique processes that enlist methods already proven, and to discard those that don’t work.

EAI products have changed from tactical middleware to e-business enabling platforms faster than many realized. Application servers, message brokers and assorted integration engines now occupy the leading role in the center of a multitier topology. Most EAI products offer interfaces to back office systems, and a few offer interfaces to front office products. Expect more front office interfaces to become available from EAI vendors as IT ramps up for CRM projects.

For the most part, front office products were not designed to work with many back office systems. Back office systems were designed for intra-enterprise operation. In the new e-business economy, all systems need to work together. This is the space where EAI technologies make a difference. In a business-to-business scenario, the business architecture needs to remain stable, while the underlying technologies must enable cross-enterprise applications.

Everyone is deploying multitier architectures for new applications. Middleware is the key, and most EAI platforms today contain a large degree of traditional middleware. Middleware provides flexibility and scalability. Host-centric architectures are scalable, but not particularly flexible. Client-server is flexible, but not as scalable. Multitier architectures mix the best of each, and mitigates the respective weaknesses. Whereas TP Monitors, ORBs and MOM were the middleware leaders five years ago, today, application servers and integration brokers are the mindshare leaders for integration.

The intersection of applications within a business process is where organizations can derive new value using EAI. In a study of 1,000 organizations using EAI techniques, nearly 50 percent were integrating the back office with electronic trading partners, and about 45 percent were extending front-office applications out over the Web. Well over half were engaged in tying together front- and back-office applications within their enterprise. Companies aiming for interoperability among applications start with messaging.

Those building new composite applications, as is usually the case with Web initiatives, start with application servers. It’s a fast moving marketplace for technology platforms, with CAGRs through 2004 predicted as high as 50 percent, and billion dollar market capitalization for very new product companies positioned in the Web EAI market.

From 1999 until 2004, middleware expenditures are expected to double from about $3 billion to about $6 billion. The most interesting datapoints within these forecasts revolve around the roles of traditional middleware technologies. Message-centric (i.e., message and integration brokers) products will double in popularity, as will application servers. Data-centric products become subsumed, losing more than half of the market share in integration projects.

This indicator about data-centric integration tools is in contrast with the integration going on between data warehouses and ERP/CRM packages. Evidently, this is primarily for strategic research and not for operational applications. For Web-based transactional systems, realtime data mining is essential to achieve personalization in customer interactions. This is attainable using techniques for partial object instantiation, memory-based messaging and replication that address potential performance inhibitors. Yet, this historical middleware capability seems to be more or less overlooked.

Middleware has traditionally been used for operational systems, and, as such, has emerged as the key enabler for multitier distributed systems. Messaging middleware has proven itself over the years as a flexible, reliable and, when required, very high-performing solution paradigm. Even the most conservative organizations have specified architectures that decouple the consumption, processing, distribution and storage functions of information systems. On the middle tier, coupling of service-oriented architectures takes place, often using messages. Commonly, you find application servers as hosts for composite applications, with message brokers extending the reach into the mainframe and IDL type interfaces to UNIX and NT.

Integration portals are increasingly being used to assist application servers as a performance cache and by federating back-office information into a unified service, allowing application servers to focus on the front end logic and looking to the integration portal as the proxy for everything in the back office. As these architectures become widely used in an organization, it has not escaped notice that a lot of application components are being developed and deployed, with a variety of different tools, that have to be managed in the context of integration.

A new style of application development, based on EAI and components, is emerging. Component-oriented thinking is certainly not new. Widely practiced in the Windows environment, and more recently in the Java environment, the benefits are well known. The issue is that in the heterogeneous B2B world, where you cannot mandate a singular software architecture, you need to be interoperable with services that change at regular and unpredictable intervals. This is a baseline requirement in the e-business age. New supply chain partners will be recruited, and IT must be able to minimally exchange messages with partner applications. Of greater value can be the development of a composite application using elements from applications on both sides of the firewall.

Multiple strategic product vendors co-exist today in an IT infrastructure. E-business implies that at least three to five EAI products will be present inside any organization. They will come into place in support of tactical projects. Perhaps as a prerequisite for a purchased application, due to the technical bias of project developers or by acquisition of a competitor, the result will be analogous to today’s DBMS reality. A cross model component architecture, without bias toward C, C++, Smalltalk, COM, CORBA, Java 2 or even COBOL can bind all of this together.

The dynamics of application development itself look much different in an EAI environment. Applications are assembled in days or weeks, but not built in weeks or months. Business process reengineering is a part of many projects. And systems are expected to be adaptable (i.e., when business policies and rules change, applications must change). This is happening within a very tight technical labor market. Skills premiums are a major issue. C/C++ and COBOL are still the leading languages for development of large-scale applications. EAI projects tend to aim for inclusion of applications with the Internet and Java/ application server platforms. By and large, Internet and Java/ application servers have brought object technologies into the mainstream. So, we’re looking at bridging two worlds – OO and non-OO, distributed and host-centric.

Application engineering environments for complex integration projects are starting to be seriously discussed in the mainstream (i.e., inclusive of Web-centered, as well as legacy environments). There are quite a few Java-centric application server products available and a mix of services for Windows server environments are broadly used. However, when you add requirements for messaging, brokering, complex and long-running transactions, support for legacy applications and process automation, then the list of available alternatives is shortened.

Hybrids of multiple-product technologies are sometimes the only course. In an e-business environment, it is hard to foresee a day when an all-Java decision or an all-DNA decision, or an all-"anything" makes sense. Services found traditionally in older middleware products, such as messaging, queuing and database normalization, will be needed to compliment the more recent application server-oriented offerings.

Frameworks, visual programming tools and code generators are on the short-term horizon as IT seeks to blend traditional development tools, modeling tools and middleware. People are starting to acknowledge that development is much more than just producing code. One must gather and manage requirements, design the solution, document, test and track bugs. You need to analyze the impact of changes. Now that e-business and EAI projects are bringing middleware to the forefront of discussion, and it is clear that heterogeneity is the only stable factor in the topology, the vendors are beginning to integrate middleware and development tools for application to bigger problems than simply Web-enabling database information.

With the advent of pervasive computing devices and Internet appliances, the heterogeneity of software will only continue to increase. Middleware and EAI products have addressed a small portion of the complexity so far. The technologies used today are falling by the wayside in favor of new ones. Middleware will always have a place in emerging landscapes where the old and the new need to work together.

The clear separation of business process from business function allows you to assemble application components in new composites. This must be achieved independently from integration style (e.g., messaging, brokering or using a portal), and the development tool choice. By decoupling the middleware, the process automation, the development and the deployment, it is possible to reassemble approaches and business logic and change techniques without incurring the cost of rewriting all the integration-related code in a system. It’s passé to insulate applications from databases, transport and operating systems. Applications also need to be insulated from integration platform choices whenever that is feasible. This it the most practical way to leverage the skills available.

EAI aligns business initiatives with IT in short timeframes. Technical diversity is the norm and will only increase. Rapid assembly of new applications is the only practical response. Middleware has proven itself as a platform for distributed computing and visual development environments have proven themselves effective at accelerating the application delivery process. The convergence of these two approaches will carry us into the next generation, where the reconstituted enterprise continually reinvents itself for e-business.

About the Author: Mark F. Creamer is a Director at Level 8 Systems (Cary, N.C.; www.level8.com), a global provider of e-business integration software.