Middleware

• 04/21/1999

In the early days of client/server computing, the term middleware was used to describe the technical glue that enabled a server presentation to be replicated on a client system. It was pretty dry stuff, the domain of internal developers and systems architects.

Over the past several years, middleware evolved to include a variety of technologies that support and integrate applications, sometimes across an enterprise. From low-level plumbing connections to analyst-oriented, GUI-based application integration, middleware today encompasses a broad, and growing, range of technologies that can help enterprise NT sites link disparate platforms, operating systems and applications.

Most middleware is transparent to end users, but its design and support is no longer the exclusive province of technical gurus. Today, popular middleware products require input from and the participation of NT managers, who must help determine where and how to deploy the technology.

"Middleware was developed as a bridge to enable distributed applications to be built and still have the service of legacy applications. Anyone who is building enterprise systems today almost has to look at a middleware solution," explains Joe Conron, director of development at NasTel Technologies Inc. (www.nastel.com).

Choosing a middleware solution, however, requires up-to-date information about the types of middleware available, the applications for which it is best suited and the pitfalls to avoid.

With the rising popularity and sales of middleware comes increased confusion about what different terms mean. Middleware traditionally included technology designed to tightly couple applications, such as ODBC, and object request brokers (ORBs), such as CORBA and DCOM. These early flavors of middleware were and still are appropriate for small applications that rely on request or reply operations.

With larger systems or those needing higher performance or asynchronous communications, messaging is the dominant solution. The publish or subscribe variety of messaging is appropriate for reproducing information on a one-to-many scale, such as distributing a stock quote to multiple applications.

The most popular and well-understood type of messaging is queuing, in which information is put into a message and written to a queue for later processing. Unlike ORBs and publish or subscribe, queued applications do not require the sender and receiver to be simultaneously active. The publishing portion of the application deposits the message into the queue, where it waits until the subscriber picks it up. The dominant message queuing product is IBM Corp.’s MQSeries, which had about $80 million in 1997 sales, according to Dataquest (www.dataquest.com), a subsidiary of GartnerGroup.

Another version of middleware is transaction processors, such as Microsoft Corp.’s Transaction Server, IBM’s CICS and BEA Systems’ Tuxedo. This type of middleware synchronizes multiple databases and provides real-time response and consistent updates to all databases.

Application servers are the type of middleware getting the most industry attention now. While the definition sometimes varies according to the vendor, application servers provide integration across the Web among applications that reside behind corporate firewalls. Although application server technology offers both integration and development functionality, it is typically considered a form of middleware, explains Kimberly Knickle, senior analyst at AMR Research Inc.(www.amrresearch.com).

Regardless of the definition, "Middleware always has an integration focus. There may be some development features or tools, but the primary focus of middleware is integration," she says.

At the top of the middleware family is a technology called business integration, also known as enterprise application integration (EAI).These types of products combine the data transport and transformation of middleware with additional intelligence, business process support and the adapters, connectors and gateways to handle integration interfaces.

A common EAI project is the integration of enterprise resource planning systems. Because this kind of project requires significant assistance from consultants, EAI is the most expensive and time-consuming of all middleware implementations.

Yet experts agree, the EAI market is the likely future of middleware. AMR estimates that EAI generated $450 million in sales last year and projects 50 percent annual growth for the future.

Most enterprise NT sites do not use middleware to pursue EAI, but to integrate discrete applications. In homogeneous NT environments, tools such as Microsoft Message Queue are popular, while heterogeneous NT environments are more likely to use non-Microsoft middleware, such as MQSeries.

The application of a middleware solution varies with the needs of the NT site. Vendors report middleware-enabled solutions are often used with new applications, such as Internet and Web connectivity or electronic commerce. But the majority of middleware is connected to older applications.

"We’re definitely seeing middleware more for enabling old technology to new technology," such as Web-based banking applications connected to legacy financial applications, says Barry Ader, director of middleware solutions at Candle Corp. (www.candle.com).

More NT sites are using middleware to connect old applications rather than to integrate new e-commerce programs, he adds. For example, the state of North Carolina is using Candle middleware to link data from various agencies. The middleware shares data about child support payments with the licensing agency, preventing the state from issuing driver’s licenses to non-paying parents.

Even if middleware becomes used for more exotic integration tasks, the massive investment in application software will ensure its primary purpose will remain as an integrator of existing applications. "The investment in application software is huge: $1 trillion. Companies must leverage that by integrating their existing applications with middleware," says Alfred Spector, general manager for IBM’s application integration middleware marketing.

Is using middleware to leverage these investments too difficult for the average NT manager? Trade press reports focus on the complexity of the solutions and the cost of hiring the required consultants.

"People should know that middleware is much easier to use now," Spector counters. Over the past two years, vendors invested an enormous amount of resources to simplify the usage of middleware. MQSeries, for example, is now designed to work with NT. The transport-level product has been supplemented with MQSeries Integrator, which automates business rules as well as middleware development and integration, Spector says.

The increasing maturity of the middleware market has encouraged systems management vendors to offer tools that help manage complex middleware implementations. Yet NT managers need to assess the ease of use of the middleware tools they consider buying.

"All applications go through a development phase and a deployment and maintenance phase. It’s the same for middleware," says Conron of NasTel Technologies. "You have to ask yourself: How easy is it to develop and maintain? When applications are deployed on top of the middleware, what are the administrative issues? How hard is it to deploy? Can components fail? How does it manifest failure?"

Management of middleware is also impacted by the complexity of integration, especially when homegrown applications are involved. With up to 90 percent of all enterprises using custom-developed programs, the need for additional middleware programming can be substantial.

"There is no way to build a general-purpose GUI to define interfaces for all systems," but NT sites can build simple application integration products quickly, explains Tom Laffey, founder and CTO for Talarian Corp. (www.talarian.com).

"It’s easy to learn and start with. But as the number of applications increases, the complexity increases exponentially," which is where scalability and performance issues become paramount, Laffey says.

Servicing customized collections of applications that are connected by middleware can be difficult. "One of the downsides of distributed computing is that there are lots of moving parts. When middleware works, it does so easily; but if there are problems, it can be hard to pinpoint them," warns John Senor, vice president and general manager of middleware technologies at Information Builders (www.informationbuilders.com).

Words to the Wise

Getting educated about middleware development before beginning a project can help many NT sites avoid common pitfalls. "Once customers have even a small success under their belt, it can be deployed very quickly. It’s a matter of doing it right up front," says Paul Roth, CTO at MessageQuest Inc.(www.messagequest.com).

Some managers attempt to cut costs, resulting in problems early on that compound as time passes. Hiring an expert to help, even if only for the first few weeks, is a wise investment, Roth says.

In addition, carefully consideration of all the middleware options is likely to produce a more efficient solution. Some managers opt for asynchronous communication when synchronous is more appropriate, or vice versa. Others purchase low-level middleware, thinking that less functionality will eliminate complexity. The problem with this approach is that it usually requires custom programming on each end of the middleware.

Another mistake is buying one type of middleware and expecting it to address every integration need. Where multiple types of middleware solutions are inevitable, some NT managers buy solutions from vendors who offer a variety of products, rather than those who specialize in just transport or messaging. This strategy can make it easier to buy complementary products later, as enterprise middleware needs evolve.

NT managers should also look for middleware products that match -- as closely as possible -- existing skill sets in the organization. "It can be hard to teach people new middleware technologies," says Anish Dhanda, president of NetNumina Solutions (www.netnumina.com). Investing in technology that matches existing applications can simplify the process.

Even when the right type of middleware is secured, NT managers must carefully prototype projects before building a full-scale solution. "People get fooled by building small applications, then the scalability just kills them. The more applications you add, the more complex it gets," Laffey says.

Carefully consider relevant scalability issues before expanding a prototype into production usage. "How much intelligence and processing can you put into one hub," AMR’s Knickle asks. "How much can you distribute among hubs? Can you distribute intelligence among interface points -- such as adapters, connectors, and gateways -- to help make it more scalable? Are you building in functionality to handle failover, recovery, load balancing?"

Completing the technical middleware acquisition is no guarantee of smooth sailing. Addressing the business and personnel needs is just as critical. At the high end, for example, some sites aspire to use EAI to support business processes. Yet defining the resources to be integrated and the business processes to be implemented is not something a middleware vendor can handle. "You need to realize there is still a lot of work you need to do, and that you may need some consulting help," Knickle says.

As the middleware market continues to evolve, NT managers can expect to see more emphasis on sophisticated business integration and EAI solutions, which will extract much of the complexity and add business value to traditional middleware offerings.

Buy carefully from middleware vendors, who are expected to undergo a rash of mergers and acquisitions during the next year or two as the market heats up and then consolidates. "It’s still a crazy market," Knickle agrees. "The product is still evolving."