In-Depth

Should We Be Thinking DAI? Planning for the Future of EAI

• 03/01/2000

Enterprise Application Integration (EAI) is a squeaky new, "slap-the-baby-and-get-him-breathing" market. EAI has gone from nothing to a hot thing almost overnight. There are too many vendors (hundreds?), too much VC money, too much confusion and incipient activity in mergers and acquisition.

Digital Consulting Inc.’s (DCI) recent EAI conference serves as a case study in early market development. The conference was tiny, the vendors were "visionary," and the attendees were confused. The vendor presentations all painted a picture of the new enterprise, where business process and workflow models, coupled with advanced integration technology, efficiently and effectively ride over the menagerie of installed best of breed, client/server, Web and legacy application systems. A vision of a "new technology tier" which will provide unprecedented control, flexibility and power to compete in the new millennium, and beyond. Yes, it’s … Enterprise Application Integration … EAI !

It seems that every few years we embrace some new "thing" that trumpets the dawn of the corporate information technology revolution (remember re-engineering?). The new "thing" could very well become EAI.

What else became obvious to anyone who took a few minutes to actually talk to the attendees of the EAI show? Although the vision of EAI is interesting, potentially strategic and possibly a future mandate … back at the ranch, we all have some very tactical stuff we have to get done.

And we have to get things done fast, like … today.

If you asked an attendee what challenges justified coughing up the money for a ticket to the show, you would hear something like this:

"Our telemarketing system is not sending the daily leads to our sales system, and we are losing revenue every day, and all I need is a way to move four fields of information from one system to another every hour, and I don’t have the staff to do it, and I hate using system integration firms that want to turn every little mole hill into a big mountain, and I thought I might get some real answers today, but I guess I was wrong."

Yikes! So what do we do? What is this market all about, and what should forward thinking professionals do to stay close to the action, solve today’s problems, and not waste time or money on visions that cannot be delivered just yet?

First, we should all continue to stay close to the development of the EAI market. Vendors are starting to deliver solutions to early adopter customers. Real world war stories are always the best way to understand new technologies and their real impact.

Second, we should pay attention to the very recent talk about business-to-business integration (cleverly referred to as B2B). B2B proposes taking the technologies of EAI and using these new tools to push integration outside the enterprise. I heard one well-known analyst group actually proclaim that "EAI is dead." This group believes that Business Community Integration (of course … BCI) is the next really big "thing."

Third, it’s time to get up to speed on XML. If you haven’t started learning what all this XML stuff is about, you better get moving. XML is gaining momentum. The Microsoft BizTalk initiative is real. Pay attention. Of course, there are a lot of overblown expectations that always accompany any new "standard" (remember EDI?), but don’t let any of that keep you from reading and learning more.

These three assignments belong in your strategic or professional development binder. Unfortunately, most of us spend more time in the trenches than in the library.

Implement point data integration technologies that provide rapid answers to today’s long list of one-off, important projects.

All of us have more mundane Departmental Application Integration (DAI) issues than we can stand. Let’s spend some of our precious time and money implementing cost-effective, easy-to-use technologies to solve application integration problems tactically and rapidly. I don’t care what you call it, but we should realize that there is a need to separate "Enterprise" from "Application Integration" and ensure that some level of tactical technology is in play now.

If you agree that some level of tactical attack is appropriate, then what do you look for in a solution? What matters?

Application integration is a technically complex task. Key technologies can make or break the ability to move rapidly and achieve success on tactical application integration tasks. Here are the top five "tech things" that matter most.

Business Rules Encapsulation. The difference between "data integration" and "application integration" is that applications have complex business and data rules that affect, and often control, the underlying application database. "Talking" directly to an application database is big-time risky. Application Software Vendors are scurrying to build application programming interfaces (APIs), but API availability and capability is pretty limited. The technology required to make tactical, point-to-point, and rapid data integration a reality must be cognizant of the application business rules challenge and should provide:

• The ability to interact with application vendor APIs (like the SAP BAPIs) using business object capabilities.

• The ability to externally encapsulate business and data rules to ensure that any inserts, updates or deletes maintain application source and target data integrity.

Frequently, the more competent application integration technology vendors address these issues with their adapter technology. Adapters house the connection technology and can be bought (or created) for both generic (ODBC, SQLServer, etc.) and vendor-specific (PeopleSoft, Siebel, etc.) sources and targets.

Dynamic Interaction. Application databases have a very annoying characteristic. They change. It’s extremely rare to find anyone who has installed a complex application out of the box without needing to modify the database schema, field lists, etc. It is also rare to find any installed application that is not modified over time as design inadequacies are identified and as the needs of the business change. This is a reality. Technology that is designed based on this reality is advantageous. The adapter (or connection) technology should be capable of dynamically interacting, and adjusting, to database changes automatically. The better vendors support dynamic adapter technology. It saves lots of time and energy as things change. And they always change.

Abstraction. Application integration is complex and can require the most talented technical people around. This is a problem. Extremely talented "techies" are in short supply, expensive and not always interested in mundane data integration work. You have to get your hands on technology that a non-programmer techie can use successfully.

Look for products that support a high degree of abstraction. Abstraction is the ability for the technology to present to the user source and target information logically (not just physically). The ability to map field migrations and integrations to logical views, stored procedures, business objects or other abstractions makes a huge difference in the level of talent that you need to complete a project. When you combine business rules encapsulation with abstraction, you dramatically reduce the demands on the techie. He or she does not have to be the rare guru that understands every quirky design aspect of an application database. You no longer need in-depth understanding of the complex physical entity maps and physical field placements (some of these applications, like SAP, can have 10,000 tables).

Flexibility and Bandwidth. Look for technologies that provide the most capability and flexibility. You never know what an integration job will require, so shop for robust tools and get a lot of "stuff," such as:

Graphic Interfaces. The more graphic support available, the better. I would not jump at any tools that require high-level programming skills, proprietary script languages, or any command line-type programming. Most of the quality products have easy-to-use graphical interfaces. The objective is to allow non-programmers to get some of this work done. Graphical interfaces help a lot.

One more item: The next time I write an article like this there will be a sixth item on the list, and it will be XML support. XML support is not a must have today, but it will be soon. Everyone is talking about it all the time. I have not been to any integration conference, heard any integration speech or read any integration article recently that did not reference XML. Microsoft is embracing XML as the backbone technology for BizTalk. This will only add fuel to the fire.

In this market, where the "talk" is often more impressive that than the "walk," the basics of vendor selection are as important as ever. Scrub the vendors’ track records, finances and customer references. Always talk to at least three customer references, and ask tough questions. Talk to references that have accomplished what you want to accomplish.

Don’t expect to get the total solution from any one vendor. No matter what any vendor claims, the EAI market is an early market. The vendors are new, and all of them are still learning. The EAI space is too new and complicated to easily segment and pigeonhole vendors. The technical challenges are diverse. There are lots of vendors and each have competence in a piece of the puzzle. Here’s my view of how it breaks down:

Big EAI Guys. Most of the high-profile EAI companies are targeting relatively large end users and offering a combination of application connectors (adapters), Message Oriented Middleware (MOM) support and Business Process Modeling Engines. These vendors have a "hub and spoke" technical vision controlled by business process models that handles all message transport and routing.

Messaging can be provided via proprietary technology or by incorporating other vendor platforms (most notably, MQ Series from IBM and/or MSMQ from Microsoft). These are the guys at the shows talking the big talk. Most are moving from talking EAI to e-business. Their solutions are service-intensive and are offered directly or with SI partnerships. They are typically competent in messaging and business processing. They view the "e" solution as a comprehensive business process and technology design effort. These are NOT DAI guys.

B2B Thinking Guys. There is a set of vendors that are focused on business-to-business integration. These guys typically started out with a "supply chain" orientation. These players are more Internet-based and more inclined to be ahead on XML technologies and business process. They are less inclined to be strong on adapter and messaging technologies.

DAI Guys. Most vendors are not taking a DAI view of the world. There is still too much big money on the table for big deals with big service.

Summary

There’s no doubt that the need to integrate application "islands" is universal. Everybody has the problem, it is expensive, and it is a big pain. Surveys indicate that as much as 40 percent of corporate IT budgets are consumed on application integration tasks. Until recently the solution was custom code, accomplished internally or with an integration firm.

The combination of the Internet, XML and e-commerce is going to put added pressure on front office to back office integration and on Web to enterprise integration. As XML matures, B2B integration opportunities will increase exponentially. The pressure for "e" everything is going to continue to be a huge integration driver. The new dimension to this challenge is that we can no longer attempt to solve integration challenges behind closed doors. Everything is out there, right in front of the customer.

EAI is the umbrella term for a new class of software technologies targeted at taking some of the pain and expense out of application integration. If you ask five experts to define EAI, I’m not sure you wouldn’t get five different answers. The long-term vision of EAI may prove to be the "transforming shift in our paradigm," but I doubt it. I’ve been in this industry for 23 years and I’m beginning to think that the fundamental problems never change, we just keep re-labeling them.

I do think that there are smart people creating pieces of the puzzle, and there’s a fair amount of pioneering going on in the field. Vendors and analysts will start to figure this space out. Some EAI vendors have had successful IPOs and they have cash. There have already been some interesting mergers and acquisitions (NEON buying Convoy, ONDisplay buying Oberon, etc.). Acquisition and consolidation will continue, market models will mature, and leaders will emerge.

For now, stay close to the action, stay in touch with your peers, and remember that pioneers get both rewards and arrows. Chip away at the daily, painful problems with Departmental Application Integration (or whatever it should be called) using less exotic, less expensive tools that can rapidly make a difference next week, not next millennium.