In-Depth

Q&A: Event-Driven Analytics

Informatica's co-founder and ex-president calls business activity monitoring a global positioning system for business health

• By Stephen Swoyer
• 07/16/2003

Informatica co-founder and former president and chief operating officer Diaz Nesamoney is back in action (http://info.101com.com/default.asp?id=1998) with another start-up: business activity monitoring (BAM) pure-player Celequest Corp.

Mr. Nesamoney explains how BAM differs from BI and why traditional BI architectures won’t cut it for BAM.

Most BI players have leapt on the business activity monitoring bandwagon, and yet you’ve said that traditional approaches to BI won’t cut it for BAM. What do you mean by this?

After I left Informatica [in July 2002], I took a step back and thought about the BI market, about where it is, where it might be headed. It’s a mature market, all of the needed solutions [and] pieces have been built. So where are the opportunities?

So the first question was what does BI do today and where does it need to go as companies make this evolution? BI has been by and large more of the rearview mirror approach in terms of looking at things. However, these companies that are trying to become more adaptive and responsive really need more of a [Global Positioning] system, which continually monitors your position and will tell you what you need to do at each point to get where you’re going. I concluded that the way that BI has historically been done does not address this requirement.

In what ways does it come up short?

Most BI today is batch-oriented. It’s done by taking data and storing it in a physical database, in a data warehouse, such as Oracle. Most of what is called “BI” is really just delivering reports based on this data. One problem is that there is quite a bit of latency involved in this [approach].

The challenge with business activity monitoring is to change from being batch-oriented to an event-driven approach. In BAM, as soon as an event occurs, like a customer deposit, analytics get applied to it immediately. So it’s an always-on architecture as opposed to a scheduled architecture.

In what ways does a BAM architecture differ from a BI architecture?

Most of the big BI vendors have developed infrastructures based on batch-oriented architectures originally designed for a data warehouse.

With business activity monitoring, our approach is to use a sort of memory-oriented architecture, or cache, so that you can make changes to the data model very easily because you’re not dealing with the limitations of physically stored data. With a data warehouse, it involves dropping in data and then indexing it, which could take some time. Secondly, storing data before you report on it reduces quite a bit of latency, even if you’re trickle-feeding it.

So it’s kind of like an in-memory version of the same operational data store (ODS) that’s used in many batch-oriented BI architectures?

It’s taking the concept of the ODS and operational reporting and creating a new stack of technology and a number of new components, like our Streaming Data Server, which operate on streams of events or messages coming into a message queue system. Or adaptive intelligence, which allows you to define analytics and business rules for a stream of events. Or, lastly, real-time metrics, as well as alerts or exceptions that get generated by the system. This is very different from how customers are doing this today.

Why not simply accelerate the rate at which data is loaded into a data warehouse?

Sometimes, yes, people have said, “If we want real-time, a lot of times we can load data faster into the data warehouse.” But customers find a whole host of problems with doing that, because these [data warehouses] are designed with certain performance characteristics in mind. So when they see this technology, they say, “This is great, we can leave the data warehouse the way it is.”

How do existing data warehouses fit into your BAM architecture?

To really do business activity monitoring, you need to look at a trend of events over a window of time in order to get a sense for where the trend is heading or whether the temporary spike is actually a trend that’s developing.

In our architecture, persistence [of event and message data] is optional. You can create a persistent ODS for audit purposes or for historical purposes so that you can go back and find out what happened at this point in time.

The reason persistence is optional, a lot of times we find that the history is already being stored somewhere else, it could be in a staging area, it could be in the data warehouse itself. One of the things that we’ve found when we talk to customers is that when we ask them how they are doing this today, most of them said that they are just taking current information from an operational data store directly and loading it into an Excel spreadsheet and basically implementing a very similar interface [to what we have].

What are the drawbacks to this approach?

The problem is that there’s a time dimension underlying this that you don’t get here. For example, You can say: “This rule holds for six hours, don’t send me an alert unless this condition has held true for six hours”—things like that. Inventory levels keep dropping and increasing in most high-volume environments, it doesn’t make sense to send an alert every time inventory drops, as you would do [with this approach], so what we do is reflect true trends in which inventory may be below a threshold for a significant period of time.

On top of this, you’ve got business rules that can be built on these views. If inventory’s down, for example, maybe you’ll get a list of suppliers who can expedite inventory to you.

No disrespect intended, but why should customers bet the farm on a small start-up such as Celequest when vendors such as Informatica, Tibco, and others are marketing similar solutions?

[Research firm] Gartner, their view is that since business activity monitoring requires a very different architectural approach, meaning one that’s event-driven, they really feel that [it will be] 2008 before some of the larger BI and other vendors can develop the technology and get it to market at a point where it’s usable by customers. Customers should consider pure play vendors (such as ourselves), just because we have the innovation and the technology to solve these problems. My belief is that we’ve taken the right architectural approach and probably solved this better than existing BI vendors.