In-Depth

Q&A: Is Real Time Right for You?

The pain of integrating systems in real time often outweighs business needs

• By Stephen Swoyer
• 05/28/2003

But Tho Nguyen, director of data management strategies with BI powerhouse SAS Institute Inc., thinks it may be too early in the game for most enterprise customers to start thinking seriously about real-time systems. In Nguyen’s account, the pains associated with real-time integration projects—which typically introduce disruptive changes in people, processes, and technology—often outweigh the business benefits.

Instead, Nguyen stresses, customers should try to limit their real-time aspirations to critical applications or processes, and in many cases should compromise on near-real-time infrastructures that “trickle” fresh data on an hourly or semi-hourly basis.

During a recent interview, Nguyen discussed these issues and others, including the problem of ensuring data quality in real-time environments.

Q. There’s been a lot of talk recently about the need for businesses to move to real-time infrastructures Yet you’ve suggested that not everyone needs to integrate their systems in real time.

A. There is a lot of hype out there for real time. Everybody wants to jump on the bandwagon. Not necessarily to listen to the customers to see if the customers really need a real-time process for real-time implementation. An organization that does sales might not need a real-time enterprise—they might just need a component to deliver real-time or near-real-time information to their sales force. The reality of the deployment is that not many of our customers are focusing on real time right now because of the economy, because of their workforces being laid-off. Instead, they’re trying to make the best of what they have.

Q. But you undoubtedly have some customers who have identified a specific need for real-time integration?

A. We have several customers of SAS (one is the Energy Department) who have specifically identified with us, yes. There is one specific aspect that we need to better communicate to our sales force that they need to elaborate with or share with potential customers.

Q. Do you help your customers make this decision?

A.When our customers ask us about real time, we try to go in and understand their infrastructures. Change takes time over months or years, and implementing in real time requires change not only in the process, but also the infrastructure. So we need to go in and understand the needs, the pain, the processes, the people—and then we help our customers make a decision.

Q. So what are the criteria for deciding if the real-time enterprise is right for a specific customer?

A. They have to ask themselves how much is a pain and how much is a need. Sometimes, there’s only the need to automate the process, make it near-real-time and modify the process by automating it, having the data trickle every hour or every fourth hour. That is one very inexpensive way to approach real-time—simply have a scheduler to append the data that’s coming into an existing system. Another is to try to integrate various components of the application in real time. Other ways would be to have a data feed where there’s an alert message coming in and automating that.

All of this [making data available in real time or near-real time] raises other issues, though.

Q. Such as?

A. I know that many CRM implementations have failed due to poor data quality, and I think that because of these data quality problems, the benefit of what the users might or might not see, they don’t trust what they have, they can’t validate that what they have is correct. That’s one aspect of it on the back-end side

How much of the data do you have to integrate and cleanse in the back-end? I think you talk about a lot of it being soft issues, people issues, process issues, and one of the processes that we’ve really struggled with is having cleansed data versus un-integrated, unclean data.

Q. I’ve spoken with at least one analyst [Mike Schiff, principal of data warehousing consultancy MAS Strategies] who raised the same issue. How do you ensure the quality of data when you’re integrating systems in real time?

A. To be honest, there’s really no process there to ensure that the data is cleansed or integrated. The question for customers is: Are we going to sacrifice the integrity of the data or process the data so fast that we can’t validate that it is correct? We have an integrated data quality and ETL suite, and what happens there is that we are trying to cleanse the data at the source. But when we’re talking about near-real time, we also have to look forward. There is a time when the infrastructure, the hardware, and also the technology can combine to truly handle real time, so what can we do with it then? These are things that we’re trying to be innovative about.

Q. Earlier, you mentioned the failure of many CRM systems and the problem of users trusting their data. This goes back to the problem of driving end-user uptake of business intelligence software—in terms of actually getting them to use the stuff. What are your thoughts?

A. One thing that customers need to be aware of is that data quality on the back-end influences user expectations on the front end. If you can set expectations so that the users know what they’re getting out of the system is quality—usable data—it can help to change the mentality [of resistance to change]. But if users don’t trust the data, how are you going to convince them to use this stuff?

There’s also another way of setting expectations. If you set expectations from the beginning of how you can help users as well as ease their day-to-day tasks more from the strategic execution side, they’re more prone to say, ‘Hey, this can help me," and they’re more prone to accept it.