Q&A: Melding Predictive Analytics and Big Data Makes Sense
Predictive analytics and big data "make perfect sense together," says analyst Fern Halper, even as some vendors are struggling to catch up with users.
- By Linda L. Briggs
- 05/08/2012
"There's definitely an overlap [between big data and predictive analytics]. These are two areas that make perfect sense together." That's according to analyst and researcher Fern Halper with Hurwitz & Associates, who has studied data analysis for over 20 years.
Halper is a partner at the consulting, research, and analyst firm Hurwitz & Associates. She has over twenty years of experience in data analysis, business analysis, and strategy development, and has held key positions at AT&T Bell Laboratories and Lucent Technologies. She is also the author of numerous articles on data mining and information technology, and an adjunct professor at Bentley College, where she teaches courses in information systems and business. She blogs about data and analytics at http://fbhalper.wordpress.com/.
In this interview, the second of two parts, Halper focuses on the overlap between big data and analytics. [Editor's note: Part 1 of our discussion can be read here.]
Shifting gears a little bit, let's talk about so-called "big data" in relation to predictive analytics. What do you see happening in that area?
There's definitely an overlap -- these are two areas that make perfect sense together.
I did a study [recently] in which I asked about big data. It's still early on the whole big-data front, but having said that, when I asked companies about their plans for advanced analytics, something like 25 percent of respondents were talking about analyzing complex, real-time data streams. That would be big data.
Companies that are analyzing huge amounts of data either have a large volume or large variety of disbursed data, or it's coming at them in real time at a huge volume, and they're doing complex things with it. A lot of times in many use cases, it is about predictive analytics and big data.
At their recent Information On Demand conference, IBM was talking a lot about big data, and showing that there were a number of customers out there talking about it.
The vendors now are doing interesting things with big data in terms of trying to put the analytics into the databases, and putting the analytics into data streams as they're coming through, to be able to analyze the data that's coming through.
Do you see big data and predictive analytics as an area where vendors are scrambling to catch up?
Some vendors, like SAS, have been evolving to this point all along. They've been evolving to [handle] what we're now calling big data. They've been doing it for a long time because they've always been pushing the envelope, more or less. They're known for scalability and performance, and for being able to deal with complex problems. They're architected that way and they've been evolving all along.
Let's face it. It's a marketing term in some ways. People have been dealing with big data problems for a long time. Back when I was doing my own research, I thought I was dealing with big data. We were dealing with large volumes at the time, and it was coming at us daily. Some of it was minute-by-minute, but we didn't have the compute power to deal with really real-time data. We were pushing the envelope regarding what we could do, using what we had at the time. It's really about what the IBM executives like to call "the art of the possible" -- what can be done at any one time.
I do think that regarding vendors, it seems everyone's now trying to say that they're doing big data. So many vendors are trying to say, "You know, we do big data."
I said last year that for the most part what I saw in terms of analytics around big data was not that impressive from many of the vendors out there. It was slicing and dicing -- basic sorts of things.
More recently, for some reason, at least the bigger-name vendors seem more impressive with what they are doing. Some vendors are just trying to catch up, because they need to have a story around big data and big data analytics.
It's interesting that a year or 18 months ago, you didn't see much happening around big data -- at least much that impressed you -- but now it seems that things are moving.
I was scratching my head last year, thinking, "OK, you have an appliance that can store big data," and everyone was talking about Hadoop. No one was dealing with the fact that Hadoop could store unstructured data. They didn't even seem to be thinking about that. They were more thinking, OK, I have my structured data, it's going to be stored in my appliance, then they were slapping something on top of it that wasn't that impressive in terms of analytics.
This year, I'm seeing a lot [of talk about big data and analytics] even though it's still early, but at least there are use cases now that are being touted -- that sort of thing.
As I said, though, there are the vendors that are trying to catch up, and then there are the vendors that have been doing this all along.
Even with SAS, were they calling it "big data" in 2010? They were talking about their partnerships with Teradata and others, but they've just put more of an emphasis around it. The fact is, they've always been doing it, but they didn't call it that. It's often like that with these terms, it seems to me.
It's great because the technology, the computing resources, are really catching up, thus enabling companies to do all sorts of things around big data and analytics. However, I think a lot of them are still scratching their head about what to do, and still trying to deal with modeling churn or other basics.
There are companies in any industry that are going to be early adopters, those pushing the vendors out of their comfort zone to be addressing big data analytics problems that the vendor perhaps hadn't even thought about. There's some of that going on as well with big data and analytics.