In-Depth
Search and Text Analysis Lead BI Push
Thanks to demand for cutting-edge features such as sentiment extraction, text analytic technology is primed for growth.
Although no one knows how the prospect of a deep and lasting recession will affect the IT industry, market watchers predict that the business intelligence (BI) and data warehousing (DW) segments should be insulated against the worst ravages of recession.
Gartner Inc., TDWI, and others predict that organizations will paradoxically spend more on BI and DW during a downturn, at least relative to their other IT investments, as they try to squeeze as much as possible out of their information assets.
Interest in analytics in general -- and predictive analytics in particular -- will almost certainly surge, experts say (see http://www.tdwi.org/News/display.aspx?ID=9212)
Interest in BI search and discovery segment is also strong. Several vendors -- SAS Institute Inc., SPSS Inc., IBM Corp., Microsoft Corp., Oracle Corp., and SAP AG (via its Business Objects subsidiary) -- could reap huge benefits. That's the conclusion of a recent study (Worldwide Search and Discovery Software 2008-2012 Forecast Update and Vendor Shares: Bloom Amid Economic Gloom) from market watcher IDC, which found that demand for search and discovery software outpaced software spending as a whole through 2007 and the first half of 2008. That's a trend IDC expects will continue.
"Despite the current economic crisis, the search and discovery software market is a bright spot that so far shows no signs of slowing down," said Susan Feldman, vice president, Search and Discovery Technologies at IDC, in a statement.
At the same time, IDC cautions, growth in IT spending will contract -- including growth in the otherwise red-hot search and discovery software segment. BI search and discovery sales will keep growing, albeit it at a slightly less accelerated pace.
"Although this market has continued its rapid growth in the first half of 2008, economic indicators for IT spending are bleak," she comments. "IDC expects slower growth in all markets, including search and discovery software," Feldman continues. "Although these external factors will impinge on what is a fast growing market, IDC still predicts 17 percent growth for search and discovery software in 2008 and 12.9 percent growth for 2009. This is down from the 28 percent growth we saw in 2007, but certainly a healthy increase. With the economy so volatile, this could change if economic conditions worsen."
That said, IDC expects that sales of text analytic technologies will outpace those of general BI search and discovery offerings, thanks to demand for cutting-edge features such as sentiment extraction, eDiscovery, and geolocation.
Search and text analytic vendors say this jibes with their own experiences.
Ann Milley, senior director of analytics strategy with SAS, says companies are increasingly capturing unstructured data, including non-traditional unstructured data such as call center logs, which they're either subjecting to speech-to-text technologies or transcribing by other means.
"Companies talk about data, and they talk about content, and they usually think of them separately. But you know, content is data. Even though they're managed separately, it helps you close the loop. Take a call center, where you're collecting [structured] data [such as the number of calls, both inbound and outbound]. [Many] call centers are recording [calls] now, so they have this unstructured content, too," she says.
Ambitious customers, Milley points out, are starting to do something with all of this unstructured call data, transcribing it (by any of several means) and subjecting it to sentiment analysis or other new-fangled ways of analyzing data.
"You can find patterns in those conversations, you can glean more value from the actual unstructured data which ultimately does get some level of structure added to it. At the end of the day, you're able to look at these patterns from these conversations, you're able to get a more complete picture of any given customer, [and] you're able to marry up that contextual data with your structured data."
Sentiment analysis is an especially hot topic in the Web world, thanks to the ascendance of social networking sites such as Facebook.com and MySpace.com.
Companies want to study social networking sites (and other Web resources) to better understand who their customers (or non-customers) are, where they're coming from, what they like (and what they buy), and (of course) what they think of them.
Consider SPSS. "We can link our data mining and text mining [technology] to word blocks [and] to blogs," says Kris Hackney, vice-president of enterprise solutions with SPSS. "We're looking at [contributors to social networking sites] not just from the perspective of who talks to whom, but we also have the capability to dive into what are they saying and analyze that data from an unstructured perspective. The ability to act on that data once you understand it is absolutely crucial for the clients that we are talking to."