In-Depth
On the Advantages and Disadvantages of BI Search
Can search deliver on the promise of ubiquitous BI? Maybe. As with any highly hyped technology, however, a few myths first need to be busted.
To listen to Cognos Inc., Google Inc., IBM Corp., Oracle Corp., and others tell it, business intelligence (BI) search is fast emerging as The Next Big Thing in an ever-expanding BI marketplace.
As with other BI Next Big Things, search proponents are yoking the technology to that most elusive of BI El Dorados: ubiquitous—or, at least, mostly pervasive—usability. The idea, backers argue, is that search gives users both a familiar UI and a proven paradigm—namely, the Web browser and its proven Web search model-- in which to perform query and analysis. Business Objects SA, Cognos, IBM, Information Builders, Microsoft, Oracle, SAS, and others all deliver BI search offerings, while a bevy of startups—along with some significant non-BI players, such as Google—also ply the search trade.
But can search deliver on the promise of ubiquitous BI? And, if not ubiquitous BI, how or why will search change the manner in which organizations generate and consume BI? Not all of the votes have yet been tallied—but early returns are promising. Search vendors, for example, cite the can’t-miss Web search model on which the technology is based and say that enterprise search has a proven usability track record.
"If you think about it, [search is] a proven [technology]. It works. The same [model] you [use to] surf the Web for information—that’s how you’ll search [your enterprise information assets] for information," said Jake Freivald, vice-president of marketing for IBI, in an interview earlier this year.
Freivald was promoting WebFOCUS Magnify, IBI’s Search 2.0 offering—which, curiously enough, it bills as the industry’s "first BI search tool."
A strange claim, admittedly: after all, IBI introduced its first search offering (jointly, with Google) in September of 2005 and followed that up with a branded search offering about six months later. More to the point, a host of players—including Business Objects, Cognos, IBM, Microsoft, Oracle, and SAS—have also fielded enterprise search entries over the last 24 months. But by first, Freivald and IBI likely meant "pervasive": WebFOCUS Magnify boasts connectivity not just into garden-variety RDBMSes and other common repositories, but into ERP systems, flat files, and other application-specific data sources, too.
"We don't just search relational databases and text files. The [Google Search Appliance] can do that out of the box," Freivald said. "We also search ERP systems, legacy systems, queries that may contain data from multiple sources—relational, non-relational, ERP, legacy, and so on—and more." Elsewhere, WebFOCUS Magnify can search and index EDI documents, CVS files, XML files, standard files, Web services, and what Freivald described as "a huge variety of other messages, documents, and other transient data."
The Connectivity Myth
Magnify is by no means the only tool on the block in this respect. To be sure, it taps the data integration expertise of IBI’s iWay subsidiary—which gives it connectivity into more than 300 different data sources—but it’s one of several solutions, including offerings from Business Objects, Cognos, IBM, Oracle, and SAS, which boast connectivity into heterogeneous (or extra-relational) data sources. Not surprisingly, most of these vendors also field credible—and in some cases (e.g., IBM, SAS), market-leading—data integration tools.
That’s one reason why search is poised to go supernova, backers say: The tools are maturing—some, like offerings from Google, IBM, and SAS have several years of maturity under their belts—and the connectivity is there, too.
Of course, connectivity of this kind isn’t simply a turnkey proposition: search vendors might promise out-of-the-box access to structured, semi-structured, and even unstructured data sources, but experts caution against deploying BI search solutions in the absence of a mature data management infrastructure.
In other words, says Philip Russom, senior manager of research with TDWI, the best search and text analytic deployments will be built on top of mature, reliable data warehouse (DW) and BI infrastructures. All the same, Russom concedes, search does introduce a new twist—namely, the consumption of semi-structured and unstructured information.
"The growing use of BI search and text analytics is part of a larger trend toward leveraging unstructured data in BI and DW, fields that previously have relied almost exclusively on structured data," writes Philip Russom, senior manager of research with The Data Warehousing Institute (TDWI) in a recent report, BI Search and Text Analytics. "Another way to put it is that unstructured data is playing a larger role in BI and DW over time, and that role is today supported largely by tools and techniques for BI search and text analytics."
In this respect, Russom says he doesn’t see search or text analytics as revolutionary or even evolutionary technologies; instead, he argues, they’re more accretive.
"BI search and text analytics certainly won’t replace the traditional BI/DW technology stack. And it’s unlikely that they will replace any components of the stack," he comments. "Instead, BI search and text analytics are being added to BI/DW infrastructure to accommodate unstructured data—via text analytics—and related techniques [such as search]."
The upshot, Russom stresses, is that search isn’t for DW dilettantes. Even if most BI search tools do promise exhaustive connectivity into a host of semi-structured and unstructured data sources, that doesn’t mean organizations can ignore the shortcomings of their existing BI and DW implementations: "Most user organizations should have a mature BI/DW implementation in place before attempting to add BI search and text analytics to it."
Elsewhere, TDWI director of research and services Wayne Eckerson expands on this idea. "BI search is great when you don’t know what you’re looking for, or can’t find what already exists," he says. "But if the average business user needs a search tool to find basic information or reports about the processes they manage on a daily basis, there is something wrong with the business!"
In other words, Russom reiterates, search’s ideal use case is as a complement to an existing BI and DW implementation.
"Managers should know exactly what information will help them run a line of business and receive this information customized for them in a performance dashboard. If they don’t know what they need or where to find it, this is a symptom of a grander malaise than BI search can cure."
In fact, experts say, a number of adopters currently tap search technologies to do just that—i.e., to complement existing (and typically mature) BI and DW implementations. It’s for this reason, says Jill Dyche, a partner with and co-founder of BI and DW consultancy Baseline, that many search adopters tap the technology to augment established reporting practices.
"Most of our clients embarking on search are using it to track and manage their reports using search appliance technology," she confirms. "Hyperion—now Oracle—does this very well, using a Google search appliance. By exposing the metadata from BI tools, the search appliance can find reports and other documents and make them available to anyone with web access."
Properly used, search tools can also reduce confusion and redundancy in large organizations, she maintains. "The bigger the client and the more diverse the organizations, the more of the same type of report is likely to be floating out there in the ether," Dyche comments. "By leveraging search, a user can find an existing report and parameterize it rather than re-creating it from scratch and thereby doing with reports what they're doing with data: creating new versions of the same thing."
Most BI platforms today embed a search capability of some kind – although most of these facilities only support a single BI platform, says TDWI’s Russom.
A more advanced BI search use case involves indexing reports across multiple BI platforms. It’s in this respect, he adds, that BI search has a unique value proposition. "When the same business entities and processes are represented in reports from multiple BI platforms, this is a barrier to gaining a complete view of corporate performance. Business analysts and other report consumers must hit multiple systems one at a time, hoping that they’ve found all the reports that are relevant," he points out. "This task is simpler, faster, and less prone to error when BI search encompasses all BI platforms. Furthermore, this configuration of BI search helps end users associate related reports, regardless of their points of origin." For this reason, Russom concedes, "BI search might be a compelling stop-gap alternative to an expensive and disruptive BI consolidation project."
At the same time, cross-platform BI search isn’t yet a turnkey proposition.
"The search functions embedded in a vendor’s BI platform support only that vendor’s brand. So, indexing multiple BI platforms will probably require a third-party enterprise search engine. In order for the complex system integration to work, that engine must support all the BI platforms involved, or have interfaces that can be adapted," Russom concludes. "Other sticky details involve security, scheduling crawlers, and the permission of platform owners. Despite the challenges, this use case provides a unique solution for organizations with multiple BI platforms."