In-Depth
In a Surging Analytic Market, Will Demand Outpace Supply?
Demand for BI and analytic technology could soon outpace demand, creating both risk and reward for IT professionals
With market watchers Gartner Inc., IDC, and Forrester revisiting and revising their IT spending forecasts downward for 2009 and beyond, spending on business intelligence (BI) and data warehousing (DW) appear, paradoxically, to be holding steady or poised for growth.
If BI and DW spending do, indeed, increase, spending on analytics -- the number-crunching, partially revelatory technology that accounts for much of the value of associated with BI and DW -- should surge.
A recent report from IDC -- Worldwide Business Analytics Software 2008-2012 Forecast and 2007 Vendor Shares published in the midst of last fall's severe economic crisis -- painted a picture of a thriving analytics segment, one in which real growth clearly outpaced projected growth in 2007. The upshot, IDC analysts say, is that the analytics market should buck an otherwise downward economic trend, with bright growth prospects through 2012.
The analytics segment, which encompasses both performance management (PM) tools and applications and DW platforms -- grew by 12.7 percent through 2007, outpacing IDC's own projection by 1.5 percent. All told, the PM tools market amounts to a staggering $15.4 billion, while that for DW systems clocks in at $6.7 billion. Oracle was the analytic market leader, growing its share by an ever-so-slight .1 percent in 2007, which boosted it to nearly 18 percent (17.6 percent, to be precise) of the market entire.
Redwood Shores was tops in both the PM tools segment and the DW systems space, which isn't surprising, given the DW reach of its Oracle database and the credible analytic technology it inherited from both Siebel Systems and Hyperion Solutions. SAS Institute Inc. -- a company with a rich analytic tradition -- was the number two overall analytic player, clocking in with less than half of Oracle's tally, at 8.2 percent overall. SAS, too, grew its share of the overall analytic segment by just .1 percentage points -- although SAS' fractional growth was more significant than Oracle's, given the latter's bigger share of the overall analytic pie. (SAS was the number three player in the PM segment and number five in the DW platform space.)
The third largest company was SAP, which significantly improved its position, growing its analytic share by half a point, to close with nearly 8 percent of the market. SAP's growth isn't quite as impressive as it might seem, however: the IDC study, which charts the performance of analytic vendors through 2007, takes SAP's acquisition of Business Objects, which occurred in October of 2007, into account.
All the same, SAP was the number two analytic vendor; it didn't crack the Top 10 in IDC's tally of the top DW players. Rounding out the Top 5 were Microsoft (#6 in PM tools and #3 in DW platforms) and IBM. Intriguingly, the IDC tally has Big Blue unranked in the PM tools segment -- even though its survey claims to take IBM's acquisition of PM power player Cognos into account. IDC's ranking of Big Blue as number two in the DW segment made a lot more sense, given the popularity of DB2 and IBM's demonstrable integration chops.
Bright Growth Prospects
Where analytics are concerned, IDC predicts growth at an astonishing clip. One upshot of this, the market watcher says, is that the demand for analytic talent could far exceed the available supply. "[Our] analysis suggests that we are at the beginning of a new wave of business analytics deployments that will materialize over the next decade and will be focused on addressing two primary demands [i.e., an ability to handle both more data and more users]," write IDC analysts Dan Vesset and Brian McKnight.
Vesset and McKnight predict that unstructured and semistructured data will finally get their due. "As the awareness of the potential of business analytics solutions to influence performance increases, the need to combine structured transactional data with various other forms of unstructured, semistructured, and rich media content becomes more acute," the pair indicates.
What's more, BI -- and analytics, in particular -- is poised to get much more pervasive. "Traditionally, the business intelligence tools market has addressed the needs of business and quantitative analysts, with less attention paid to managers and supervisors, line-of-business … staff, and stakeholders external to the organization," write Vesset and McKnight. "To achieve pervasive business intelligence, end-user organizations and technology vendors will have to rethink their approaches to technology deployment by taking into account expectations that users have for information access and interactivity on the Web and by embedding business analytics functionality into operational applications."
Will Analytic Demand Outpace Supply?
On the supply front, however, the industry seems ill-equipped to meet this demand, the IDC analysts indicate. As a result, many shops will embrace automation as a (not-quite-stopgap) measure to address the shortfall. This could place IT -- which is highly sensitive to disintermediation -- at odds with its line-of-business customers. "[T]he number of internal IT developers and analytics experts … doesn't seem to be keeping pace with the increased demand from end users," they write. "The widening gap between supply and demand … will need to be filled with automation, external services firms, … [software-as-a-service] solutions, or outsourced business analytics."
One upshot of this, Vesset and McKnight urge, is that IT must make its peace with automation: the potential analytic workload will just be too much for it to handle by itself.
"IT groups must overcome this perception given their existing and future high and potentially unsustainable workload requirements," they note. "The outcome of further intelligent process automation is not that IT staff will be eliminated due to automation. Instead, IT groups will be freed to perform the higher-value-added tasks of developing new applications and enhancing existing applications to support innovation and internal process efficiencies."
The analysts contend that "automation must be viewed by IT as a means to show its true value to the organization." (The two concede that "some internal IT staff" will likely be affected by the automation trend, but such displacement could itself lead to additional opportunities -- in outsourcing or systems integration, for example.)
Not Everything Counts
Consistent with its prediction that forward-thinking shops will try to push more data to more users, IDC urges organizations to radically expand the scope of their BI and analytic investments, chiefly by crunching more data (from structured, unstructured, and semistructured sources alike) and by rolling out BI and analytic capabilities to new user constituencies.
"[O]rganizations should expand their view of business intelligence and analytics beyond traditional query and reporting tools to include advanced analytics, search and discovery, business process automation, collaboration, and workflow management," Vesset and McKnight counsel.
Of course, with so much data and so many sources -- and with the increasing ability to actually connect to and crunch through it all -- there's a tendency to analyze everything. This would be a mistake, Vesset and McKnight observe. Of particular importance in the current milieu, they say, is to identify analytic projects with obvious revenue- or profit-generating potential.
"Not all performance indicators are 'key.' Select only those that are actionable and impact the business," they point out, citing an aphorism from Albert Einstein: "Not everything that counts can be counted, and not everything that can be counted counts."