In-Depth

BI Decision-Making: The Human Factor

TDWI Keynote speakers Jonathan Wu and Jon Koomey focused on the ways in which human behaviors can drive, or impede, BI decision-making.

• By Stephen Swoyer
• 02/28/2007

LAS VEGAS—The "human" dimension of business intelligence (BI) served as a prominent theme in last week’s TDWI Winter World Conference, held amongst the trappings of a modern-era, smoke-filled, surreal, Roman-forum-like palace called Caesar’s.

First, there was Jonathan Wu, a senior principal with business intelligence (BI) consultancy Knightsbridge Solutions (recently acquired by HP), who kicked off his keynote by challenging attendants to be successful with "every single one" of their BI implementations.

That’s a tall challenge, as any project manager can attest: project failure is a fact of life, after all, and nearly half of organizations say they’ve abandoned one or more projects over the last several years. (http://esj.com/enterprise/article.aspx?EditorialsID=1501)

But Wu urged attendees to focus on the intangible (and abundantly human) reasons many BI projects fail: the inability of organizations to effectually manage people, processes, and change. And one aspect of managing people, Wu says, is to understand how much and how well they’re using the BI solutions at their fingertips. This isn’t always as straightforward as it might sound, he said.

"Are they using it or are they just accessing it? You need to dive into how people are using it as well as the corresponding frequency of usage," he indicates. "Are they generating queries? Are they generating reports? How are they engaging with that application?" Organizations should also consider the corresponding time period, Wu said: "If you’re only updating the data warehouse on a monthly basis, you can’t expect people to use it on a daily basis. [Their usage] has to correspond" to update frequency. The lesson, he says, is that there’s a "high correlation between update of the data and usage of the data."

On the change management front, Wu indicated, organizations need to think beyond garden-variety training and education. "I’m talking about a comprehensive program that allows people to evolve their skill sets, so ideally you’re evolving a curriculum plan, a skills matrix for the …user community," He said. "[You’re] understanding what are the data sets that they need to have access to and help[ing] them evolve [those] skill[s]."

There’s also the issue of behavioral change, which is perhaps the most difficult aspect of successful BI project implementation. "Changing behavior [can involve] putting together a series of incentives and rewards for people; recognizing people, giving people the recognition that they’ve done something wonderful with the BI application," Wu explained. "At the same time, you’ve got to also put in penalties, more deterrents, [such as] cutting off access to how they previously obtained their data … so that they are focused on the BI applications."

Keynote II

Thursday’s keynote speaker, Jonathan Koomey, a staff scientist with the Lawrence Berkeley National Laboratory and a consulting professor with Stanford University, emphasized the human dimension of another BI-related field—statistical analysis. "A lot of the work [in BI] … relates to the tools [and] the data, but I want to talk about some of the aspects of improving the way people use that information," Koomey said.

One way in which human beings use—and misuse—information is by telling stories, Koomey said. He cited one particularly infamous case in kind: a 1999 claim by analysts Mark Mills and Peter Huber that the Internet was consuming 8 percent of U.S. electricity. Among other findings, Mills and Huber claimed that a Palm Pilot which is plugged into a network consumes as much electricity as a conventional refrigerator. Koomey said that on several occasions he asked Mills and Huber to furnish evidence in support of this claim; the duo never responded. Nevertheless, the Palm Pilot canard—which Koomey demonstrably debunked—became accepted as conventional wisdom.

One lesson is that human beings like stories—and like a good story a lot more than a middling one. More to the point, Koomey says, stories inevitably influence decision-making. He says Mills and Huber’s claims—which the duo outlined in a Forbes article and a Wall Street Journal op-ed—had a demonstrable effect on decision-making. The duo’s claims were widely disseminated—without attribution—in many newspaper dailies.

Nevertheless, Koomey stressed, Mills’ and Huber’s claims were clearly untrue. "Humans are hard-wired in some ways to remember stories and use them in their decision-making, but … [stories] are not particularly amenable to decision-making," he indicated. "What happens when statistics get cited without attribution is that they get disembodied. The original analytic work that was responsible for the creation of those numbers gets separated [decoupled, if you will] from how those numbers are reported." And once an authority—such as Forbes or the Wall Street Journal—cites a figure, "then it becomes fair game."

As a result, Koomey urged attendees to question—and to be downright skeptical of—all their data. "Humans matter," he said, and "human behavior was key to the propagation of these [erroneous] statistics."

Nor is decision-making as straightforward as BI and performance management (PM) vendors would have us believe, Koomey said, who stressed that—even in companies—"analysis occurs in a political context. People use information to their advantage, to gain a bureaucratic advantage, to gain power, to gain resources. It’s not just a neutral set of activities that you’re doing."

More to the point, he argued, every decision involves a value-choice. "Every decision involves some evaluation of facts to make the choice that you’re going to make. If you agree on the values and you agree on the facts, it’s very simple," Koomey explained. "If you disagree on the values and agree on the facts, negotiation is what’s needed. If you disagree on the values and the facts, that’s where you can have paralysis or chaos. [And] you … see this in companies that don’t have a clear sense of what their mission is." [One could argue we also see this in the hot debate around "Global Warming."]

Finally, Koomey counseled, don’t simply do data collection and analysis for data collection and analysis’ sake. "If you focus your analysis and data collection on a specific decision, it’s more likely to lead to an effective outcome," he said. "I think there’s always core work that you need to do to make sure the data you’re collecting are accurate. There needs to be analytical input into that process, but that will be more effective if you know the decisions that you’re going to have to make. Focus the process on analytical decisions and you’ll be better off."