Q&A: The Drive to Unified Information Access
Imagine querying all enterprise information with the precision of SQL and the fuzziness of search. That’s the promise of unified information access.
Information access is a vital requirement for any enterprise -- but given the wide range of data types (and consider the vast challenges of unstructured data), how can an enterprise support querying of data with SQL precision but with the fuzziness of search that all users are used to? The answer just might be unified information access. To learn more, we spoke with Sid Probstein, CTO and co-founder of Attivio.
Enterprise Strategies: What do you mean by unified information access?
Sid Probstein: We refer to a single point of access for all information in the enterprise: structured, related, transactional, numeric data ( typically stored in relational databases) and unstructured text data (typically stored on file systems, content management systems, etc.). Our goal is to support querying this information -- and all information -- with the precision of SQL and the fuzziness of search.
What’s driving IT’s interest in UIA? What are the benefits IT expects?
There is tremendous, unending demand for more detailed understanding of the various forces that act on a business. The focus for the last few decades has been on structured data -- numbers, transactions, etc. Progress has been made, but a company that has invested in this type of analysis has only achieved parity with its competitors.
Competitive advantage will flow only from getting ahead, which means understanding not just what happened or where it happened but rather why it happened. Those answers are often in text. Recently, the CTO of one of the leading BI vendors said that bringing unstructured information into the “business intelligence stack” was the top priority for the industry.
IT knows that demand for unstructured information, and -- more importantly -- the ability to correlate what happened in transactions with what was said in text will be something they have to deliver. There are also some selfish (in a good way) reasons: wouldn’t it be great to actually have one silo to maintain instead of hundreds or thousands of silos with different eco-systems supporting them?
The benefits of UIA are hard to overstate. The fact is that decision making is difficult enough when you have all the data. Most organizations are forced to make decisions without all the data because they have silos working against them.
Here’s a concrete example. I recently rolled over my 401K fund. I had 30 days to do it, and actually did it on the 29th day. It was easy -- I picked up the phone and called my financial services company and it took 30 minutes. The next day, a different associate called me and asked me if I wanted to roll over my 401K. I was quite upset by this. I asked the rep “How is it you don’t know that I did that yesterday?” He told me that he did not have access to that information.
The cost of not having the information available was high in this case; not only a wasted inbound call to me, but then fielding a third call from me verifying that I had, in fact, rolled over the 401K correctly, not to mention damaging my confidence in the company.
What are the biggest challenges that get in the way of a UIA implementation?
The biggest challenge is trying to put content into data warehouses that are built on relational databases. They just don’t deal with unstructured content very well. The few technically successful examples I’ve seen are very domain specific (and thus don’t typically cut across silos) and involve “structuring the unstructured” via brittle tools such as text mining.
Another challenge is motivation. IT is always under pressure to deliver something specific, so the time needed for finding ways to break down silos often ends up without priority or resources. In fact, this pressure makes silo creation appealing. Visionary companies that invest in UIA quickly realize the gains; rather than one month to create a silo, which is a pain to maintain thereafter, you can make “slices” of data on demand, as fast as the business can make use of them.
What’s the impact of UIA to an enterprise? For example, does unified information access change the way a business makes use of its data or just make it more widely available?
Ultimately, both. The first step is to change the way data is used. Imagine what your business could do if it could query all data. What kinds of advantages could you create? What kind of products could you rapidly bring to market? What kinds of new processes could you create?
The second step is to make it widely available. People often speak of the “BI pyramid” ... a few users, at the top, who understand the business and can use the complex query tools, supported by many IT and other staffers who get the data together, keep the systems running, etc. UIA is an important step to flipping the pyramid over such that there are many users consuming information, and few supporting them.
Is a UIA project a complete rip-and-replace affair, or does it include access to legacy systems?
Rip-and-replace requires serious organizational commitment and likely would be palatable only to visionary companies that have identified an advantage that would result from such an effort. UIA is best done incrementally, starting with a specific goal or application in mind, and then building on it, migrating data as appropriate. Companies should be able to protect and realize return on their investments as they do it.
Given our uncertain economy, funding for a UIA project is key. What costs are involved in a UIA project, and is there a way to calculate the return on investment?
The cost to rip-and-replace would be significant of course, with ROI depending on some market advantage. For the incremental approach, it likely isn’t different from any other project around information access: back-end, front-end, integration costs, training, and so forth. Working with an agile solution that can be easily and quickly prototyped and deployed is key to minimizing these costs.
Calculating the ROI on UIA depends highly on the goal of the project. Let’s use the 401K example. If there are 100 rollovers per day, 10 percent of them result in a second call, and 2 percent of them result in a third call, that’s 12 extra calls per day which is 3,000 calls per year. If those calls cost $75 each to handle, that’s $225,000 per year.
Are these economic benefits enough to motivate CIOs to support a UIA project?
Visionary CIOs we are working with already see the need for UIA and leading analysts predict that by 2011 up to 75 percent of IT spending will be on unifying information. The economic benefits are clear, especially in challenging economic times. Holding on to your customers requires good service, and that requires access to all information. Competing for new customers is just a different side of the same coin: if you want to win the deal, you need to know everything possible.
What are the biggest mistakes IT makes when undertaking a UIA project?
I think there are three common, major mistakes made:
- Not talking to a UIA vendor. The giants of industry talk about UIA but at the end of the day they are busy integrating their newly acquired BI companies and information access products. They don’t have UIA; at best they will have integrations between their favorite silos. Their UIA story will be “eliminate the other silos.” This is not practical.
- Trying to do UIA with a relational database. It’s generally believed that 80 percent of data in the enterprise is unstructured. This is equivalent to trying to build something 4x bigger and far less structured than the average data warehouse, which deals with only 20 percent of structured data.
- Planning to federate between sources. This can work in some scenarios, but as an overall strategy it incurs many of the costs of the data warehouse, as the differences between silos must be factored in.
What best practices can you suggest to avoid these problems?
An enterprise should look for a UIA solution that is open and vendor neutral. It should also use a project that will deliver value to pull the overall UIA story along. Doing UIA in the abstract will be hard to sell, but solving a problem and/or creating some advantage for your company will be much easier and will educate the organization on how to do it.
How does UIA give an enterprise a competitive advantage?
First, it delivers an advantage because not that many companies are doing it - yet. Few and far between are companies that can really correlate what people do with what they say, and exploit it via some technical channel.
Second, it creates an advantage by supporting rapid prototyping, “what if” scenarios, etc. Imagine being able to create, in days, a micro-site for customers with a particular challenge. That’s how conversion rates rise, loyalty increases, etc.
What products or services does Attivio offer to solve these problems?
Attivio offers a complete unified information access platform: the Active Intelligence Engine (AIE). Our patent-pending, query-side JOIN operator lets you combine SQL/relational analysis with full-text search. Moreover, our “Active” capability allows you to create business processes that integrate systems together. For example, let’s say you write a query that identifies a group of customers who are good prospects for a new product. Rather than printing out that list, faxing it to marketing, having IT load it into the CRM, etc, you can use AIE and save the query as an alert and tie it to a workflow that writes the data directly into the CRM system. Thereafter, customers who matched would instantly appear in the CRM system and receive an outbound call.