In-Depth

Q&A on Emerging Tech: Analyzing the Voice of Your Customer

Making the most of the 80 percent of data stored as text is the job of this increasing popular technology.

• By James E. Powell
• 09/29/2010

With so much customer data tucked away in social media sites or in large text fields -- think customer feedback stored in your CRM system -- how can an enterprise get to the data and mine its value? The answer may be in the emerging field of text analytics. Chris Jones has been helping companies implement data warehouses, Web analytics, and text analytics systems for over 15 years.

During the past four years, his focus has been in the area of text analytics; using these tools to help companies improve the customer experience. He has made presentations on the topic at conferences such as TDWI, SSPA, TSIA, Customer Feedback Week, and the Text Analytics summit. He will be speaking at TDWI's World Conference in Orlando we turned to him to learn more about how this technology can empower BI professionals and boost the quality of their decisions.

BI This Week: I hear a lot about "voice of the customer" lately, but what does that really mean?

Chris Jones: Voice of the customer is as much about the culture of your company as it is about the technology. Companies used to talk about being data driven, using data to make informed business decisions: now they are talking about customer experience and voice of the customer. These means not only looking at how many widgets you sold last quarter, but what do your end customers feel about those widgets? What features do they love and which ones do they struggle with?

What is text analytics and what benefits does it offer my enterprise?

I try to think about text analytics as having three parts. 1) Using computers to do what we learned how to do in fifth grade -- namely, break down a sentence into its parts, understand the nouns, verbs, adjectives, prepositions, and who did what to whom. 2) Being able to categorize or group sentences into meaningful categories or hierarchies which are using to the enterprise. 3) Understanding the tone or sentiment of those sentences. Text analytics is a great tool for analyzing the voice of the customer.

How do text analytics projects impact DW/BI teams? How will it change how BI professionals (end users) do their job?

Data warehousing professionals have spoken for a long time about how 80 percent of their data is unstructured and stored in text, memo, blob, and clob fields, but all of our reports are based on the other 20 percent.

In many cases, the DW will become the primary source for text analytics tools as well as the main target of the analytics you perform. Just as companies use data mining to pull a set of customer data from the DW, build a model, score the customers, then load that score back into the DW, text analytics tools will be used on that voice-of-the-customer data (among other sources). Many of the tools include reporting tools, but the most value to the enterprise comes when we link the categories data back to all the structured data we report on today.

What data should I analyze with text analytics? What data isn't suited to this technology?

To quote almost any consultant I have worked for the answer is that it depends on what business questions you are trying to answer. If you are working with the marketing team and looking to monitor/track the brand of your companies, social media is going to be a key input. If you manage a call center and want to understand the main reasons customers contact you, it will be case notes from your CRM.

As far as what data isn’t suited I continue to see more companies finding more applications for this technology which is very exciting, but I would caution BI pros that just because text analytics tools can analyze all the available unstructured text about your products or company doesn’t mean you should.

Start with one or two sources and use those to find value in your organization, then start adding more. If you start looking at everything all at once, you will overwhelmed and be pulled in too many different directions.

I don’t have a Ph.D. in linguistics. How hard is it to perform text analytics?

If you can use Google, you can use most text analytics tools. Yes, the interfaces are a little harder, but being able to look for "this" or "that" AND "the other" but NOT "these" is about all it takes.

Obviously all of these tools get more sophisticated in their approaches from there, so as you get more familiar or skilled, you will start exploring the tools in the arsenal and maybe -- just maybe -- you will end up talking with that Ph.D. in linguistics and you might understand one or two things that he says.

What mistakes do enterprises typically make when they undertake a text analytics project?

In the excitement to start using their new tool, they go after the largest, most complicated, hardest source first. If you went out and bought a bunch of tools, would you start by building a house first? No, you might start with a dog house, then a shed or garage, and as you started to understand things like plumbing, wiring, and/or structure you would try to build a house.

What best practices can you recommend to avoid these problems?

Have a set of business questions you are hoping to answer. Make it somebody’s job -- text analytics isn’t a Friday afternoon experiment. It takes focus and dedication to get off the ground, both from a technology standpoint and a cultural one.

What's the average duration of a typical text-analytics project, and what is the typical ROI?

My very first project took almost six months, mainly because the business couldn’t agree on what the correct categories should be. I have also done projects in 60mins, when the set of data is fairly small and the model is simple.

The return on your investment can be measured in two ways. First, how many people does your company have doing this manually (believe me, there are people performing the analysis manually). You need to be able to repurpose these people, but allow them to spend more of their time on insight/analysis instead of manual categorization of data.

Second, have you provided your enterprise the quantifiable data workers need to properly prioritize projects?

The ROI approaches are very similar to justifying the needs of building an enterprise DW: by using automation and providing the business with reliable data to make informed decisions, your enterprise will be more efficient and effective in delivering better products/services to their customers.