In-Depth

Q&A: Integrating Social Media and BI

We explore the benefits and drawbacks of integrating data from social media with your existing BI investment.

• By James E. Powell
• 05/11/2011

Can social media really make your business intelligence better? What are the drawbacks, and what data should you try to integrate? How do you incorporate the data in reports, and will this truly give your enterprise a competitive advantage? For insight and answers, we turned to a company with experience in this process, arcplan, and its president, Roland Hoelscher.

BI This Week: What are the benefits of integrating social media information into BI applications? What are the drawbacks?

Roland Hoelscher: I think many businesses are starting to recognize that social media tracking needs to be part of their overall business strategy, whether or not they've considered integrating social data into their corporate BI systems. Social media data is like any other data your organization might collect except that it's often unstructured and it comes from an external system. Now we're starting to see a lot of companies realize that there is so much that can be gained from viewing social media data as another data source in their BI systems.

By collecting and analyzing social media data alongside the rest of their corporate BI, companies get an enhanced view of the people who purchase (or don't purchase) their products and services. It offers an understanding beyond "Product X sells well in the Midwest." Social data can explain why Product X sells so well in the Midwest, and can even help you spot early trends that can drive product development, and product delivery, marketing messaging. A BI application will give you a visual representation of this data, making it easy to identify these trends.

One drawback is the sometimes difficult task of marrying your qualitative social media data with the classic quantitative data that's housed in BI systems. However, as social data becomes increasingly important to many organizations, it is possible to bring the two together, especially if your BI system can manage both structured and unstructured data.

What are some of the misperceptions about social media integration? What benefits do companies mistakenly think they'll gain?

Most companies that implement business intelligence are likely see ROI fairly soon -- revenue enhancement, cost reduction, etc. -- so they may think that tracking social media data is also going to bring them an ROI that's quantifiable in dollars and cents. Some companies will have to live with the fact that re-tweets and YouTube video views will always be leading indicators that may bring future financial gains. In addition, you'll only get better insight into your customers and prospects if you map your customer interactions on social media with your CRM records, thereby putting them into context of existing relationships. This is often no small challenge.

What types of data from social channels should companies track? What information should be ignored?

With over 100 million active blogs, more than 65 million tweets per day, and countless Facebook status updates, there's a lot of information out there for you to potentially track. What you end up tracking depends on the goals you're trying to meet with your social media efforts. We suggest that companies explore some of the free or self-administered tools on the Web to derive a list of metrics that can always be revised if you find that it's not quite the right data.

Here are some guidelines we suggest to our clients who want to begin social media monitoring in arcplan:

• If your goal is branding and awareness, then measure the changes in your "share of voice" over time. Share of voice is brand mentions divided by total mentions (where total mentions are those of brand and your competitors' brands). Lower-level metrics to track include your number of Facebook fans, Twitter followers, and blog subscribers.


• If your goal is customer/prospect engagement, there's a metric exactly for that. "Audience engagement" correlates to comments + shares + trackbacks divided by total views. You can track this for specific campaigns generated by your marketing team or for hot issues generated by your audience. You can also track more granular metrics such as your number of active followers, conversation reach, comments on your blog, and views of your YouTube videos.


• If your goal is improving customer service, monitor negative customer sentiment and complaints, then respond to them in a timely manner. Measure the time to resolution and post-support satisfaction scores.

Isn't there a lot of bluster on social media sites? How can you filter out this content from the valid content you want to include in your BI app?

It's true -- social media content needs to be filtered and "massaged" in order to be meaningful. Depending on the media, re-postings or copies of original text in answers need to be subtracted from the content. Ambiguous text data needs to be consolidated as well -- the process is called sentiment analysis. There are specialized vendors such as Cymfony and Converseon that use technology or "human analysis" to crawl the Internet, extract relevant social media data, and categorize it. Most of them also store this data on their own premise or provide excerpts to clients for further analysis. Using a vendor to aggregate social data is probably the best option for most companies -- then they can simply input the data into their BI system and analyze it alongside other metrics, such as sales volume or financials.

There are many social media monitoring vendors you can choose from, depending on your company's goals: Do you want to analyze social media data to be reactive to the market or proactive? For public relations or customer insight?

How long have business intelligence tools been able to integrate data from social channels into traditional corporate BI applications?

As long as social media monitoring service providers have been around. These vendors have been creating databases with historic detail about social media activities for about two years. These databases can easily be accessed by business intelligence tools for analysis within dashboards and reports. Such data collection can amount to huge datasets, by nature of the billions of social media activities. BI tools are the most natural solution to handle analysis of this type of data volume, as BI vendors have decades of experience handling large data volumes. Though this doesn't exclude the need for a human touch when it comes to analyzing social media data, whether that comes from your social media monitoring service or internally at your organization (especially to derive sentiment, which is not an exact science).

How is this social media content used? Is it displayed as a field on a report? Can users filter for key words or phrases (and how do they know what words to select)? Given the variability of freeform text, how can social media information be used?

First and foremost, social media data needs to be "standardized" so it has a common structure. Terms and phrases need to be adjusted to a common glossary, which is also where your social media monitoring vendor comes in handy. Storing this data over time -- making it a historic archive of social media activity -- allows for different types of analysis -- real time, over time, within a given period, etc.

Typically we suggest identifying a set of key performance indicators (KPIs) such as those I've mentioned that align with business objectives. These social media KPIs can then be analyzed next to traditional KPIs such as ROI and profit margins to associate social media activity with sales volume, revenue highs and lows, and other relevant metrics.

Will organizations that embrace social BI be at a competitive advantage compared to companies that don't?

Companies can have a competitive edge if they do not see social media analysis as an information island. This information needs to be reviewed in context with other business information, which is where BI comes into play. If done correctly, integrating social media analysis and business intelligence gets you immediate insight into Web activities that have an impact on business.

There's another angle to consider here as well. The tools that we're using every day on Web 2.0 and social media sites -- such as rating, tagging, commenting, and search -- can also be thought of as "social BI." Why not take advantage of these existing mechanisms in the context of your BI system to allow the best data and the most useful reports to rise to the top? Taking one of the principles of social media (everyone with Internet has access) and applying it to BI, which is typically only used by 15 to 20 percent of an organization, means information access for everyone, translating into better decision-making. You're surely at a competitive advantage if your employees are making the best decisions on matters that align with corporate objectives or that affect the bottom line.

What should organizations consider before moving forward with implementing social media monitoring?

They should consider the costs of monitoring social media data and the costs of not doing it. Will your competition pass you by because they used social data to get to know their customers and prospects better? Did they take customer feedback gained from social channels into account in their latest product release and your company didn't?

Something else to consider is how to manage your data. For smaller companies that need to monitor just a handful of social media platforms, a service provider with standardized reporting can work. If you need to monitor hundreds of blogs, forums, Twitter-ers, etc. -- typically B2C companies -- your social data should be handled data warehouse-style, which provides a consistent and persistent view.

What are the biggest mistakes organizations make in their social media/BI projects? What best practices can you recommend to overcome these problems?

We've seen companies decide to measure every metric they can think of, down to the most trivial digital activities that have no business impact. Measuring everything often means you can't act on anything, and collecting data just for the sake of collecting it is useless. Deciding on the most meaningful KPIs upfront is important. You can always go back and add more later on, or revise your list in the future.

The good thing about integrating your social media data into your business intelligence means that you can customize the data however you like. If you get a set of granular data from your social media monitoring vendor and you want to perform calculations similar to those I've described, you have the flexibility to do that in your BI tool.

What products or services does arcplan offer to integrate social channels with BI?

We have a couple of products that fit the bill. Our BI engine, arcplan Enterprise, allows organizations to take their social media data and report on it, analyze it next to other corporate data from various sources, and distribute it to desktops and mobile devices so it becomes meaningful and actionable. Our next-generation BI search and collaboration tool, arcplan Engage, takes advantage of learned Web 2.0 and social media skills including rating, tagging, commenting, and search, and makes data widely available to improve decision-making. Our company focuses on making use of a company's existing investments -- whether in social media monitoring or traditional data sources like ERP, relational databases, or OLAP -- and integrating them into a single, streamlined interface.