In-Depth

Q&A: Charting the Course of Innovation

What new technologies are affecting BI professionals, and how can we be prepared for these changes and the innovations they bring?

• By James E. Powell
• 04/10/2012

From Big Data to mobile BI, technologies that affect BI professionals are changing rapidly. Innovation is the subject of the opening keynote at the TDWI World Conference in Chicago on May 7. We discussed the subject with the keynote speaker, Shawn Rogers, to understand what BI trends are the most important and how they’ll bring innovation to our work.

BI This Week: Why is it more important than ever to plan for the future with regards to our BI/DW environments?

Shawn Rogers: In the all my years in this industry, I don’t believe I have ever seen such dramatic change in technology and culture surrounding business intelligence ecosystems. New analytic opportunities have forced IT and business practitioners to stop and appraise where their strategy is heading and how best to address the inevitable challenges that accompany these new opportunities.

In past years, the enterprise data warehouse (EDW) was the focal point of our business intelligence and data warehouse strategy. Most companies built and maintained an architecture that serviced the reporting and early analytic needs of their end users. Many companies experienced early growing pains with architectures or solutions that wouldn’t scale, but overall data warehousing technology did a fine job meeting most workloads that were required.

Today, the demands of a maturing user base have stretched EDW architectures beyond their limits and have forced a paradigm shift away from the central EDW as the primary data structure to an environment that is driven by the practical approach of matching data and workloads to the best possible platforms to meet the analytic needs of the users or data consuming applications.

This shift has opened the door for analytic database and appliance technology, Big Data, mobile platforms, and cloud technology. Each of these has found a home along side the EDW in our data management landscape, offering new ways to solve problems but adding new challenges to managing what was once a highly focused environment. It’s critical for business intelligence and data warehouse professionals to research these new opportunities, understand how and where they might fit into their specific data landscape, and implement what will add the greatest value to their business.

How are these new technologies driving change in our BI and data management landscape?

Most companies have already created an environment that delivers analytic capabilities. Their systems span data acquisition, data management, business analytics, knowledge delivery, and actionable intelligence.

Figure 1. Enterprise management business intelligence continuum

Each new technology requires a plan for change across many of these disciplines. For instance, adding cloud based technology to a BI environment may have an immediate positive impact on speed to implementation, upfront capital expenditures, and (in some cases) overall project adoption. At the same time a strategy for data acquisition/integration will need to be implemented with special attention to details surrounding security, governance, and regulatory issues.

Traditional data management tools are not necessarily suited for cloud interaction, and managing relationships with cloud vendors will demand a new skill set from IT management personal responsible for running these projects. In each case, as new technology is adopted that will impact the five major areas of business intelligence and each will need a strategy for success.

How are these new technologies driving change in our BI and data management landscape?

There are several that are important. Cloud, Big Data, mobile, and analytic platforms all offer significant impact to companies who adopt the technology. Working closely with end users will help to identify which of these will deliver the largest value for your specific circumstances.

In some cases, these new platforms will merge to deliver hybrid solutions that combine technologies. Big Data analytics supported by cloud infrastructures are a good example of data and workload finding the best possible platforms to meet specific use scenarios. Companies who embrace adding new platforms to their data management environments will benefit the most.

Big Data may deliver the highest innovation impact. Adoption is still early, but the ability to leverage data in high volume, at high velocity, from a variety of sources and structures is a compelling recipe for complex analytic innovation.

Big Data isn’t necessarily a new technology, but advances in computing power and reduced adoption costs have allowed companies to add it to their data line up. Early Big Data projects were done with super-computing platforms and were so expensive they generally required federal grants to operate. Today, Big Data analytics is executed on commodity hardware and often uses open source software as a foundation. Hadoop is a leading software framework for Big Data and offers many answers to the challenge, but Big Data can also be found in traditional RDBMS systems and analytic platforms. This is an excellent example of why it’s critical to strategically add these platforms to you your existing environments so you can bring all your tools to bear on these problems and leverage the best platforms for the task at hand.

Can you share a Big Data use case example that illustrates innovation?

Supply chain analytics has long been a complex model-driven challenge for companies looking to save money through smarter inventory management. Big Data has opened new horizons for these companies. Introducing social data into the process can add great value to decisions but the volume, velocity, and variety of data from the social sphere can overwhelm most traditional systems.

Retailers are listening to the social signal and analyzing the data flow in Hadoop to determine social sentiment and purchasing indicators. By combining this data with customer information, historical store sales, and current sales, a company can gain excellent insight into how best to stock stores. They’re taking another step forward in the process by integrating these results with product-based RFID and geolocation data that allows companies to take action and apply the sights to the supply chain process while it’s in motion.

This use case is an excellent example of how an expanded data ecosystem can be leveraged to execute sophisticated and complex business analysis. The platforms involved in this scenario include operational systems, EDW data, supply chain application data, and social analysis from Hadoop. The final analytics were executed on an analytic platform and resulted in significant savings for the retailer.

What is primary takeaway for innovation in business intelligence?

By accepting that our once EDW centric world is quickly expanding and identifying the best of the new technologies to add to our environment, a company is building a foundation for innovation. Remember to be selective and allow your users to drive the overall direction for new technology adoption. As you address these new opportunities, remember to plan for how it will affect each area of your BI abilities.