Q&A: IT Automation's New Directions
The evolution of IT automation, how the technology has changed, where it is today, and where it's headed.
Automation helps IT meet its "do more with less" mantra. How has IT automation evolved, how is it used today, and where is the technology headed? For answers, we turned to Ben Rosenberg, president and founder of Advanced Systems Concepts, Inc.; the company's ActiveBatch solution automates IT and business processes.
Enterprise Strategies: How has IT automation evolved from traditional job scheduling/batch processing into the solution it is today?
Ben Rosenberg: Over time, the introduction of distributed computing meant scheduling jobs over a broad set of machines and platforms. Shared data and interdependencies between systems and technologies meant the need for a centralized scheduling solution. The result: distributed job scheduling transformed into workload automation. Just as distributed systems signaled the need to coordinate different forms of job scheduling via workload automation tools, the integration of runbook automation, or the automation of IT infrastructure management (such as server provisioning), with workload automation meant that workload automation solutions had to mature into a single, integrated automation solution that brought the various forms of IT automation under one roof.
Cloud computing and virtualization bring together yet another form of automation and infrastructure management into the mix. Running jobs across diverse machines requires resources. Resource allocation, based on finite physical capacity, can present limitations. The flexibility of virtual environments, whether it's a public or private environment, presents complexity challenges where automation has found another niche. Bringing the management and provisioning of virtual and public cloud into the equation is an elegant way to automatically allocate resources to workload processing where and when it's needed.
Finally, to better align IT automation with the needs of the business, IT automation solutions have further matured by incorporating the ability to set business policies or service levels to jobs and workloads and automatically provision and manage resources accordingly to ensure they're available for that workload to execute on time.
Overall, the result is taking an integrated, centralized approach to job scheduling, workload and runbook automation, and resource management, which is now commonly referred to as IT automation or IT process automation.
Where is IT automation headed? What additional capabilities and functionality will be its prime focus?
Three of the biggest areas of focus now and for the foreseeable future will be around self-service automation, policy or service-level agreement-driven workload management, and predictive functionality. In many ways, all three are interlinked
Self-service automation is where IT process automation meets business process automation. The idea is what an end user can use an IT automation solution to choose from a service catalog and initiate a process or workflow themselves -- without the need to involve someone from IT operations. Ideally, end users could plan the execution of business processes themselves, and the IT automation solution will self-provision resources and load applications to achieve business objectives.
Policy based or SLA-driven workload management means the ability to tie workflows to a specific business deadline or service-level agreement (SLA). The reason for this is because ultimately the workflows IT is running are to achieve a business goal. Being able to tie that business deadline to the workflow has become critical. From a functionality standpoint, this means more advanced monitoring and alerting capabilities to notify users of an impending SLA breach and critical path analysis to give workflows that are tied to an SLA priority in terms of execution and resources.
Playing into both of these is predictive scheduling and resource management, or proactively provisioning virtual and cloud-based resources in advance of a workflow executing. You're combining historical analysis and workload forecasting to ensure that adequate resources are reserved for the successful execution of workflows tied to an SLA. If your IT automation solution has a history of when workflows execute, of their runtimes, and the resources they consumed, why not view the upcoming schedule and use this historical information to allow your automation solution to make "intelligent" decisions about resources required and allow it to automatically provision them? If certain workflows are tied to SLAs, give them priority over less-critical jobs.
Many IT automation vendors now offer these capabilities, but expect them to continue to expand on these areas moving forward.
What role does IT automation play in managing virtual and cloud environments today, and how do you see that changing in the next 1-2 years?
IT automation is changing the way virtual and cloud-based resources are managed by tying their provisioning and allocation to specific workflows. In many regards, IT process automation is altering the management of cloud and virtual environments just like workload automation altered traditional date/time scheduling. It's making the allocation and management of virtual and cloud resources more dynamic, based on events, policies, or demand as circumstances require.
As I mentioned, IT automation solutions can both predict when an increase in workloads will occur and proactively provision virtual and cloud-based resources in advance of that spike. The real benefit, in addition to ensuring resources are allocated for the completion of workflows, is resource optimization and cost savings. By enabling an IT automation solution to take over the provisioning and management, virtual and cloud resources are only being brought online only when and where they're needed and then being taken down with "just-in-time" provisioning. Cloud automation matches workload demand with capacity supply in a "pay-for-what-you-consume" model. As a result, it means ensuring that the capacity curve tracks to the workload demand curve, effectively eliminating the costs associated with idle assets between workload bursts.
What role will predictive analytics and predictive scheduling play in IT automation in the next year?
Predictive analytics will become a key enabler in successfully tying workload processing with business success. IT environments are growing in complexity, SLA delivery times are getting shorter, and business demands are becoming increasingly event driven. The growth of cloud and virtual environments has only added to the complexity. Being able to coordinate scheduling across various IT applications, data sources, and technologies, all the while managing the resources these workflows consume, is going to require the ability to "look ahead" based on how these workflows have executed in the past.
Analytics will be used to optimize workloads and satisfy SLAs by using historical analysis to determine the execution patterns and statistics about jobs that have run and make future predictions about possible execution times and durations for future batches. Moreover, analytics can interact with physical, virtual, and cloud resources to ensure they're prepared and jobs are executed to eliminate wait times.
Why is reporting becoming increasingly important for IT automation solutions?
If the idea of IT automation solutions is to provide a single point of control for building and executing workflows, a single point of monitoring is also required, which is why the role of reporting and dashboards has becoming increasingly important to workload automation solutions. Just like predictive scheduling will be the vehicle by which IT can better understand performance and allocate resources accordingly, reporting will be the vehicle by which a user can get the "big picture" to better understand the performance of executing jobs and batches, see if SLA deadlines are being met and which are being missed, and, as a result, what steps are required to ensure they are met.
The idea is to provide IT operations with a central dashboard from which to monitor key metrics and key performance indicators (KPIs), such as determining peak operating periods when workflow execution and resource usage is high, thus enabling a user to take action such as allocating additional resources or altering workflow runtimes.
What issues are companies facing with big data, and how can automation help?
The size, complexity of formats, and speed of delivery of data is starting to exceed the capabilities of traditional data management technologies. In recent years, the democratization of analytics and BI solutions has become a major driving force for data warehousing, resulting in more data sources and the use of self-service data marts. Along those same lines, use of big data is driving the idea that enterprises are replacing the single data warehouse model with a model that means bringing together information from multiple sources as needed. Combined with the fact that enterprise IT departments are continually moving towards distributed computing environments, the need for IT process automation to automate and execute the integration and movement of data between these disparate sources is more important than ever.
Many of the leading enterprise data warehousing and BI solutions come equipped with job scheduling and automation capabilities, but these are typically limited in their respective capabilities to scheduling on just that system, leaving IT to rely on error-prone and time-consuming scripting to pass data and manage dependencies across the vast array of BI and data warehousing solutions.
Workload automation solutions can provide a single automation framework to integrate data pathways into automated, repeatable, and schedulable processes that deliver a high degree of control over all steps in the BI/data warehousing process. The benefits are numerous, including being able to more seamlessly update data warehousing and downstream BI for improved reporting and to better meet business demands. Rather than taking months to update an end-to-end process, resulting in "lag time" between when a business requests new reports and when IT can deliver on it, an IT automation solution provides the agility and centralized control to update processes quickly and seamlessly.
How and why are IT automation solutions playing a bigger role in automating business process automation?
This really gets back to what we've talked about regarding SLAs and policy-driven workflows. Business process automation and IT process automation are now really two sides of the same coin. At the center of this intersection are workload automation solutions, which are executing and managing the workflows that integrate the various mission-critical application and technologies that business policies are now so intimately tied to.
One of the keys for any workload automation solution is to provide the integration points with these systems to enable IT to easily construct and automate workflows that pass data and manage dependencies between these technologies. For years, IT has been relying on custom scripting to accomplish this, but scripting can be time consuming and prone to error. The demands of today's businesses require a simpler, more elegant way to build and manage these workflows, and that's the job of workload automation.
What products or services does Advanced Systems Concepts, Inc. offer for IT automation?
Advanced Systems Concepts, Inc. offers ActiveBatch Enterprise Job Scheduling and Workload Automation. ActiveBatch is what we call an IT automation solution without boundaries. It provides a single IT automation framework for job scheduling, runbook and workload automation, resource management, and business process automation. Users can develop and deploy end-to-end workflows faster and more reliably than relying on custom scripting or "closed" scheduling systems. ActiveBatch's Integrated Jobs Library has over 100 production-ready job steps, allowing a user to drag and drop job steps into organized workflows that integrate business and IT operational processes. The Integrated Jobs Library and ActiveBatch Extensions support dozens of applications, databases, technologies, and services, including Oracle, SAP, Microsoft, PeopleSoft, VMware, Teradata, Web services, stored procedures, and .Net Assemblies.