Q&A: Getting the Most from IT Automation
How can IT do more with less? By automating tasks. We explore the financial and productivity benefits, look at the downsides of automation, and examine how Big Data, virtualization, and cloud computing benefit from automation.
How can IT do more with less? By automating tasks. We explore the financial and productivity benefits, as well as the downsides and misconceptions of the technology. We examine how Big Data, virtualization, and cloud computing benefit from automation with Ben Rosenberg, founder and president of Advanced Systems Concepts, Inc.
Enterprise Strategies: Why has IT automation become such a hot topic for 2012?
Ben Rosenberg: Complexity is on the rise. With global commerce causing many organizations to conduct business on a 24 by 7 basis, IT organizations are operating in increasingly complex environments. For IT managers and developers who have to manage workflows and processes that span technologies that were never designed to work together, they're often forced to rely on custom scripting and/or manual intervention that can be time consuming, error prone, and expensive.
You add cloud computing and virtualization into the mix, IT budgets that are staying relatively flat, and the increasing costs of managing IT and it quickly becomes clear that IT is being asked to do more with less. IT automation solutions are becoming a catalyst for providing IT with a solution to streamline operations and increase productivity that reduces costs.
IT automation is a broad term; what do you mean by it?
IT automation is a term that has its roots in batch processing and job scheduling. IT automation is also broad in terms of its purpose. Today, two main IT areas employ the term for somewhat different ends. IT application developers are interested in IT automation in terms of business process automation. IT operations are interested in IT automation in terms of the repetitive nature of many of the administrative tasks performed. Of course, many times the areas overlap -- in particular the Operations group because their charter is to ensure the reliability and timeliness of the business processing they oversee.
Over time, the introduction of distributed computing meant scheduling jobs over a broad set of machines and platforms. Shared data and interdependencies between systems and technologies meant that the need for a centralized scheduling solution. The result: distributed job scheduling transformed into workload automation. Just as distributed systems signaled the need to coordinate different forms of job scheduling via workload automation tools, the integration of run book automation, or the automation of IT management processes such as server provisioning, with workload automation meant that workload automation solutions had to mature into a single, integrated automation solution that brought the various forms of IT automation under one roof.
Cloud computing and virtualization bring together yet another form of automation and infrastructure management into the mix. The result is taking an integrated, centralized approach to workload, run book, and cloud/virtual automation which is now commonly referred to as IT automation or IT process automation.
From a vendor perspective, product offerings have matured with the marketplace to mimic these changes and have resulted in advanced solutions that allow for the creation and management of workflows and processes that span job scheduling, workload and run book automation, and cloud/virtual management from within a single user interface. These solutions also provide a level of "intelligent automation" that includes event processing and workflow triggers that allow the solution to use a specific event, such as the arrival of a file, to automatically start a process without manual intervention.
What are the financial benefits that automation can deliver to an IT organization? What are the biggest benefits for IT?
The business benefits are numerous. Reducing costs of operations is one. The number of disparate technologies comprising today's IT environment is growing. IT automation can drive efficiency and reduce the costs of operations through the automation of time-consuming and resource-intensive processes. Improving service levels is another. The ability to execute mission-critical business processes are often reliant on IT's ability to deliver that resources necessary to execute them.
IT automation solutions assist in improving IT service levels and meeting service-level agreements (SLAs) through the automation of resource management and "intelligent automation" or the ability to execute workflows based on historical performance analysis.
Finally, IT automation allows for the alignment of business IT processes for improved flexibility. In today's 24/7 business world, businesses have to be more agile when it comes to building and adjusting business and IT processes to respond to competitive and customer demands. IT automation can provide an organization with a single automation engine by which to build and modify workflows quickly and seamlessly to better align IT with business needs.
For IT, there are numerous benefits. Scripting takes time, is error prone, and requires constant updating by its original author. IT automation can reduce the reliance on scripting, reducing the errors associated with scripting and saving time for IT staff. It can also reduce what I call "slack" time in IT operations. Relying simply on data and time scheduling to trigger jobs can waste time and reduce server utilization/resource management. IT automation solutions provide event automation to allow users to execute workflows based on specific events, whether they be downloaded, memory availability, or even historical performance of workflows to take into account peak operating periods for IT resources.
Are there any drawbacks/downsides to or misconceptions about automation?
As with all large systems, it is important to make careful plans. Many times "islands of automation" are created, sometimes inadvertently, which further complicates IT and application development. IT automation should be thought of as a "utility" in which both related and standalone systems are enabled.
Of course, this requires that the product chosen for IT automation be robust enough to cover a wide variety of platforms, systems, and both old and new technologies. Products that allow for integration of older and/or product-specific scheduling support the best of both worlds. Monitoring can be performed from a central location without the necessity of migrating the older systems to the new product and platform. For example, SAP jobs could be executed under SAP but controlled and monitored through a more robust and interconnected IT automation platform.
The concept of "Big Data" and the challenges associated with data warehousing/data integration is another hot topic for 2012. How can automation help address/deliver Big Data?
The democratization of analytic and BI solutions and the fact that IT departments are continually moving towards distributed computing models has meant an increase in the volume of data and the number of data sources. You add to that the idea of "Big Data," complexity of formats, and speed of delivery of data and you quickly realize it's starting to exceed the capabilities of traditional data warehousing and data management tools, and the need for IT process automation to automate and execute the integration and movement of data between these disparate data sources is more important than ever.
Today, many of the tools used to automate the processes to update data stores have significant limitations. For example, a DBMS has job-scheduling capabilities, but DBMS options typically focus only on data maintenance. Data integration solutions such as Teradata and Informatica are excellent solutions, but their scheduling abilities are typically limited or "closed" to automating tasks only within their own environments, not others.
IT automation will become a cornerstone in allowing IT operations to efficiently integrate and manage key resources in their data warehousing environments for improved data quality and reporting by providing a single automation engine for improved data quality and reporting. For example, an IT automation solution could be used to develop a workflow that automates the uploading of data via an ETL job from various data sources into a single data warehouse, upload that into a BI solution, analyze it, run reports, and then distribute those reports via e-mail in a specified format, all without manual intervention.
How will virtualized and cloud computing environments benefit from IT automation?
Unlike mainframe and distributed computing systems, virtualized and cloud computing systems have few, if any, resource restraints. Resources can be assembled as needed to meet the task at hand. In the case of hybrid and cloud environments, pricing is a la carte -- and capacity is theoretically limitless. The trouble with underlying management systems is they aren't designed to predict what capacity will be needed. The result is slower completion times of processes run on virtual and cloud environments, excessive physical requirements, and cost inefficiency.
IT automation solutions will become a critical component in automatically allocating resources to processes being run within cloud or virtual processes where and when they're needed. Taking things a step further, IT automation will provide IT organizations with the ability to use predictive analysis -- or the ability to anticipate resource needs -- to change the way cloud resources are assembled and used. This "intelligent automation" -- the marrying of historical analysis and predictive resource management -- will allow enterprises to leverage historical processing information to proactively configure their public or private clouds to ensure that enough resources have been allocated before running a critical process to ensure its success as well as on-time completion.
By ensuring that systems will be rapidly and accurately provisioned, IT organizations can leverage the cloud to rapidly provision its internal infrastructure for upcoming load increases. Cloud-based systems will positively affect their hosts in terms of utilization and power consumption. Because a customer pays only for what it uses, there is an even greater incentive to obtain incremental resources in real time and to disengage when they're no longer needed to reduce expenses.
How can data center operations improve as a result of IT automation?
The two groups I referred to earlier (application developers and operations) have the most to gain. IT operations would require much less manual intervention for the various administrative aspects that govern data center operations. Monitoring, provisioning of systems, and configuration management could share a common automation platform. Application developers could use enterprise-class IT automation for their business process automation that ensured a highly reliable and secure facility as well as meeting any required SLAs imposed on IT.
What are some of the biggest mistakes IT makes when implementing an automation project? What best practices can you recommend to avoid these pitfalls?
The four biggest mistakes we see are:
- Not creating generic and reusable jobs. For example, an SFTP job that transfers files from one machine to another can be made generic through the use of variables in which the variables hold the actual files to be transferred rather than hardcoding these values.
- Determining what should be a "job step" versus a "job." Most IT automation platforms support job restart/recovery, so the key is to use best practices when deploying related operations as steps within a job, then using multiple jobs to implement non-isolated operations. In other words, it is best to say "what operations must operate together such that either they all succeed or none is processed" when determining a job versus job step.
- Integrating security early in the implementation. Sometimes projects consider security very late in an implementation without considering the impact to overall workflows.
- Governance. What operations should be allowed and who should be able to deploy them?
What are some of the key features to consider when evaluating a job scheduling and workload automation solution?
When looking into an IT process automation solution, you'll find a broad selection of vendor offerings that range in price, functionality, and capability. The key is finding a solution that does the best job supporting your IT environment, both in terms of technologies that are supported and the types of processes that you're looking to automate with this solution. As detailed in your first question, there are many types of processes that make up IT automation, including job scheduling, workload automation, run book, and more. Subsequently, vendors have approached the IT automation market from these various angles; their product's strengths and weaknesses are typically representative of that approach.
In addition, IT automation is about managing complexity in today's IT environment, and making it as simple as possible. Finding a solution that provides a consistent, easy-to-use GUI that leverages capabilities (such as drag-and-drop interfaces) is critical in saving time and building accuracy and efficiencies into constructing workflows.
Finally, another key aspect is ensuring that the IT automation solution provides a single automation framework. Several vendors have acquired different automation solutions over time. Make sure that their various offerings have been brought under a single umbrella and can share data and other interfaces through a single framework. If the idea behind IT process automation is to provide a single, IT automation solution, you want a solution that's been built from the inside out to create a single automation engine with consistent user interfaces and process modeling.
What is Advanced Systems Concepts, Inc. and what does your product, ActiveBatch, do?
Advanced Systems Concepts, Inc. is the developer of ActiveBatch Enterprise Job Scheduling and Workload Automation, which allows IT organizations to develop and roll out end-to-end workflows faster and more reliably than relying on custom scripting or "closed" scheduling systems. ActiveBatch's Integrated Jobs Library has over 100 template job steps, allowing a user to drag and drop job steps into organized workflows that integrate business and IT operational processes. The Integrated Jobs Library and ActiveBatch Extensions support dozens of applications, databases, technologies, and services, including Oracle, SAP, Microsoft, PeopleSoft, VMware, Teradata, Web services, stored procedures, and .Net Assemblies.