Q&A: Working with Multiple Job Schedulers
Dealing with multiple job schedulers can be a challenge. We explore the problems, how IT goes about solving them (and how to avoid the typical mistakes), and the solutions available.
It's tough to keep all your jobs running smoothly. You have to deal with dependencies, production windows, and abends. Now imagine how much more complicated things become when you have more than one tool scheduling jobs across your enterprise. That's just the dilemma we wanted to learn more about, so we turned to Mark Loehr, director of professional services for Stonebranch, a provider independent scheduling agents solutions.
Enterprise Strategies: Why do companies have multiple job schedulers?
Mark Loehr: There are often many contributing factors, and most companies have fallen victim to more than one:
Acquisitions: For many companies that integrate acquisitions, consolidating IT tools such as scheduling is not their highest priority. Although it will save money in the long run, it is a big project that poses risks. Consequently, scheduling staff members are burdened with managing multiple schedulers.
Departmental purchasing: The lack of centralized purchasing or company-wide standards often results in the independent departmental acquisition of different job scheduling solutions that are unable to interoperate with each other and with existing tools.
Access to tools: Application departments circumvent standard tools and use easier scheduling tools such as Cron. It is simple and easy to set up a Cron tab or schedule a process in Windows, but it is only a quick fix. Moreover, these one-off fixes decrease overall business visibility.
Built-in application capabilities: This is one of the biggest culprits. Companies deploy applications and tools (such as backup products) due to the perceived benefits of a self-contained system that can be managed by its own tools. This thinking ignores the fact that most applications are not self-contained. In fact, they "touch" other applications by exchanging data and have dependencies based on the overall business workflow. Implementing these "point" solutions prevents the ability to control business workflows from end-to-end.
Lack of management mandate: Without clear mandates from senior management, there is no ability to enforce centralized control. Thus, departments and applications developers will take the path of least resistance.
In many cases, sooner or later, IT operations inherit these different applications and tools and are stuck managing them.
In what environments are multiple job schedulers most common?
One of the most common is the heterogeneous operating infrastructure. This environment results in a company having multiple schedulers due to legacy operations on z/OS mainframes and Windows, UNIX, and Linux distributed operations.
Additionally, you see multiple scheduling systems in companies that are not centrally managed or use third-party vendor applications. These applications either have limited built-in scheduling capabilities or have business partners with scheduling systems that they recommend for their applications.
Staff members with various backgrounds also play a role. Let’s face it -- people tend to prefer things that are familiar to them. Scheduling systems are no different. This can lead to multiple scheduling systems based on personal preferences.
Finally, companies that have not fully automated their business operations are more likely to have disparate scheduling systems. Business groups that work independently from each other seldom have common scheduling toolsets.
What problems arise from having multiple job schedulers? What factors or technologies -- for example, new Web technologies -- exacerbate the problems?
A fundamental problem with multiple job schedulers is that they provide a view of operations based only on what is defined in each separate scheduling database. Therefore, business and IT only see a partial technology view and not an end-to-end view of the information the business needs to operate.
This lack of visibility is compounded by the need of the business to maximize the use of newer Web technologies while continuing to process its other workload based on more established technologies. To maximize its competitiveness, the business needs visibility based on how it operates regardless of the technology used.
Because most Web technologies were not developed with integration into scheduling systems as a core requirement, integration can be a challenge that is made even more difficult if multiple scheduling systems are involved.
Undoubtedly, one of the most significant issues with a multiple scheduler environment is a higher operational cost. Each scheduling system has its own scheduling engine, agents, terminology, and staff requirements. They are not designed to be interoperable. The bottom line: multiple scheduling systems mean more complexity and more products to learn, implement, and maintain. Therefore, they require more people to operate. This adds up to greater cost and less visibility.
How does a disparate job scheduling environment affect the business?
One of the most devastating effects on businesses is the delay of mission-critical business information. As I mentioned, having multiple scheduling systems lead to complexity. It also leads to multiple points of failure and delays in supplying the business with the information it needs to competitively operate.
Additionally, unnecessary complexity leads to more money spent on IT. There are always investments in new technology needed to help businesses stay current and compete. The problem is that the money available to IT is limited. Money that could be spent on staying competitive is instead spent on operating a complex, multiple scheduling infrastructure, depleting the budget for more revenue-driving technologies.
Moreover, the complexity of multiple scheduling environments affects the mindset of IT. The effort required by IT to maintain a multiple scheduling environment requires it to focus resources on managing day-to-day issues instead of preparing for the evolving needs of the business.
What factors affect IT’s ability to solve these problems?
Lack of funding. With the current economic environment, IT funding is under increased pressure. Sometimes it is difficult to meet the day-to-day needs of the business. The thought of spending money and resources today to save money tomorrow is a luxury that IT does not have.
Another problem is that there appears to be limited options. One option is to keep the status quo and continue to invest in disparate scheduling systems that provide a complex environment and limited business-level visibility. Alternatively, you can pick one scheduling system to consolidate your schedulers. The first option is non-strategic and expensive in the long run. The second option is strategic, but potentially disruptive to the business and very expensive.
Solving the problems of multiple scheduling environments is about risk management. Are the cost and problems with visibility more risky to the business than the potential disruption associated with change? This is a question that each data center must answer before making a decision.
Lastly, in a multiple-scheduling environment, IT is often more focused on products instead of the core technology required to meet the functional requirements of the business.
How does IT typically address the problems from having multiple job schedulers?
One approach is to do nothing. Many IT organizations see no practical alternative than to continue to operate a multiple scheduling environment. However, there are those IT organizations that experience enough pain from having multiple scheduling solutions that they must consolidate. In some cases, this results in one scheduling system for the entire environment. In other cases, consolidation just means fewer scheduling systems.
Others take a “fresh start” approach. These IT organizations stop to rethink the business requirements. They then go back to the business and update requirements and search for solutions that best map to those requirements.
What mistakes does IT make, and what best practices can you suggest to avoid or correct them?
The first mistake is to do nothing. If you continue to operate multiple scheduling systems, IT must invest in scheduling system solutions that are proprietary and don’t interoperate with other schedulers. The current inefficiencies in multiple scheduling systems are not solved. Later, if you find that the problem must be solved, it becomes even more expensive to consolidate because you lose more of your investment.
Another mistake is consolidating to a single vendor. Vendor lock-in can lead to higher costs due to price increases and licensing structures when it’s time for your contract renewal. Additionally, IT is often forced to trade visibility for vendor lock-in. Managing vendor lock-in involves risk management. The central question being, is it more risky for you to tie your scheduling to a single vendor’s solution than it is for you to trust multiple vendors to work and play nice together? Will you take the risk that your operations will suffer due to incompatibilities between multiple scheduling vendors?
Another approach is to consolidate to fewer products/vendors. By reducing the number of products/vendors, business visibility can increase and costs can be lowered. It is, however, only a partial solution. There are still issues with partial visibility and incompatibilities between multiple scheduling vendors.
When taking the “fresh start” approach an IT department takes a fresh look at requirements and matches them to a solution. This often involves buying a new scheduler or scheduling system to displace the current multiple scheduler infrastructure. It may be the best way to meet the needs of the business, but it is also the most expensive and most disruptive to the IT operations.
There are several best practices IT can take to smartly navigate through this process. First, understand your realties. Anything is possible given enough resources and time. However, resources and time are two things that IT is sorely lacking.
Next, be sure to look at the big picture and not the quick fix. What is the ultimate goal? What needs to be done to minimize costs and maximize benefit to the business?
Finally, take an evolutionary approach. However you plan to improve your multiple scheduling environments, do it in a series of well-planned steps that allow you to balance benefits and risk.
What solution does Stonebranch offer that addresses the problems of multiple job schedulers?
Stonebranch has taken a different approach to addressing these problems by upgrading your multiple scheduling environments and not replacing them. The Stonebranch solution is a vendor/product/platform-independent scheduling agent. Because Indesca, Stonebranch’s independent scheduling agents solution, works as a standard agent for multiple scheduling systems, IT can minimize the cost and risk associated with maintaining scheduling agents.
Allowing an IT department to consolidate without getting locked into a single vendor, both price and functionality-wise, minimizes risk. With Indesca, you can also re-use your existing technology investments in software, processes, and skill sets. And, by consolidating your multiple scheduling environment, you simplify administration allowing you to focus on business operations.
Stonebranch also provides professional services that can help you with implementation, provide expertise to consolidate your existing schedulers, or help you solve your specific issues.