In-Depth
5 Steps for Auditing Outsourced Operations
How to audit the quality of your outsourced operations.
Editor's Note: IT Auditing: Using Controls to Protect Information Assets (Second Edition) authors Chris Davis and Mike Schiller (with Kevin Wheeler) provide a handbook for creating an organization’s IT audition function and for performing their IT audit. This excerpt examines the steps you should take to monitor the quality of ongoing outsourced operations, including examining governance and disaster recovery processes.
Step 1: Review and evaluate your company’s processes for monitoring the quality of outsourced operations. Determine how compliance with SLAs and other contractual requirements are monitored.
Although you have hopefully dictated expectations in your contract, unless you monitor for compliance with those expectations, you will have no way of knowing whether they’re being met. If those expectations are not met, the availability, efficiency, and effectiveness of your operations and the security of your systems and data can be impacted.
This step is applicable to all forms of outsourcing.
How: Review the contract to understand requirements. Interview your company’s internal management to determine their processes for monitoring that each of those requirements is being met. Obtain and review metrics, slides from operations reviews, and other materials, and compare the results to the requirements stipulated in the contract. Where deviances have occurred, review for corrective action plans and evidence that those plans have been implemented and were effective.
If requirements have not been dictated in the contract, determine how the quality of services is monitored and how the vendor is held accountable. The inclusion of SLAs should be a requirement when the contract is renewed.
Ensure you cover the following basic topics in performing this step:
- Availability (such as expected uptime)
- Performance (such as speed of transaction response after the ENTER key is pressed)
- Response time (such as whether the vendor will respond to problems 24/7 or only during normal business hours)
- Issue resolution time (such as how quickly you should expect issues to be fixed)
- Security and compliance requirements
- Other key metrics and performance indicators that can be used by your company to measure the quality of the service
Step 2: Ensure that adequate disaster recovery processes are in place to provide for business continuity in the event of a disaster at your service provider.
Just as with internally-hosted systems, you must to prepare for recovery from a disaster when outsourcing operations. Failure to do so will likely result in extended outages and business disruptions if a disaster occurs with your vendor.
This step is most applicable to cloud computing, dedicated hosting, and offsite service outsourcing.
How: You should expect that your vendor will follow sound disaster recovery disciplines, such as those you would look for when auditing your internal operations. This includes steps outlined elsewhere in this book, such as reviewing for offsite backups, up-to-date documented recovery procedures, periodic testing, hardware redundancy, and so on. Your first option should be to determine whether an evaluation of this area is available via a third-party assessment (such as SAS 70). If not, you’ll need to work with your operations, procurement, and legal departments to determine your rights to audit the vendor in this area. Ideally, that right is spelled out in the contract. If not, your company will need to attempt to press for that right, possibly using the next contract renewal as negotiating leverage.
If the area is not covered by an assessment such as a SAS 70 and if you have the right to audit it, you will need to interview the vendor and review their documentation regarding their controls and processes, testing those controls as you’re able. You will also want to see the requirements for disaster recovery controls, including recovery time objectives (how quickly your systems should be back up after a disaster) and recovery point objectives (how many days’ worth of data you’re willing to lose), spelled out in your contract. Determine how the vendor ensures compliance with the requirements in the contract.
While it is important that you understand your vendor’s disaster recovery procedures, you should also expect that your company will have documented procedures regarding how they would recover in the event of a disaster at your vendor. This should include notification and escalation procedures, any necessary hand-offs between your company and the vendor during the recovery, and potential manual workarounds while waiting for recovery. It should also include contingency plans should the vendor be unable to recover for an extended period of time (or ever). Request information regarding the location of your data and regarding any replication in the architecture. If the data and infrastructure are replicated across multiple sites, your vulnerability and need for contingency plans decrease. If your systems are at a single location, it becomes more critical for your company to document contingency plans, which need to include a method for obtaining your data and bringing it back in-house if necessary.
Step 3: Determine whether appropriate governance processes are in place over the engagement of new cloud services by your company’s employees.
Cloud computing makes it easy for business unit personnel to meet their needs without ever engaging corporate IT. Because most cloud services can be accessed via an Internet-connected browser, a business unit can engage a cloud vendor and outsource the systems and data related to one of their business processes without really having to tell anyone else. This has the potential to bypass all of the governance processes normally in place to ensure proper security of company data, interoperability of systems, appropriate support capabilities, and so on.
This step is most applicable to cloud computing.
How: Review company policies to determine whether this topic has been addressed. Policies should be in place requiring company personnel to follow specific procedures when engaging vendors for this sort of service. If this policy exists, review it for adequacy. It should require that IT be engaged and that specific security and operational needs be addressed. Determine how employees are made aware of the policy. Also, determine how the policy is enforced. For example, if your company has a centralized procurement organization that must be engaged to sign contracts and pay invoices, you can use them as the gatekeeper for ensuring that proper procedures are followed for new engagements.
Step 4: Review and evaluate your company’s plans in the event of expected or unexpected termination of the outsourcing relationship.
Your company might terminate the outsourcing relationship in the future for many reasons. The provider could go out of business or discontinue the service you’re using. You could be unhappy with the provider’s cost or performance. You might engage in a new competitive bid at the end of your contract and another vendor may win the business.
If you can’t bring the service back in-house or switch it to another vendor, you’ll find yourself locked in with your vendor, which greatly damages your leverage to influence price and service quality. And if that company goes out of business, you’ll experience significant business disruption.
This step is applicable to all forms of outsourcing.
How: Determine whether your company has a documented plan indicating how they would bring the functions back in-house (or move them to another vendor) if necessary. If bringing the function in-house is unrealistic, you should see evidence that alternative service providers have been identified. Ensure that an analysis has been performed regarding how long it would take to transition the services and determine whether appropriate contingency plans are in place to keep the business running in the interim.
Look for contractual requirements for your vendor to return your data and assets upon request. If this has not been indicated in the contract, the vendor can hold your data hostage or can comingle it with other customers’ data in such a way that it’s nearly impossible to extract your data. Your company should require that your vendor deliver copies of your data to you periodically in an agreed-upon format (one that can easily be ported to a new application). Where applicable, ensure that code is put in escrow to protect against the vendor going out of business.
For IaaS and PaaS, your systems should be developed and deployed so that they are easily portable to new environments. Review your company’s processes for ensuring that portability is a key goal in any development for cloud-based services.
Step 5: If IT services have been outsourced, review the service provider’s processes for ensuring quality of staff and minimizing the impact of turnover. If those services are being performed offshore, look for additional controls to ensure employee attendance and effective communication and hand-offs with the home office.
If service provider employees aren’t qualified to perform their jobs or the provider experiences high levels of turnover, the quality of IT services will obviously be poor. This risk generally increases with outsourced operations, where turnover tends to be higher.
Outsourced operations that are performed offshore contribute to the risks of communication breakdowns and absenteeism that can impact the quality of service received.
This step is most applicable to IT service outsourcing (onsite and offsite).
How: Review the contract to ensure job descriptions and minimum qualifications for each position are documented (such as education level, skills, experience). Pull a sample of supplier employees and verify that these minimums have been met. Review the provider’s employee screening process to verify that appropriate background checks and qualification reviews take place prior to employment offers.
Determine how continuity of services is ensured in the event of turnover of service provider employees. Review staffing assignments and determine whether any single points of failure exist. Review cross-training processes.
Review the vendor’s processes for providing training to update skills and knowledge. Request evidence that the training policy is being followed for a sample of employees.
Review the vendor’s processes for monitoring attendance. This is particularly important if the services are being performed offshore, where absenteeism tends to be high. This should include reviews of physical security logs and system access logs. Request copies of these logs and verify the attendance of a sample of employees.
For offshore outsourcing, determine how appropriate language skills are ensured. This could include a language test with minimum test score requirements defined, conducting spoken and written interviews in the required language, and so on. You should also determine how the inherent complexity of communication and hand-offs is mitigated. Look for the existence of periodic hand-off and status meetings between countries. SLAs should be documented and monitored. An employee of your company at the offshore site (or at least in the same city with easy access to the site) should be available to act as your liaison and perform monitoring and oversight of the operations.
Requirements for all of these items should be dictated in the contract. Review the contract to verify this.
Excerpted from IT Auditing: Using Controls to Protect Information Assets; copyright 2011 by The McGraw-Hill Companies. Used by permission of McGraw-Hill.