In-Depth

Automated Quality During Big Bang SAP Implementation

• 09/01/1999

While those in scientific circles continue to debate theories regarding the creation of the universe, folks at Siemens Information and Communication Networks (Siemens ICN; Boca Raton, Fla.) have discovered, and proved, the truth behind the "Big Bang" theory ... At least the truth when it comes to implementing a quality, across-the-board switchover from its former legacy systems to a full-scale SAP (SAP AG; Mannheim, Germany) system.

The plain and simple fact discovered by the company is that a successful, big bang SAP implementation can only be made possible through the deployment of quality practices and automated testing technologies.

Siemens ICN provides integrated voice, data and video communications networks to carrier and enterprise customers. Last year the company’s 7,000 U.S. employees generated sales of nearly $2 billion. Its parent firm is Siemens, the pioneering electronics giant with more than 400,000 employees worldwide and global sales of over $66 billion.

As a major player in the highly competitive telecommunications industry, Siemens ICN has long been recognized for its quality products and services. The same vision of quality permeates throughout the organization and was a powerful driver for its successful, enterprisewide SAP implementation.

"It was vital that we validated the various SAP application modules and ensured that they could successfully handle our expected volume of transactions," explains Ken Battles, Project Manager/Process Modeler for Siemens. "Manual testing would have been too time-consuming and undependable. Automated stress and performance testing was really the only logical option."

After evaluating several different testing methods and tools, the company implemented LoadRunner (Mercury Interactive Corporation; Sunnyvale, Calif.). LoadRunner is a load testing tool that predicts system behavior and performance. The product exercises an entire enterprise infrastructure by emulating thousands of users to isolate problems, optimize performance and accelerate application deployment.

"Automated load testing with LoadRunner enabled us to detect and diagnose any performance problems long before our production implementation," says Battles. "Based on test results, we were able to rapidly adjust our configurations to ensure the stability of the system and to ensure optimum throughput."

Today’s competitive business requirements demand that tasks be performed faster, cheaper and more efficiently than ever before. These requirements also dictate the need to constantly evolve the business to meet customer demand.

And in the telecommunications industry, competition is fierce and evolution is nearly revolution.

After evaluating methods for improving IT services to the business, Siemens ICN elected to fully replace its aging, mainframe-based legacy systems with an efficient, integrated SAP system that spans all critical areas of the business. According to Battles, Siemens ICN shared the same legacy-system challenges as many other organizations: Throughout the years the applications had grown highly customized and remained unintegrated, resulting in a great deal of redundancy in data and programs. These challenges caused the applications to become unwieldy and made it exceedingly difficult for IT to deliver a timely response to constantly changing business requirements.

"We needed to shorten our logistics cycles and deliver information faster to support the business," says Battles. "Although we had many online systems, we were also heavily into batch processing. We needed to become more interactive."

An additional prompt for the SAP decision was a Year 2000 compliance issue within an existing manufacturing software package. The product had been extensively modified and customized throughout the years and an analysis revealed that the compounded costs for upgrading and re-customizing the package would far exceed any realized value.

The decision by any company to replace existing systems usually has far-reaching implications. However, Siemens ICN went light years farther, not just by opting for an SAP conversion, but by reaching the decision to bring the new system online in a one-time, big bang, enterprisewide implementation, effectively flipping the switch that turned off the old system and launched the new.

"The business was on the old production system one day and the next we were on the new SAP system," says Battles.

With the future of their business depending on a successful deployment, the company launched a project that created the foundation for ensuring quality through automated load testing with LoadRunner.

One of the initial tasks was the formation of a corporate project team chartered with the responsibility of identifying and reviewing the business processes supported by the legacy system. These processes were viewed with an eye toward developing effective migration strategies and implementing quality testing strategies.

For example, while it was possible to perform a straightforward migration into SAP for some business processes, other processes required reengineering efforts to include new, expanded functionality, while others became candidates for elimination.

"Our review of the business processes helped us identify exactly what we needed in the new system and what could be eliminated as we migrated from the legacy system," says Battles.

In some instances, the company performed business process modeling with the ARIS Toolset (IDS Scheer; Chaddsford, Pa.), a group of products that help companies quickly design business processes, measure their impact and apply them across the enterprise.

"The tool proved valuable in helping us map out some of our migration strategies," says Battles. "We plan to extend its usage in future activities."

After the business processes were identified and streamlined, the next step was the creation of a matrix of key transactions with a description of how they would be used in the new environment. The goal of this crucial task was to estimate the volume of online transaction processing (OLTP) that would occur during specific times of the day, and to build the LoadRunner test scripts and scenarios that would emulate magnitudes of processing.

Also included in the matrix were transaction codes, the types of users employing the transactions, an estimate of the number of transactions per hour per user, and the time of day that the transactions would be performed.

The end result of the analysis was the identification of 45 key business processes, an anticipated load of 350 concurrent users out of a total SAP user base of approximately 1,000, and nearly 12,000 transactions per hour against 22 SAP application servers and two mirrored database servers – all of the information required to create quality LoadRunner test scripts and scenarios.

"We wanted to target the transactions that would involve the largest number of concurrent users or require the highest database or CPU usage per transaction," explains Battles.

Although Siemens ICN recognized the importance of performing load testing on the new SAP applications, it was first vital for those responsible for the migration to gain insight on exactly what steps could be used to perform high-volume, enterprisewide testing.

But for Siemens ICN – and just about any company – it’s probably an impossibility to coordinate a large number of users for high-volume manual testing. Instead, by exploiting the LoadRunner Controller component and its Virtual User Generator, the company was able to emulate SAP performance for 350 concurrent users.

"We knew that LoadRunner could automate stress testing activities," said Battles, "but we also needed a better understanding of how to maximize the efficiency of the product in our environment."

The customized education provided by consultants from Mercury Interactive’s Professional Services Team provided the company with the knowledge required to effectively transform the created transaction matrix into the test scripts and scenarios that would provide the most value, coverage and quality. Because LoadRunner requires no programming skills, it was easy to begin creating the appropriate criteria. For example, while LoadRunner automatically created individual test scripts during the capture of user transactions, good stress testing disciplines required those individual scripts to be reusable and included within any number of testing scenarios that would model the future SAP production environment. Using the LoadRunner Controller, it was possible to seamlessly drive the mix of defined Virtual Users through the different testing requirements.

"We needed the scripts we captured to work within the scenarios we created in our transaction matrix," says Battles. "The scripts would be constantly reused to simulate and validate our matrix estimates."

The Scenario Wizard in LoadRunner Controller also provided the ability to establish testing situations for different hardware and network configurations. For some companies, that means stress testing enterprise applications with hundreds or thousands of desktop, terminal, Web and Java clients to see, for example, the effect of Java users on a system when accessing an ERP application over the Web. In other cases it means setting up a scenario that shows different bandwidths to determine what type of response time a user will receive over a 56K connection versus an ISDN or T1 connection.

For Siemens ICN, the Scenario Wizard was used for custom server definitions. These definitions directed LoadRunner to perform scenario testing for specific business processes using specific client, application or database servers. Most important, these scenarios were used to help pinpoint any potential bottlenecks.

"During testing, we could easily add virtual users and scripts, manipulate scenarios, and zero in on specific problem areas."

After performing a few limited tests to grow familiar with the LoadRunner product and processes, the company’s test team launched several full load tests that ran for approximately one hour each. Immediately, the LoadRunner tests produced critical results that required action.

Says Battles: "One of the first things we discovered was that we hadn’t created a proper distribution of users across application servers."

It turns out that some of the transactions that had been earlier identified as "heavy hitters" weren’t used as expected, while others that had been tagged as low usage were the exact opposite. According to Battles, the analytical capabilities of LoadRunner helped the testing team quickly discover that, in many situations, a primary transaction previously identified as high usage actually operated as a trigger for a secondary transaction that was also launched by other transactions.

In effect, many of the secondary, often unseen transactions, were actually the "heavy hitters." "The performance problems signaled that we had to alter the way we were assigning users for high-use modules," he says. "Once we made certain adjustments, we saw a significant improvement in performance."

Battles notes that the ability to identify potential configuration problems with LoadRunner provides the auxiliary benefit of helping to protect the investments made in its server technology. "Instead of rushing out to buy more servers whenever we run into bottlenecks," he says, "we can gain an accurate picture of where the performance problems may reside in the environment and reallocate servers as necessary."

Expanding the use of LoadRunner’s analytical tools, Siemens ICN was able to pinpoint slow-running transactions and exploit various diagnostic techniques to isolate the steps that were taking the greatest amount of time in order to implement the proper remedy.

In one instance, a review of a LoadRunner transaction response time report highlighted that some steps were just taking too long to process. A quick review of the SAP programs in question revealed that query processing was based on keys and that those keys weren’t present in the database index, thereby forcing the program into a much slower, sequential read mode.

"It was a matter of re-indexing some of the databases," says Battles. "Our results were much improved during later tests."

Following this basic stress testing process, the team also performed a few tests that added either WAN traffic or a small number of PCs logging in manually. "This helped us get a feel for how the network might affect performance, and allowed us to test our results with live users," comments Battles.

The corporate team approach used to migrate to SAP resulted in a greater level of communication within the organization – especially between IT and the business units. "We in IT have gained a better understanding of the functional requirements for our business and our business people have gained a better understanding of technical requirements," says Battles. "The roles many people have within the organization have changed to better support the business."

Most important though, the completion of the LoadRunner tests substantially increased corporate confidence in the ability of the new SAP system to handle the ever-evolving business requirements. "This kind of testing helps the company maximize its investment by ensuring that the hardware and software technology we adopt will deliver top performance," says Battles. "The goal is to provide users with faster, more efficient access to accurate data so that they can continue to make timely, intelligent business decisions."

About the Author: Philip E. Courtney is a marketing consultant and technical journalist with nearly 20 years of experience in the IT industry. He can be reached at (570) 588-5242, or at www.philcourtney.com.