In-Depth

Case Study: Integrating Mainframe and Web-based Data Entry

Mainframe integration tool improves customer service, productivity, accuracy

Companies that maintain complex mainframe-based applications face difficult choices. They often need to continue supporting the mainframe architecture, while at the same time offering efficient new front-end services to users that tie into the mainframe backend. Often, companies need solutions that support service-oriented architectures as well.

For the National Rural Electric Cooperative Association, the situation was more complex because the NRECA will eventually move away from its mainframe, possibly in the next three to five years.

The association needed a mainframe integration product that allowed continued support of the existing mainframe system as the IT department gradually shifted the complex mainframe applications residing there elsewhere.

NRECA, founded in 1942, represents over 900 cooperative electric utilities serving 39 million customers across the country. It runs three critical applications on its IBM zSeries mainframe at the central offices in Arlington, VA. One is a custom-built benefits administration application used for maintaining health, 401(k), and pension plans for member co-ops.

When the mainframe applications were first created in the 1980s, benefits data from members arrived on paper via the U.S. mail, to be entered into the system by clerks via “green screens.” By the 1990s, NRECA had built a Web site for members to enter much of that data directly.

However, the workflow still contained a significant manual step. “It was smoke and mirrors,” according to Linda Scotto, manager of IT applications development and support in the NRECA IT department. “We were doing database extracts every night from the mainframe, and loading that to the Web site’s SQL database server” to have the data needed to collect individual transactions. The system would eventually send the new transaction information via e-mail to a clerk, who would print it out—and then type the data into the mainframe via the old-style green screens.

Clearly, Scotto says, NRECA needed a way to eliminate the manual data entry step and deliver data directly to the mainframe. She also wanted a solution that fit within the service-oriented architecture (SOA) that NRECA is gradually moving to.

After weighing two competing mainframe connectivity products, NRECA selected Shadow z/Services from DataDirect Technologies. The solution is designed to support mainframe integration through Web services for SOA, real-time events for event-driven architectures, and SQL for direct SQL data access and transactional support.

For NRECA, Shadow z/Services made it relatively easy to create a Web service from a mainframe green screen. When NRECA adds new features to the Web site, those features can be easily integrated with the backend system.

A key selling point: Shadow z/Services uses .NET architecture, which has allowed in-house developers familiar with that environment to quickly come up to speed. The other product under consideration used a proprietary language.

An IBM and Microsoft Shop

A mostly Microsoft shop except for its mainframe, NRECA runs Microsoft’s object-oriented programming language, C#. Shadow z/Services was implemented in tandem with Microsoft BizTalk servers and SharePoint Portal Server technologies. Shadow z/Services sits atop the mainframe system, with Microsoft BizTalk Server on top of that as a service bus.

The new product fits into the SOA that NRECA is moving to, Scotto says, and allows continued support of the existing mainframe system for now. With the Shadow z/Systems system in place, “we can keep dual processing going without double data entry,” she explains.

In a three-to-five-year process, Scotto plans to separately migrate each of the three applications still on the mainframe to another system (which is still in the planning phase). During that time, she says, “I’ll have to keep the old system going through all three phases, and still have that data fed into the backend mainframe system.”

With the new system, data entered online by co-op members is stored in a Microsoft SQL Server database and distributed by BizTalk to the correct endpoint. “We put the workflow logic in BizTalk,” Scotto explains, “so that as a transaction comes in, BizTalk makes the decision whether it needs to be looked at by a human or sent straight through to the mainframe.” If the transaction requires human intervention, it’s sent directly to a Windows SharePoint portal. With some 80 percent of transactions, however, the data is immediately sent to the mainframe for processing. In those cases, Shadow z/Services enables interaction with the mainframe data as a Web service.

Challenges Documenting Workflow

Once the software and hardware was in-house (in July 2005), the rollout proceeded quickly. NRECA started development in August and went live in November. “Our weakest link,” Scotto says in hindsight, “was [working with] our user community—[for example, supervising] while they tested to make sure everything was tested thoroughly.”

Indeed, the user integration proved to be the biggest challenge during the rollout. Working with clerks to define and describe their work took nearly three months, longer than Scotto had anticipated.

“We were making them define the workflow, which they had never had to do before,” she says. “It was all new. … We’d asked, ‘OK, when you’re sitting in front of that green screen, what do you do?’” The work had become so automatic that it was difficult for data entry clerks to break it down into steps and describe it. The eventual solution: Scotto’s staff sat down with clerks in front of the old mainframe green screens, then worked together through the new Web site transaction screens, field by field and screen by screen.

Business Metrics

Now that the system is in place, users are pleased. “They’re so happy; they can’t believe how well it works for them,” Scotto says. A plus for users: Through BizTalk and SharePoint Portal Server, the new system can produce useful business metrics, such as what percent of transactions were successful.

Another useful measurement and one that highlights the success of the system: Scotto can now see how long a transaction sits waiting for manual intervention. “We had nothing [to measure with] before,” she says. “All we knew was the date/time stamp of when the transaction was entered on the Web site, and the date/time stamp of when it was updated in the mainframe system. But we didn’t know what happened in the three weeks in between.” Those three weeks have been reduced to a week at most, Scotto says.

The new system has also eliminated a common problem when double data entry is involved: clerical errors. Co-op members would enter data into the Web site correctly, but a clerk would occasionally introduce an error while entering the information the second time. “That was really giving us egg on our faces,” Scotto says. “Now, what goes into the mainframe is what they entered on the Web site.”

Must Read Articles