In-Depth

Keeping the Trains on Track

AS/400s help a Georgia railroad company keep track of where loaded trains are within the system.

• By Julie Miller
• 08/01/2001

Imagine having trouble locating a fully loaded train roaring down the tracks at 2 a.m. That’s the situation that Railcar Management Inc. (RMI) has made strides to prevent, using a real-time high-availability solution that works with its AS/400s.

Headquartered in Atlanta, RMI has about 500 clients, including a substantial number of regional and local railroad companies. The company processes all transportation movement and revenue settlement for these railroad clients. When trains move, RMI tracks where they’re going, when they’ll arrive, where the cars are throughout the trip and when cars are switched from train to train as they move across the United States. RMI also uses an AS/400-based solution to calculate complex shipping fees—if one railroad ships from the East Coast, it may go through five railroads to reach the West Coast.

The company, which is both an Application Service Provider (ASP) and a Business Service Provider (BSP), rents its software for a license fee. About 90 percent of its clients, each with its own database, use RMI as their complete data center. RMI has 12 AS/400 processors at three data centers in Atlanta, with a redundant telecommunication lead with fiber optic connection to all locations.

Planning the Trip
Van Rownd, RMI’s vice president of information services, describes the importance of RMI’s role: "If the system is working fine, the train goes down the track and the staff receives all relevant data by EDI. They’ve admitted the train and are ready to send it out, but the computer goes down. Now the next railroad [team] doesn’t know the train is coming, and they don’t know where it is on the line. Even worse, the engineer doesn’t have a printed report to account for the cars he’s transporting."

Since their application manages train schedules, RMI needs to know what’s on the train and where it’s going at all times. A system failure could mean trains sitting on the track wasting time and money, or an inability to locate specific trains in motion.

Because their business requirement is to get as close to 100 percent uptime as possible, RMI knew that it needed a high-availability solution, and it chose Vision Solutions’ Vision Suite. "Railroads run in a 24x7 world, and when we’d take them off-line for eight to 10 hours, it was a disaster," Rownd says.

RMI wanted to offer its clients a "disaster business continuity services offering," which automatically provides a redundant state in another data center if, for example, one of their data centers is hit by severe weather, power outages or other communication problems. RMI also wanted to eliminate much, if not all, of the downtime associated with planned outages, such as those needed for maintenance and software or hardware upgrades.

RMI offers its clients two levels of data availability. Level One mirrors each client’s database to a second processor within the same location, giving a reliable level of business disaster and continuity services. Level Two mirrors the railroad data between two processors and between two locations. If one location goes down, users can be switched to one of the other locations, ensuring that the system is constantly up and running.

"When Vision Suite was installed, our customer satisfaction increased one thousand percent, and that was just with Level One. When we offered Level Two, our customers were ecstatic," says Rownd. "We’ve been testing Level Two on a regular basis and can demonstrate to our clients that should a location go down, we can have the railroad up and running within an hour, with full processing and telecommunications."

Leaving the Station
Vision Solutions supplied a project manager for the RMI implementation. The role of the project manager, David Mee, was to gather the correct information, schedule the resources, identify the application and any problems that might occur, identify the requirements for role-swaps, bandwidth and user requirements, and schedule the resources needed on-site.

Vision Solutions typically schedules three weeks to perform an installation, but Mee says the installation for RMI was shorter than expected. "The advantage we had with this implementation was that the resellers had a very good technical resource who knew our product very well, and RMI had a technical staff who was highly sensitive to the needs of mirroring because they were already familiar with journaling," he explains.

When RMI first started out, they tried handling large groups at the same time. They later moved to TCP/IP addressing, with every railroad having its own IP address in the system. RMI now can manipulate each railroad individually, therefore avoiding working with whole processors at a time. By using additional techniques on the AS/400 along with Vision Solutions’ products, RMI can manage any road anywhere in the system.

Mee says that during the implementation, there were several small issues in the role swap itself. One problem was finding the best method for moving specific users from one system to the other. Since RMI was using TCP/IP addressing, it offered a simple solution. The other issue was moving its communication lines from a local site to a remote one. It solved these problems by working with a local carrier.

Staying on Schedule
"We have virtually eliminated downtime. In fact, my clients do not consider the rollovers downtime," says Rownd. "We have been so good at it, and so consistent at always meeting our deadlines, that now we can call our clients at lunchtime and tell them we can have them back up in 30 minutes—no problem."

RMI minimizes downtime by using mirroring for every client on the system. If the company needs to upgrade a processor, it simply switches the particular railroads on that processor to a backup processor. These "rollovers" take about half an hour. If RMI has just two upgrades a year for processors, software and hardware, the most the system will be down during the year is one hour.

"Downtime is not productive time. The railroads are very touchy when you don’t have systems running, and you don’t know where the trains are," says Rownd. "When the systems go out, or the communications go out, all hell breaks loose. We have redundant communications, as well as systems. We have been able to provide that in a cost-effective manner for all our clients, and they’ve come to expect that."

After installing Vision Solutions’ products, RMI has never had a system outage. Although it has had outages due to weather-related issues or communications failures, the company now has redundant communications, so that clients remain unaffected. Rownd adds that 95 percent of his outages are planned—for maintenance and upgrades.

Rownd says he has talked to other mirroring clients (regardless of what product they use) who thought they could complete a mirroring project simply by reading the instruction book. "We don’t do that at RMI. When I took this job, my number one requirement before we did anything was to get educated on the entire process," he cautions.

The Trains Keep Rolling
A properly implemented high availability solution offers advantages by eliminating both planned and unplanned downtime. Before RMI installed its solution, the idea of disaster recovery or business continuity services didn’t exist.

At the time, they had over 12 systems in the data center, and each one was filled to capacity. When it got full, RMI couldn’t shut the system down to upgrade it and load more railroads; instead, they had to buy another system.

Vision Solutions has eliminated this cycle. Rownd says he has been able to reduce his systems budget by almost 15 percent and expects that percentage to continue growing over time. He estimates that over the next three to five years, RMI will save close to a million dollars in systems expenses (hardware and software).

He is now using two AS/400 systems with four processors each and is adopting resource sharing. RMI can move resources around as it moves the roads around, which no longer requires a one-to-one redundancy. "Some people think you have to have one-to-one redundancy—you don’t," says Rownd. "My redundant areas comprise less than half of my total system. I’ve got 1.8 terabytes on disk with eight processors, and only 40 percent is dedicated to the mirroring side. That has eliminated all but 0.2 percent of my downtime."