In-Depth

New Life for Old Software

The federal Jet Propulsion Laboratory is porting old VMS software onto new PC hardware running Linux, extending the life (and value) of legacy programs.

• By Drew Robb
• 08/01/2001

"We were asked to port the Corps Battle Simulation (CBS) program to new hardware so it could continue into the future, much longer than expected," says Joe Provenzano, JPL's project manager for CBS. "The requirement for increased speed and the very high cost of maintaining legacy hardware were the reasons for the port to a newer, faster, cheaper system."

Funded by NASA and run by Cal Tech, the Jet Propulsion Laboratory is the nation's premiere institution for unmanned exploration of the universe. It's 5,000 researchers and employees built the Hubbell telescope, dispatched a rover to Mars and surveyed the solar system and beyond with space probes and earthbound telescopes.

Bloodless Warfare
Beginning in the 1970s the military began building computer-based simulation systems; Corps Battle Simulator is one of the most effective programs. CBS is managed under the sponsorship of the Army's Simulation, Training and Instrumentation Command (STRICOM), based in Orlando, Fla. First developed in 1983, it has since gone through annual revisions to improve its features and accuracy.

NASA's JPL develops and tests the software and ships it out to military sites, such as the National Simulation Center at Fort Leavenworth in Kansas, the Korea Battle Simulation Center in Seoul and the Warrior Preparation Center in Germany. Once the software is developed, JPL supports the first few uses before turning it over to a military contractor, BTG Inc., for ongoing support.

"Despite the need for support, CBS is really production software," says Jay Braun, a JPL contractor specializing in simulation software. "In other words, we don't make changes to support the ‘next' exercise. Each year's release is exactly that: a software release that can be distributed on CD."

From Mainframe to Desktop
CBS was originally developed to run on what were then top-end machines, DEC VAX 7800 series computers, at a cost of about $100,000 each. "The VAX and VMS systems have served CBS very well over the years," says CBS Project Manager Dave Zedo, a civilian employee of STRICOM, "but since the VAX original equipment manufacturer (OEM) has abandoned that product line, the only way to maintain the CBS platform is to use after-market vendors, purchasing used and refurbished spare and repair costs at high prices." As a result, the CBS team constantly scrambled to find parts and equipment on the used market. Maintenance on the 7800s and smaller VAX machines costs millions of dollars per year.

Besides the expense, the hardware limited the usefulness of the system. The VAX had already reached its limits in terms of speed, and coalition or multi-faction warfare requires lots of additional processing. CBS needed extra power to address scenarios in which allies might be capricious or where many neutral factions might exist—such as in the Balkans or the Middle East. Due to these limitations, the Army had scheduled CBS for termination and replacement.

As a new simulation program won't be ready for several more years, however, senior military officials decided to extend the life of the software by porting it to a more robust platform. "Operationally, the Army needed to ensure that the CBS system would remain the highly functional training tool it has come to depend on," says Zedo. "As such, it became necessary to re-host the CBS software on a new platform. Since standard IBM-compatible PCs have become technologically advanced and much cheaper, it was decided that the new CBS platform should be as standard as possible."

Accordingly, JPL wrote this year's version of CBS software to run on PCs, a move that has dramatically improved overall simulation quality. It tested the new system using a Korean battle scenario, a particularly complex battle situation that requires the modeling of thousands of Air Defense Artillery sites. Rather than stretching the system to the limit, however, the new computers performed well above expectations. In February, on a $4,000 PC with a 1.2 GHz AMD Athlon processor, the CBS team ran the largest CBS ever—the latest Korea scenario—three to four times faster than the most powerful VAX, without sacrificing model fidelity.

Downsizing the OS
Switching from VAX to PC also necessitated a change in operating system. After considering the options, the team decided to port the software to Linux. "Linux is a flexible, open, non-proprietary operating system, designed to operate on a myriad of Intel-based architecture platforms," says Zedo. "Our developers chose Linux because of its flexible open architecture capabilities."

The simulation is written using a high-level language called SIMSCRIPT II.5 from CACI. "In the past 10 years, the vendor has emphasized portability, and it has paid off for us," says Braun. "They have turned Simscript into a C pre-processor, which means that the same, no-nonsense C code gets compiled on the various platforms without portability concerns."

Simscript also supports a checkpoint/modify/restart feature. This means that when a bug is found, the programmer can pause the program, modify the code and continue running the exercise from that point, rather than having to start again at the beginning. This ability is particularly crucial with CBS, since a single simulation can run for days or weeks.

According to Braun, it took about one person-year to make the switch, going through intermediate stages on Solaris and Windows NT, before the final version was running on Red Hat Linux Version 7.0. (JPL is now shifting to 7.1.) The labs used a single-processor machine for development and testing, but the program will probably be running on dual processor boxes out in the field to allow additional simulations to run.

"Our model has plenty of features that are pushing the limits of Linux," says Braun. "We've configured the kernel to support 4GB applications—the theoretical limit on 32-bit architecture—all readily accessible in physical RAM at any instant."

JPL also made adjustments to make a huge portion of the program contiguous so that Simscript can do its checkpoints quickly (20-second save times for the largest exercises, 3 seconds for small exercises—an order of magnitude faster than the VAX). Even so, JPL may need to switch to a 64-bit architecture in the future to increase memory addressing. Fortunately, Linux supports 64-bit processing, so JPL will not have to change the operating system again.

Development Tools
Beyond the basic port, Eric Tailor, the JPL team's "system software guru," also created new tools to improve performance. One, called IMON, is a real-time application profiler that tells the user how much processor time is being spent in each software module while the model is running. With standard tools, the user can instrument an application beforehand and then get a batch report on how much time is spent on each function. But there is no way to distinguish between low-level library functions and the main application. With IMON, however, the user gets a continually refreshed histogram of the functions that are taking the most time right now. When there is a slowdown or if suddenly the simulation is using 100 percent of the CPU when it's been using 20 percent all along, the developers can immediately find out what is executing. As an added bonus, they can exclude library routines from consideration, allowing them to focus on the application.

Another tool Tailor developed is called the "Hub." The Hub allows a person to arbitrarily connect programs together, with one or more programs publishing data and one or more programs subscribing to that data. The Hub acts as a server with all the applications connected to it running as clients.

"The big innovation is that the server itself can be observed to find out what kind of traffic is being passed between applications," says Braun. "If you connected two applications directly, you wouldn't have that kind of flexibility. This has been of enormous debugging benefit."

Into the Field
The switch to Linux and PCs has gone well. The new system has been tested in the field and is being shipped this summer to simulation centers worldwide. Although software ports often result in unexpected functional changes, that hasn't happened here. "Our tests to date have shown that the military training audience is getting exactly the same feedback that they have come to expect from CBS," says Braun, "but from a faster, cheaper and arguably better system. The door is wide open to higher-fidelity modeling and new capabilities."

With its lifespan now extended for several more years, CBS will continue to provide the military with a swift and cost-effective means of maintaining its operational readiness for whatever wars may arise.