In-Depth

Banking on Better Storage

To consolidate over 300 servers in three data centers, the Bank of Montreal turned to IBM, Inrange Technologies and StorageTek.

• By Drew Robb
• 11/01/2002

Founded in 1817, the Bank of Montreal (BMO) was Canada's first bank. With headquarters in Toronto, Ontario, the bank has grown over the past two centuries into a multinational financial services company with 34,000 employees in 32 lines of business and total assets of $246 billion (Canadian). Its long and venerable traditions don't keep it stuck in the past. More than one-fifth of its employees (about 7,500) provide IT services to the rest of its staff and its customers.

As the company has grown in size, so too has the complexity of its IT platform. Overall, the company now has over 3,500 servers. Its three data centers, two in Toronto and one in Chicago, have grown from 22 servers in the late 1990s to over 300 servers today with a combined storage capacity of 43TB.

Though capacity proved adequate, inefficiency was taking its toll. Each of the many operating systems in use came with its own type of direct-attached disk. This wasted disk space made storage management a nightmare.

With its storage needs growing 15 percent to 20 percent annually—and predicted to accelerate from 20 percent to 30 percent from 2003 to 2006—the bank faced ever-increasing costs and management difficulties unless it changed its approach to storage.

Robert Smalley, senior project specialist in the Mid-Range Services Department, summed up the dilemma: "Hard coupling of storage to the servers and applications results in low storage utilization, increases management complexity, and continually drives up the cost for operational support. We needed to centralize and simplify the storage management."

Product Information Assorted Storage Products: IBM TotalStorage Enterprise Storage Server (ESS) Inrange FC9000 switches StorageTek 9840 tape drives

Five Year Plan
Rather than looking for a quick fix, the company has embarked upon a wide-ranging strategy to streamline its IT operations to better meet growing needs. BMO has mapped out an IT plan that will cost approximately $20 million (in Canadian dollars) stretching to the year 2006.

"Our strategy is to deploy solutions that support cross-platform resource sharing and measurable service and management improvements," says Smalley. That, in turn, will "provide cost-effective, highly available, ‘on demand' networked-storage solutions, which facilitate automation of business continuity operations."

Storage Switch
The major change, however, is to switch from direct-attached storage to a series of storage-area networks (SANs), one at each data center. The bank will also use separate SAN islands for its different platforms, office towers and business needs. These will provide storage for all the servers regardless of platform, will increase reliability, and will shrink the underutilized capacity inherent in direct-attached storage.

In doing so, BMO wanted to leverage its existing investment in a fibre management infrastructure. The migration began in 2001 at the two data centers in Toronto. In September, the bank upgraded its open systems tape infrastructure StorageTek STK 9740 system by switching its 9840 tape drives from SCSI to fibre connections. Fibre supports approximately 125TB in tape storage. BMO also replaced its TSM server with an IBM p660 server to host Tivoli Storage Manager software. It also installed 64-port Director Class switches at both data centers.

The bank evaluated products from several vendors including Brocade Communications Systems Inc., Storagetek and McData Corp. before settling on FC9000s from Inrange Technologies Inc. "We already had favorable past experience and familiarity with Inrange technology since the bank was utilizing an Inrange CD/9000 switch with the z/OS mainframe platform," Smalley notes. "This implementation allowed BMO to utilize existing cabling and ESCON [Enterprise Systems CONnection, an IBM 17Mbps fiber-optic channel] infrastructure and installation practices."

Two of the switches are used at the primary data center to provide redundancy. The other data center, serving as a backup and test facility, was initially set up with only one switch, but will get a second unit so that both facilities have a complete dual-fabric structure.

Inrange Global Consulting (IGC), the vendor's professional services organization, assisted in planning the SAN. The consultants spent a couple of days doing a preliminary investigation offsite, then visited the sites for a few days to become familiar with the BMO environment. After that, it only took a few hours to actually implement, test, verify and put the equipment into production. A few minor host bus adapter compatibility issues within the tape infrastructure portion of the SAN were quickly resolved.

With this infrastructure in place, in February the bank installed an IBM TotalStorage Enterprise Storage Server (ESS), providing 10TB of shared storage.

Looking Ahead
It will still be a few more years before BMO has completely switched over to a SAN environment, but Smalley reports that what is currently in place has improved the connectivity to disk space, almost providing "storage on demand," and BMO has seen performance throughput gains.

While the equipment installed so far has been doing the job, Smalley says that BMO still hasn't found storage management software with the maturity needed to manage the entire SAN environment—both infrastructure and open-systems disk storage.

BMO also sees a challenge in getting the storage in place fast enough to keep up with rising capacity demands. Nevertheless, he has no doubt that implementing a SAN is necessary to replace the dead-end of direct-attached storage.

"The risk of doing nothing is worse than the risk of doing something," Smalley says.

Details: Bank of Montreal (BMO) Team Leader: Robert Smalley, Senior Project Specialist, Mid-Range Services Department, Network and Systems, EMFISYS (the bank's technology and e-business group). Organization: Bank of Montreal, Network and Systems Division. Location: Toronto, Ontario, Canada Business/Mission: Financial Services Web Site: www.bmo.com Goal: To install a scalable corporate storage system that's flexible enough to accommodate future technology changes. Scope: Three data centers with 300+ servers (3,500+ servers total enterprisewide) having 43TB combined storage serving 34,000 employees in 32 lines of business, plus customer needs. Five-year plan to migrate direct-attached storage to storage-area network. $1 million (Canadian) initial capital investment; $20 million over five years. Equipment/Platform: A variety of direct-attached storage for z900, AIX, Solaris and Windows NT/2000 servers. Solution: Consolidate servers and reduce the number of supported databases; opting to stick with a few vendors, including Oracle Corp. and Sybase Inc. Simplify support by reducing the number of supported operating systems, eliminating certain flavors of Unix, choosing IBM Corp.'s AIX and Sun Microsystems Inc.'s Solaris as its focus. Consolidate storage into a SAN connected to servers via director-class switches. Products: IBM TotalStorage Enterprise Storage Server Inrange FC 9000 StorageTek 9840 Other Products Considered: Switches from Brocade, McData (6064) and StorageTek (SN6000). Results: To date, BMO has created its Fibre Channel infrastructure including the Inrange FC9000 64-port switch, a 10TB IBM Enterprise Storage System RAID box, and 125 TB tape library. Project still ongoing. Interoperability Issues: Some minor problems with the HBAs in the Tape Area Network. Lessons Learned: Continue to do your homework There is no one perfect SAN management tool for all situations Build/have access to a SAN box Document a detailed architecture Continue to communicate and educate