In-Depth

Storage Insecurity

Bank of America's security lapse could happen to you, too

• By Jon William Toigo
• 04/28/2005

We are having trouble getting e-mail to a friend of this column who works for (or perhaps worked for) the Bank of America at the company’s Northern California operations center. We have tried the e-mail address we have always used, as well as every variation we can dream up. Every message bounces back. We called BoA to find out whether our friend is still employed there and have been told that the company can neither confirm nor deny the existence of any employee. We even Googled the guy and found hundreds of hits associated with his name. Invariably, these contacts have led us to folks we don’t know—possibly relatives, but probably persons unrelated to our friend.

Our inability to reach Phil is especially distressing for us given the light he might have been able to shed on a recent incident at the largest bank in the United States. We refer specifically to the recent report that the personal data of a large number of bank customers may have been exposed owing to the misplacement of backup tapes somewhere between Bank of America and its remote storage facility.

Phil, if you are out there, please contact us to let us know that you haven’t “disappeared” while visiting some Third World country.

This column is not just about the whereabouts of our friend, of course. It is about storage security—or better yet, storage insecurity. For the past several years, I've discussed in this column the need for increased storage security. We have reported the vendor community’s efforts to build a business case for storage security that goes beyond the protection afforded by LAN firewalls and application passwords. I have tracked the largely ineffective efforts of the Storage Networking Industry Association to develop storage security reference profiles. I have also followed regulatory developments, such as the Graham-Leach-Bliley Act, HIPAA, and others that have placed a priority on data integrity and personal privacy.

The recent accidental data loss by Bank of America, and its inherent potential for the unauthorized disclosure of detailed customer information, has brought the subject of data protection and privacy back to this column. Details surrounding the data loss are sketchy at best. But, in full compliance with regulatory disclosure requirements, the bank admitted that it had misplaced backup tapes containing private information.

It further admitted that the data on the backup tapes, which it insists is not readable, was not encrypted.

The incident has caused huge embarrassment for the bank, according to a number of insiders I chatted with, off the record, at recent industry conferences. I was told that the bank is conducting its own investigation to discover the root cause of the incident, and that it will act swiftly to correct any flawed work processes determined to have permitted the occurence in the first place. I was also told that Bank of America is considering all options, including storage encryption, that might provide a technical fix going forward.

All of this is off the record. Sadly, the company is reluctant to further expose itself to ridicule by conducting investigations in the open. Making matters worse is the slippery language being used to describe the disappearing tapes as “non-readable.” This is technically true, since no one can read information just by looking at a tape cartridge. The slippery part of the claim is that the data might be readable by technical means (a tape drive and software) after considerable effort. Truth be told, if bad guys have the tapes, they probably have the skill, the equipment, and the motivation to pluck the information they want from the recorded data. As a result, this incident may well reverberate for a long time, to the tremendous inconvenience of both bank management and customers.

Most people can understand the company's plight. Somebody screwed up, to be sure. The law required disclosure of the screw up, causing the otherwise good folk who work at Bank of America to be subjected to humiliating press inquiries and the company’s reputation to be compromised, however temporarily.

The incident is very likely the tip of the proverbial iceberg when it comes to accidental and deliberate data disclosures by companies. Incidents already encompass information stored to both production and backup media. There were, in fact, several other “lesser incidents” (in terms of numbers of customers impacted) involving companies other than the bank mentioned in just about every press account of the debacle. True to form, the press has paid a lot of attention to this event while largely ignoring press releases from little storage security vendors with big ideas.

Storage encryption vendors and their leaders, such as NeoScale's CEO Barbara Nelson, have been lobbying for stronger security measures for the past five years. NeoScale’s technology could have saved Bank of America and the others (not to mention their collective customers) a lot of grief had it been deployed in their overpriced SAN environments and in the data path between primary and backup storage devices. An encrypted backup is of no use to anyone who lacks the decryption key. At most, the loss would have been an inconvenience to the companies involved.

Hindsight is 20/20, as they say. There were probably tough calls made at Bank of America to decide how to allocate increasingly scarce IT resources. Maybe, though the bank hasn’t said this, some storage security technologies were in the process of being reviewed. Maybe NeoScale, or competitor Decru, or some other vendor’s security products had been tested and found to be deficit in some important way.

Who knows—the absence of a storage encryption solution might have been the result of a legitimate concern about the delay that encryption might have visited on the backup process itself, or worse yet, on the process for restoring data from encrypted backup in the wake of an unplanned interruption. Tape backup and restore are a lengthy enough processes already—when they work at all. If encryption technologies would impose further delays, it is understandable why Bank of America and others might have been circumspect about their use.

A key concern, suggested by Oscar Ernst, CEO of SouthernLink-IT, at a security conference in Brazil this past week, is that the very laws and regulations that require companies to disclose their data accidents do nothing to compel the companies to disclose the details of how the accidents occurred or what repairs are being made to workflow processes or infrastructure vulnerabilities to prevent such mishaps in the future. As was the case with disaster recovery planning a decade or two ago, says Oscar, the best information about storage security incidents is being locked away in the exit interviews of those who were fired in the heat of the moment. The handling of these events by embarrassed CEOs and their suited minions is producing no useful insights to leverage our understanding.

Smart guy, that Oscar. We just hope that our friend Phil wasn’t one of the fall guys at Bank of America-- sacrificed as a tribute to someone’s warped view of corporate reparation.

In fact, storage insecurity is as much the result of a combination of poorly conceived legislation, budgetary constraints, and operational trade-offs as it is of any action or inaction instigated by any individual. Instead of lopping off heads, Oscar asked, why don’t we focus on building a good knowledgebase of the efficacy of techniques and technologies for securing storage itself?

We’d like to help. If you are using any special products or processes to secure your company’s data from unauthorized disclosure, please tell us about them. Write to jtoigo@intnet.net.