Surviving The Slashdot Effect
When I bought my 1998 Ford Explorer, the last thing I thought it would provide me was a topic for discussion in this column. You see, I’m one of the lucky few million people with a set of Firestone Wilderness AT tires on my vehicle. These tires are the ones that allegedly disintegrate as the tread peels off of the tire, causing some nasty crashes.
The connection between tires and this column is something often called the Slashdot Effect. In the event that you’ve never heard of or experienced the Slashdot Effect first-hand, it occurs when a link to your e-mail address or Web server gets posted on a heavily trafficked site such as www.slashdot.org. These postings are frequently accompanied by encouragement for surfers to hit the link and voice their opinion. I know first-hand how debilitating it can be getting Slashdotted; I had my e-mail address posted there about a year ago as the contact person for an anti-Linux piece that ran in ENT. It was impossible to even try and read all the e-mail I received.
Back to Firestone. I’ll bet a week before the day I wrote this column, the folks that manage Bridgestone-Firestone’s Web operations had things nicely under control. Traffic was probably reasonably predictable, the Web servers were probably sized and replicated according to these traffic levels along with some margin for error. Life was pretty much a daily routine. After all, IT managers are used to planning for 1.5 to 1.6 times the normal workload to make allowances for things like month-end and year-end processing. The workload for Web-based systems, unfortunately, can jump several hundred times with the Slashdot Effect.
The first hint of trouble arose in the first week of August when an article in USA Today exposed a series of accidents involving disintegrating Firestone Tires on Ford Explorers. Quickly, the problem received widespread coverage: CNN, local news programs, and radio broadcasts.
Firestone launched a recall program. To get the story right from the horse’s mouth, I went to www.firestone.com. Why was I not surprised that my browser responded with the dreaded "Cannot find server or DNS error" message?
Along with what must have been millions of customers, reporters, and competitors, I was trying to get important information from a company that apparently experienced a collapse of its Web infrastructure. Who could have predicted, expected, or prepared for such an incredible surge in Web traffic?
Of course nobody could predict such an occurrence, but overload situations should be expected. Often the promotion for sites, for example, can result in out-of-control events. Such was the case with some well-known consumer-oriented Web site overloads: the launch and subsequent crash of the Martha Stewart site right after Martha went on TV to plug it, and the first Victoria’s Secret virtual fashion show, which was overwhelmed by viewers -- most of whom probably were not customers.
We're getting to the point where organizations have to prepare for such monumental increases in traffic. In the Windows 2000 world, Microsoft’s Network Load Balancing capability -- in Advanced Server and Datacenter Server -- and the new Application Center 2000 server product are geared up to enable a Web site manager to respond to this kind of scenario. Using these technologies, one could roll additional machines into a Web farm with only several minutes notice. Other operating environments also have IP load balancing capabilities and the ability to set up server farms. Furthermore, there are open source software products, such as Squid, that also offer this function.
While the Web has become a first-tier form of communication, it is clear that not all of the world’s Web infrastructure is up to the task. Bridgestone-Firestone’s Web operations were not all down. Foreign-language sites did responded to my browser requests, although in local native languages. Too bad the company couldn’t tap those resources to aid its overloaded English-language Web site servers.
This is a subject for you to think about within your own computing infrastructure. If you have not developed contingency plans for how you will quickly respond when your Web traffic jumps by a factor of 100, 1,000 or 100,000 times, maybe it’s time to start thinking along those lines. Don’t think it can’t happen to you just because it hasn’t. --Al Gillen is research manager for infrastructure software at IDC (www.idc.com) and former editor in chief of ENT. Contact him at agillen@idc.com.