In-Depth

Crawling the Internet to Find and Stop Spyware

Researchers find spyware lives especially on adult, game, and wallpaper sites. The enterprise security mandate is clear: start blocking those sites.

• By Mathew Schwartz
• 02/14/2006

How easy is it for a PC to “catch” a spyware infection?

Unfortunately, spyware is a significant threat whenever users browse the Internet. That’s one major finding from “A Crawler-Based Study of Spyware on the Web,” a new study from four researchers in the University of Washington’s (UW) Department of Computer Science and Engineering. Their research analyzed the real-world likelihood of a user encountering spyware, and the result.

Spyware infections on PCs are pervasive. According to a recent study from AOL and the National Cyber Security Alliance (NCSA), 80 percent of surveyed consumers’ PCs contained spyware infections, as well as an average of 93 spyware components, many of which are far from innocuous. Indeed, according to the UW report’s authors, “the consequences of spyware infections can be severe, including inundating the victim with pop-up ads, stealing the victim’s financial information or passwords, or rendering the victim’s computer useless.” Furthermore, the severity of spyware is increasing.

The UW research suggests how to better protect enterprise users from spyware: isolate them from sites known to harbor spyware-laden executable code and drive-by downloads. Doing so protects not only users, but also a company’s bottom line. For example, a recent Webroot survey of 228 organizations found 61 percent of companies had experienced malicious attacks via spyware. In addition, notes Webroot’s recently released “2005 State of Spyware” report, “11 percent of companies said that spyware caused a loss of sales in 2005.”

According to “Crawler” report co-author and UW professor Henry Levy, that current spyware reality—pervasive infections, leading to financial losses—drove the researchers “to understand and quantify the presence of spyware in the Internet,” as opposed to just its effects after infecting a PC.

Finding Spyware Infection Vectors

Spyware typically ends up on a PC in one of two ways: either a user downloads a file that contains piggy-backed spyware, or else when browsing the Internet, a user encounters a Web page able to automatically exploit a vulnerability on the PC. The latter infection vector, known as a drive-by download, is sometimes completely surreptitious.

To study both types of infections, the UW researchers used software to crawl Web sites, looking for both executable files with piggy-backed spyware, as well as Web pages containing “malicious objects.” They ran the crawler tests in both May and October 2005, and used the then-most-current versions of the anti-spyware software AdAware to identify all malware.

Some interesting spyware statistics from the report: with 18 million URLs (individual pages) crawled in May, researchers found 21,200 types of executable files, of which 13.4 percent contained spyware. They also found drive-by-download attacks on 5.9 percent of Web pages. Then from the October 2005 research, which crawled 20 million URLs, researchers found a drive-by download attack in about 1 in 62 Web domains, and discovered roughly 4.4 percent of Web domains harbor software containing piggy-backed spyware.

Most spyware attacks require users to first authorize a download, or allow a drive-by-download script to run—though users may not know what they’re agreeing to. Still, the researchers note that “approximately 0.2 percent of the pages crawled in October exploited browser vulnerabilities to install spyware even when the user denied permission for a download or script execution.”

One piece of good news, however, is that from May to October 2005, the number of pages containing drive-by attacks decreased from 5.9 percent of all pages, to just 0.4 percent of pages. What could have caused that reduction? “The specter of lawsuits, broad use of XP Service Pack 2, and FTC enforcement seems to be helping,” opines Alex Eckelberry, president of security software maker Sunbelt Software, on Sunbelt’s blog.

Even so, the UW researchers stress spyware’s continued threat. “Overall, our results show that even with some of the reductions we have seen, the density of spyware on the Web is still substantial,” and that spyware is becoming, on average, more malicious. For example, while 91 percent of spyware in May 2005 was actually adware, by October 2005 the amount of spyware that hijacked browsers had increased substantially, as had keylogging software, dialers, and Trojan download programs.

Those results square with findings noted in Webroot’s “2005 State of Spyware” report. In particular, Webroot researchers found the prevalence of malicious spyware increased throughout 2005, with system monitors and keystroke logger infections growing about 50 percent, consecutively, per quarter. Also from March to December 2005, Trojan code infections increased by nine percent, then spiked to three times that—due to exploits aimed at the Microsoft WMF vulnerability—until a patch was released in early 2006.

Countering Spyware on the Internet

When spyware ends up on corporate PCs, organizations can see the results in myriad ways. According to Webroot’s survey of 228 organizations, repercussions from spyware infections included diminished PC performance (for 61 percent of respondents), disabled PCs (43 percent), reduced productivity (33 percent), and increased help desk calls (24 percent).

What can IT departments do to better counter spyware? Note that UW researchers found the highest concentrations of executable files with piggy-backed spyware on celebrity and adult sites, as well as sites offering free desktop wallpaper. In May 2005, the researchers also found that “14 percent of ‘pirate’ sites installed drive-by infections when a user responded ‘yes’ to prompts.”

In other words, users of celebrity, games, adult, pirate, and wallpaper sites are at much higher risk for getting a spyware infection. By contrast, kid sites and news sites harbored few, if any, spyware attacks.

So consider moving beyond just an “anti-spyware on the desktop” paradigm, and installing anti-spyware on network gateways and proxies to block the inevitable drive-by downloads, plus the executable files with piggy-backed spyware users might download. Furthermore, contemplate filtering the URLs employees can access, to block or restrict access to the types of sites known to harbor spyware.

Related Articles:

• Beyond Malware, SOX, and Data Breaches: The 2006 Security Forecast


• Spyware Hampering Compliance Initiatives