In-Depth

Why Vista Was Vulnerable to Animated Cursors

Despite considerable work on improving security, “old, errant code” exposes flaw in Microsoft Vista.

• By Stephen Swoyer
• 05/08/2007

That's why it surprised many that Vista—like its Win32 and Win64 predecessors—was listed as susceptible to the Windows Animated Cursor Handling vulnerability that Microsoft patched last month (see http://redmondmag.com/news/article.asp?editorialsid=8397).

That vulnerability stemmed from a basic flaw (a stack-based buffer overrun) in Microsoft's Windows GDI implementation—the kind of thing you'd expect Microsoft to have eliminated, especially in light of Vista's top-to-bottom Desktop Window Manager (DWM) overhaul, anyway.

Microsoft officials were surprised, too. As it turned out, the errant code happened to be very, very old errant code. "First of all, this code is pretty old; [it's] in Windows 2000 and predates the SDL," wrote Michael Howard on Microsoft's new Security Development Lifecycle blog.

Microsoft's SDL practice is part of its much-ballyhooed Trustworthy Computing push. As Howard pointed out, no SDL is perfect, and—in the case of last month's Windows Animated Cursor exploit—some vulnerabilities will still slip through.

"In the Windows Vista process, we banned certain APIs, like strcpy and strncpy, and changed well over 140,000 calls to use safer calls. Memcpy [the affected call] wasn't on that list," Howard said.

How did Microsoft miss the Windows Animated Cursor Handling flaw during its Vista DWM coding (and legacy GDI recoding) processes? For one thing, Howard said, the errant call in this place didn't trigger a -GS cookie, which makes it difficult to identify in the first place.

"Because there are no candidate buffers on the function's stack, there is no -GS cookie added to the stack, even though the code is compiled with -GS," Howard wrote. "This is not the first time we've seen code with no cookie, and this has made us rethink the heuristics used by the compiler when it determines whether to place a cookie on the stack or not. But changing the compiler is a long-term task. In the short-term, we have a new compiler pragma that forces the compiler to be much more aggressive, and we will start using this pragma on new code."

In an ironic twist, Howard said, one of Vista's new security improvements—namely, address space layout randomization (ASLR)—actually helped mask the vulnerability.

"The next issue is that the code is wrapped in an exception handler that catches code failures," he said.

Enter ASLR: "[T]he goal of ASLR is to reduce the likelihood that an attacker can determine the address of critical functions," Howard said. "This makes it harder to make exploits run correctly. But if the vulnerable code is wrapped in an exception handler that catches many errors, a failed attempt will not crash the component and the attacker can try again with a different set of addresses."

Elsewhere, Howard explained, Microsoft missed the vulnerability because of an utterly pragmatic reason.

"Code that uses calls such as memcpy is hard to flag as vulnerable without generating a great many false positives. This is a research problem that no one has solved, here or elsewhere," he wrote. "Another angle is to replace calls to memcpy with memcpy_s, which forces the developer to think about the destination buffer size. We may ban memcpy for new code, but we still need to analyze this further."