Embracing Microsoft Vista for Enhanced Network Security

Effective implementation of server and domain isolation requires complete network visibility throughout the OS migration process

by David Arbeitel

Microsoft Vista offers tremendous opportunity for organizations to improve network security by limiting access at the operating system level. With native support for the IPsec protocol and new facilities for creating IPsec-based group policies within Active Directory, Microsoft has made network security an integral part of the operating system. Thanks to these new capabilities, organizations can now employ server and domain isolation (SD&I), a powerful security technique that logically separates computers to ensure that only “trusted” computers can communicate. Using SD&I, IT executives and security managers can manage security policy at the operating system level through the authentication and, optionally, encryption of client-to-client, client-to-server, and server-to-server communications.

SD&I allows organizations to mitigate threats to the perimeter and the core by limiting access according to user affinity and role, not just one’s physical location on the network. Thus, SD&I complements network security devices and hosts in the enterprise. The new policy management add-on to Active Directory in Vista provides organizations with a highly centralized and efficient way to define and manage these groups or domains over time, which will potentially limit risk by requiring fewer changes within the enterprise to enforce policies. Organizations will also be less vulnerable to configuration errors on disparate network devices.

Challenges to Vista Adoption

However, powerful as it is, SD&I does not fully eliminate the risks created by gaps between policy and configuration, nor does it provide a way to validate that policy and configurations are functioning effectively. Essentially, each computer in a domain has a personal firewall embedded in the operating system that enforces IPsec-based policy. A great deal of automation exists in Vista for management of these policies; nevertheless, there is potential for error within the category of “border machines” which connect a “trusted” domain to “untrusted” domains within the enterprise and beyond.

Since most organizations are heterogeneous in nature, Windows machines need to communicate with machines running other operating systems. Just as firewalls, IPS systems, or routers enforce policy on a physical network, border machines sit at the edge of a domain and enforce the “exemptions” to the established isolation policies that determine whether a computer may connect to the outside world. Improper implementation of these exemptions will result in one of two outcomes:

  • All of the computers residing in a domain become vulnerable
  • or

  • A restriction of connectivity which results in IT’s inability to fulfill critical requirements for a user or support multiple users in a domain.

The ultimate success of an SD&I solution built on Vista and Longhorn requires careful planning before, during, and after the migration process to ensure that policies are well defined and properly configured.

The Importance of Network Visibility to Project Success

Organizations implementing SD&I in Vista should begin by creating an accurate picture of both the devices on their network and how traffic flows from domain to domain, and from a domain to the Internet. This picture enables one to measure the effectiveness of current policies as well as the connectivity requirements for the business.

Since business requirements will continue to evolve at a frenetic pace, organizations must have a way to understand the impact of change over time to group policies and exceptions on risk and compliance. Establishing a baseline of connectivity is the first step. This baseline can be used before, during, and after implementation to validate group policy.

Building a baseline is no simple task. One must be able to visualize the flow of traffic on the network quickly in order to discover unauthorized connectivity between trusted and untrusted networks, a critical capability because IPsec Group Policies are very explicit in defining which Vista and Longhorn machines can talk to each other in terms of the network, using IP addresses, CIDRs, and protocols. Without a thorough understanding of these items, it is unlikely that IPsec group policy will be implemented correctly, or that they will be resilient to change in a manner necessary to support the business.

Meeting this challenge requires the use of a comprehensive discovery solution for network devices and segmentation, traffic, and hosts.

From a network point of view, IT must know which network devices will need to be upgraded for cost estimation purposes, but it is just as important for implementation, since in many cases IPsec will break access control lists (ACL’s) on devices that do not support it. Sometimes the device hardware footprint will not allow a device to function properly under peak loads in a network running IPsec, so identifying network hardware that can receive a RAM upgrade is important.

Understanding how traffic flows at different times can help identify potential weaknesses in the infrastructure that will cause significant bottlenecks once IPsec traffic begins to flow through a particular device. An effective network discovery solution will show where all of the devices and ACLs on the network are located, report on their configurations and profiles, and show how traffic flows through them.

From a host point of view, understanding which devices are connected to the network and gathering basic information about their operating system, services, and configuration allows one to understand which version of the operating system is running on Windows servers and desktops for upgrade planning purposes. This information is also necessary to understand the basic profiles for non-Windows machines, since these machines represent the bulk of the untrusted network.

IPsec creates a great deal of overhead that could cause performance problems for any machine in the infrastructure that lacks the performance capabilities to run IPsec. The challenge is to pinpoint these machines, since in a large network, hosts tend to disappear from management. Finding and managing these “unknown and untrusted” hosts is perhaps the greatest single factor in reducing the risk profile of an SD&I migration.

To take full advantage of the SD&I security potential of Vista, IT groups should:

  • Understand your current network connectivity and whether it is aligned with security policy
  • Understand what is physically connected to your network
  • Understand which network assets can support Vista and Longhorn
  • Define the architecture and policy for the future state of the network for SDI implementation
  • Deploy and continuously measure group policies to ensure that the configuration supports enterprise requirements for security and availability

The network security features inherent in Microsoft Windows Vista represent a tremendous opportunity for organizations to improve their enterprise security posture. However, like most systems, group IPsec policy enforcement in Windows Vista is only as good as its weakest link. Implementing policies as part of an SD&I scenario is potentially a risky proposition in a large heterogeneous enterprise, if the proper planning steps aren’t taken. Building a comprehensive view of the network infrastructure and enterprise connectivity is a critical success factor for planning and implementing SD&I.

- - -

David Arbeitel is chief technology officer at Lumeta Corp. You can reach the author at

Must Read Articles