In-Depth

Removing the Risks of SSL -- Part 2 of 2

To address the security risks of SSL, we look at how transparent SSL proxies work and how they are used in enterprises today.

• 04/13/2010

by David Wells

In part 1 of this series, we explained the risks that arise from increased use of SSL within enterprise networks. This week we explain how transparent SSL proxies work and provide examples of common use cases in enterprise networks.

Transparent SSL proxies allow existing enterprise security appliances to “inspect” the decrypted contents of SSL flows. Transparent SSL proxies need to perform a number of functions including:

It's important to distinguish between transparent SSL proxies that allow the associated security appliance to “inspect” the decrypted session and those that allow “modification” of the session. As the transparent SSL proxy is “invisible” to the client and server, the best approach is to only allow “inspection,” as this will prevent the associated security appliance from being able to change the content without the knowledge of the client or server. A proxy that only allows “inspection” will decrypt the data from the client, send the unencrypted version to the security appliance, and recrypt the original data for transmission to the server. Any changes made in the decrypted data by the security appliance and sent back to the proxy will be ignored.

Detecting SSL Traffic

Typically, traffic to an HTTPS Web server uses TCP port number 443, but this is not guaranteed. Although it is possible to control the port number used for SSL by servers within the enterprise, there is nothing preventing servers outside the enterprise from using other port numbers.

Relying on the port number to identify SSL traffic will not detect all SSL traffic in the network. A better approach is to inspect all traffic, looking for the SSL handshake sequence. This approach will detect all SSL traffic in the network regardless of the port number in use. As a transparent SSL proxy needs to be “in-line,” it already sees all the traffic making it feasible to find SSL traffic by payload inspection (even at Gigabit line rates).

Determining If an SSL Session Should be Inspected

Not all SSL traffic can be inspected by a transparent proxy. For example, SSL sessions that use both client and server certificates cannot normally be inspected. SSL can choose from a wide range of cipher suites when determining which suite to use for a particular session. If the suite chosen is not supported by the transparent proxy, inspection will be impossible.

In addition to SSL traffic that cannot be inspected, there is also traffic that the enterprise chooses not to inspect. For example, an enterprise would not want to inspect HTTPS traffic from an employee to their Internet banking service. Decisions about what traffic to inspect need to be under the control of the enterprise and will be influenced by corporate policies and/or local regulatory and legal environments.

A typical enterprise deployment may involve inspecting all SSL traffic going to servers in the enterprise that are providing HTTPS services to external users. At the same time, SSL traffic from employees in the enterprise to external SSL-based services may be inspected depending on the service being accessed. Employees' HTTPS access to Gmail or Salesforce.com would be inspected but traffic to Internet banking sites would not.

Inspecting Traffic to an Enterprise SSL Server

In a typical scenario, a transparent SSL proxy is deployed alongside an existing intrusion prevention system (IPS) to stop external attacks on an enterprise server that is hosting HTTPS services. In this situation, the transparent proxy requires a copy of the server's certificate and its private key which are loaded into the proxy and used in a “man-in-the-middle” (MITM) session. Because the enterprise has control of the SSL server(s) and of the transparent proxy, obtaining the server certificate/keys should not be an issue.

When operating in this mode, the client will receive the server's certificate and public key (from the transparent SSL proxy) and will authenticate the certificate just as if it had come from the server directly. There is no requirement to make changes to the client.

Inspecting Traffic to an SSL Server on the Internet

Typically a transparent SSL proxy is deployed alongside an existing data loss prevention (DLP) or network forensic appliance to detect and prevent leakage of sensitive information from the enterprise. In this situation, the transparent SSL proxy needs to operate differently because there is no way for the enterprise to obtain the SSL server certificate and private key.

To inspect SSL traffic in this scenario, the transparent SSL proxy needs to:

• Intercept the server certificate message that the server sends to the client
• Replace the servers' public keys with public keys from the transparent SSL proxy
• Re-sign the server certificate with a certificate authority (CA) that is installed in the transparent SSL proxy
• Send the re-signed certificate to the client

If the client system already trusts the CA used by the transparent proxy to re-sign the certificate, it will authenticate the server certificate without any problem. If the client browser does not trust the CA, a warning will be generated to the user that they should not trust the session.

As the client in this instance is a machine inside the enterprise, there are two ways to avoid the browser generating warning messages. One approach, common in larger enterprises, is to take an existing enterprise CA that the clients already trust and load this into the transparent proxy for use when re-signing server certificates. An alternate approach is to export the CA being used by the transparent SSL proxy and add this CA to the set of trusted CAs on client systems. Either of these approaches is simple to implement and will ensure that no warning messages appear on the client system as the clients are part of the same enterprise that has control of the proxy.

Policy Control over SSL traffic

As a transparent SSL proxy is deployed in the enterprise network, it is in a position to detect all SSL traffic and to apply policies that are specific to SSL.

One example of this is enforcing a policy to control the types of HTTPS services that clients in the enterprise can visit. An SSL server has a server certificate that can be signed by a trusted CA, such as Verisign, so the client can authenticate the server and trust it. SSL servers can create their own SSL server certificate, a self-signed certificate, that is not signed by a trusted CA. Clearly a server using a self-signed certificate is less trustworthy than one with a certificate signed by a publicly trusted CA.

Configuring a policy into the transparent SSL proxy to prevent any SSL sessions that are using self-signed certificates is a simple way to enhance enterprise security. This method prevents enterprise users from setting up HTTPS connections to less trustworthy servers on the Internet. Many other types of policies can be configured, specific to particular SSL services or types of SSL connections.

A Final Word

Hopefully we've shown that the additional security threats posed by increasing levels of SSL traffic in the enterprise network can be removed by deploying a transparent SSL proxy allowing flexible inspection and control over SSL. Providing a decrypted copy of SSL traffic to the existing security appliances in the network maximises the use of existing assets and minimizes deployment costs.

David Wells is vice president of technology and general manager of Europe for Netronome with 18 years of experience in the data and telecommunications equipment sector. He has successfully led product development activities in both start-ups and large companies. At a corporate level, he has experience developing strategy, supporting M&A activities, corporate finance, and leading technology and marketing activities. David has a Ph.D. from Cranfield Institute of Technology and holds several patents.