The average IT admin needs to be concerned about a wide range of security threats, such as the prospect of a security breach and denial of service (DOS) attacks. In this post we shall look at five steps admins need to take to protect their Exchange Server deployments from security attacks.

1. Be persistent about security updates

Ensuring that important patches and security updates are applied in a timely fashion is a must when it comes to protecting an Exchange Server from security breaches.  On the downside, if done manually, the installation of a security update can be a time-consuming affair for larger deployments due to the need to take systems offline. David Kelleher touches on this in his post where he suggests the best practices for running server patch management.

On the other hand, the judicious use of virtualization can minimize downtime by allowing administrators to easily test new updates before an actual rollout.  And assuming that mailbox databases are stored on a SAN, the option exists to perform a rollback should catastrophic problems surface at a later stage. Of course, other benefits such as higher scalability and rapid disaster recovery apply.  Indeed, virtualization vendor VMware has put together some nice pages on using Exchange Server with virtualization.

Maintain separation using firewall

The creation of server roles in Exchange Server has served to greatly alleviate the challenges of protecting a general purpose email server against external attacks.  Regardless, it would be foolhardy not to place an Edge Transport Server behind properly configured firewalls, preferably within a DMZ.

The concept is simple: to reduce the attack profile by allowing only essential services to be exposed to the Internet.  This is the same philosophy that Microsoft applied to its upcoming Windows Server 8 operating system where the software vendor removed the GUI from the basic base Server Core installation so as to reduce security risks to an absolute minimum.

And while we’re on the topic of narrowing the attack profile of an Exchange Server, it makes sense to tweak things on the network front such as the disabling of HTTP (allowing only HTTPS), as well as ensuring that default digital certificates are not used on Internet-facing server roles.

3. Protecting against DoS attacks

The hard truth is that there is really no easy way to defend against DOS attacks without huge investments to acquire the requisite expertise and to bolster one’s underlying infrastructure capabilities. For most companies faced with a determined and competent attacker, the only viable solution would be to seek the assistance of a DDoS mitigation vendor.

Fortunately, there are a number of tricks that an administrator can employ to foil the occasional troublemakers.  On an Exchange 2010 Transport Server, for example, the Set-TransportServer cmdlet can be used to modify the default control message processing rates, SMTP connection rates and SMTP session time-out values. Moreover, the Set-ReceiveConnector cmdlet can be used to configure inactivity timeouts, maximum number of connections and allowable SMTP protocol connection errors.

Finally, the Set-POPSettings and Set-IMAPSettings cmdlets can be used to configure parameters related to POP and IMAP.  The last two are particularly useful for organizations that don’t implement VPN security but allow users to download their emails from external networks. Ram Mohan’s post on how to defend against DDoS attacks touches on generic techniques further.

Have external parties conduct penetration tests

The simplest way to know what hackers are thinking would be to hire someone who can reason in the same way and then task them with finding ways to break into your system.  It is an acceptable practice these days to hire penetration testing engineers, also known as ‘white hats’, to find weak spots in a company’s IT setup.

Protecting against zero-day vulnerabilities

By definition, zero-day vulnerabilities are not detectable with current antimalware defenses. It is therefore unfortunate that an increasing number of attacks have been shown to utilize novel exploits.  One possible way of defending against zero-day vulnerabilities would be to install antimalware defenses known as whitelisting software. While nothing is absolute, the use of whitelisting software should offer a level of additional protection against the execution of ‘helper’ software such as RAT (Remote Administration Tool) commonly installed to facilitate hackers’ entry into a compromised server.

Following these five steps may not guarantee ultimate protection, but it will definitely mean you are making the best out of the technologies and methods available to protect your Exchange Server.

Like this post?

If you like this post and would like to receive more Exchange Server tips, as well as the latest Exchange Server posts from across the web, plus a free ebook with 42 Exchange tools, subscribe to the IT Dojo – Exchange Sensei series!