How The Spam Jam Became Business As Usual

Where there’s spam there’s brass, or at least spammers think so. The technical reason there is spam is that the SMTP protocol used to transport e-mail on the Internet is unauthenticated. A few years ago a solution to this problem was put in place, but it doesn’t seem to have made a difference.

DKIM (DomainKeys Identified Mail) emerged from a battle several years ago as the leading standard for SMTP domain authentication. It uses public key encryption to sign messages with a domain’s private keys and stores the public keys for domains in the DNS. Receiving e-mail servers can use that public key and the digital signature in the message to confirm that a message does indeed come from the domain it purports to come from. This doesn’t prove that the domain it comes from is trustworthy. For that, you need reputation data, which you either gather and manage internally, or acquire from a public reputation service.

Click here for a more detailed paper from the Messaging Anti-Abuse Working Group on authentication and trust. I’ve also been inspired to this by a recent series of articles on DKIM on CircleID.

This is the way it was all supposed to work. It’s years now since DKIM was effectively finalised and almost two years since the DKIM standard was published. DKIM is definitely out there, but not in the way I had hoped. It’s implemented and used by several large e-mail providers, such as Gmail and Yahoo Mail. Many bulk e-mail senders, both large and small, sign their e-mail because it helps to get their mail into the inbox of users on those systems, But DKIM in the real world so far seems to be much more about preventing false positives than blocking spam.

To use DKIM to allow a good sender in, you merely have to have some reputation data about them and track abuse complaints to keep that data current. Then new mail comes in, the signature matches, and zoom, it’s in (you might still want to spam-filter it and certainly you should malware-scan it). To use it to prevent spam you have to deal with senders who either have no signature or who have no reputation, and both these questions are difficult ones. If you’re Gmail you can, over time, develop a pretty extensive reputation database, especially since you also own Postini which hosts enterprise e-mail security. If you’re Acme Average Enterprise and you’re doing your own e-mail you need access to a public e-mail reputation service.

This service business hasn’t developed in the way it might have, and even to the extent that it has, you can’t always use the reputation in the way it’s intended. If verizon.net has a lousy reputation are you going to block e-mail coming from it? This is one reason why domain reputation can’t displace IP-based reputation systems such as Spamhaus any time soon.

Perhaps the answer is to push more e-mail processing into the cloud. It’s seemed to me for a while that large, institutional e-mail processors such as Google were in a better position to implement solutions such as DKIM for large numbers of users, and indeed GMail does this.

Cloud-based e-mail security is increasingly in style, too, as evidenced by Sendmail’s recent announcement of cloud-based e-mail services. Sendmail has long been a proponent of authentication and has long-supported DKIM, so you can use their products to sign your outbound mail as well.

Sendmail argues that cloud-based e-mail security has become commoditised, but the actual catch and false positive rates are probably still more important to customers than price. So if DKIM positively affects catch rates it’s a winning strategy. I just doubt that it’s much of a factor in catching spam, even in the large clouds.

So unfortunately, it seems that this is as good as e-mail is going to get. It’s disappointing, but it gets enough of the job done, and that’s why it’s not going to get any better. I mean, how much more time and money were you really willing to take it to the next level anyway?