‘Biggest DDoS Attack’ Did Not Hurt The Global Internet – This Time

The massive distributed denial of service (DDoS) attacks on Spamhaus, and a variety of other connected organisations, gave rise to plenty of blarney and dodgy use of facts this week.

But looking at the attack now it has been countered, the following is what we should take away: no, there was no noticeable impact on the global Internet, but this was still the biggest DDoS attack on public record, and there is genuine cause for concern.

First, let’s put the idea that there was any serious impact on the global Internet to bed. There is scant proof this was the case. In fact, the opposite is true. This was a localised issue, one that largely affected Spamhaus, along with at least one of its security suppliers, and caused minor damage to affiliated parties. The eventual trickle down effect to end users of the World Wide Web would have been very minimal indeed.

High level data appears to back up this view. Note the lack of major fluctuations on the Internet Traffic Report site (the only noticeable shift on 22 March could have been caused by the attacks, but a major undersea cable was also damaged in Egypt, which could have accounted for the mini aberration):

Looking deeper, we can see stronger evidence of how the distributed nature of the Internet shrugged off the attacks. CloudFlare, the content delivery network (CDN) which was called in to protect against the DDoS on Spamhaus, told us the attackers couldn’t hit the anti-spam not-for-profit straight on once the added security was implemented. According to the content delivery network provider, they couldn’t hit CloudFlare head-on either, so instead the attackers went after connected networks they believed were closely related enough to Spamhaus to have an impact.

London at centre of major DDoS

But even those affiliated parties, as named by CloudFlare in its blog on what happened, have reported little operational impact. The two Internet exchanges TechWeekEurope spoke to, those based in London and Amsterdam, say they saw some increased traffic, but their platforms remained largely unaffected. Neither said “congestion impacted many of the networks on the IXs”, as CloudFlare had suggested.

“The only thing I can say is that we saw some activities in the amount of traffic, but we didn’t have any problems whatsoever on our platform,” said a spokesperson from the Amsterdam Internet Exchange. “It was not an issue.”

The graph below again shows there was no major fluctuation on the Amsterdam end:

This below shows an anomaly at LINX, the London Internet Exchange, on the 23 March:

Yet, according to our sources, that disruption would not have impacted general Internet users. There was a small amount of collateral damage, but nothing significant, TechWeekEurope understands. LINX itself remained up, though it was believed the attack traffic did in fact hit its corporate network, thanks to an infrastructural decision made years ago. Similar things are believed to have happened at the DEC-IX exchange in Frankfurt. That has now been addressed. Yet very few customers or end users would have noticed any network jam, according to our sources. Even if they did, it was only for a short time.

Here’s why: what is key here is that the attackers were going after the ports CloudFlare was connected to at those exchanges. Somewhat ironically, it means that it would largely have been CloudFlare customers (and in turn their customers, as well as any others connecting to that IP block) who were affected by the attacks on exchanges. The company has essentially admitted to a flaw in the system that allows DDoSers to circumvent protections offered by the CDN. It’s unclear how much damage the alleged overspill into LINX’s own network caused, and it isn’t talking about it.

It should also be remembered that much of the world’s Internet traffic does not go through such exchanges.

Tier 1 attacks

CloudFlare also noted the attackers went after its upstream peers – those organisations it gets and shares bandwidth with. Tier 1 providers, of which there are around a dozen in the world and who ensure every network is connected to one another, sucked up a lot of the traffic, one of them telling CloudFlare the attacks reached 300Gbps. According to the CDN firm, this led to congestion across a number of major Tier 1 organisations.

This is where things get even more interesting. Tier 1 network owners have reported some operational impact at their end, but not to the extent that end users were seeing slow Internet services.

TechWeekEurope spoke to one of those Tier 1 firms, which preferred to remain anonymous. It saw a significant attack between an internal router port and another port that served customers in London – actually reaching a little over 300Gbps, and in two different attacks.

There was a short period of service degradation, our source told us, but the security team was able to apply filters quickly to mitigate the attack, sending traffic across different networks in other cities to lessen the impact.

But, as noted, end users wouldn’t have noticed much. Though it appears London was at the centre of this massive DDoS, here in the UK, Thinkbroadband.com, which tracks users’ speeds via tests on its site, said there was no evidence of any major degradation for the average user. It even saw a slight increase in speeds on 25 and 26 March.

It’s clear that certain pipes in some major organisations serving the global Internet were clogged up for a period of time. But as for end user impact, which is what really matters when it comes to the practical use of the Internet, it wasn’t big. As our source from the Tier 1 provider noted, anyone not connected to the handful of affected parties wouldn’t have seen a thing.

What’s promising is that the architectural design of the Internet’s backbone, its resilience, was able to cope with massive attacks. Traffic is shared to such an extent, across the different tiers and networks, and Tier 1 providers are adapt so quckly, that it is close to impossible to cause any Internet “apocalypse” on a global scale just with some DDoSing.

Everyone should also be wary of claiming the attackers were Cyberbunker. The company sent over a statement on the matter, neither confirming nor denying anything. “The only thing we would like to say is that we (and/or our clients) did not, and never have been, sent any spam. We have no further comments,” said general manager of Cyberbunker, Jordan Robson. Indeed, the firm has now flat out denied the attacks.

Why we should worry

Now, looking at the DDoS attack, let’s talk about how and why this is worrying. As TechWeekEurope noted in our original piece yesterday, the size of the attack at 300Gbps is fairly astonishing. Such mammoth attacks are possible in many cases because millions of open DNS resolvers, integral to how the Internet works, can be exploited.

Attackers send requests to  “open recursive resolvers”, used in the DNS process, where URLs are translated to IP addresses, so people can access websites by typing in names (e.g. Google.com) rather than numbers (e.g. 216.239.51.99).

They do this whilst masquerading as their target, by spoofing an IP address. Once they have made a large number of requests for DNS  files from these open DNS servers, the resolvers respond and send back a load of traffic to the victim, clogging up their pipes and taking them offline.

There are as many as 25 million of these open recursive resolvers at the disposal of attackers, according to the Open Resolve Project, which is being run by Jared Mauch, an engineer at Tier 1 network provider NTT. They can sit anywhere someone sets up a DNS resolver, so ISPs and data centre owners are common owners. Mauch thinks he can convince a lot of them to reconfigure their DNS servers so they can’t be used for huge DDoSes.

“We can get it down to five to 10 percent of 25 million,” he says. “I’ve seen a lot of positive response so far.” Computer Emergency Response Teams (CERTs) and ISPs from across the world have already contacted Mauch asking what they can do to fix the problem.

With great power…

The sheer power of these attacks remains frightening. Since the story broke yesterday, we’ve been hunting for any signs of bigger DDoS attacks, and it appears this really is the most powerful, at least on record. Josh Corman, director of security intelligence at Akamai, believes the largest on record previously was one that went off in the US, which reached 200Gbps.

There are rumours of much bigger attacks. Engineer at CloudFlare Tom Paseka tells us of “war stories” of 500Gbps attacks going on internally in China. He admits he’s “unsure if it’s just conjecture or reality”, but believes DDoSes of that magnitude are not far off.

“I definitely do think that DDoS will hit 500Gbps scale and keep growing. Two years ago, to hear of a 100Gbps DDoS was huge. To now see 300Gbps or more. I don’t expect it’ll be long before we see that double,” Paseka says.

Now, let’s consider that US banks were knocked offline by attacks between 70Gbps and 100Gbps. Wells Fargo is still getting battered as we publish. If banks can’t handle that level of traffic, then how are they going to cope when massive botnets start firing hundreds of thousands of requests at open DNS servers, and financial institutions are hit with DDoS attacks over five times as powerful as what they’ve seen? That’s why the Spamhaus attacks are worrying.

Are you a security expert? Try our quiz!