Cloud & Donation Platform Help Comic Relief Cope With Red Nose Day

Comic Relief CTO Zenon Hannick explains how downtime on Red Nose Day could cost charity tens of millions of pounds and how its IT is set up to avoid that

For more than 30 years, Comic Relief has been raising money for good causes and its Red Nose Day and Sports Relief events are television institutions.

Comedians, celebrities and athletes offer their time and services to help encourage the public to donate through various entertainment segments.

Over the course of three decades, the charity’s appeals have raised more than £1 billion and at the last Red Nose Day, £99.4 million was generated alone.

Revamping the donation platform

A significant amount of the donations are collected on the night of the telethons. This means two things for Comic Relief CTO Zenon Hannick. The first is that it has to be as easy as possible for people to donate and the second is that there can be no downtime.

As a result, Comic Relief has embarked on a major transformation of its donation platform and it also employs a multi-vendor strategy to ensure that if one supplier fails, the organisation can still accept contributions.

“If our platforms went down, we would lose tens of millions of pounds, he tells TechWeekEurope. “We have redundancies across all layers of our application. We can survive the loss of a payment provider or a cloud provider, but what we couldn’t survive is if a major part of the internet went down.”

Hannick initially joined Comic Relief in 2010 as a developer, working on the donations platform, before he assumed his current role.

“At that point those platforms were handcrafted,” he says. “Essentially, it was a Java application that required the coming together of 14 different partners.

“HP would loan us a load of servers for free and we would ship them to a data centre company. IBM, Oracle and some other names [were involved].

Real time payments

“[But in 2011] we nearly broke the platform. It was extremely successful for us but we flooded our internal network with traffic and it was very close to topping over. Year on year, donations were doubling. We looked at it and decided to rebuild it.

“What’s unique about our event is that immediately after a film [showing a cause], we get spikes in donations. These spikes are short-lived and if you don’t capture the money at that point [people] won’t come back. It’s not like Glastonbury, people won’t click refresh.”

Under the old system, donors’ details would be collected and then stored so they could be processed later. Hannick and his team wanted this to be done as close to real time as possible.

High risk platform

In his role as CTO, Hannick oversees a permanent tech team of around 45, which oversees IT operations, human resources, internal systems and web development. It partners for infrastructure, while other aspects – including the online shop and donations platform itself– are third party.

In 2012, Armakuni was approached by Comic Relief to help it create a platform capable of processing 2,000 donations per second. It uses CloudFoundry’s Platform-as-a-Service (PaaS) so it’s Infrastructure-as-a-Service (IaaS) can be spread out across multiple providers and availability zones to minimise the threat of downtime.

Comic Relief also had the support of WorldPay, but again the spirit of a multi-vendor approach led to to seek more help. But this wasn’t easy as the organisation’s demands, coupled with the high profile of its campaigns meant that if something went wrong, a supplier could be “tarnished by association”.

Braintree platform

However the arrival of new entrants into the payment service arena changed things, Hannick says, claiming younger providers were not encumbered with legacy systems and understood the new API-driven direction Comic Relief wanted to take. Braintree was one such vendor and led to the two companies agreeing a partnership.

“The first tangible benefit is redundancy,” he says of the tie-up. “The other thing is we haven’t had a close working relationship with PayPal [parent company of Braintree], so them being part of the same group means the majority of our traffic goes through them.”

“We don’t have the capacity or money to look at what’s going on in the rest of the world. [Braintree’s] business is based on this: if there’s a new payment mechanism, they know about it. We want to make it as easy to pay us.”

Braintree can also use its experience with large enterprises to help Comic Relief deal with security threats too.