Met Office Shift To The Cloud Fuelled By Demand For Weather Data
IN DEPTH: Met Office discusses how demand for data is fuelling a move to the cloud and new super computer will improve the weather forecast
Brits are obsessed with the weather. It might be a clichéd subject for small talk but it can also be a serious matter. From impacting our daily travel plans to freak incidents endangering our lives – it matters.
Since 1854 the Met Office has acted as the UK’s national weather service and has operated as an agency of the Ministry of Defence (MoD) and the Department for Business, Innovation and Skills (BIS).
Such is its influence that the Met Office accounts for as much as a fifth of all government web traffic on a peak day. And this demand is driving a significant IT transformation at the organisation.
Technical shift
“Almost any activity you can think of its impacted by the weather and we provide a wide range of services, many of which are 24-7, 365 days a year,” James Tomkins, chief enterprise architect at the Met Office said at AWS Summit in London.
“Every day we collect more than 300 million observations from observing networks around the world. We use this to perform 20 quadrillion calculations on a super computer.
The supercomputer referred to by Tomkins is a £97 million Cray CX40 system, based at the organisation’s Exeter headquarters, and due to reach full capacity next year.
The system is 13 times more powerful than the previous system and will be able to improve the current status of forecasting out to the coming months, rather than days or weeks.
Super computer
It will also provide highly detailed information for precise geographical areas, which should help in preparation for extreme weather events such as floods and heavy snowfall.
The ability to produce high resolution models with a spatial resolution of 300 metres will also help to track and predict the impact of small scale, high impact weather, such as heavy fog or high winds preventing airplanes from taking off. In total the system will be capable of 23 trillion calculations per second.
“We see growth of data and a huge appetite for the data,” continued Tomkins. “Weather data has moved from a few specialists into the heart of decision making for businesses and government.”
This demand has not just seen the purchase of a new supercomputer but driven a move away from on-premise to cloud. Tomkins said there was a desire for rapid innovation and scalability and eighteen months ago, the Met Office moved from an FTP to an API-led development strategy.
“We engaged a strategic partner, Cloudreach, to develop these capabilities and help us navigate the unknowns of moving to the cloud,” he explained.
Cloud migration
“We also leveraged the power of the AWS platform. At the time there was a fear of lock-in, but we feel the architectural benefits of using pre-existing services outweigh the risk.”
The Met Office already has some APIs which Tomkins says allow it to scale and rollout new services quickly, but also fail fast and cheaply if they don’t work out.
There are plans to build a cloud ops team to replace outsourcing, best security practices, and look at financial options like the AWS spot market. This involves bidding for spare capacity, which can be cheaper than the on-demand price.
“As we move more APIs into the cloud, we’re looking to improve connectivity with Direct Connect,” he added.
Earlier this year the Met Office launched its new forecast mobile app, which it claims delivers faster, more accurate information.
