Categories: CloudDatacentre

UK Hosting Giant 123 Reg ‘Deletes’ Customer Sites

Customers of registrar and web hosting company 123 Reg have expressed outrage after the company admitted to having accidentally deleted an unspecified number of virtual servers over the weekend.

The incident is potentially significant in scale – 123 Reg is the UK’s largest domain name registrar, having registered more than 3 million domain names, and also provides website hosting services for more than 1.7 million sites.

The company is part of Host Europe Group (HEG), which claims to be Europe’s largest privately owned hosting company.

Server deletion

The error, which occurred on Saturday morning, involved a routine maintenance procedure that went awry, resulting in the erasure of customer virtual private servers (VPS), effectively deleting customers websites and online operations, according to an email sent to affected customers on Sunday.

As part of a “clean-up process”, the company ran a script on Saturday morning at 7 am that was intended to indicate virtual server activity, but the script erroneously showed some VPSs as running no servers, and as a result automatically deleted what was in fact running on the host, 123 Reg brand director Richard Winslow said.

“For those customers, this created a ‘failure’ scenario – showing no VM’s and effectively deleting what was on the host,” he wrote, adding that the result was “data loss” for affected customers. 123 Reg didn’t indicate how many customers were affected.

The company’s system status page said some customers were experiencing server “connectivity issues”.

‘Business killer’

“We apologise for any inconvenience this may cause,” 123 Reg stated in a Saturday morning status message.

The company said it has been working since the incident with both its own staff and external consultants to restore the affected servers, indicating the likelihood that a large number of sites were affected.

As of Monday morning customers were still indicating that their services hadn’t been restored.

“INCONVENIENCE …. Understatement of the century!!” wrote one user on the company’s Twitter page.

“This is a business killer,” another wrote, with a third user saying eight of his sites had been affected.

An online retailer selling mobile phone accessories and audio equipment said via Twitter that its site had been affected by the “catastrophic” event, and asked users to send queries to its Facebook page. As of Monday morning the company’s website was still unavailable.

‘Mass deletion’

Winslow didn’t indicate whether all the affected data was sure to be restored, leading some users to speculate via social media that the incident might represent not an outage, but effectively a “mass deletion” of customer sites.

One user pointed out that last September 123 Reg itself posted a Twitter message warning of the dangers of data loss. “This has come back to haunt you,” the user wrote.

Winslow said the company has now instituted measures designed to prevent such an incident from occurring again, including requiring human approval for the deletion of virtual servers. He said the company plans to roll out an automatic backup platform by the end of the year.

“I understand that some customers may have lost some confidence in the service that we offer,” he wrote, adding that users can manually restore their own sites if they have a backup.

123 Reg wasn’t immediately available to provide further comment. The company plans to provide updates directly to users as they become available, according to Winslow.

IT security experts recommend online businesses have back-ups of their sites and have a disaster recovery plan in place.

Think you know everything about IT fails? Try our quiz!

Matthew Broersma

Matt Broersma is a long standing tech freelance, who has worked for Ziff-Davis, ZDnet and other leading publications

View Comments

  • There has been no 'speculation' about deletions, 123 themselves described what happened in detail. They had scripts set up to automatically delete (WTF) empty servers, and another faulty script reported them all as faulty. And they have no backups. But as a result of all this, there planning on implementing the ability for them to back up their servers.. By the end of 2016.

Recent Posts

Apple Sales Rise 6 Percent After Early iPhone 16 Demand

Fourth quarter results beat Wall Street expectations, as overall sales rise 6 percent, but EU…

21 hours ago

X’s Community Notes Fails To Stem US Election Misinformation – Report

Hate speech non-profit that defeated Elon Musk's lawsuit, warns X's Community Notes is failing to…

22 hours ago

Google Fined More Than World’s GDP By Russia

Good luck. Russia demands Google pay a fine worth more than the world's total GDP,…

23 hours ago

Spotify, Paramount Sign Up To Use Google Cloud ARM Chips

Google Cloud signs up Spotify, Paramount Global as early customers of its first ARM-based cloud…

2 days ago

Meta Warns Of Accelerating AI Infrastructure Costs

Facebook parent Meta warns of 'significant acceleration' in expenditures on AI infrastructure as revenue, profits…

2 days ago

AI Helps Boost Microsoft Cloud Revenues By 33 Percent

Microsoft says Azure cloud revenues up 33 percent for September quarter as capital expenditures surge…

2 days ago