Spotify Shifts To HortonWorks’ Hadoop Platform

Online music service Spotify has moved its giant database of music preferences and playlists to a different big data platform, going to Hortonworks’ Hadoop offering.

Spotify maintains a large database of recommendations and playlists, and uses it to deliver music to legitimate customers, in a fashion that competes with other easy options such as free piracy, according to Spotify officials.  The London-and-Stockholm based firm is shifting from one Hadoop firm, Cloudera, to Hortonworks in October, apparently in the expectation of better support and responsiveness.

Playing a different tune

“Spotify is moving  to Hortonworks from a competitive distribution,”  Hortonworks’ President Herb Cunitz told TechWeekEurope. “Spotify is a very experienced user. They’ve worked with Hadoop for years. They want to have support and want to get things fixed if they have a problem.” Hortonworks was also promising  to get features implemented in the open source Hadoop ecosystem that Spotify needed, he said.

For its part, Spotify said it was impressed by Hortonworks’ efforts to build the Hive data warehouse on top of Hadoop: “The work they have done to improve the Apache Hive data warehouse system aligns well with our needs, as we use Hive extensively for ad-hoc queries and for the analysis of large datasets,” Spotify’s head of infrastructure Wouter de Brie said in a statement.

The implementation will be the largest commercial Hadoop cluster in Europe, running on 700 nodes, Spotify said.

Hadoop is open source, of course, but players compete in the level of support they can offer to the free code, and the level of their commitment to developing the project and tailoring it to their customers’ needs.

Cloudera appears to be the market leader in Hadoop installations at the moment, according to figures from data integration firm Syncsort, with Hortonworks normally reckoned to be running second. However, Hortonworks claims to be the firm that commits most to the open source Hadoop projects, something which Cloudera hotly disputes.

“We have the majority of the committers and wrote the majority of the code,” Cunitz told TechWeekEurope. Earlier in the year, Cloudera chief operating officer Kirk Dunn made a  competing claim: “We have 75 percent of the contributors across all the projects. So we dominate,” he told us, reminding us at the same time that Hadoop founder Doug Cutting founded Cloudera.

Do you know about open source? Try our quiz!

Peter Judge

Peter Judge has been involved with tech B2B publishing in the UK for many years, working at Ziff-Davis, ZDNet, IDG and Reed. His main interests are networking security, mobility and cloud

View Comments

Recent Posts

Apple, Google Mobile Ecosystems Should Be Investigated, CMA Told

CMA receives 'provisional recommendation' from independent inquiry that Apple,Google mobile ecosystem needs investigation

2 days ago

Australia Rejects Elon Musk Claim About Social Media Ban For Under-16s

Government minister flatly rejects Elon Musk's “unsurprising” allegation that Australian government seeks control of Internet…

2 days ago

Northvolt Files For Bankruptcy Protection In US

Northvolt files for Chapter 11 bankruptcy protection in the United States, and CEO and co-founder…

2 days ago

UK’s CMA Readies Cloud Sector “Behavioural” Remedies – Report

Targetting AWS, Microsoft? British competition regulator soon to announce “behavioural” remedies for cloud sector

2 days ago

Former Policy Boss At X, Nick Pickles, Joins Sam Altman Venture

Move to Elon Musk rival. Former senior executive at X joins Sam Altman's venture formerly…

3 days ago

Bitcoin Rises Above $96,000 Amid Trump Optimism

Bitcoin price rises towards $100,000, amid investor optimism of friendlier US regulatory landscape under Donald…

3 days ago