IBM Launches Apache Spark-as-a-Service Offering

IBM launched a new Apache Spark-as-a-Service offering to help organizations better wrangle big data for real-time insights.

Announced at the IBM Insight 2015 conference here, the availability of IBM’s Spark-as-a-Service offering – IBM Analytics on Apache Spark – on IBM Bluemix follows a successful 13-week beta program with more than 3,000 developers using it to build intelligent business and consumer apps fueled by data. The company also redesigned more than 15 core analytics and commerce solutions with Apache Spark — helping to dramatically accelerate their real-time processing capabilities.

IBM said today, about 80 percent of all the available data — images, voice, literature, chemical formulas, social expressions, external – is not being utilized and companies are struggling to effectively analyze and capitalize on its value. To address this challenge, IBM is unveiling the industry’s first real-time Insights Services on the cloud, a key shift in its analytics strategy through the redesign of its portfolio based on Spark and new cognitive capabilities to advance capture and extract more information and insight from enterprise documents.

Speed

Apache Spark is an open source data processing engine built for speed, ease of use, and sophisticated analytics. Spark is designed to perform both batch processing and new workloads like streaming, interactive queries, and machine learning.

“Spark is undoubtedly a force to be reckoned with in the big data ecosystem,” said Beth Smith, general manager of the Analytics Platform for IBM Analytics. IBM has invested heavily in Spark.

IBM noted that some analytics vendors respond to the big data challenge by focusing on one product to handle traditional types of data, offering grab-bags of hard-to-use analytics tools and selling proprietary products that are out of step with today’s open-standards ethos and rapidly changing technology environment. IBM is working to make data analytics easier for business leaders and developers alike, to embrace openness, and to accelerate innovation by spreading cognitive technologies to every corner of business and society.

Big Blue’s new Insight Cloud Services deliver data and data-science based insights using external data about people, events, geospatial and businesses from sources such as Twitter and The Weather Company. In what IBM refers to as the “Cognitive Era,” organizations can use Insight Cloud Services to gain clarity about their business, find new demand signals and act more confidently. These services don’t require deep domain knowledge to deploy, can be used in action-oriented decision-making or embedded in mobile apps, enterprise applications and business processes.

By applying a combination of advanced imaging, natural language processing, and machine learning technologies, IBM is introducing a new cognitive offering, IBM Datacap Insight Services, that can automatically classify and understand a document – including format and structure, as well words and numeric information – from any document type to quickly and accurately determine the appropriate action to take, IBM said.

“By embracing the insights-as-a-service model, you’re not buying raw data or analytics tools. You’re purchasing solutions to the problems and challenges you face every day,” said Rob Thomas, vice president of IBM Analytics, in a statement. “There is no need for you to make and manage separate arrangements with numerous data providers, to stay on top of the latest analytics technologies, or to recruit and retain the most expensive professionals in the insights economy–data scientists. IBM handles all of that for you.”

In June, IBM announced a series of moves to invest in and further commit to Spark as a centerpiece of its big data platform.

“IBM is building Spark into the core of our analytics and commerce platforms,” Joel Horwitz, director of the IBM Analytics Platform, told eWEEK. “Additionally, we’ll offer Spark as a Service on IBM Bluemix, host Spark applications, and offer free Spark online courses to educate a million people worldwide. In addition, IBM will also offer enterprise level support and consulting to our clients. Spark enhancements will extend well beyond IBM Analytics into all part of the business.”

IBM also said it will commit more than 3,500 researchers and developers to work on Spark-related projects at more than a dozen labs worldwide.

Moreover, the company opened a Spark Technology Center in San Francisco for the data science and developer community.