EMC Embraces Open Source With New EMC {code} Projects

EMC has shared details of the latest projects in a series of open source contributions under the EMC {code} umbrella, including the Polly open source framework that enables storage allocation in scheduling environments such as Docker, Cloud Foundry, Mesos and Kubernetes.

At EMC World in Las Vegas today, the company also revealed new integrations and enhancements to EMC’s REX-Ray open source storage orchestration engine for containers.

48 projects

EMC’s Community Onramp for Developer Enablement, known as EMC {code}, was founded in 2014 with the mission to support third platform development and open source communities through contributions to critical open source projects, engagement and technical solution leadership. Since its inception, the team has released 48 projects, with the EMC {code} community contributing more than 350,000 lines of code to the open source community in 2015 alone.

Through this work, EMC technologies are gaining increasing relevance to open source infrastructure communities such as Docker and Mesos. These new technologies represent an emerging and viral market for persistent applications in containers, and EMC {code} is dedicated to assuring continued relevance and affinity for EMC’s software and physical infrastructure products.

Josh Bernstein, VP of technology, EMC {code}, EMC Corporation, said: “Open source and software-based infrastructure is becoming critical to our customers. Early adopters are seeing tremendous value through integration and operating infrastructure as code. In a challenging and quickly evolving eco-system, the EMC {code} team is making it possible for customers to leverage open source solutions and containers as a pillar in their IT strategy.”

Borne out of the need to expose storage as a first-class citizen in scheduling environments, Polly – the name of which derives from polymorphic volume scheduling’ – is an open source framework for Cloud Foundry, Docker, Kubernetes, Mesos, and others. Polly implements a centralised storage scheduling service that connects to container schedulers. It can simultaneously be used to explicitly offer resources to any number of these schedulers.

Polly will be further developed to create a framework that enables the scalable offer-acceptance pattern of consuming volumes across the emerging eco-system of container and storage platforms. The ability to offer storage with other compute resources is an evolutionary leap past other container projects, and elevates storage to become as accessible as other resources.

Container-based infrastructure represents a substantial evolution in the way applications are developed, deployed and managed in production. Adding persistence extends the types of applications that can be containerised and opens the door to new opportunities for databases, key-value stores, infrastructure services such as DNS, and more. For efficiency, the scheduler needs to understand the underlying storage infrastructure to properly allocate storage resources within orchestration engines at scale.

Previously, container schedulers focused solely on compute, memory, and network resources for container deployments. As applications within containers begin to require persistent back-end storage, the need arises for storage to be available as a scheduled resource. Polly fills this role to integrate storage as an open framework to multiple container scheduling solutions.

The open source container ecosystem is diverse, and implementations vary; a polymorphic solution, which allows code to evolve while keeping the original algorithm intact, is critical to enabling common features and approaches of integrating storage into the supportive ecosystem.

Larry Rau, director of architecture and infrastructure, Verizon Labs, said: “REX-Ray and its integration in Apache Mesos enable simplicity in Verizon’s application deployments and efficiencies throughout our Mesos cluster. Storage persistence capabilities from REX-Ray help take advantage of benefits that containers provide when applying them to Cassandra and other applications.”

CJ Desai, president, emerging technology division, EMC, said: “EMC has long supported the open source community and through these new projects we’re focused on enabling the development, deployment and maintenance of modern software. With container-based technologies designed for storage allocation and management, EMC is laying the foundation for users to address some of their most pressing challenges to support innovation for 3rd Platform use cases.”

Key Features of Polly

Centralised control and distribution of storage resources
Offer-based mechanism for advertising storage to container schedulers

Framework supporting direct integration to any container scheduler, storage orchestrator, and storage platform

Polly supports the following storage platforms:

EMC: ScaleIO, XtremIO, Isilon, VMAX

Cloud: Amazon EC2, Google Compute Engine, OpenStack, RackSpace

Laptop: VirtualBox

New REX-Ray Enhancements

EMC {code} also announced REX-Ray 0.4, the latest version of the EMC open source project that delivers persistent storage across containers for runtimes, such as Docker and Mesos. REX-Ray offers vendor-agnostic persistent storage for containers and provides a simple and focused architecture for enabling advanced storage functionality across common storage, virtualisation and cloud platforms.

REX-Ray 0.4 contains a variety of new updates through community and developer advocate contribution including updates to driver packages, security and client/server models. This release includes significant architecture updates to REX-Ray designed to ensure greater flexibility when deploying and centrally controlling containers. As an open source project, new features and functionality will continue to be added to REX-Ray aimed at delivering value and driving the container ecosystem to help make storage directly integrate with container platforms.

REX-Ray’s Key New Features:

New optional client/server model architecture for centralisation of control and Polly integration

Compatibility with Docker 1.11 Volume API

Support for EMC ScaleIO v2.0

Try our open source quiz!