SAN FRANCISCO, CA and SAN JOSE, CA--(Marketwired - Feb 11, 2015) - Mesosphere, creators of the Mesosphere Datacenter Operating System (DCOS) for managing datacenter and cloud resources at scale, and MapR Technologies, Inc., provider of the top-ranked distribution for Apache™ Hadoop®, today announced Myriad, the first resource management framework that allows Apache YARN jobs to run alongside other applications and services in enterprise and cloud datacenters. Based on an open source and collaborative development effort between MapR, Mesosphere and eBay, Myriad (available on GitHub) is an open source project built on the vision of consolidating big data with other workloads in the datacenter into a single pool of resources for greater utilization and operational efficiency. Concurrently, there are plans to submit Myriad as an Apache Incubator project with the Apache Software Foundation in the first quarter of 2015.
To date, Hadoop developers have been forced to run big data jobs on dedicated clusters, leaving those resources isolated from other applications and services in production, and typically resulting in poor server utilization rates. Myriad leverages both Apache YARN and Apache Mesos, allowing big data workloads to run alongside other applications including long-running Web services, streaming applications (like Storm), build systems, continuous integration tools (like Jenkins), HPC jobs (like MPI), Docker containers, as well as custom scripts and applications.
"Big data developers no longer have to choose between YARN and Mesos for managing clusters," said Florian Leibert, CEO and co-founder of Mesosphere. "Myriad allows you to run both, and to run all of your big data workloads and distributed applications and systems on a single pool of resources. Big data developers get the best of YARN's power for Hadoop-driven workloads, and Mesos' ability to run any other kind of workload, including non-Hadoop applications like Web applications and other long-running services."
Apache Mesos is a distributed systems kernel that abstracts CPU, memory, storage and other compute resources allowing developers to program against the datacenter like a single pool of resources. It forms the core of Mesosphere's Datacenter Operating System (DCOS). With Myriad, organizations running Mesos or Mesosphere DCOS can now run YARN and other big data frameworks on the same Mesos cluster. For YARN users, Mesos unlocks resource scheduling and isolation beyond just CPU -- and allows memory, networking and storage to be dynamically managed on the same cluster that runs their big data jobs.
"Myriad enables businesses to tear down the walls between isolated clusters just as Hadoop enables businesses to tear down the walls between data silos," said Jim Scott, director, enterprise strategy and architecture, MapR Technologies. "Developers can now focus on the data and applications which the business depends on, while IT operations can manage compute resources to maximize business agility and minimize operating expenses."
Additional Resources
- Project Myriad
- Apache Mesos
- Apache Hadoop NextGen MapReduce (YARN)
- How does YARN compare to Mesos? (Quora)
About Mesosphere
Mesosphere (Twitter: @Mesosphere) is building the datacenter operating system (DCOS) to help enterprises unlock the next generation of scale, efficiency and automation. The Mesosphere DCOS pools datacenter and compute resources, gives IT operators a much simpler administration model, and improves developer velocity with more modern abstractions and APIs for writing distributed systems. Backed by Andreessen Horowitz, Data Collective, Fuel Capital, Khosla Ventures and Kleiner Perkins Caufield & Byers, Mesosphere is headquartered in San Francisco with a second office in Hamburg, Germany.
About MapR Technologies
MapR delivers on the promise of Hadoop with a proven, enterprise-grade platform that supports a broad set of mission-critical and real-time production uses. MapR brings unprecedented dependability, ease-of-use and world-record speed to Hadoop, NoSQL, database and streaming applications in one unified distribution for Hadoop. MapR is used by more than 700 customers across financial services, government, healthcare, internet, manufacturing, media, retail and telecommunications as well as by leading Global 2000 and Web 2.0 companies. Amazon, Cisco, Google, Teradata and HP are part of the broad MapR partner ecosystem. Investors include Google Capital, Lightspeed Venture Partners, Mayfield Fund, NEA, Qualcomm Ventures and Redpoint Ventures. MapR is based in San Jose, CA. Connect with MapR on Twitter, LinkedIn, and Facebook.