Apr 17, 2015
D2iQ
D2iQ
3 min read
In case you missed it on Thursday, Twitter released some details of its large MySQL implementation and a proposal for a new Apache Software Foundation project called Mysos. As the pormanteau suggests, Mysos is a framework for running MySQL (my) on Apache Mesos (sos), based on technology Twitter has developed in-house to manage its own MySQL environment.
Mesosphere is very excited about Mysos because it's potentially very important to whole a lot of companies. In fact, we already have two committers -- Tobi Knaup and Sunil Shah -- on the project.
Why are we so excited? The first reason has to do with MySQL itself. MySQL is still a crucial piece of the puzzle at many companies. It's useful, it's open source, it's adaptable and lots of people know how to work with it.
While web companies large and small have experimented with various database technologies over the years -- and even created several of their own -- many have yet to find a suitable replacement for MySQL for situations where a relational database is necessary. Last year, Alibaba, Facebook, Google, LinkedIn and Twitter united on a project called WebScaleSQL that aims to share with the world the lessons they've learned, and the code they've developed, while pushing MySQL beyond its natural limits in terms of data and performance.
After all that work getting MySQL to scale, the last thing you want to see is your application crash because of problems at the infrastructure level. This is where Mesos, and Mysos, come into play. Here's how Twitter describes the capabilities of Mysos:
It dramatically simplifies the management of a MySQL cluster and is designed to offer:
- Efficient hardware utilization through multi-tenancy (in performance-isolated containers)
- High reliability through preserving the MySQL state during failure and automatic backing up to/restoring from HDFS
- An automated self-service option for bringing up new MySQL clusters High availability through automatic MySQL master failover
- An elastic solution that allows users to easily scale up and down a MySQL cluster by changing the number of slave instances
Of course, where Mesos really shines is not just on providing this type of reliability to a single application, but to a whole collection of supported ones all running on the same cluster. On clusters that can scale to tens of thousands of nodes, no less. Natively supported Mesos frameworks (or services) include Hadoop, Spark, Marathon, MPI, Jenkins, Elasticsearch, Cassandra and more.
Mesosphere's Datacenter Operating System, a commercial product with Mesos at the core, makes it drastically simpler to launch and manage all of these services and more.
Even with all the talk of next-generation databases and big data systems, MySQL is still just as important as ever in many cases. It's actually being asked to do a whole lot more. If Mysos can keep MySQL up, running and scaling just like everything else running on Mesos, it could be a game-changing moment in the history of an already storied database.
