For more than five years, DC/OS has enabled some of the largest, most sophisticated enterprises in the world to achieve unparalleled levels of efficiency, reliability, and scalability from their IT infrastructure. But now it is time to pass the torch to a new generation of technology: the D2iQ Kubernetes Platform (DKP). Why? Kubernetes has now achieved a level of capability that only DC/OS could formerly provide and is now evolving and improving far faster (as is true of its supporting ecosystem). That’s why we have chosen to sunset DC/OS, with an end-of-life date of October 31, 2021. With DKP, our customers get the same benefits provided by DC/OS and more, as well as access to the most impressive pace of innovation the technology world has ever seen. This was not an easy decision to make, but we are dedicated to enabling our customers to accelerate their digital transformations, so they can increase the velocity and responsiveness of their organizations to an ever-more challenging future. And the best way to do that right now is with DKP.
May 29, 2015
Michael Hausenblas
D2iQ
3 min read
This installment of the Mesosphere HackWeek series covers the exciting work being done to integrate Apache Kafka and Apache Mesos.
Kafka is a high-throughput, low-latency publish-subscribe messaging system. Originally developed at LinkedIn, Kafka was open-sourced in 2011 and has been adopted widely across the industry at companies such as Cisco, Goldman Sachs, Netflix, and many more. It forms the backbone of critical real-time data flows in these companies, processing hundreds of billions of messages per day and is typically used in tandem with stream-processing systems such as Storm, Spark or Samza.
During the HackWeek, we worked with our partner Confluent (founded by the team that developed Kafka at LinkedIn) on a scheduler that runs on Marathon and allows users to launch any version of Kafka. The scheduler takes care of operational automation, manages broker configurations, exposes a REST API, and provides for a command line interface that enables elastic scaling and smart partitioning.
To learn more about simplifying the operation of Kafka with Mesos and the Mesosphere Datacenter Operating System, see:
- The Kafka-Mesos GitHub repo.
- Slide deck on Apache Kafka, HDFS, Accumulo and more on Mesos by Joe Stein
