Flink Forward in San Francisco
For more than five years, DC/OS has enabled some of the largest, most sophisticated enterprises in the world to achieve unparalleled levels of efficiency, reliability, and scalability from their IT infrastructure. But now it is time to pass the torch to a new generation of technology: the D2iQ Kubernetes Platform (DKP). Why? Kubernetes has now achieved a level of capability that only DC/OS could formerly provide and is now evolving and improving far faster (as is true of its supporting ecosystem). That’s why we have chosen to sunset DC/OS, with an end-of-life date of October 31, 2021. With DKP, our customers get the same benefits provided by DC/OS and more, as well as access to the most impressive pace of innovation the technology world has ever seen. This was not an easy decision to make, but we are dedicated to enabling our customers to accelerate their digital transformations, so they can increase the velocity and responsiveness of their organizations to an ever-more challenging future. And the best way to do that right now is with DKP.
The Apache Flink community recently improved support for Apache Mesos and DC/OS in the latest release. Though there were always ways to run Flink with Mesos (and a user survey by data Artisans shows that 30% of recent poll respondents were doing just that), Flink 1.2 can now run as a Mesos framework, which makes deployments on both Mesos and DC/OS simpler.
During his opening remarks, Jamie Grier of data Artisans mentioned that Flink has really great support on Mesosphere's DC/OS and it was a real pleasure for me to see DC/OS listed as a supportive vendor. It made me really proud to be part of a project that's helping other communities to grow by making it easier to deploy and run their technologies, like Flink, in production.
In other morning keynotes, Netflix and Uber showcased their streaming-data journeys, which each culminated in Flink adoption.
Following the keynotes Ravi and I gave our presentation, titled Flink Meet DC/OS: Deploying Flink at Scale. At the beginning of the talk I asked the audience how familiar they were with Mesos and DC/OS, and was glad to find that they were the perfect audience for the introductory-level DC/OS presentation Ravi and I had prepared.
I started by introducing DC/OS itself, and key components like Mesos and Marathon. Ravi then gave a streaming data pipeline demonstration, and to ran through our Flink example, which details the simple and advanced install options with the DC/OS GUI . Finally, I highlighted some of the key features in the DC/OS 1.9 release: unified logging and metrics, along with the new pods and GPU capabilities.