Product, Partners, Events

Real-time stream processing on DC/OS with Concord

For more than five years, DC/OS has enabled some of the largest, most sophisticated enterprises in the world to achieve unparalleled levels of efficiency, reliability, and scalability from their IT infrastructure. But now it is time to pass the torch to a new generation of technology: the D2iQ Kubernetes Platform (DKP). Why? Kubernetes has now achieved a level of capability that only DC/OS could formerly provide and is now evolving and improving far faster (as is true of its supporting ecosystem). That’s why we have chosen to sunset DC/OS, with an end-of-life date of October 31, 2021. With DKP, our customers get the same benefits provided by DC/OS and more, as well as access to the most impressive pace of innovation the technology world has ever seen. This was not an easy decision to make, but we are dedicated to enabling our customers to accelerate their digital transformations, so they can increase the velocity and responsiveness of their organizations to an ever-more challenging future. And the best way to do that right now is with DKP.

Jun 17, 2016

Shinji Kim

D2iQ

Today, we're excited to announce that Concord is available in the DC/OS Universe. Concord is a real-time stream processing framework built on Apache Mesos.

 

As more organizations are embracing stream processing, demands for better performance and the ability to easily operationalize real-time applications have emerged. Concord addresses developer efficiency by automating devops work like provisioning or managing servers when scaling applications.

 

While open source stream-processing systems like Apache Storm and Spark Streaming focus on running Hadoop MapReduce functions in real-time, Concord runs real-time applications as asynchronous services. Some of the unique features that Concord provides on DC/OS include the following.

 

Dynamic deployment of applications

 

Change application logic or scale jobs at runtime (no downtime when changing jobs / topologies).

 

[caption id="attachment_5805" align="aligncenter" width="800"][

Dynamic topology in Concord.

Dynamic topology in Concord.[/caption]

 

Multi-language support

 

Concord supports a broad set of programming languages including Python, Ruby, Java, Scala, C++ and Go with its open-source APIs.

 

2016-06-17-concord-2_multi_lang

 

Performance and built-in tooling

 

Concord's core is built in C++, which provides predictable performance as you scale. It also allows you to debug your application easily using its built-in distributed tracing and logging tools.

 

[caption id="attachment_5905" align="aligncenter" width="800"][

Tracing in Concord.

Tracing in Concord.[/caption]

 

So what would you use Concord for? Concord is designed to run real-time applications like fraud detection, real-time bidding, IoT sensor monitoring and alerting.

 

We decided to integrate with DC/OS as it is the easiest way to run distributed systems frameworks on the cloud. The idea is that Mesos is the "operating system kernel" of your distributed system, abstracting away computing resources and providing distributed programs with APIs for resource management and scheduling; while DC/OS is the operating system existing on top of this "kernel." This operating system can automate resource management, schedule new processes, simplify deployments, and even comes with a web and command line interface.

 

[caption id="attachment_5907" align="aligncenter" width="800"][

Concord in the DC/OS Universe.

Concord in the DC/OS Universe.[/caption]

 

We're excited to be a part of the DC/OS community with a true real-time stream processing framework. To learn more about Concord + DC/OS and how to get started, take a look at our blog post.

Ready to get started?