Certified Packages for Production-ready Data Services on DC/OS
For more than five years, DC/OS has enabled some of the largest, most sophisticated enterprises in the world to achieve unparalleled levels of efficiency, reliability, and scalability from their IT infrastructure. But now it is time to pass the torch to a new generation of technology: the D2iQ Kubernetes Platform (DKP). Why? Kubernetes has now achieved a level of capability that only DC/OS could formerly provide and is now evolving and improving far faster (as is true of its supporting ecosystem). That’s why we have chosen to sunset DC/OS, with an end-of-life date of October 31, 2021. With DKP, our customers get the same benefits provided by DC/OS and more, as well as access to the most impressive pace of innovation the technology world has ever seen. This was not an easy decision to make, but we are dedicated to enabling our customers to accelerate their digital transformations, so they can increase the velocity and responsiveness of their organizations to an ever-more challenging future. And the best way to do that right now is with DKP.
Are you just starting out with DC/OS? Do you want to know more about one of our central features, the DC/OS Certified catalog packages? In DC/OS 1.10, we released a set of Certified packages with the main goal of making it easier for our customers to design and build production-ready data services. The most commonly deployed Certified packages include Apache Spark, Apache Kafka, Elasticsearch, Apache Cassandra, and Apache Hadoop. In this blog post, we are going to explain the benefits of using Mesosphere's Certified packages, outline the certification criteria, and help you get started using a data service today.
If you would like to follow along and test Certified packages, we have some recommendations for setting up a DC/OS cluster. First, your cluster of nodes should be installed with DC/OS version 1.10 or newer. The cluster should have at least one Master node, three Private agents, one Public agent, and one Bastion host. The Bastion host is often called a Bootstrap node. Here is a quick start guide for installing a cluster. Finally, we recommend installing the DC/OS CLI on your Bootstrap node and have it pointed towards your Master node.
Benefits of Certified DC/OS Services
There are a multitude of benefits when using Mesosphere's Certified packages. Possibly the greatest benefit is consistency. All the Certified packages have the same core set of features and operations. After learning the operational features of one package, you can apply then to the others. Shared features and operations include:
- Single-line install command. In addition to having a simple installation process in the DC/OS UI, you can also install any Certified package with the dcos package install command. For example, dcos package install cassandra or dcos package install kafka.
- Built-in health monitoring. By default, the application definition of every Certified package includes a readiness check and a health check. Check out our docs to learn more about health checks.
- DC/OS CLI subcommands. The subcommands allow you to interact with your package, discover information about the service, and manage the package.
- Rolling updates. A package can be upgraded with a rolling update. The DC/OS CLI subcommands makes this process straightforward and simple. For example, dcos cassandra update start --package-version="1.1.6-5.0.7".
- Rolling configuration updates can initiated with a DC/OS CLI subcommand. For example, dcos cassandra update start --options=options.json. This process also allows you to scale out the service.
- Configurable placement constraints. These allows you to dictate where each service instance should be located within the DC/OS cluster. For example, one instance per DC/OS agent.
- Designed for security. Application folders were designed for security and multi-team deployments.
- They are supported. The Certified packages are maintained by Mesosphere. Additionally, if you have DC/OS Enterprise you can request assistance from Mesosphere Support.
These are the core benefits of using a Certified package. To learn more about an individual package, please refer to the service documentation.
To ensure that each Certified package is production ready, they also must meet a set of certification criteria. For each of the following categories there are a set of requirements:
Certified DC/OS Service Example: Apache Kafka
Next, let's take a few minutes to set-up the Kafka package. This will allow you to become familiar with the benefits of using a Certified package. If you are unfamiliar with Kafka, it is a publish-subscribe messaging system released under the Apache project. First, let's install Kafka with the DC/OS CLI. On the Bootstrap node, issue the command dcos package install kafka. The deployment process should take a few minutes. To monitor the process, issue the following command, again on the bootstrap node, dcos kafka plan show deployment.
Once Kafka is installed, a couple of commands to help you learn more about the cluster are dcos kafka describe and dcos kafka endpoints. The describe command displays configuration information and the endpoints command shows connection information. Now that you have your first data service running, feel free to test the other Certified packages!
If you would like to learn in-depth, technical details about each Certified package visit the Service Documentation. The documentation provides more information about the service settings, customizing the install, and managing the cluster. If you would like to get involved with the community join the DC/OS Community channel on Slack.