Continuous deployment is the holy grail of datacenter automation. In this post, we demonstrate a simplified example of how we use Mesos, Marathon, TeamCity, and Docker internally to deploy applications automatically to our internal staging environment. The team is automatically notified of new deployments by using Slack.
The following flow chart shows the high level sequence of actions. A check-in to GitHub triggers a build of a Docker image in TeamCity. On a successful build, a special TeamCity Deploy build is triggered, which uses Marathon's REST API to trigger a new deployment.
For this example, we are using the presentation located at
appfolder contains our presentation along with a
marathon.jsonwhich describe how to build and deploy our presentation.
nginxbase container which provides the
nginxwebserver. Finally, we
ADDthe presentation directory to the container to be served by
MAINTAINER Mesosphere email@example.com
ADD app/ /usr/share/nginx/html
The simple Marathon app definition in
mesosphere/cd-demo-appDockerHub repository as the source of our Docker image. Note the
$tagplaceholder, which is required by TeamCity to deploy the latest built tag of our presentation.
We also specify health checks in this definition, which allows Marathon to gauge when the newly deployed container is up and running correctly. The health checks use HTTP requests to a defined endpoint. You can read more about how Marathon health checks work here.
[ "hostname", "GROUP_BY" ]
In TeamCity, we have a build that builds and pushes the Docker image for our presentation and another build that deploys the most recently built image to Marathon.
Before setting up the build you should develop a scheme to use for generating the Docker image tags.
Here's the scheme we're going to use for our demo:
The components of the tag are as follows:
%env.BUILD_NUMBER%: The build number so that we can always track down the TeamCity build that created the Docker Image
%teamcity.build.branch%: The branch of the Git repo that the Docker Image was built from
%env.BUILD_VCS_NUMBER%: The Git SHA of the code that was added to the Docker Image
This scheme ensures that we have a new tag for every build, meaning that we are creating immutable containers. Also, the scheme ensures that we have visibility into the branch that the code came from and the Git SHA of the commit.
Docker Image Build
The first build in TeamCity has these steps:
docker login: log in to our DockerHub account
docker build: build the specified
Dockerfilewith a new tag
docker push: push the built image to DockerHub
- Report tag into new
jqto write the tag into the Marathon app definition
The first three steps are fairly standard, but the fourth step is the magic that makes this work. Our script looks like this:
mkdir -p target
echo %TAG% > target/docker-tag
cat marathon.json | \
jq '.container.docker.image |= "%DOCKER_IMAGE%:%TAG%"' > target/marathon.json
After the build completes you can view the generated artifacts on the
Artifactstab for the specific build:
After the first build completes successfully and there are new artifacts, TeamCity automatically triggers a second build where we do two things:
- PUT to Marathon's REST API
- Send a message to the team on Slack
Marathon's REST API makes it easy to kick off new deployments of an application. Using HTTP
PUTwith the newly generated
marathon.jsonartifact creates a new deployment. You can read more about Marathon deployments here. The deployment starts a new instance of the application and stops the old instance only when the new application is deemed "healthy".
APP_ID=$(cat %MARATHON_JSON_FILE% | jq -r '.id')
http --check-status PUT \
%SCHEME%://%MARATHON_HOST%:%MARATHON_PORT%/v2/%MARATHON_JSON_TYPE%/$APP_ID < %MARATHON_JSON_FILE%
Sending a message on Slack is easy because of their accessible API. The process involves pushing a JSON encoded message to a pre-configured Slack webhook URL (see Slack's documentation for more details).
}' | http --print=HhBb --json POST %SLACK_WEBHOOK_URL%
Of course, by parameterizing all of these values, it is easy to create a TeamCity template and extend this to other services.
|SLACK_MESSAGE||Heads Up! %teamcity.build.triggeredBy% just deployed <https://teamcity.mesosphere.io/viewLog.html?buildId=%teamcity.build.id%&tab=buildResultsDiv&buildTypeId=%system.teamcity.buildType.id%|%system.teamcity.projectName% :: %system.teamcity.buildConfName% #%build.counter%>|
Each time a successful Docker image build completes, a deploy build is triggered, the
PUTto Marathon, and the app begins rolling out across your cluster. A notification that an application rollout is happening is sent to the team.
Here you can see the generated
marathon.jsonthat is sent to Marathon. [highlight code='json']
[ "hostname", "GROUP_BY" ]
The new version of the application is scaled up, the new task is shown in the
After Marathon considers the new instance healthy, it cleans up old instances of the application:
A notification is then sent to the team indicating that a deploy has been triggered:
- Set up an automatic build for your project and generate a new Docker tag for each build.
marathon.jsonas a build artifact.
- Configure TeamCity to collect the generated
- Configure firewalls to allow TeamCity build agents to communicate with Marathon.
- Create a TeamCity build to
- Notify your team of the deployment.
This post shows how to easily set up automated builds that deploy Dockerized applications automatically to a Mesosphere cluster. We use this same procedure in many of our projects to make life easier for our engineers. Give continuous deployment on Mesosphere a try today!