Quality assurance is an essential part of authoring great software. To help our community write great services that run on top of DC/OS, we are releasing Shakedown
, a lightweight Python library that can be imported into your tests to aid and streamline your software testing workflow.
According to Wikipedia
, a shakedown
is a period of testing or a trial journey undergone by a ship, aircraft or other craft and its crew before being declared operational.
In the DC/OS world, Shakedown consists of both a testing harness as well as a Python library which abstracts many common Mesos and DC/OS-related tasks such as installing packages from the Universe, querying a framework service for associated tasks, running commands on individual DC/OS agents, and more.
Why we developed Shakedown
We had a few goals as we developed Shakedown:
- Develop and achieve widespread adoption of an extensible, scalable testing harness that will serve automated testing need within the DC/OS community.
- Deliver a well-understood, well-documented, user-friendly solution to software engineers and quality personnel, both internally and externally.
- Optimize for test stability, usability, and speed, in that order.
Using the Shakedown tool to run tests
$ git clone github.com:dcos/shakedown.git && cd shakedown && pip3 install -e .
Shakedown requires a Python 3.4+ environment. Installation is as straightfoward as cloning the repository
from GitHub and using pip
to install the required libraries, as shown above.
Shakedown can be configured either via arguments or via a .shakedown configuration file in the user's $HOME directory.
See below for the arguments that you can pass to Shakedown on the command line (as described with shakedown --help):
--fail [fast|never] Specify whether to continue testing when
encountering failures. (default: fast)
--ssh-key-file PATH Path to the SSH keyfile to use for
Print the standard output of tests with the
specified result. (default: fail)
--stdout-inline Display output inline rather than after test
Alternatively, you can create a .shakedown configuration file in the user's $HOME directory to set your options:
ssh-key-file = '~/.ssh/id_rsa-mesosphere-shared' fail = 'never' stdout = 'all' stdout-inline = false
The Shakedown testing tool is an opinionated wrapper around the popular pytest
testing tool. Some default behavioral assumptions Shakedown makes are:
- Fail fast. If any test fails, immediately abort the test run and report on what failed, and why.
- Only verbosely report on what failed. Details and output of passing tests is not necessary.
We are testing DC/OS. First and foremost, make sure we can talk to the specified cluster. Secondly, if it is a cluster requiring authentication, all communications with the cluster should be authenticated.
$ shakedown --dcos-url http://dcos.example.com [options] [path_to_tests]
As shown above, --dcos-url is the only required Shakedown argument; all other configuration options can be specified on the command line or via a .shakedown file as detailed above under Configuring Shakedown.
Shakedown tests are pytest tests, and Shakedown will automatically discover and run any test files (filenames beginning with test_) contained within the recursive search of the current working directory.
Writing tests using the Shakedown Python library
Shakedown, aside from being a testing tool, includes a Python library which abstracts many common Mesos and DC/OS-related tasks. Full documentation on available methods can be found in the Shakedown API documentation
Example acceptance tests
"""Install the Jenkins package for DC/OS.
assert package_installed(PACKAGE_NAME), 'Package failed to install'
end_time = time.time() + WAIT_TIME_IN_SECS
found = False
while time.time() < end_time:
found = get_service(PACKAGE_NAME) is not None
if found and service_healthy(PACKAGE_NAME):
assert found, 'Service did not register with DCOS'
In this example, install_package_and_wait, package_installed, get_service and service_healthy are all Python methods provided by Shakedown.
As shown above, Shakedown aims to help speed up and improve framework authoring and testing. As with the rest of DC/OS, Shakedown is an open-source project and contributions and enhancements are welcome!