Product

Building Dispatch, A Cloud Native GitOps Platform

Why write 2,000 lines of YAML, when you can write 1,000 lines of Dispatch YAML? Better yet, write only 250 lines of a Python derived, declarative language.

Apr 29, 2020

Seth Mason

D2iQ

Why write 2,000 lines of YAML, when you can write 1,000 lines of Dispatch YAML? Better yet, write only 250 lines of a Python derived, declarative language.

Enterprise customers are faced with an exploding ecosystem of cloud native technologies that enable them to run modern applications. The applications they're building require a flexible and scalable infrastructure to meet the increasing demands of scale, security, and reliability. Core technologies like D2iQ's Konvoy address the needs of these demanding applications with an easy to operate enterprise-grade Kubernetes distribution. However, the tools the developers are using for developing, testing, and deploying their applications are often rooted in technologies that were designed for a pre-cloud native era. To address the needs of the modern developer, D2iQ created Dispatch, a cloud-native CI/CD solution that supports software lifecycle management using GitOps.  

  

I had a chance to sit down with D2iQ’s Dispatch engineering team to get a better understanding of the platform. Here are four key benefits that Dispatch provides to its users: 

 

Flexibility of architecture  

Cloud native isn't just about being able to run on Kubernetes, but being able to leverage all of the inherent abilities that your underlying platform provides. How cloud native is Dispatch? Modifying two lines in a yaml file deploys it via Helm, all pipeline tasks are pods, the steps within those tasks are executed as containers, and leverages Kubernetes native Role Based Access Controls (RBAC) and secrets management. With a foundation of Tekton and Argo, Dispatch was purpose built from the ground up for CI/CD, leveraging expertise and insight from D2iQ’s years of expertise in building distributed systems for some of the largest organizations in the world and leveraging our own internal CI/CD team.

 

Most importantly, Dispatch is designed to give a clear separation of responsibilities between the software developer who relies on speed in their build/test iterations and the production, and the Site Reliability Engineer (SRE) or operator who's focused on the deployment requiring tremendous flexibility and control.    

 

Improved developer’s CI loop velocity

Dispatch provides a higher level layer of abstraction on top of the foundational pipeline of Tekton, a foundational Kubernetes native pipeline building block. Developers should have the ability to leverage the tooling they’re familiar with in traditional software development. This includes things, such as schema validation, unit testing, and modularity. So why not use these to build their pipelines?  By offering multiple declarative pipeline languages, such as  CUE (extended JSON) and Starlark (Python flavored), JSON, and YAML, developers can rapidly get up to speed, regardless of skill level. You can start with 2,000 lines of terse Tekton YAML, but wouldn't you rather write 1,000 lines of Dispatch YAML?  Personally, I would take it a step further and write 250 lines of simplified Dispatch DSL (Domain Specific Language). With CUE and Starlark I can use common programming conventions such as functions, templating, and inheritance, a big improvement in usability over standard YAML. With per team configurations, Dispatch projects are also democratized. Developers can have their own sandbox environments, which helps with access control, governance, and security. For example, some  developers may run their pipelines locally for speed, and then run the exact same pipeline on a large cluster for more scale based testing.

 

True cloud native GitOps methodologies 

Chances are, you're already checking your code into a Git repository. With Dispatch, you can describe your build pipeline in code and again check it into Git. Applications deployments can also have their desired state stored in Git, thereby enabling easy auditing. The different types of deployments, such as blue/green or canary, can be automatically triggered or manually approved via standard Git processes by an SRE or operator.  

 

Ease of management  

Your Git repositories' webhooks are managed via a Kubernetes controller, making it easy to manage your CI system via GitOps. Dispatch's controllers manage your webhooks for the build and deployment activities, as well as translating from CUE/Starlark/JSON/YAML to a lower level pipeline language. Source code management statuses based on task results are made available via a controller, and can be made available to your code repository. Ease of management and abstraction layers make it easier for enhancements to be rolled out.

 

D2iQ, Powered by Dispatch

One bit that I found especially interesting is that the Dispatch team itself has been using Dispatch since the very beginning of the development process. It's proven to be a huge time saver for the team to rapidly build their pipeline in the DSL that matches their expertise and then locally execute it.  Conversely, other teams using Dispatch picked the different DSL that matched their strengths. Once executed, task notifications are dynamically updated and appear in Slack so that other team members can quickly review code, thereby simplifying their workflow. In addition, automation notifies operators on other teams when deployment candidates are ready for production. 

 

After speaking to the Dispatch developer team, I came away seeing a purpose built solution, built by developers for developers.  Dispatch developers took a solid foundation of cloud native tools and enabled any developer, working in any location to get their job done easily.

 

Want to see it in action? Head on over to D2IQ.com for your free trial of Dispatch on Konvoy.

Ready to get started?