GitOps was originally defined by WeaveWorks as a way to have Git at the center of your delivery pipeline. After you implement GitOps, Git becomes the source of truth for your deployment and the delivery activities of your application and infrastructure.
While the original purpose of GitOps was to manage deployments and tasks for Kubernetes, you can use GitOps to improve Continuous Delivery practices for any application or change in your delivery pipeline.
How Would GitOps Work for Your Application’s Deployments?
For reference, I created a sample repository to show how you can use GitOps for your team’s application deployments. The contents of this blog post are also located in that repository’s README.
The sample repository contains the following files:
- manifest.json: Versioned, source-controlled file that describes which version of the application should be deployed to any environment.
- Jenkinsfile: Pipeline script to read in the manifest.json and parse environment variables such as your application version and artifact location.
- dev-env.json, staging-env.json, prod-env.json: Files that describe the environment and deployment information for your applications. These files could be replaced by results from API calls to configuration management or other methods to provide your delivery system with endpoints for deployment or testing.
Please note that this pattern is not specific to a particular technology or application stack but instead defines a “GitOps-like” process to promote your application through environments that correspond to branches in a git repository. You could extend this pattern to any "as-code" deployment for any application or infrastructure change.
The GitOps Process for Application Deployment
You will be creating a manifest that contains application data, such as a specific version or application type, to help define what will be deployed. This manifest can have one or many applications listed and can be used as a representation of a stack or “environment” of connected products. Each of the applications you need in a product release could be represented here. To promote the application or stack through environments, you will “pull” the application through its CD pipeline by creating Pull Requests (PRs) of a manifest to Git branches that correspond to environments in the application's path to production.
By using this process, you can define exactly which version of your application(s) and which deployment steps are defined in code. In this way, it will always be clear which applications exist in which environment, and you will be able to control who has the ability to deploy to specific environments by simply having certain team members manage PR merges to specific branches.
Treating everything as code is super important, and GitOps allows you to apply this practice when deploying any of your applications or infrastructure code.
Many CI/CD tools could be used to accomplish this functionality. The basic requirements are to be able to parse a manifest and perform certain tasks based on git branch-specific logic. I have chosen to use a Jenkinsfile (declarative Jenkins pipeline) to illustrate the process, as described below.
Preparation
1. Create a starter manifest in a “master” branch to outline the information needed to deploy any artifact to any location.
- Refer to manifest.json as an example on how to outline which applications need to be deployed
- The version and artifacts are the sections that would change most often. They may be different because you may choose to deploy multiple artifacts/versions of different applications.
- Remember, this process is for deploying anything anywhere or promoting from env to env. Make sure you’re using versioned artifacts.
2. Define your application’s “path to production” prior to deploying your application to any environments by creating a branch per environment in your “GitOps deployment repo”. Create one branch per environment defined in your app’s path to prod (like dev -> staging -> production). Use this repo as an example.
- The manifest/gitops repo is independent of your actual “code” repository. (We're not talking about the application code here; instead, you’re deploying the immutable artifacts or infrastructure it creates.)
- I have chosen to reference environment information from some static files: dev-env.json, staging-env.json, prod-env.json but your environment information can come from anywhere. Having it in Git, if the environments are static, is a great way to have it tracked at all times.
- This manifest/gitops repo may be completely disconnected from your application builds but could also be updated by CI to update a version if you like. It is for deployment processes only.
3. Create a deployment script like the Jenkinsfile to outline the specific stages and parse values from the manifest corresponding to the branch logic applied to each environment.
- You can use any CI tool/process you like here. The declarative Jenkinsfile makes it super simple to enable branch-specific logic you can use for GitOps.
- You will need to script methods to pull your artifacts from artifact storage locations such as Nexus or Artifactory. Those methods aren't defined here because they will be specific to your implementation.
Execution
1. Trigger the deployment of your application in one of two ways: via automated "Continuous Delivery" system updates or via a PR.
- The CI system can update the manifest in the dev branch. After validation by your team, the automated system can update the manifest with the latest version of your application. This process is very important if you wish to support Continuous Delivery or Continuous Deployment.
- Initiate deployment to your dev environment by updating your manifest and creating a PR to the Environment of your choice. Normally you'll PR from environment to environment to promote your application, but you can also PR from any branch to any branch. What if you want to take what's in production and deploy it to QA? The main point here is to make any change traceable.
2. If you’re deploying/promoting your artifacts manually by PR, assign reviewers to help validate the correct application version in the manifest or any other changes that need review.
3. Merge into the environment yourself after approvals or allow others to merge into target environments.
4. Upon PR merge (or automated commit), the process will kick off, and the application listed in the manifest will deploy to the environment aligned to the branch you’re merging into.
- Dev deployment:
- Staging deployment:
- Production deployment:
Taking It to Prod: Restricting Deployment to Production or Protected Environments
You may wish to make the production deployment more restrictive. Merge restrictions come into play in an enterprise environment where you may wish to assign certain team members the capability to deploy to some environments but not others. Alternatively, you may want to restrict production deployments to a certain team. You can lock down merge capabilities to some branches/environments and require certain reviewers in others.
GitHub enables users to control these restrictions at a very granular level.
NOTE: You can lock this process down further for production deployments by creating another repository where a separate build system that only deploys to production has access. Doing so will enable you to have a protected set of credentials or an entire deployment system dedicated to production deployments.
Summary
Whether you create your environments dynamically, run them in Kubernetes, or simply deploy to the same static environments with each release, you’ll always need a way to know which applications are deployed in which environments/regions.
Using GitOps for your application’s delivery is a viable solution to help build a repeatable delivery system while honoring the core principle of treating “everything as code.”
Remember: the example above is just a starting point for how GitOps could be implemented in your environment. You can reimagine this process for your own environment using more or less complicated manifests and deployment processes.
