GitOps Fundamentals

There has been a lot of talk around Gitops and how it is the next big thing for DevOps/NoOps. When I started looking at GitOps, I had a few questions.

• What is GitOps?
• Where all can it be used?
• Why use it, and what are the benefits?
• How do you implement GitOps?
• Why not use Jenkins/custom build scripts or any combination of CI/CD tools?

What is Gitops?

GitOps is a solution to the management and deployment of cloud-native applications. It eases the operational aspects of managing and deploying applications/infrastructure for developers, by using tools/techniques developers are already familiar with, namely CI/CD, merge requests and Git operations.

Using GitOps, developers maintain the declarative state of their infrastructure and application in Git repositories. GitOps recommends that all changes made to the application, environment, deployment, and infrastructure should be handled via git.

Typically application constructs like source code are already maintained in git. Besides, cloud-native applications are deployed using a docker image expressed using a Dockerfile. Deployment artifacts are maintained via Kubernetes constructs as manifests(Deployments, Services, Statefulsets, DaemonSets, etc.). Configuration is maintained as Kubernetes ConfigMap objects and infrastructure can be maintained using Terraform, CloudFormation, etc.

Where all can it be used?

Most of the examples on the GitOps site and other blog posts are about Kubernetes. That leads to the question, is the applicability of GitOps limited to Kubernetes?

GitOps can be used if the system fulfills the following two criteria.

• The system should be declarative. A declarative system describes the end state or the computation of logic without specifying the control flow. Kubernetes and Terraform are good examples of declarative systems. When you specify a manifest in Kubernetes, you specify the desired state of an application, like, how many pods to run, what services to expose, which port the service should expose, etc. Similarly, the state file in Terraform specifies what the desired state of the infrastructure should be. On the other hand, imperative systems specify the commands that should be run to change the system's state to reach the desired state.
• The system should be convergent. A convergent system is one in which a diff can be calculated based on the current state and the desired state. The system works to reach the desired state if there is drift in the current state from the desired state, i.e., reach a state where the diff does not exist. Kubernetes, for example, is an eventually consistent convergent system. It runs a reconcile loop to ensure that the current state matches the desired state specified in the manifest.

The applicability of GitOps can be to any system which fulfills the above two criteria; however, Kubernetes is the best example of a declarative convergent system, hence is the major use case.

Why Use GitOps?

GitOps uses tools to compare the desired state of your whole application with what’s under source control, and then it tells you when your current state doesn’t match the real world.

The benefits of GitOps are

• GitOps speeds up the meantime to deployment, which increases developer productivity. Besides, since developers are pushing code and are using familiar tools like Git and CI/CD tools like Jenkins and Spinnaker, it enhances the developer experience.
• Using Git workflows provides an audit log of all cluster changes outside of Kubernetes. An audit trail of who did what and when to your cluster can be used to meet compliance and ensure stability.
• Git’s capability to revert/roll back and fork enables teams to achieve stable and reproducible rollbacks. Git serves as the single source of truth because the entire system is described in Git. It also enables faster rollbacks, which can be as simple as pushing a previous/next stable version in case of any issues.
• Agent-based GitOps can enable automated drift correction. For example, consider a case where a user bypasses GitOps and directly changes the cluster by running a kubectl command. Other CI/CD tools would be oblivious to this change, which leads to configuration drift. With GitOps, the agent will detect that the cluster configuration and the desired state are out of sync. Agent’s have an auto-sync configuration, which, if enabled, tells the agent to automatically reconcile the cluster's state with the desired state automatically, thereby avoiding drift.

How do you implement GitOps?

There are two common ways to implement GitOps.

• Agentless or Push-based: This is the traditional approach, where the CI/CD pipeline pushes changes to the environment. Agentless GitOps requires firewalls to be opened to the cluster and admin rights to apply the changes. This might open up security loopholes. However, the upside of this approach is that it is generally applicable and can be used for VM’s or physical machines.
• Agent or Pull based: In a pull-based model, there is an agent installed on the cluster. The agent pulls changes whenever there is a change and ensures the current state matches the desired state. Agent-based implementations prevent configuration drift. They are inherently more secure since agents only carry out operations permitted by Kubernetes RBAC, policy, and security. Trust is shared with the cluster and not managed separately. However, it does need an agent running within each cluster. The agent-based approach is limited to Kubernetes manifests only. ArgoCD, Flux, Jenkins X are all examples of Agent-based implementations.

Why not use Jenkins/custom build scripts?

Strictly speaking, you can accomplish what GitOps provides using Jenkins and custom build scripts. That has been the traditional approach. You may have a team dedicated to creating custom scripts, managing Jenkins pipelines, and monitoring and fixing clusters. However, development, testing, and maintenance overhead are associated with each of these activities.

Handling failures can also be challenging using the traditional approach. If your CI pipeline pushes changes into the cluster using scripts, it can take time and sometimes breaks. And if there is a failure, the cluster state may be unknown. In the event of partial failures, the only recourse is to start from scratch all over again. These issues only get compounded with scale.

As the size and number of clusters you are managing increases, throwing people at the problem is not an ideal solution. Scale mandates newer approaches to problem-solving. With declarative and convergent systems like Kubernetes, a lot of the overhead can be moved from humans to software constructs that automate the often boring, repetitive, and error-prone tasks. GitOps provides an elegant solution to manage your clusters using software constructs and automation.