Production GitOps with GitLab: Continuous Delivery with the GitLab Agent for Kubernetes

For years, continuous delivery to Kubernetes from CI/CD pipelines relied on a push-based model:

  1. You created a long-lived kubeconfig or ServiceAccount token inside your cluster.
  2. You pasted that secret into your CI/CD repository settings.
  3. Your CI/CD runner reached through your firewall into the Kubernetes API server to run kubectl apply.

While push-based deployment is simple to set up, it introduces severe enterprise security liabilities:

  • Cluster Exposure: Your Kubernetes API server must be reachable from the internet or public CI runners.
  • Credential Sprawl: Storing cluster-admin credentials inside CI variables creates a high-value attack vector if a pipeline or dependency is compromised.
  • Configuration Drift: If someone manually changes a pod or service with kubectl edit, the cluster silently drifts away from Git.

To solve this, GitLab introduced the GitLab Agent for Kubernetes (KAS) — a modern, pull-based GitOps engine.

In this guide, I will show you how to configure the GitLab Agent to manage your Kubernetes deployments securely with zero open inbound firewall ports.

1. Push vs. Pull-Based GitOps Architecture 

+-------------------------------------------------------------+ | TRADITIONAL PUSH-BASED DEPLOYMENT | | | | GitLab CI Runner ---(Exposed API Port 6443)---> K8s API | | * High attack surface; credentials stored in CI runner. | +-------------------------------------------------------------+ vs. +-------------------------------------------------------------+ | PULL-BASED GITOPS (GITLAB AGENT / KAS) | | | | GitLab Repository (GitOps Manifests / Kustomize) | | ^ | | | Outbound Secure gRPC/WebSocket | | | (No inbound firewall openings) | | v | | In-Cluster GitLab Agent (Polls & Syncs State Directly) | | - Reconciles desired state vs. actual cluster state | | - Prevents configuration drift automatically | +-------------------------------------------------------------+

2. Setting Up the Agent Configuration in Your Repository

In GitLab, the Agent for Kubernetes is configured entirely via version-controlled files inside your repository.

Create the configuration file at the mandatory path:
.gitlab/agents/<agent-name>/config.yaml

# .gitlab/agents/production-agent/config.yaml gitops: manifest_projects: - id: "my-org/cloud-infrastructure" paths: - glob: "manifests/**/*.{yaml,yml}" reconcile_timeout: "5m" prune: true # Automatically deletes resources removed from Git! ci_access: projects: - id: "my-org/microservices-app"

Key Parameters:

  • prune: true: Ensures declarative synchronization. If you delete a deployment.yaml from Git, the agent automatically deletes the deployment from the cluster.
  • ci_access: Allows authorized GitLab CI pipelines to securely interact with the cluster via the agent without storing raw cluster credentials.

3. Installing the Agent Inside Your Kubernetes Cluster

Step A: Register the Agent in GitLab UI

  1. Navigate to your GitLab project $\rightarrow$ Operate $\rightarrow$ Kubernetes clusters.
  2. Click Connect a cluster (agent).
  3. Select your agent name (production-agent) from the dropdown.
  4. GitLab displays a registration token and an auto-generated Helm command.

Step B: Install via Helm

Run the Helm command directly inside your cluster (or via your local kubectl admin context):

helm repo add gitlab https://charts.gitlab.io helm repo update helm upgrade --install production-agent gitlab/gitlab-agent \ --namespace gitlab-agent-production \ --create-namespace \ --set image.tag="v17.5.0" \ --set config.token="YOUR_AGENT_REGISTRATION_TOKEN" \ --set config.kasAddress="wss://kas.gitlab.com"

Within seconds, the agent pod connects outbound to GitLab over secure WebSockets (wss://). In your GitLab dashboard, the cluster status instantly turns Connected (Green).

4. Declarative Synchronization in Action

Create a simple deployment manifest in your Git repository under manifests/api-service.yaml:

apiVersion: apps/v1 kind: Deployment metadata: name: api-service namespace: production labels: app: api-service spec: replicas: 3 selector: matchLabels: app: api-service template: metadata: labels: app: api-service spec: containers: - name: web image: registry.gitlab.com/my-org/microservices-app:v2.1.0 ports: - containerPort: 8080 resources: limits: cpu: "500m" memory: "512Mi" requests: cpu: "100m" memory: "128Mi"

Commit and push this file to main:

git add manifests/api-service.yaml git commit -m "feat(deploy): provision api-service with 3 replicas" git push origin main

The GitLab Agent detects the commit within seconds, validates the manifest, and applies it to the cluster.

5. Drift Detection & Self-Healing

What happens if an engineer enters the cluster and accidentally deletes a pod or modifies replicas?

# Simulating accidental drift kubectl scale deployment api-service -n production --replicas=1

Because GitOps treats Git as the single source of truth, the GitLab Agent detects that the cluster state (replicas: 1) diverges from Git (replicas: 3).

Within the configured reconcile_timeout window, the agent automatically scales the deployment back to 3 replicas, self-healing the cluster without human intervention!

6. Secure CI/CD Pipeline Access (Hybrid GitOps)

Sometimes a CI/CD job needs to run an imperative command against the cluster (like executing database migrations or running integration smoke tests).

With the agent configured under ci_access, your .gitlab-ci.yml can interact with the cluster securely:

deploy_migration: stage: deploy image: name: bitnami/kubectl:latest entrypoint: [""] script: # Set context to the registered GitLab Agent - kubectl config get-contexts - kubectl config use-context my-org/cloud-infrastructure:production-agent - kubectl run db-migration --image=registry.gitlab.com/my-org/db-tools:latest --restart=Never

No tokens are exposed in pipeline logs, and permissions are strictly bounded by Kubernetes RBAC.

7. Key Takeaways

  1. Zero Inbound Attack Surface: The agent connects outbound over WebSockets (wss://), allowing private clusters to remain completely shielded behind firewalls.
  2. Immutable Git Audit Trail: Every deployment, scale event, or configuration change is tracked in Git commits with cryptographic signatures.
  3. Automated Self-Healing: Configuration drift is reconciled automatically to match Git.

Resources & Links

Are you using push or pull-based deployment for your Kubernetes clusters? Share your architecture in the comments below!