Day 7 — argocd & github-runner: How Code Reaches Production

Series: Zero to Production on AWS EKS with Terraform — Part 9 · Day 7 Layer 4 of 5 — delivery. Add-ons of the day: ArgoCD and the GitHub Actions Runner Controller, both inside eks-addon.

Last updated: 26 October 2026

In short: ArgoCD and the GitHub Actions Runner Controller (ARC) are the delivery layer of an EKS platform. ArgoCD continuously syncs the cluster to what Git declares, so merges become deploys and drift is visible. ARC runs CI jobs on self-hosted runner pods inside the cluster that scale to zero. Both are toggles in the eks-addon Terraform module.

We have a network, a cluster, storage, certs, load balancing, secrets, and smart scaling. The cluster is fully usable. But how does your app actually get onto it — and stay up to date — without someone running kubectl apply by hand at 11pm?

That's today: delivery. Two add-ons.

1. ArgoCD — pushes app changes from Git to the cluster, automatically (GitOps).
2. GitHub Actions Runner Controller (ARC) — runs your CI/CD pipelines inside the cluster.

Part 1 — ArgoCD: GitOps delivery

Why replace manual deploys with GitOps?

GitOps makes the Git repo the single source of truth, so what's in Git is what runs in the cluster. Manual deploys are fragile: someone forgets a step, dev and prod drift, and there's no clean record of "what's actually running right now." You want the Git repo to be the single source of truth — what's in Git is what's in the cluster, always.

What is ArgoCD?

ArgoCD is a robot that continuously compares "what Git says should be running" against "what's actually running" and fixes any difference. Merge a change to your manifests → ArgoCD notices → rolls it out → reports health and sync status. Someone manually changed something in the cluster? ArgoCD flags the drift (or reverts it). The repo is the truth.

   git push (new manifest / new image tag)
            │
            ▼
   ArgoCD detects diff ──► applies to cluster ──► reports Synced/Healthy
            ▲                                            │
            └──────────── reconcile loop ◄───────────────┘

Which inputs does ArgoCD need in eks-addon?

ArgoCD needs an on/off toggle, install basics, and the ingress hostname and ACM certificate for its UI.

enable_argocd               = true
argocd_enable_ingress       = true
argocd_server_host          = "argocd.example.com"
argocd_acm_certificate_arn  = module.acm.certificate_arn   # ← Day 5 wiring
argocd_enable_applicationset = true

Why is ArgoCD token-minting in a separate argocd-token module?

It keeps the shared eks-addon module GitOps-engine-agnostic. ArgoCD often needs an API token (to register itself, or for automation to create apps). Minting that token is split into a dedicated argocd-token module, not baked into the shared eks-addon. Why? So that the shared add-on stack stays GitOps-engine-agnostic — a pure monitoring cluster that doesn't run ArgoCD has no reason to carry token-minting logic. Keep the common module clean; isolate the product-specific bit.

Part 2 — GitHub Actions Runner Controller (ARC)

Why not just use GitHub-hosted runners?

Hosted runners cost money per minute, can't reach private resources, and don't give you control of their environment. CI/CD jobs (build, test, deploy) need machines to run on. GitHub's hosted runners cost money per minute, can't reach your private resources, and you don't control their environment.

What is the GitHub Actions Runner Controller (ARC)?

ARC runs self-hosted GitHub Actions runners as pods inside your own cluster. When jobs queue up, it spins up runner pods; when the queue empties, it scales back down (even to zero). You get cheaper CI, full control of the runner image, and direct private access to your VPC resources.

  GitHub job queued
        │  ARC sees it
        ▼
  Runner pod spawned in cluster ──runs the job──► pod terminated when done
        (scales with demand, down to zero when idle)

Which inputs does ARC need in eks-addon?

ARC needs the GitHub org/repo URL, a token, and a map of runner scale sets.

enable_github_runner = true
github_config_url    = "https://github.com/<org>"
github_token         = var.github_token   # sourced from a secret, never hardcoded
runner_container_mode = "dind"
github_runners = {
  default = { min_runners = 0, max_runners = 5 }
}

What are the gotchas with self-hosted runners on EKS?

The main ones are memory sizing for dind, Karpenter evictions, and token handling.

• dind runners are memory-hungry. Docker-in-Docker needs real headroom. If a runner pod's memory limit exceeds the node's actual capacity, it OOM-kills or never schedules — and the node can flap to NotReady. Size runner pods and their nodes together. (This bit us: 6Gi runner pods on ~3Gi-usable nodes.)
• Protect runners from Karpenter consolidation. A runner mid-job that gets evicted by node consolidation = a failed pipeline. Give runner pods a karpenter.sh/do-not-disrupt annotation so they finish their job before the node can be reclaimed (ties straight back to Day 6).
• Never hardcode github_token. Pull it through External Secrets (Day 6) or a Kubernetes secret. It's a credential.

How do ArgoCD and ARC tie the platform together?

They form the push-to-live loop: ARC builds and pushes the image, and ArgoCD deploys the updated manifests.

  Developer ──git push──► GitHub
        │                    │
        │                    ├─► ARC runner pod builds image ──push──► ECR (Day 3)
        │                    │   and updates the manifest's image tag
        ▼                    ▼
   Manifests repo ◄──── ArgoCD watches ────► deploys to EKS ──► ALB + ACM serve it (Day 5)

This is the full loop: push code → CI builds it on your own runners → ArgoCD ships it → it's live behind https. No manual kubectl, fully auditable, identical in every environment.

How do you verify ArgoCD and the runners?

Check that ArgoCD applications show Synced/Healthy and that the runner scale set appears in GitHub.

# ArgoCD
kubectl -n argocd get pods
kubectl -n argocd get applications        # Synced / Healthy
# open https://argocd.example.com (the ingress + ACM cert)

# Runners
kubectl get pods -n <runner-namespace>
# check Settings → Actions → Runners in GitHub: scale set shows Idle/Active

Frequently asked questions

What does ArgoCD do when someone changes the cluster by hand?

ArgoCD continuously compares what Git says should be running with what is actually running. A manual change shows up as drift, which ArgoCD flags or reverts, because the repo is the source of truth.

How do I expose the ArgoCD UI over HTTPS on EKS?

Set argocd_enable_ingress = true, give it a hostname with argocd_server_host, and pass the Day-5 ACM certificate as argocd_acm_certificate_arn. The ALB controller then serves the UI over HTTPS.

Can self-hosted GitHub runners scale to zero?

Yes. ARC spins up runner pods when jobs queue and scales back down, even to zero, when the queue empties. Set min_runners = 0 on the scale set in github_runners.

Why do my dind runner pods get OOM-killed or never schedule?

Docker-in-Docker is memory-hungry, and a runner pod whose memory limit exceeds the node's usable capacity either OOM-kills or never schedules. Size runner pods and their nodes together.

Where should the GitHub token for ARC come from?

Never hardcode github_token. Supply it through External Secrets or a Kubernetes secret, because it is a credential.

What's the takeaway?

• ArgoCD makes Git the source of truth — merges become deploys, drift is visible/auto-corrected. Token-minting is isolated in a dedicated argocd-token module to keep the shared add-on engine-agnostic.
• ARC runs CI/CD on self-hosted runners inside the cluster — cheaper, private-reachable, controllable — but size them and shield them from consolidation.
• Together with Day 3 (ECR) and Day 5 (ALB + ACM), they form the complete push-to-live pipeline.

Next — Day 8 (finale): eyes on everything. The observability stack — Promtail, Loki, Grafana, Prometheus agent, Tempo — plus the ec2 module and a full one-command environment wrap-up. 📊