Issue #025 - Four ways to put secrets in Git: three are wrong for you
SOPS and age, Sealed Secrets, External Secrets Operator, Vault and OpenBao, threat models
The repo had been public-inside-the-company for years, the kind every platform team has - charts, kustomizations, the lot. I was greping it for an unrelated ConfigMap and the match landed in a file called secret.yaml, committed in 2021, with a data block that was very obviously base64. I ran it through base64 -d out of reflex, the way you do, expecting nothing. It was the production database password for a service that was still very much running. Not a sealed anything, not encrypted - a plain Kubernetes Secret, which is to say base64, which is to say plaintext with an extra step, sitting in Git history forever, readable by everyone who'd ever had a clone.
The uncomfortable part wasn't that someone did it. The uncomfortable part was that for three years the repo's whole story had been "secrets live in Git, that's our GitOps model, it's fine." Nobody had lied. They'd just never written down what "fine" was protecting against, so base64 had quietly counted as protection. The fix that afternoon was easy. The thing that took longer was the question underneath: if we're putting secrets in Git on purpose - and GitOps means we are - what exactly are we defending against, and which of the four standard tools actually defends against that?
Because there are four. SOPS, Sealed Secrets, External Secrets Operator, HashiCorp Vault and its operators. Every team picks one, usually the one a teammate used at their last job, and then defends it as if it were the obviously-correct choice. It isn't. Three of the four are wrong for any given team, and which three depends entirely on a threat model most teams never write down.
🏗️ Architectural Pattern: where the trust root lives and what's in Git
The question nobody writes down
Every one of these four tools answers the same question - "how do I get a secret from my laptop to a running pod, through a Git repo, without it leaking" - and they answer it completely differently because they disagree about one thing: where the key that can read the secret lives, and therefore who, exactly, you're trusting.
When I finally drew this on a whiteboard for my own team, the question that mattered wasn't "which is most secure" - that one has no answer. It was two narrower ones: what is actually committed to Git, and what can turn it back into a plaintext secret. Once I had those pinned, the right tool stopped being a debate. The 2021 secret.yaml was what you got by skipping them - a model nobody chose, defending against nothing in particular.
The four sort into two camps along that axis, and in practice the camp mattered more than the specific tool.
Camp one: the ciphertext is in Git
SOPS and Sealed Secrets both put encrypted bytes into the repo. The secret material is there, in a commit, forever. What differs is who holds the key that reverses it.
With SOPS, you encrypt each value against a key you control - an age keypair, or a cloud KMS key (AWS KMS, GCP KMS, Azure Key Vault), or PGP. The repo holds ciphertext; the decryption key lives wherever you put it. The threat this defends against is precise and limited: someone reading your Git repo learns nothing, because the values are encrypted. But anyone who holds the age private key or can call that KMS key decrypts everything, including every secret that ever appeared in history. The trust root is the key, and the key is off in KMS or on a few laptops. Recover the key, recover the entire history.
With Sealed Secrets, the trust root moves into the cluster. The Bitnami controller generates an RSA keypair on first boot and keeps the private key as a Secret in kube-system. You encrypt with the public half using kubeseal - anyone can, the public key is public - and only that one controller, holding that one private key, can ever decrypt. The ciphertext in Git is genuinely useless to anyone who isn't that controller. That's a stronger property than SOPS in one specific way: there's no decryptable-by-a-human key sitting in KMS. And it's a much sharper footgun in another, which I'll get to, because that private key is now the single most important thing in your cluster and most teams don't know where it's backed up.
Camp two: the secret is not in Git at all
The other two refuse the premise. Nothing secret goes into the repo - only a reference.
External Secrets Operator commits an ExternalSecret manifest that says, in effect, "the value for db-password lives in AWS Secrets Manager at this path, go fetch it." ESO runs in-cluster, authenticates to the external store, pulls the real value, and materializes a normal Kubernetes Secret. Git holds a pointer and nothing else. Clone the repo, get the entire repo, and you have learned the names of secrets and where they're stored - never a byte of the secrets themselves. The trust root is the external store plus ESO's credential to read it.
Vault (via the Vault Secrets Operator, the CSI provider, or Vault Agent) is the same shape - reference in Git, value in the store - with one capability the other three structurally can't have: the secret can be dynamic. Vault can mint a Postgres credential that exists for one hour and then revokes itself. There's no long-lived password to leak, because by the time anyone exfiltrates it, it's dead. That's a different threat model again: not "protect the secret in transit through Git" but "minimize the lifetime of the thing worth stealing."
So the four are really two questions stacked. First: is the ciphertext in Git (SOPS, Sealed Secrets) or is only a reference in Git (ESO, Vault)? Second, within each camp: who holds the key, or how long does the secret live? Answer those honestly about your own threat model and the field of four collapses to one. Answer them with "it's what my last team used" and you're back to base64.
Links
🆚 The Showdown: SOPS vs Sealed Secrets vs ESO vs Vault
I've run three of these four in anger and watched a team get badly burned by the fourth, so let me be specific about where each one earns its place and where it bites.
SOPS plus age or KMS
SOPS is the one I reach for when a team wants encryption-in-Git and doesn't already run anything heavier. It's a CNCF Sandbox project now - donated out of Mozilla in 2023, after seven-odd years of life, picked up by a fresh group of maintainers under the getsops org. That matters because the Mozilla-era SOPS had gone quiet and a lot of people wrote it off; it's actively maintained again.
The thing SOPS gets right that nothing else does: it encrypts values and leaves keys in plaintext. Your secret.yaml still diffs cleanly in a PR - you can see that db-password changed without seeing what it changed to. No in-cluster component is needed for the encryption itself, which is why Flux integrates it natively: Flux decrypts at reconcile time with a key you've handed the controller. ArgoCD is clumsier - you bolt on ksops as a Kustomize plugin or run a SOPS-aware sidecar, and it's never as clean as the Flux path.
Where it bites: the decryption key is a human-holdable thing. An age private key on a laptop, a KMS key a CI role can call. Whoever has it reads the whole repo's history. SOPS gives you encryption-at-rest in Git and not one thing more, and teams routinely forget that and treat "it's SOPS-encrypted" as if it meant "it's safe even if the key leaks." It does not.
Sealed Secrets
Sealed Secrets is the simplest mental model: encrypt to the cluster, only the cluster decrypts. The controller's RSA private key never leaves kube-system. You seal with kubeseal and the public key, commit the SealedSecret, and the controller unseals it into a real Secret. By default the seal is strict - it binds the ciphertext to the exact namespace and name, so a secret sealed for prod/db-password won't decrypt as staging/db-password, which is a genuinely nice property and the source of most "why won't this decrypt" tickets when someone renames a namespace. There are namespace-wide and cluster-wide scopes if you need to relax that, and relaxing it is a deliberate, auditable choice.
Here's the part that bit a team I watched. They rebuilt a cluster - new nodes, restore the GitOps repo, let it reconcile. Every SealedSecret came back as a decryption error and nothing started. The controller in the new cluster had generated a new keypair, and the ciphertext in Git was sealed against the old private key, which had only ever existed inside the cluster they'd just torn down. Nobody had backed it up, because nobody had internalized that the controller's private key was now the single most precious object they owned. They got lucky - the old cluster was still half-alive and they pulled the key out. If it hadn't been, every production secret would have had to be regenerated by hand. Sealed Secrets moves the trust root somewhere very safe and then dares you to forget where you put the only copy.
One bit of good news on maintenance, because there was a scare. When Broadcom restructured the Bitnami catalog on August 28, 2025 - the same change that broke a wave of bitnami/* Helm charts and container images industry-wide, the one I mentioned back in the Valkey issue - a lot of people assumed Sealed Secrets was caught in it. It wasn't. Sealed Secrets lives under bitnami-labs, not the commercial Bitnami catalog, its images stay on docker.io/bitnami, and it kept shipping. If you deferred adopting it last autumn out of license nerves, that fear was misplaced.
External Secrets Operator
ESO is the one I'd call the default for most teams that already have a secret store, and it's also the one with the most interesting 2025. The pitch is clean: nothing secret in Git, ever, just ExternalSecret references, and the operator syncs from AWS Secrets Manager, GCP Secret Manager, Azure Key Vault, Vault, and a long tail of others. Compromising the Git repo gets an attacker the topology of your secrets and not the secrets. It's a CNCF Sandbox project, accepted back in July 2022.
And in August 2025 it nearly died. On the 13th the maintainers announced they were pausing all releases - no features, no security patches, no image publishing - because the project had effectively one maintainer carrying it quasi-full-time and he'd burned out. The quote that stuck with me was Gergely Brautigam's, roughly: people feel entitled to say "I use this, it's broken, fix it," and that's not how any of this works. For a few months ESO's future was a live question, which is a deeply uncomfortable thing to read about a tool you've wired into every cluster you run. By mid-2026 it recovered - new maintainers stepped up, releases resumed, it's actively maintained again. But the episode is the most honest argument in this whole comparison: the trust root for ESO isn't only the external store, it's also the bus factor of an operator that spent a season at exactly one.
Vault and the OpenBao question
Vault is the heavyweight, and the only one of the four that does dynamic secrets - credentials minted on demand with a lease, auto-revoked when the lease ends. The Vault Secrets Operator understands Vault's lease model natively: it renews leases, caches the client token, and rotates dynamic database credentials at 67% of their TTL by default, in-cluster, declaratively. If your threat model is "a stolen credential should be worthless within the hour," nothing else here competes, because the other three all hand you a long-lived secret and call it done.
The cost is operational weight - Vault is a system you run, seal, unseal, back up, and staff, not a controller you helm install and forget. And there's a license asterisk that's now load-bearing. HashiCorp moved Vault to the BUSL on August 10, 2023 (1.15 was the first BUSL release, 1.14 the last under MPL), and IBM closed its acquisition of HashiCorp on February 27, 2025, so Vault is an IBM product under a source-available license today. The open-source answer is OpenBao, forked from Vault 1.14.0 - the last MPL release - now under the Linux Foundation and an OpenSSF Sandbox project since June 2025. If you want Vault's model without the BUSL, OpenBao is the fork to look at, the same way Valkey was the fork for Redis. As of mid-2026 the BUSL hasn't been reversed and there's no sign it will be, so the choice is real, not hypothetical.
Links
👮 The Policy: a manifest each, and which threat model picks which
Enough comparison. Here's what the two camps actually look like on disk, and then the part that matters - which threat model lands you on which.
A SOPS-with-age secret, after sops --encrypt, keeps the structure readable and hides only the values:
apiVersion: v1
kind: Secret
metadata:
name: db-creds
namespace: prod
data:
password: ENC[AES256_GCM,data:9f3k...,iv:7a2c...,tag:b1e4...,type:str]
sops:
age:
- recipient: age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9aqmcac8j
enc: |
-----BEGIN AGE ENCRYPTED FILE-----
...
-----END AGE ENCRYPTED FILE-----That commits to Git as-is. The recipient line is the age public key - safe to publish. The decryptable half lives on the laptops and CI roles you've handed the private key to.
An ESO reference commits no secret material at all - just a pointer and a refresh interval:
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata:
name: db-creds
namespace: prod
spec:
refreshInterval: 1h
secretStoreRef:
name: aws-secrets-manager
kind: SecretStore
target:
name: db-creds
data:
- secretKey: password
remoteRef:
key: prod/db/passwordClone that repo and you've learned that a thing called prod/db/password exists in a store called aws-secrets-manager. You have learned nothing you could use.
When I've had to make this call for real, I started from the threat model rather than the tool I already knew:
You're air-gapped or have no external secret store, and you want the simplest GitOps model. Sealed Secrets. The trust root is the in-cluster controller key, you depend on nothing external, and kubeseal is a five-minute setup. Just write down where the controller's private key is backed up before you ship anything, or the cluster-rebuild story from the Showdown becomes yours.
You're air-gapped or store-less but want readable diffs and multi-recipient encryption (different keys for prod and staging, say). SOPS with age. Lighter than Sealed Secrets if you're already on Flux, and the encrypt-the-value-keep-the-key property is worth a lot in review. Accept that the key is human-holdable and guard it accordingly.
You already run a secret store - AWS Secrets Manager, GCP, Azure, or Vault - and you want nothing secret in Git. ESO. It's the boring-enterprise default for a reason, and the recovery in 2026 means you can stop worrying about it disappearing. Trust shifts to the store and ESO's read credential, which is where most mature teams want it.
Your threat model is credential lifetime - the secret worth stealing should be dead within the hour, or you need per-request database credentials for compliance. Vault, via VSO, with dynamic secrets. Or OpenBao if the BUSL is a dealbreaker. This is the only camp that makes the stolen-secret problem go away by construction, and it's also the only one that's a system you have to staff. Don't pick it for static API keys you could've put in any of the other three.
The trap, every time, is picking by familiarity and then writing the threat model backwards to justify it. Pick the threat model first. Three of these four will be visibly wrong for it, and the fourth will feel obvious.
Links
What secrets actually are
Strip the tooling away and a secret is a trust root - the thing everything downstream believes without checking. Every one of these four tools is really an argument about where that root should sit: in a KMS key, in a controller's memory, in an external store, in a one-hour lease. They aren't competing on features. They're competing on where you're willing to put the thing that, if it leaks, ends you.
That base64 in the 2021 repo was a trust root nobody had decided to place anywhere, so it had ended up everywhere - in the Git history, in every clone, in the laptop backups of everyone who ever pulled the repo. Picking SOPS or Sealed Secrets or ESO or Vault is, underneath, the act of deciding. The wrong tool for your threat model still beats the non-decision that base64 represents, by a wide margin.
Go find one secret in your repos that nobody ever decided where to trust. There's at least one. See you Tuesday.
- Ilia