In May 2026 I added Argo CD to a small startup's k3s cluster. Shortly after, I removed it on purpose. It was reverting the team's canary releases within seconds of them starting.
I still like GitOps. This is a post about where it doesn't fit, and how I should have spotted that before installing it.
I was moving a bare-metal platform from hand-run shell scripts to Terraform. The cluster already ran Argo Rollouts for canary releases, and that part worked well. The engineer who shipped the app had a simple, tag-driven routine: build an image, then run kubectl argo rollouts set image to start a canary with the new tag, watch it, promote it.
Git held the Kubernetes manifests, but git wasn't the source of truth for which image was running. The manifests said newTag: latest. The real version lived in the cluster, set by hand.
While I was rebuilding everything as code, adding Argo CD looked like a natural upgrade. Manifests in git, a controller that keeps the cluster matching them, drift corrected automatically. I turned on selfHeal: true, because that's most of the point.
When a canary release started, within seconds Argo CD saw the Rollout's image differ from git. Git said latest. The cluster said the new tag. Self-heal did its job and put it back.
The rollout controller saw its spec change mid-canary and aborted the steps. The release never progressed. From the engineer's side, a deploy they had run many times before now silently undid itself.
Nothing was broken in either tool. Each one was doing exactly what it was configured to do. They just disagreed about who owned the image tag.
There were a few ways to keep Argo CD:
The third option is the clean one. It also meant changing how the team shipped, adding a commit and a sync to every release, for a small team that was happy with its release flow and had no problem with it. I'd be swapping a routine they trusted for one I preferred.
On 25 May 2026 I removed Argo CD. Releases stayed manual and tag-driven through Argo Rollouts, the way they had been before I arrived.
That sounds like a step back. I don't think it was. GitOps solves a specific problem: several people or pipelines changing a cluster, and nobody sure what's running. This team had one engineer doing the releases, a clear canary routine and a small number of services. The problem GitOps solves wasn't the problem they had.
What they did need was the infrastructure underneath in code: servers, bootstrap, secrets plumbing, databases, DNS. That's where the hand-run scripts had been, and that's where Terraform went. The release step on top could stay manual without hurting anyone.
Not much in time, because it came out within days. It cost some trust, though. The engineer's first experience of my "upgrade" was a deploy that undid itself. If I'd left it in and asked them to adapt, I would have been optimising for my architecture diagram, not their week.
I also had to be honest in the documentation afterwards. "Drift is corrected automatically" was in early drafts of the design. It isn't true anymore, and saying so matters more than the diagram looking modern.
Before adding any reconciler, I'd ask one question: how does a new version reach production today, step by step? Then I'd look for anything in that answer that changes the cluster outside git. A kubectl set image, a Helm upgrade from a laptop, a patch run during an incident. Each of those will fight a self-healing controller.
If the answer is "a person runs a command", there are two honest choices. Move that step into git first, with the team, and then add GitOps. Or leave GitOps out and put the effort where the risk actually is.
I did it the other way round: added the controller, then found the conflict. It was cheap here. On a bigger team, with more people releasing, it would have been a bad afternoon.
I'd still add it, and early, when:
If none of those apply, a well-understood manual release with a canary is fine. Boring and understood beats elegant and fought.
Fit the delivery model to how the team actually ships, then automate that. Don't install a controller that assumes a workflow nobody agreed to.
If you're deciding how releases should work on your cluster, I'm happy to look at it with you. See Kubernetes & Platform.
Production releases failed 50% of the time. Dashboards showed where they broke, and fixing those causes took July to 10%. Duration did not improve.
Spot nodes were reclaimed, the on-demand fallback hit its own limit, and production was down for about 25 minutes. Then I moved production off spot.
A production rollout sat blocked for 8 days while the site looked healthy. The first week of real alerts found it, and stuck queue messages too.
