No One Should Be Deploying Code at 2 AM
How to Fix Your Release Process
Deploying code at 2 AM is a cry for help. It signals a process problem, not a people problem. If your team is routinely making production changes in the dead of night to minimize impact on users, then you’re not reducing risk; you’re just shifting it onto tired, overworked developers who are more likely to make mistakes.
Deploying code in the middle of the night is a cry for help.
A recent Reddit post captures this frustration:
"…[M]ost production releases degrade our site performance, so developers release at night to limit impact. But this leads to unhappy, stressed engineers and slows company growth. Obviously this leads to very unhappy developers, more stress, more mistakes and it's ultimately a problem for the company growth, because we're not delivering features effectively. I really want to help stop this before it becomes normalised."
We get it. It’s easy to say “just automate more” or “embrace DevOps best practices”, but when you’re knee-deep in firefighting, those solutions feel abstract. Let’s break this down into concrete steps to fix your release process and make deploying to production feel as uneventful as sending a Slack message.
🚨 The Root of the Problem: Risky, Unpredictable Releases
When deployments cause performance degradation, it’s a sign that changes are too large, poorly tested, and not easily rolled back. In this case, the Reddit post mentions:
Big-bang releases: Deploying large, complex changes all at once instead of incrementally.
No feature flags: No way to toggle new features on/off safely in production.
Push-based deployments: Likely manual or ad-hoc deployments rather than automated, predictable ones.
When you don’t trust your deployment process, you try to minimize damage by releasing at 2 AM. But this is a high-toil bandaid, not a fix.
🔧 How to Fix It: From Chaos to Confidence
Let’s talk about practical solutions you can start implementing today.
1. Deploy in Small, Frequent Batches (Trunk-Based Development) 🚢
Instead of waiting weeks or months to ship a giant release, aim for small, incremental deployments. This reduces the blast radius of any single change.
How to do it?
Developers push changes to
mainmultiple times per day.Use short-lived feature branches (if needed), but integrate them frequently.
Avoid long-lived branches that accumulate changes over weeks.
Why does it help?
Small changes are easier to test, debug, and roll back if needed.
You build confidence in deploying often, so a single release isn’t a high-stakes event.
2. Use Feature Flags to Decouple Deployment from Release 🎛️
Feature flags let you deploy code without making it visible to users right away. You can turn features on/off dynamically without needing a new deployment.
How to do it?
Tools like LaunchDarkly or Flagsmith let you implement feature flags easily.
Wrap risky changes in a flag and turn them on only when you’re ready.
Use gradual rollouts: Enable the feature for 1% of users, monitor, then expand if everything looks good.
Why does it help?
You can release during normal hours without affecting users.
If something breaks, just turn the feature off instead of rolling back the whole deployment.
3. Implement Canary Deployments 🐤
Instead of rolling out changes to everyone at once, canary deployments let you test changes on a small subset of users first.
How to do it?
Deploy to a small percentage of users and monitor.
If performance drops, stop the rollout and investigate.
If everything looks good, expand to more users.
Why does it help?
You catch performance issues early without impacting the whole system.
It reduces risk, so you can deploy during normal working hours with confidence.
4. Automate Deployments with GitOps & CI/CD ⚙️
Manual deployments are slow, inconsistent, and prone to human error. A solid continuous integration/continuous deployment (CI/CD) pipeline removes these risks.
How to do it?
Use GitOps: Your infrastructure and deployments are defined in Git, making changes auditable and repeatable.
Automate tests (unit, integration, performance) in your CI/CD pipeline.
Check out Caparra’s guide to Continuous Integration for a deep-dive on this topic.
Why does it help?
Developers can trigger safe, automated deployments with minimal effort.
Changes are tested and validated before reaching production.
5. Shift-Left on Performance Testing 🏋️
If deployments consistently cause slowdowns, you need better pre-release performance testing.
How to do it?
Add load tests to your CI/CD pipeline using tools like JMeter, k6, or Locust.
Run chaos testing (Netflix’s Chaos Monkey is a great example) to simulate failure scenarios.
Set up real-time observability (Prometheus, Datadog, Grafana) to catch issues before they hit production.
Why does it help?
You catch performance issues before they impact users.
Developers gain confidence in their code changes.
🚀 The End Goal: Deploying at 2 PM, Not 2 AM
A great deployment pipeline is one where:
✅ Releases happen during working hours because they’re predictable and safe.
✅ Developers trust the process and don’t fear deployments.
✅ Performance issues are caught before production, not after.
✅ Rolling back is simple, so no single deployment is a high-risk event.
This isn’t just about making developers happier, it’s about making your company more effective. Faster, safer releases mean you can ship features quicker, delight customers, and scale smoothly.
If you’re stuck in the 2 AM deployment cycle, start small. With a few key changes, deploying to production can become just another Tuesday afternoon task with no drama, and no lost sleep.
At Caparra, we help DevOps teams reduce toil and streamline deployment processes. Want to learn how to improve your release pipeline? Try out the free Caparra chatbot to explore better DevOps practices today!
