How Smoke Tests Catch Broken Deployments Before Users Do
A short post-deploy check suite can stop a bad release in minutes. Here is what to test, where it fits, and how to make failure roll back on its own.
On this page 6 sections
Smoke tests are a short set of fast checks that run right after a deploy and confirm the application starts, answers requests, and completes its most important paths. They catch broken releases, such as bad configuration, missing secrets, and failed migrations, within minutes. A rollback then happens before customers notice.
Every team has shipped a release that passed all its tests and still broke in production. The unit tests were green. The staging run was green. Then a missing environment variable took the login page down. This guide explains how smoke tests close that gap, what to put in them, and how to keep them fast enough that nobody skips them. It is written for developers and engineering leads who own a deploy pipeline. Backtrace covers the unglamorous parts of shipping software, and this is one of the cheapest safety nets in the stack.
What Do Smoke Tests Check After a Deploy?
A post-deploy smoke test checks that the new version is alive and that its critical paths work against the real environment. It does not try to prove the code is correct. Unit and integration tests do that earlier, so the smoke test only needs to answer one question: is this release safe to leave running?
Google's Site Reliability Engineering book describes smoke tests as very simple but critical checks that short-circuit more expensive testing. That framing is useful. If the basics fail, nobody should spend an hour running the slow suite against a broken build.
The reason these tests matter is that most deploy failures are environmental. A service can compile, pass every test, and still fail because a secret was never rotated into the new environment. Another common case is a database migration that ran in staging but timed out in production. Code review cannot see either problem. A request to the live URL sees both.
Consider a worked example. A web service has three dependencies: a database, a cache, and a payment provider. A good smoke test makes one request that touches all three, such as loading an account page that reads the database, hits the cache, and renders a payment method. If that single request returns a 200 with the expected content, the release is probably sound. If it returns a 500, the pipeline stops.
Five Smoke Tests Worth Running on Every Release
The following list covers the checks that catch the largest share of broken deployments. Each one is cheap to write and takes seconds to run.
- Health endpoint. Request the service's health route and confirm a 200 response from the new version, not the old one. Expose the build SHA in the response so the test can verify it.
- Home page render. Fetch the main page and assert that a known string appears. A 200 status alone is not enough, because an error page can also return 200.
- Authentication path. Log in with a dedicated test account. This single step exercises sessions, secrets, the database, and often an identity provider.
- One write operation. Create a record and delete it. Reads can succeed against a replica while writes fail on a broken primary.
- Critical third-party call. Hit the payment, email, or search dependency in its sandbox mode. Expired API keys show up here first.
Browser-level tools make the render and login checks straightforward. The Playwright assertions documentation shows how to assert on page content, so the test fails on an error screen even when the status code looks healthy.
Keep each check independent. If the login test fails, the home page test should still run, because the combination of results tells you where to look. A failed health check with a passing home page usually points to a stale cache. A passing health check with a failing login usually points to configuration.
One practical rule from the incident reviews Backtrace's contributors have run: if a failure has happened twice in production, it earns a smoke test. This keeps the suite grounded in real outages instead of imagined ones. For a structured way to capture those lessons, see our guide to an incident response runbook.
Where Smoke Tests Fit Next to Canaries and Feature Flags
Smoke tests are one layer in a release strategy, and they work best alongside progressive delivery. The SRE book describes the canary approach as upgrading a subset of servers, watching for unexpected variance, and reverting quickly if something goes wrong. A smoke test is the gate that runs first, and it is much cheaper than waiting for a canary to bake.
The table below compares how each safeguard behaves, so you can see why teams use all of them together.
| Safeguard | When it runs | What it catches | Typical time to signal |
|---|---|---|---|
| Smoke test | Immediately after deploy | Dead service, bad config, broken critical path | Under 2 minutes |
| Canary release | During gradual rollout | Error rate and latency regressions under real traffic | Minutes to hours |
| Feature flag | At runtime, per feature | Faulty new behavior, contained to a cohort | Instant to switch off |
| Full regression suite | Before deploy | Logic errors in code | 10 to 60 minutes |
Notice that none of these replace the others. A smoke test cannot see a slow memory leak, and a canary is a slow way to discover that the app never started. Our comparison of blue-green and canary deployments goes deeper on rollout mechanics, and the piece on feature flag best practices explains how to contain a bad feature once it is live.
The strongest setup runs the smoke test against the new environment before it takes traffic. In a blue-green deploy, that means testing the idle environment and only then switching the router. GitHub documents this kind of gated flow in its guide to deploying with GitHub Actions, where a job can run checks and block the next stage on failure.
Can Smoke Tests Catch Technical SEO Mistakes?
Yes. A smoke test can fetch a few public URLs and confirm that crawlers can still reach and index them. This matters because a deploy can break search visibility without producing a single error in your monitoring dashboards or alerts.
The classic case is a staging flag that leaks into production. A noindex directive or a blanket disallow rule is copied across with the release, and nothing looks wrong to users. Google's own documentation on the noindex tag confirms that a page carrying it is kept out of search results once crawlers see it. Rankings then decay quietly for weeks.
Teams that run search programs for software companies treat these technical SEO regressions that follow a releaseSponsored as a deploy problem as much as a marketing problem. That is the right instinct, because the cheapest fix is a test that fails the build.
A crawl-aware smoke check needs only four assertions:
- The
robots.txtfile returns a 200 and does not contain a site-wide disallow. - The home page and one article page return no
noindexdirective in the meta tag or the response header. - The canonical tag on a sample page points to the production domain, not a staging host.
- The XML sitemap returns a 200 and lists at least one URL.
Google explains what the robots.txt file controls in its crawling documentation, which is the reference to keep open when you write the first assertion. The same logic now applies to AI answer engines, which also depend on being able to fetch your pages.
How Do You Keep a Smoke Suite Fast and Trustworthy?
Keep the suite under ten checks and under two minutes, and delete any check that fails without meaning something. A flaky smoke test is worse than none, because people learn to rerun it and ignore it. Speed and trust are the two properties that decide whether the suite survives.
Start with these habits:
- Run against the real deployed URL, never a mock or a local build.
- Use a dedicated test account and test data that production cleanup jobs leave alone.
- Retry a failed request once to absorb a cold start, but never more.
- Print the failing URL, status code, and response snippet in the log.
- Wire failure to an automatic rollback, not to a Slack message.
Automatic rollback is the step teams skip most often. A failed smoke test that only posts to a channel still relies on a person noticing at 5 p.m. on a Friday. The point of the suite is to remove that dependency.
AI-assisted development adds pressure here. As more code ships faster, the pipeline carries more changes per day, and a cheap post-deploy gate matters more. Our analysis of how AI coding is creating a CI bottleneck shows why teams are looking for checks that add confidence without adding queue time.
Finally, treat the suite as production code. Review changes to it, keep it in the same repository as the pipeline, and run it on a schedule against production too. A scheduled run turns the smoke suite into a lightweight uptime monitor that uses the exact same assertions.
Conclusion: Start With Three Checks This Week
Smoke tests work because they are small. They confirm the new release is alive, that users can sign in and write data, and that search engines can still reach the site. Everything deeper belongs in canaries, flags, and monitoring.
The next step is simple. Pick the health endpoint, the home page, and one authenticated action. Script them against your deployed URL, add the crawl checks if the site is public, and make a failure roll the release back. You can add more checks the next time production teaches you something.
