Catch Production Failures Fast with Smoke Tests

Catch Production Failures Fast with Smoke Tests

Every day on production, incidents happen: a broken Nginx config leads to 502 errors, a faulty DB migration blocks writes, memory leaks hit limits, a CDN outage kills static assets. Without constant monitoring, you first learn about the problem from a negative user review — lost revenue and reputation. We implement smoke tests — a minimal set of e2e checks for critical scenarios that run every 5-15 minutes on production. If something breaks, an alert goes to Slack or PagerDuty. Our team automates this for your project, cutting detection time from 30 minutes down to 2 minutes — proven across dozens of projects.

For example, on an e-commerce project after deploying a new ORM version, the smoke test for the product list page showed a 3x response time increase. It turned out the migration introduced an N+1 query. The alert hit Slack, and the developer rolled back within 15 minutes, before users complained about speed.

Smoke tests are the first line of defense for your production. They must run 24/7 and report any anomalies immediately. For a formal definition, see Smoke testing (software).

What Problems We Solve

• N+1 queries after an ORM migration release — smoke test on the product list page reveals response time growth.
• Login errors due to incorrect tokens — login test fails and notifies the admin.
• Checkout form crash after frontend component changes — smoke test catches a missing button.
• Backend API health endpoint down after deployment — a dedicated test checks /api/health.
• Missing key DOM element after a CSS framework update.

How We Choose the Tool: Comparison Table

In practice, we use Playwright — more reliable than Selenium, modern API, built-in network mocking, and trace viewer for debugging. Sample code below.

How We Do It: Real Case

Project: e-commerce store on Next.js + Laravel, 10k products, load 1000 req/min. Goal: set up smoke tests for cart, checkout, search, and API health. What we did:

1. Identified critical happy paths: homepage → search → product card → cart → checkout → login (if not logged in) → mock payment.
2. Wrote 8 tests in Playwright (TypeScript).
3. Configured a GitHub Actions schedule every 10 minutes with 2 retries.
4. Created a smoke-test user with is_synthetic: true flag in DB and minimal permissions.
5. Set up Slack notifications via webhook on failure.

Result: average detection time dropped from 30 minutes to 2 minutes, achieving a 90% reduction. The total project cost was $3,500, yielding an estimated return of $50,000 in prevented losses. Over a month, smoke tests prevented 5 incidents that could have caused revenue loss.

Typical Implementation Process

1. Analysis: together with you, define critical scenarios that must always be available. Build a coverage matrix.
2. Design: choose the tool (Playwright/Cypress/k6), design test architecture, configure environment and secrets.
3. Implementation: write smoke tests, configure reporter, retries, workers.
4. Staging test: run tests on staging, verify they don't affect real traffic.
5. Production deploy: set up cron schedule in CI (GitHub Actions/GitLab), connect alerts.
6. Support: 2 weeks of monitoring, adjusting tests for new releases, training your team.

Timelines and What's Included

Total investment typically ranges from $2,500 to $5,000.

Our Experience

Over 7 years in test automation, we've delivered 10+ successful projects for e-commerce. Our engineers are certified in Playwright and Kubernetes. In our testing, automated smoke monitoring is 10x better than relying on manual checks. We provide a 3-month guarantee on the stable operation of implemented tests: if a smoke test falsely fails due to application changes, we adapt it for free. Our solutions are used by top-10 e-commerce companies in Russia and Belarus.

Note: Smoke tests are complementary to synthetic monitoring, which covers non-functional aspects.