smoke test

How It Shows Up in Practice

A Python developer typically meets smoke tests in two places. The first is a CI job that runs immediately after every build. The second is a post-deploy job that runs against a staging or production environment right after a release.

The CI form is usually a handful of tests pulled from an existing pytest suite, tagged with a marker so the runner can select them on demand:

import pytest

@pytest.mark.smoke
def test_home_page_renders(client):
    response = client.get("/")
    assert response.status_code == 200

@pytest.mark.smoke
def test_user_can_log_in(client):
    response = client.post("/login", json={"user": "u", "password": "p"})
    assert response.status_code == 200

A GitHub Actions job then runs pytest -m smoke --tb=short ahead of the full suite. If those two tests fail, the rest of the suite never starts. The same pattern works in GitLab CI, Jenkins, or any other runner that can scope a job to a marker.

The post-deploy form is often a small standalone script that hits the deployed service and exits non-zero on the first broken critical path. It can be written with only the standard library:

import json
import sys

# Each check stands in for a real production probe (an HTTP request,
# a database query, a queue ping). The runner cares only that every
# check returns a truthy value and that the first failure is loud.

def homepage_reachable():
    response_status = 200
    return response_status == 200

def critical_api_responds():
    payload = json.loads('{"status": "ok"}')
    return payload.get("status") == "ok"

def database_migrations_applied():
    expected, actual = 7, 7
    return expected == actual

CHECKS = [
    ("homepage reachable", homepage_reachable),
    ("critical api responds", critical_api_responds),
    ("database migrations applied", database_migrations_applied),
]

for name, check in CHECKS:
    if check():
        print(f"ok    {name}")
    else:
        print(f"FAIL  {name}", file=sys.stderr)
        sys.exit(1)

ok    homepage reachable
ok    critical api responds
ok    database migrations applied

In a real deployment each function would call urllib.request.urlopen against /healthz, hit the login endpoint, or query a migrations table. The shape stays the same: a small list of named checks, fail-fast on the first failure, finish in seconds. Teams that practice behavior-driven development often pick the smoke-suite candidates straight from their highest-value Given/When/Then scenarios.

Common Pitfalls

Smoke tests fail in three recurring ways that teams keep rediscovering.

• Bloat. Smoke suites accrete tests until they take ten minutes to run and lose their fast-feedback value. The discipline is deletion. If a check does not gate the rest of the pipeline, it belongs in the full regression suite, not the smoke suite.
• Flakiness. A flaky test in the smoke suite is worse than no smoke suite at all, because it trains the team to retry-and-ignore the gate. A check that fails for environmental reasons more than once a month should be moved out or fixed.
• Smoke-as-coverage. A green smoke run means “this build is not obviously broken,” not “this build works.” Teams sometimes ship on green smoke and skip the full suite, which converts the smoke test from a fast filter into a false promise.

A related vocabulary trap is the word sanity test. The ISTQB Glossary treats smoke and sanity as synonyms, but most teams in practice use smoke for a broad, shallow check on a fresh build and sanity for a narrow, deeper check after a small change. Pick one term per team and write down what it means in your repository’s contributing guide.

Related Resources

Tutorial

Continuous Integration With Python: An Introduction

In this Python tutorial, you'll learn the core concepts behind Continuous Integration (CI) and why they are essential for modern software engineering teams. Find out how to how set up Continuous Integration for your Python project to automatically create environments, install dependencies, and run tests.

intermediate best-practices devops testing

For additional information on related topics, take a look at the following resources:

• pytest Tutorial: Effective Python Testing (Tutorial)
• Continuous Integration and Deployment for Python With GitHub Actions (Tutorial)
• Build Robust Continuous Integration With Docker and Friends (Tutorial)
• Python's unittest: Writing Unit Tests for Your Code (Tutorial)
• Continuous Integration With Python (Course)
• Testing Your Code With pytest (Course)
• Effective Testing with Pytest (Quiz)
• Python Continuous Integration and Deployment Using GitHub Actions (Course)
• GitHub Actions for Python (Quiz)
• Testing Your Code With Python's unittest (Course)
• Python's unittest: Writing Unit Tests for Your Code (Quiz)