Generating Python Tests With AI: A Suite You Can Trust

Say you’ve inherited a project with no tests or so few that you can’t refactor it safely. Generating Python tests with AI seems like the quick way out, but a one-line prompt often gives you a suite that passes without catching real bugs. That’s worse than no suite at all because it falsely tells you everything is fine when it isn’t.

In this guide, you’ll learn a reusable workflow for building a pytest suite you can trust. A reliable test suite should have at least three qualities:

• Accurate: Its assertions state what the code is supposed to do, not what it happens to do today.
• Complete: It covers boundaries, error paths, and edge cases, not just the happy path.
• Robust: It fails when someone changes the code’s behavior.

To get there, you’ll set up testing conventions once, map the project and extract its intended behavior, generate the suite to meet a coverage goal, and confirm that it holds up with a review and a mutation testing run.

Take the Quiz: Test your knowledge with our interactive “Generating Python Tests With AI: A Suite You Can Trust” quiz. You’ll receive a score upon completion to help you track your learning progress:

---

Interactive Quiz

Generating Python Tests With AI: A Suite You Can Trust

Check how well you can generate Python tests with AI, from behavior tables and branch coverage to mutation testing with mutmut.

Prerequisites

This guide assumes you’ve written tests before and are comfortable working with an AI coding agent. You won’t learn pytest from scratch here. Instead, you’ll focus on directing the agent and checking its work. Before you start, make sure you have the following tools and knowledge:

• Python 3.10 or newer: The sample project runs on any recent Python version.
• Working knowledge of pytest: This guide uses pytest but doesn’t teach it. If you need a refresher, then check out the tutorial on effective Python testing with pytest.
• An AI coding agent: This guide uses Claude Code, but any agent that can read code, edit code, and run commands will work.
• Git: You’ll use Git to review everything the agent writes before you accept it.
• pytest, pytest-cov, and mutmut: You’ll add these tools in Step 1. pytest-cov builds on Coverage.py. To learn pytest in depth, check out the Testing Your Code With pytest video course.
• uv: The sample project uses uv to manage its dependencies and run commands. If you don’t have it yet, follow the installation instructions.
• macOS, Linux, or WSL: The mutmut tool doesn’t run natively on Windows. If you’re on Windows, work through this guide in a WSL terminal.
• The sample project: You can download the bookstore package using the link above.

The download contains the finished project, including the tests/ folder, BEHAVIOR.md, CLAUDE.md, and the extract-behavior skill under .claude/skills/, plus the testing configuration in pyproject.toml. As a bonus, you can copy the skill into any other project and reuse it as is.

To follow along from scratch, delete tests/, .claude/, BEHAVIOR.md, and CLAUDE.md. Then remove everything but the [project] table from pyproject.toml. You’ll have the following structure:

bookstore-project/
│
├── bookstore/
│   │
│   ├── data/
│   │   └── books.json
│   │
│   ├── __init__.py
│   ├── catalog.py
│   ├── inventory.py
│   └── pricing.py
│
├── main.py
├── pyproject.toml
├── README.md
└── uv.lock

The bookstore package models a small online bookshop with three modules:

• pricing.py computes what a customer pays, including volume and member discounts, shipping, and country tax. You’ll use it as the running example.
• catalog.py loads book records from data/books.json and looks them up by ISBN or author.
• inventory.py tracks the copies on hand and reserves, releases, and ships them for orders.

This is your starting point. Go ahead and commit the project to Git before moving on so that you can inspect every change the agent makes.

Step 1: Set Up Testing Conventions and Environment

The first step is to write down the testing conventions for the project. The agent must follow these rules every time it writes a test. In this tutorial, you’ll use Claude Code, which reads CLAUDE.md from the project root. This file is a good place to put the rules.

Create CLAUDE.md and add the following sections. The rules refer to BEHAVIOR.md, which lists what each function should return or raise. You’ll create and review that file in Step 2:

## Testing Conventions

Write tests with `pytest`. Put them in `tests/test_<module>.py`, mirroring
the package layout.

### Test Structure

- Arrange, act, assert, in that order, separated by blank lines.
- One behavior per test. If the name needs an "and", split the test.
- Name tests `test_<unit>_<condition>_<expected>()`.
- Use `@pytest.mark.parametrize` for table-driven cases, not loops.

### Assertions

- Assert exact values, not just types: `assert total == Decimal("190.00")`,
  not `assert isinstance(total, Decimal)`.
- Take expected values from `BEHAVIOR.md`. Never call the code under
  test to produce an expected value, and never reimplement its
  arithmetic in the test.
- Match the full exception message, anchored with `^` and `$`:
  `pytest.raises(ValueError, match="^unknown tier: gold$")`.
  Wrap messages that contain regex special characters in `re.escape()`.
  Never `pytest.raises(Exception)`.
- Assert on returned values and raised errors, not on
  `assert_called_once_with`.

### Test Coverage

- Parametrize both sides of every boundary in `BEHAVIOR.md`. For a
  threshold at 10, test 9, 10, and 11.
- One test per error path.
- Cover empty, missing, and zero inputs.

### Mocking Conventions

- Mock only what you can't control or can't afford: clocks, randomness,
  network calls, subprocesses, slow or destructive operations.
- Never mock the module under test.
- Never mock code we own just to avoid setting it up. Use a
  fixture instead.
- For files, use `tmp_path` and real files rather than patching `open`.

### Quality Gates

Before reporting a suite as done, run these yourself and keep working until
they pass:

- `pytest` passes.
- `pytest --cov=<module> --cov-branch --cov-report=term-missing` reports at
  least 95% branch coverage on the module under test.

Then report the coverage number.

These rules tell the agent where tests go, what a useful assertion looks like, which cases it must cover, and when it’s allowed to mock. The mocking rules matter more than you might expect. Every piece of code that a test replaces with a mock is code the suite no longer checks.

The Quality Gates section is the most important part. Why put the coverage goal here rather than in each prompt? Because standing instructions give you consistent behavior across sessions. The agent knows it isn’t done until the tests pass and the suite meets the coverage goal, and your prompts can stay short.

Next, add the testing configuration to pyproject.toml, keeping the project’s existing settings: