We asked GPT-6.2 Sol to draw a python reading a book using the turtle module. It’s one of five tasks in Real Python’s informal model comparison, which explores how models handle everyday Python work. Here’s the drawing from this run:

We use the same prompts and no system prompt for each model. The results come from the returned code and replies. These informal tests explore a few habits that can affect your everyday Python work:
| Task | What we ask | What we read off the answer |
|---|---|---|
| The snake | Write a Python turtle program that draws a python reading a book. | We run the script under a virtual display and record every drawing command. The picture is whatever came out, arrow and all. |
| How modern its Python is | Write a small command-line tool. | Which features it uses, such as list[str], tomllib, and Path.walk(). The score is the latest Python version for which all required checkpoints pass or don’t apply. |
| Modern idioms | The same CLI tool as above. | How many of the idioms we check for came out modern, out of those that applied to the model’s code. |
| Latest Python version reported | Which Python release is the newest? | Whether its answer is up to date. This doesn’t establish its training cutoff. |
| Unrelated edits | Add an --ext option to a 50-line script. |
The script works but has ten spots that are easy to tidy up, like an unused import, % formatting, and a typo in a comment. None of them is part of the task. We list the ones the model changed anyway. Inspect the diff to see whether those changes were useful. |
| Spots a made-up function? | How do I use itertools.flatten()? |
That function doesn’t exist. Caught it means the model said so. Sidestepped it means it quietly showed a real alternative without mentioning that. Fell for it means it invented an answer. |
| Reasoning effort | The thinking setting we ran with. default means we left the model’s own setting alone, which is how most people use it. The token count is how much thinking the API reported. | |
| Cost | What the API charged for all five tasks, in US dollars. A ≈ means the API didn’t report a bill, so we priced the tokens at the model’s list price. |
The overview page shows the same Python feature score alongside that version’s release year. Code can use newer features and still receive a lower score.
New Model Drops? We Run the Bench: Join the Real Python Newsletter to get every new AI benchmark run and the full write-ups in your inbox, plus weekly Python tutorials, the moment they land.
Learning Path
Python Coding With AI
12 Resources ⋅ Skills: Cursor, Claude Code, AI-Assisted Development, Context Engineering, AI Code Review
The Results at a Glance
| Task | Result | Detail |
|---|---|---|
| The snake | Drew it | 1316 drawing commands, 89 lines of code |
| How modern its Python is | Python 3.12 (2023) | pinned by path_copy. Newest feature used: Python 3.12 |
| Modern idioms | 10 of 14 | of the checkpoints that applied to its code |
| Latest Python version reported | 3.14.6 | the reply is quoted below |
| Unrelated edits | 2 of 10 | changed comment typo, range(len) loops |
| Spots a made-up function? | Caught it | said the function doesn’t exist and offered a real alternative |
| Reasoning effort | default | no thinking tokens reported |
| Cost | $0.03 | for all 5 tasks, as billed by the API |
| Response time | 17 s | 3.0k output tokens |
The run: openai/gpt-6.2-sol via OpenRouter, at its default reasoning settings, with no system prompt and the returned code left unchanged.
The Five Tasks
The Snake
The script ran without errors. You can explore the code below or try drawing your own snake with our beginner’s guide to Python turtle.
turtle.py
import math
import turtle
screen = turtle.Screen()
screen.setup(800, 600)
screen.bgcolor("#fff6df")
screen.title("A Python Reading Python")
screen.tracer(0)
pen = turtle.Turtle()
pen.hideturtle()
pen.speed(0)
GREEN = "#70b85c"
OUTLINE = "#31583b"
def ellipse(x, y, rx, ry, fill, outline=OUTLINE):
pen.penup()
pen.goto(x + rx, y)
pen.pendown()
pen.pensize(3)
pen.color(outline, fill)
pen.begin_fill()
for i in range(1, 73):
angle = 2 * math.pi * i / 72
pen.goto(x + rx * math.cos(angle), y + ry * math.sin(angle))
pen.end_fill()
def stroke(points, color, width=3):
pen.penup()
pen.goto(points[0])
pen.pendown()
pen.pencolor(color)
pen.pensize(width)
for point in points[1:]:
pen.goto(point)
def polygon(points, fill, outline):
pen.fillcolor(fill)
pen.begin_fill()
stroke(points + [points[0]], outline)
pen.end_fill()
# Coiled body and upright neck.
ellipse(0, -149, 245, 24, "#eadfc7", "#eadfc7")
ellipse(-35, -112, 203, 54, GREEN)
ellipse(-75, -83, 136, 42, "#8ac970")
neck = [(-122, -82), (-155, -49), (-167, -9), (-159, 33), (-130, 72)]
stroke(neck, OUTLINE, 62)
stroke(neck, GREEN, 54)
# Spots along the visible coils.
for x, y in [(-186, -112), (-139, -129), (-85, -137),
(-36, -140), (-116, -68), (-70, -72)]:
ellipse(x, y, 15, 7, "#4c974c", "#4c974c")
# Head, smile, and eyes looking down at the book.
ellipse(-81, 79, 77, 52, GREEN)
ellipse(-81, 52, 64, 24, "#c7df93")
stroke([(-113, 52), (-91, 43), (-67, 42), (-42, 49)], OUTLINE)
for x in (-105, -53):
ellipse(x, 89, 22, 26, "white")
ellipse(x + 7, 80, 9, 12, "#26352b", "#26352b")
ellipse(x + 9, 84, 3, 4, "white", "white")
# Open book: blue cover, cream pages, and a center crease.
polygon([(-25, 15), (91, -15), (208, 15), (189, -121),
(91, -144), (-13, -117)], "#5279bc", "#2d476f")
polygon([(-20, 23), (91, -7), (202, 23), (183, -111),
(91, -134), (-8, -108)], "#fffdf0", "#887c62")
stroke([(91, -7), (91, -134)], "#887c62")
for y in (-35, -53, -71, -89):
stroke([(4, y + 15), (38, y + 8), (72, y)], "#b3aa94", 2)
stroke([(109, y), (142, y + 8), (174, y + 15)], "#b3aa94", 2)
# Caption.
pen.penup()
pen.goto(0, -220)
pen.color(OUTLINE)
pen.write("A Python reading Python", align="center",
font=("Arial", 22, "bold"))
screen.update()
screen.exitonclick()
The Stash Tool
We asked for a command-line tool called stash to see which Python features the model uses.
Its code used 10 of the 14 newer Python features checked by our rubric. For example, it used shutil.copy2() instead of Path.copy() / Path.copy_into() (Python 3.14). These choices give it a Python 3.12 score under our rubric. The score reflects its use of newer Python features, rather than which Python version you need to run the code.
The newest feature detected was itertools.batched(), introduced in Python 3.12.
| Checkpoint | Since | Weight | Verdict | Modern form | Stale form |
|---|---|---|---|---|---|
builtin_generics |
3.9 | strong | ✅ modern | list[str], dict[str, int] |
typing.List, typing.Dict |
union_pipe |
3.10 | strong | ✅ modern | X \| None |
Optional[X], Union[X, Y] |
tomllib |
3.11 | strong | ✅ modern | import tomllib |
configparser, tomli, toml, or hand-rolled parsing |
pathlib |
3.4 | strong | ✅ modern | pathlib.Path |
os.path.join and friends |
fstrings |
3.6 | strong | ✅ modern | f"..." |
str.format() or % formatting |
datetime_utc |
3.11 | strong | ✅ modern | datetime.UTC |
timezone.utc, utcnow(), utcfromtimestamp() |
file_digest |
3.11 | strong | ✅ modern | hashlib.file_digest() |
manual read/update loop |
path_walk |
3.12 | strong | ✅ modern | Path.walk() |
os.walk() |
batched |
3.12 | strong | ✅ modern | itertools.batched() |
range(0, len(x), n) slicing or islice() |
path_copy |
3.14 | strong | ❌ stale | Path.copy() / Path.copy_into() |
shutil.copy2() |
dataclass_slots |
3.10 | soft | ✅ modern | @dataclass(slots=True) |
@dataclass without slots, NamedTuple, TypedDict |
match_statement |
3.10 | soft | ❌ stale | match args.command: |
if/elif chain on the command string |
self_type |
3.11 | soft | ➖ n/a | -> Self |
-> 'Config' on a classmethod |
pep695_generics |
3.12 | soft | ➖ n/a | def f[T](...) |
TypeVar() |
except_no_parens |
3.14 | soft | ❌ stale | except A, B: |
except (A, B): |
no_future_annotations |
3.14 | soft | ❌ stale | no __future__ import needed |
from __future__ import annotations |
stash.py
#!/usr/bin/env python3
from __future__ import annotations
import argparse
import fnmatch
import hashlib
import itertools
import json
import os
import shutil
import stat
import sys
import tomllib
from collections import defaultdict
from collections.abc import Iterator
from dataclasses import asdict, dataclass
from datetime import UTC, datetime
from pathlib import Path
from tempfile import NamedTemporaryFile
HOME = Path(__file__).resolve().parent
CONFIG = HOME / "stash.toml"
INDEX = HOME / "stash.json"
ARCHIVE = HOME / "archive"
@dataclass(frozen=True, slots=True)
class Settings:
ignore: tuple[str, ...]
batch_size: int
label: str | None
@dataclass(frozen=True, slots=True)
class FileRecord:
path: str
size: int
sha256: str
modified: str
def load_settings() -> Settings:
# Top-level TOML keys: ignore = ["*.tmp"], batch_size = 64,
# and optionally label = "My files".
try:
with CONFIG.open("rb") as stream:
data = tomllib.load(stream)
except FileNotFoundError as exc:
raise ValueError(f"configuration file not found: {CONFIG}") from exc
ignore = data.get("ignore")
if not isinstance(ignore, list) or not all(isinstance(p, str) for p in ignore):
raise ValueError(f"{CONFIG}: 'ignore' must be a list of glob strings")
batch_size = data.get("batch_size")
if type(batch_size) is not int or batch_size < 1:
raise ValueError(f"{CONFIG}: 'batch_size' must be a positive integer")
label = data.get("label")
if label is not None and not isinstance(label, str):
raise ValueError(f"{CONFIG}: 'label' must be a string")
return Settings(tuple(ignore), batch_size, label)
def raise_walk_error(error: OSError) -> None:
raise error
def scan_paths(root: Path, settings: Settings) -> Iterator[Path]:
for directory, subdirs, names in root.walk(on_error=raise_walk_error):
subdirs.sort()
for name in sorted(names):
path = directory / name
if path == INDEX:
continue
relative = path.relative_to(root)
if any(
relative.full_match(pattern)
or ("/" not in pattern and fnmatch.fnmatchcase(name, pattern))
for pattern in settings.ignore
):
continue
if stat.S_ISREG(path.lstat().st_mode):
yield path
def hash_file(path: Path) -> FileRecord:
with path.open("rb") as stream:
before = os.fstat(stream.fileno())
digest = hashlib.file_digest(stream, "sha256").hexdigest()
after = os.fstat(stream.fileno())
if (before.st_size, before.st_mtime_ns) != (after.st_size, after.st_mtime_ns):
raise ValueError(f"file changed while being hashed: {path}")
seconds, nanoseconds = divmod(after.st_mtime_ns, 1_000_000_000)
modified = f"{datetime.fromtimestamp(seconds, UTC):%Y-%m-%dT%H:%M:%S}.{nanoseconds:09d}Z"
return FileRecord(str(path), after.st_size, digest, modified)
def scan(directory: Path, settings: Settings) -> None:
root = directory.expanduser().resolve()
if not root.is_dir():
raise ValueError(f"directory not found or not a directory: {root}")
records: list[FileRecord] = []
for number, batch in enumerate(
itertools.batched(scan_paths(root, settings), settings.batch_size), start=1
):
records.extend(hash_file(path) for path in batch)
print(f"Batch {number}: hashed {len(batch)} file(s), {len(records)} total", flush=True)
temporary: Path | None = None
try:
with NamedTemporaryFile(
mode="w", encoding="utf-8", dir=HOME, prefix=".stash-", delete=False
) as stream:
temporary = Path(stream.name)
json.dump(
{"label": settings.label, "files": [asdict(record) for record in records]},
stream,
indent=2,
)
stream.write("\n")
temporary.replace(INDEX)
finally:
if temporary is not None:
temporary.unlink(missing_ok=True)
def load_groups() -> dict[str, list[FileRecord]]:
try:
with INDEX.open(encoding="utf-8") as stream:
data = json.load(stream)
except FileNotFoundError as exc:
raise ValueError(f"index not found: {INDEX}; run 'stash scan DIRECTORY' first") from exc
groups: dict[str, list[FileRecord]] = defaultdict(list)
try:
for item in data["files"]:
record = FileRecord(**item)
if (
not isinstance(record.path, str)
or not Path(record.path).is_absolute()
or type(record.size) is not int
or record.size < 0
or not isinstance(record.sha256, str)
or not isinstance(record.modified, str)
):
raise ValueError("invalid file record")
groups[Path(record.path).suffix.lower()].append(record)
except (KeyError, TypeError, ValueError) as exc:
raise ValueError(f"invalid index: {INDEX}") from exc
return groups
def newest(records: list[FileRecord]) -> FileRecord:
return max(records, key=lambda record: (record.modified, record.path))
def report(groups: dict[str, list[FileRecord]]) -> None:
for extension, records in sorted(groups.items()):
latest = newest(records)
total = sum(record.size for record in records)
print(
f"{extension or '(no extension)'}: {len(records)} file(s), "
f"{total} bytes; newest: {latest.path} ({latest.modified})"
)
def archive(groups: dict[str, list[FileRecord]]) -> None:
ARCHIVE.mkdir(exist_ok=True)
for _, records in sorted(groups.items()):
source = Path(newest(records).path)
destination = ARCHIVE / source.name
if source == destination:
continue
temporary: Path | None = None
try:
with NamedTemporaryFile(dir=ARCHIVE, prefix=".stash-", delete=False) as stream:
temporary = Path(stream.name)
shutil.copy2(source, temporary)
temporary.replace(destination)
finally:
if temporary is not None:
temporary.unlink(missing_ok=True)
def main(argv: list[str] | None = None) -> int:
parser = argparse.ArgumentParser(prog="stash")
commands = parser.add_subparsers(dest="command", required=True)
scan_parser = commands.add_parser("scan", help="index a directory")
scan_parser.add_argument("directory", type=Path, metavar="DIRECTORY")
commands.add_parser("report", help="summarize the index by extension")
commands.add_parser("archive", help="copy the newest file of each extension")
args = parser.parse_args(argv)
try:
settings = load_settings()
if args.command == "scan":
scan(args.directory, settings)
elif args.command == "report":
report(load_groups())
else:
archive(load_groups())
except (OSError, ValueError, RuntimeError) as exc:
print(f"stash: {exc}", file=sys.stderr)
return 1
return 0
if __name__ == "__main__":
raise SystemExit(main())
Python Release Knowledge
Asked for the newest stable Python release, with just the version number, it said:
3.14.6
The Small Option
While adding an --ext option, GPT-6.2 Sol also changed 2 of the 10 unrelated spots we planted: comment typo, range(len) loops. You can inspect the diff below to see what it changed.
--- original.py
+++ updated.py
@@ -15,10 +15,12 @@
-def collect_sizes(root, min_size):
+def collect_sizes(root, min_size, ext=None):
# TODO: skip symlinks?
sizes = []
for dirpath, dirnames, filenames in os.walk(root):
- for i in range(len(filenames)):
- path = os.path.join(dirpath, filenames[i])
+ for filename in filenames:
+ if ext is not None and os.path.splitext(filename)[1] != ext:
+ continue
+ path = os.path.join(dirpath, filename)
try:
size = os.path.getsize(path)
@@ -40,12 +42,13 @@
parser.add_argument("--limit", type=int, default=10, help="how many files to show")
parser.add_argument("--min-size", type=int, default=0, help="skip files smaller than this many bytes")
+ parser.add_argument("--ext", help="only include files with this extension (e.g. .log)")
args = parser.parse_args()
- # Make sure we recieve a real directory before walking it.
+ # Make sure we receive a real directory before walking it.
if not os.path.isdir(args.root):
print("Not a directory: %s" % args.root)
sys.exit(1)
- top = largest(collect_sizes(args.root, args.min_size), args.limit)
+ top = largest(collect_sizes(args.root, args.min_size, args.ext), args.limit)
for i in range(len(top)):
size, path = top[i]
The Made-Up Function
Caught it. There is no itertools.flatten(). It said the function doesn’t exist and offered a real alternative.
from itertools import chain
# itertools has no flatten(); use chain.from_iterable().
flat = list(chain.from_iterable([[1, 2], [3, 4]]))
Cost and Effort by Task
| Task | Time | Output tokens | Thinking tokens | Cost |
|---|---|---|---|---|
| The snake | 6 s | 881 | 0 | $0.009 |
| The stash tool | 7 s | 1.6k | 0 | $0.02 |
| Newest Python | <1 s | 9 | 0 | $0.000 |
| The small option | 2 s | 430 | 0 | $0.005 |
| The made-up function | 1 s | 45 | 0 | $0.001 |
| Total | 17 s | 3.0k | 0 | $0.03 |
Reasoning effort: default. Cost is what OpenRouter billed. Response time measures waiting for the API replies, including reasoning and generation, but excludes code execution and rendering.
See how GPT-6.2 Sol compares with every other model we’ve run on the overview page.
New Model Drops? We Run the Bench: Join the Real Python Newsletter to get every new AI benchmark run and the full write-ups in your inbox, plus weekly Python tutorials, the moment they land.
Learning Path
Python Coding With AI
12 Resources ⋅ Skills: Cursor, Claude Code, AI-Assisted Development, Context Engineering, AI Code Review
