Real Python Podcast E302 Title Image

Episode 302: Constructing and Judging Modern Agentic Workflows

The Real Python Podcast

Jul 10, 2026 58m intermediate ai testing
RSS Apple Spotify YouTube
Podcast Addict Overcast Pocket Casts CastBox Stitcher Download MP3
Christopher Bailey
Christopher Bailey
Host
Suneet Malhotra
Suneet Malhotra
Guest

How can you improve your LLM agent systems through specification enrichment? What are the advantages of having an LLM act as a judge within an agent system? This week on the show, Senior IEEE Member and Quality Engineer Suneet Malhotra joins us to discuss building and evaluating agentic architecture.

Suneet Malhotra is an independent practitioner-researcher with 18 years of experience in Quality Engineering (QE) and test automation for consumer-scale platforms. He discusses building specification-enrichment loops, monitoring performance, and using Cohen’s kappa to measure agreement between LLM judgments.

Suneet is currently publishing multiple papers that are under peer review on these topics. He also provides links to his work and GitHub projects if you want to experiment with these concepts and methods yourself.

Quick Survey: Get more out of the podcast show notes

Topics:

  • 00:00:00 – Introduction
  • 00:00:56 – Survey: RP Podcast show notes
  • 00:02:11 – How did you get into testing?
  • 00:05:04 – Has working for large public-facing corporations changed how you approach testing?
  • 00:07:06 – Writing a paper on LLM-as-Judge
  • 00:09:22 – Looking across the Software Development Lifecycle
  • 00:14:46 – Agentic AI: theater vs methodology
  • 00:17:23 – Specification enrichment
  • 00:27:52 – Video Course Spotlight
  • 00:29:18 – Saving the specifications
  • 00:31:27 – Using the LLM as a judge & Cohen’s kappa
  • 00:39:31 – How can people try out the project?
  • 00:43:35 – What are some of the failure modes you’ve seen?
  • 00:50:26 – What’s an inexpensive way to try these ideas out?
  • 00:54:04 – What are you excited about in the world of Python?
  • 00:56:14 – What do you want to learn next?
  • 00:57:14 – How can people follow your work online?
  • 00:57:32 – Thanks and goodbye

Show Links:

Level Up Your Python Skills With These Courses:

Improving Your Tests With the Python Mock Object Library

Improving Your Tests With the Python Mock Object Library

PydanticAI: Typed LLM Agents With Structured Outputs

Building Type-Safe LLM Agents With Pydantic AI

Build a Python MCP Client to Test Servers From Your Terminal

Testing MCP Servers With a Python MCP Client

← Previous All Episodes