Extracting pydoc Comments With autodoc
In the previous lesson, I showed you RST markup and how to use it in a Sphinx document. In this lesson, I’ll be talking about the
autodoc extension that allows you to incorporate the comments in your code into your documentation.
Sphinx has a mechanism for writing extensions, and in fact, it ships with some. The one I’m going to be showing you is called
autodoc, and it adds directives for extracting pydoc comments from your code and including them in your documentation.
autodoc comes with the Sphinx package, so you don’t have to install anything additional. You do have to configure it though. Open up your
conf.py file and add
sphinx.ext.autodoc as a string to the
extensions list to enable it.
autodoc extension directive that I use most frequently is
automodule. This takes a module filename as an argument and causes
autodoc to parse that file for pydoc comments. The names of classes and functions with pydoc comments get pulled into your documentation.
01:04 There are a bunch of different options available for the directive. You can explicitly list those members of the module you want to include, or you can leave it blank and have them all included.
You can specify whether to include private members in your docs, those being things that start with single or double underscores. The
special-members option will include dunder methods from a class if present.
You can optionally list those dunder methods you are interested in explicitly as well. By default,
autodoc only looks for things that have pydoc comments.
If you add the
undoc-members option, it will also include members without pydoc comments.
exclude-members allows you to specify anything you wish to exclude.
member-order allows you to change what order they appear in. I typically have a method to my code-ordering madness when I write my code, so I like to use the
bysource argument to
member-order so that the docs show the same order as my code.
02:09 Rather than include a whole module, you can explicitly include a code thing. There are directives for classes, functions, decorators, data items, methods, and attributes.
To demonstrate all of this, I have added some source to my Serenity project. The
serenity/ directory now has two directories inside of it: the
docs/ one I created before and a new one for my source.
I’m going to add a new RST file to the docs that uses the
automodule directive to pull in the contents of my source code. The source is split into two files, one file containing a class and the other a function.
This is what the whole project tree looks like. In the project directory, I added
README.rst, and in the
docs/ directory, I’ve added
src/ directory has the Serenity code in it. This module has a ship and actions and
__init__ Python file. Inside
__init__, I’ve put a variable containing the version number for my release.
I’m going to update the
conf.py file to use this value so I don’t have to update the release number in two places.
03:25 Inside of your pydoc comments, you can use special notations to cross-reference to other pieces of code. All of these are variations on references for classes.
Relative referencing can be a bit finicky, and as you might move your classes into new files, I find the best thing to do is always just use the second one, the fully named module. Prefacing that with a tilde (
~) will mean the resulting link will only show the class name if you prefer it to be shorter.
03:55 There are similar notations for functions, attributes, and modules themselves. In HTML, all of these will end up as links to the place in your document where that specific thing has been incorporated.
In addition to cross-referencing code, there are some notations you can use to help describe part of a function or method inside of your pydoc. If you use the
param notation, Sphinx will build a little table of your arguments to the function.
returns notation is to document what your function or method responds with.
If you want to bring a bit of style to your documentation, you can choose a different theme. Themes are available through PyPI and are
Once you’ve got it installed, you need to tell the output that it is using the theme you want to do that. This is done in the
conf.py file. I’ll be using the Read the Docs theme in this demo.
You can get that by
sphinx-rtd-theme. Note that the package name uses hyphens, but the module uses underscores.
05:03 Remember that for when I show you the config. Okay, you’re all set to incorporate pydoc comments into your documentation. Let’s go take a look.
This is the
conf.py file in the docs directory. I’ve messed with it a little bit. First off, I’ve changed the
copyright variable from a simple string to a bit of code.
I grab the
datetime class, use it to calculate the current year, and then check if the year is later than 2023. That’s the year of the recording. If it is, I make the copyright value have a range. If it’s still 2023, I just use the year and author like before. All this is possible because
conf.py is, well,
.py. This file gets run every time you build, so you can make your logic as convoluted as you like.
If you can do it in Python, you can do it in your config. Recall that I added a version variable to the
__init__ file in the
serenity module. It’s named
These three lines dynamically load that file and set the value of
conf.py to the value of
This means I only have to bump my version number in one place, the
__init__ file, in my code. My docs will stay in sync because
conf.py reads it.
Scrolling down, the next change was to add the
extensions configuration. As I mentioned before, that’s just a matter of adding a string with the full module name of the extension into this list. As
autodoc comes with Sphinx, this is all you need to do to use the
automodule directive and its friends.
Finally, I’m changing the look and feel of the HTML build by setting the
html_theme value to the Read the Docs theme as promised. All right, with all that configuration in place, let’s go look at what I’ve done to my
Earlier I mentioned this little trick. Often the contents of your doc home page is going to be the same as the README for your GitHub repo. As GitHub supports the RST format, you can write a
README.rst file then use the
include directive to suck that into your
index.rst file. This way, you don’t have to write the content twice.
Although I did change how
release gets populated, this line doesn’t change. It’ll still pull it in, referencing the newly dynamically loaded variable from
I’m going to put my pydoc stuff into a file called
serenity.rst, so like with
movie.rst, I’ve added it to my table of contents.
07:54 Let’s go look at the README file …
07:59 and here it is. This time I remembered to use the title style heading the way I should have before. Nothing fancy in this file, just a few sentences, but the magic is that it can be both my GitHub landing page for the project as well as the document homepage.
08:21 This is an RST file like any other. You can write as much or as little as you like here. If all your info is in your pydoc comments, this doc might be short, like in this example.
If you need to write something more or include images or whatnot, you can RST to your heart’s content. The first
automodule directive here references the
Note that the name of the module is fully qualified. It needs both
ship to find the right one. The
members option means all the contents of this file will be parsed.
special-members option means dunder methods will also be included in the output. My second
automodule directive is similar, but this one is for the
actions file. And that’s it.
09:07 With that in place, the docs are ready to go. Let’s look at the code that this file is going to be parsing.
This is my
__init__ file. As promised, I have declared a
__version__ variable to hold my release number. This is what gets read in by
conf.py, and it can also be used by my module itself.
ship.py. Up at the top here, I have my first pydoc comment, the pydoc for the class itself. It’s actually good practice to put this here rather than in, say, the
__init__ doesn’t always get included. Speaking of
__init__, this part of the pydoc is using the
param notation to describe the
passengers argument for the
And similarly, here inside of the
.crazy_ivan() method, I’m using the
returns notation to indicate that
.crazy_ivan() returns the
direction. On to
class marker here will result in a link to the
Firefly class documentation. This second reference uses the angle bracket (
<>) style to change how the name is displayed. And this is a reference to the
.pilot attribute inside the class. Attributes don’t get linked, but they do get styled appropriately.
Note that although I’m using these references in the pydoc itself, you can use them anywhere in your documentation. If one of your RST files mentions a class or function, you can use the same notation there to deep link into the docs for that class or function. I’ll skip the boring
make html part.
10:55 Let’s go see the result.
11:00 The new look and feel here is because of the Read the Docs theme being applied. The content next to my pointer is the README file that was included. Notice how the nav on the left and the table of contents now have the links to the Serenity Module.
Let me click that. And here is the
autodoc content. A couple things to note: because I used the
special-members option to the
automodule directive but didn’t explicitly list which dunder methods, I’ve got all of them:
If you only wanted
.__init__(), you could specify that in the option to the
automodule directive. Note how the
returns markers are being presented, showing the description.
This is even clearer on the
change_pilot() function where you have multiple arguments. A nice little listing is created.
11:59 The cross-reference here isn’t fantastic, as it points to the same page, but it is a fully qualified link with a hash actor. Clicking it does send me to the top of the class. Seeing as that’s all on the same page, there’s not much to see by doing that though.
One little caveat:
autodoc actually imports your code. That means your code has to be importable. In the case of frameworks like Django, things can go wrong if you haven’t got the environment set up correctly. As
conf.py is a script, you can do whatever setup you need though. This little snippet works for Django.
It adds the parent directory to the
django, and runs Django’s required setup. Depending on your framework and where you’ve put your code, this snippet would need to change. In fact, if I recall correctly, the code I pulled this out of didn’t use an
src/ directory, so that path statement would likely need to change if Serenity was a Django thing.
You might have to fiddle a little bit based on what framework you’re using, but you get the general idea: whatever setup you need can be done in
13:10 And there you go. You’ve got some docs. Next up, I’ll show you how to host them at Read the Docs.
Become a Member to join the conversation.