Using PyInstaller to Easily Distribute Python Applications

Distribution Problems

Setting up a Python project can be frustrating, especially for non-developers. Often, the setup starts with opening a Terminal, which is a non-starter for a huge group of potential users. This roadblock stops users even before the installation guide delves into the complicated details of virtual environments, Python versions, and the myriad of potential dependencies.

Think about what you typically go through when setting up a new machine for Python development. It probably goes something like this:

• Download and install a specific version of Python
• Set up pip
• Set up a virtual environment
• Get a copy of your code
• Install dependencies

Stop for a moment and consider if any of the above steps make any sense if you’re not a developer, let alone a Python developer. Probably not.

These problems explode if your user is lucky enough to get to the dependencies portion of the installation. This has gotten much better in the last few years with the prevalence of wheels, but some dependencies still require C/C++ or even FORTRAN compilers!

This barrier to entry is way too high if your goal is to make an application available to as many users as possible. As Raymond Hettinger often says in his excellent talks, “There has to be a better way.”

PyInstaller

PyInstaller abstracts these details from the user by finding all your dependencies and bundling them together. Your users won’t even know they’re running a Python project because the Python Interpreter itself is bundled into your application. Goodbye complicated installation instructions!

PyInstaller performs this amazing feat by introspecting your Python code, detecting your dependencies, and then packaging them into a suitable format depending on your Operating System.

There are lots of interesting details about PyInstaller, but for now you’ll learn the basics of how it works and how to use it. You can always refer to the excellent PyInstaller docs if you want more details.

In addition, PyInstaller can create executables for Windows, Linux, or macOS. This means Windows users will get a .exe, Linux users get a regular executable, and macOS users get a .app bundle. There are some caveats to this. See the limitations section for more information.

Preparing Your Project

PyInstaller requires your application to conform to some minimal structure, namely that you have a CLI script to start your application. Often, this means creating a small script outside of your Python package that simply imports your package and runs main().

The entry-point script is a Python script. You can technically do anything you want in the entry-point script, but you should avoid using explicit relative imports. You can still use relative imports throughout the rest your application if that’s your preferred style.

You can give this a try with your own project or follow along with the Real Python feed reader project. For more detailed information on the reader project, check out the the tutorial on Publishing a Package on PyPI.

The first step to building an executable version of this project is to add the entry-point script. Luckily, the feed reader project is well structured, so all you need is a short script outside the package to run it. For example, you can create a file called cli.py alongside the reader package with the following code:

from reader.__main__ import main

if __name__ == '__main__':
    main()

This cli.py script calls main() to start up the feed reader.

Creating this entry-point script is straightforward when you’re working on your own project because you’re familiar with the code. However, it’s not as easy to find the entry-point of another person’s code. In this case, you can start by looking at the setup.py file in the third-party project.

Look for a reference to the entry_points argument in the project’s setup.py. For example, here’s the reader project’s setup.py:

setup(
    name="realpython-reader",
    version="1.0.0",
    description="Read the latest Real Python tutorials",
    long_description=README,
    long_description_content_type="text/markdown",
    url="https://github.com/realpython/reader",
    author="Real Python",
    author_email="info@realpython.com",
    license="MIT",
    classifiers=[
        "License :: OSI Approved :: MIT License",
        "Programming Language :: Python",
        "Programming Language :: Python :: 2",
        "Programming Language :: Python :: 3",
    ],
    packages=["reader"],
    include_package_data=True,
    install_requires=[
        "feedparser", "html2text", "importlib_resources", "typing"
    ],
    entry_points={"console_scripts": ["realpython=reader.__main__:main"]},
)

As you can see, the entry-point cli.py script calls the same function mentioned in the entry_points argument.

After this change, the reader project directory should look like this, assuming you checked it out into a folder called reader:

reader/
|
├── reader/
|   ├── __init__.py
|   ├── __main__.py
|   ├── config.cfg
|   ├── feed.py
|   └── viewer.py
|
├── cli.py
├── LICENSE
├── MANIFEST.in
├── README.md
├── setup.py
└── tests

Notice there is no change to the reader code itself, just a new file called cli.py. This entry-point script is usually all that’s necessary to use your project with PyInstaller.

However, you’ll also want to look out for uses of __import__() or imports inside of functions. These are referred to as hidden imports in PyInstaller terminology.

You can manually specify the hidden imports to force PyInstaller to include those dependencies if changing the imports in your application is too difficult. You’ll see how to do this later in this tutorial.

Once you can launch your application with a Python script outside of your package, you’re ready to give PyInstaller a try at creating an executable.

Using PyInstaller

The first step is to install PyInstaller from PyPI. You can do this using pip like other Python packages:

$ pip install pyinstaller

pip will install PyInstaller’s dependencies along with a new command: pyinstaller. PyInstaller can be imported in your Python code and used as a library, but you’ll likely only use it as a CLI tool.

You’ll use the library interface if you create your own hook files.

You’ll increase the likelihood of PyInstaller’s defaults creating an executable if you only have pure Python dependencies. However, don’t stress too much if you have more complicated dependencies with C/C++ extensions.

PyInstaller supports lots of popular packages like NumPy, PyQt, and Matplotlib without any additional work from you. You can see more about the list of packages that PyInstaller officially supports by referring to the PyInstaller documentation.

Don’t worry if some of your dependencies aren’t listed in the official docs. Many Python packages work fine. In fact, PyInstaller is popular enough that many projects have explanations on how to get things working with PyInstaller.

In short, the chances of your project working out of the box are high.

To try creating an executable with all the defaults, simply give PyInstaller the name of your main entry-point script.

First, cd in the folder with your entry-point and pass it as an argument to the pyinstaller command that was added to your PATH when PyInstaller was installed.

For example, type the following after you cd into the top-level reader directory if you’re following along with the feed reader project:

$ pyinstaller cli.py

Don’t be alarmed if you see a lot of output while building your executable. PyInstaller is verbose by default, and the verbosity can be cranked way up for debugging, which you’ll see later.

Digging Into PyInstaller Artifacts

PyInstaller is complicated under the hood and will create a lot of output. So, it’s important to know what to focus on first. Namely, the executable you can distribute to your users and potential debugging information. By default, the pyinstaller command will create a few things of interest:

• A *.spec file
• A build/ folder
• A dist/ folder

# -*- mode: python -*-

block_cipher = None

a = Analysis(['cli.py'],
             pathex=['/Users/realpython/pyinstaller/reader'],
             binaries=[],
             datas=[],
             hiddenimports=[],
             hookspath=[],
             runtime_hooks=[],
             excludes=[],
             win_no_prefer_redirects=False,
             win_private_assemblies=False,
             cipher=block_cipher,
             noarchive=False)
pyz = PYZ(a.pure, a.zipped_data,
             cipher=block_cipher)
exe = EXE(pyz,
          a.scripts,
          [],
          exclude_binaries=True,
          name='cli',
          debug=False,
          bootloader_ignore_signals=False,
          strip=False,
          upx=True,
          console=True )
coll = COLLECT(exe,
               a.binaries,
               a.zipfiles,
               a.datas,
               strip=False,
               upx=True,
               name='cli')

build/
|
└── cli/
    ├── Analysis-00.toc
    ├── base_library.zip
    ├── COLLECT-00.toc
    ├── EXE-00.toc
    ├── PKG-00.pkg
    ├── PKG-00.toc
    ├── PYZ-00.pyz
    ├── PYZ-00.toc
    ├── warn-cli.txt
    └── xref-cli.html

dist/
|
└── cli/
    └── cli

Customizing Your Builds

PyInstaller comes with lots of options that can be provided as spec files or normal CLI options. Below, you’ll find some of the most common and useful options.

--name

Change the name of your executable.

This is a way to avoid your executable, spec file, and build artifact folders being named after your entry-point script. --name is useful if you have a habit of naming your entry-point script something like cli.py, as I do.

You can build an executable called realpython from the cli.py script with a command like this:

$ pyinstaller cli.py --name realpython

--onefile

Package your entire application into a single executable file.

The default options create a folder of dependencies and and executable, whereas --onefile keeps distribution easier by creating only an executable.

This option takes no arguments. To bundle your project into a single file, you can build with a command like this:

$ pyinstaller cli.py --onefile

With the above command, your dist/ folder will only contain a single executable instead of a folder with all the dependencies in separate files.

--hidden-import

List multiple top-level imports that PyInstaller was unable to detect automatically.

This is one way to work around your code using import inside functions and __import__(). You can also use --hidden-import multiple times in the same command.

This option requires the name of the package that you want to include in your executable. For example, if your project imported the requests library inside of a function, then PyInstaller would not automatically include requests in your executable. You could use the following command to force requests to be included:

$ pyinstaller cli.py --hiddenimport=requests

You can specify this multiple times in your build command, once for each hidden import.

--add-data and --add-binary

Instruct PyInstaller to insert additional data or binary files into your build.

This is useful when you want to bundle in configuration files, examples, or other non-code data. You’ll see an example of this later if you’re following along with the feed reader project.

--exclude-module

Exclude some modules from being included with your executable

This is useful to exclude developer-only requirements like testing frameworks. This is a great way to keep the artifact you give users as small as possible. For example, if you use pytest, you may want to exclude this from your executable:

$ pyinstaller cli.py --exclude-module=pytest

-w

Avoid automatically opening a console window for stdout logging.

This is only useful if you’re building a GUI-enabled application. This helps your hide the details of your implementation by allowing users to never see a terminal.

Similar to the --onefile option, -w takes no arguments:

$ pyinstaller cli.py -w

.spec file

As mentioned earlier, you can reuse the automatically generated .spec file to further customize your executable. The .spec file is a regular Python script that implicitly uses the PyInstaller library API.

Since it’s a regular Python script, you can do almost anything inside of it. You can refer to the official PyInstaller Spec file documentation for more information on that API.

Testing Your New Executable

The best way to test your new executable is on a new machine. The new machine should have the same OS as your build machine. Ideally, this machine should be as similar as possible to what your users use. That may not always be possible, so the next best thing is testing on your own machine.

The key is to run the resulting executable without your development environment activated. This means run without virtualenv, conda, or any other environment that can access your Python installation. Remember, one of the main goals of a PyInstaller-created executable is for users to not need anything installed on their machine.

Picking up with the feed reader example, you’ll notice that running the default cli executable in the dist/cli folder fails. Luckily the error points you to the problem:

FileNotFoundError: 'version.txt' resource not found in 'importlib_resources'
[15110] Failed to execute script cli

The importlib_resources package requires a version.txt file. You can add this file to the build using the --add-data option. Here’s an example of how to include the required version.txt file:

$ pyinstaller cli.py \
    --add-data venv/reader/lib/python3.6/site-packages/importlib_resources/version.txt:importlib_resources

This command tells PyInstaller to include the version.txt file in the importlib_resources folder in a new folder in your build called importlib_resources.

You’ll want to adjust the path in the above command to match where you installed the feed reader dependencies.

Now running the new executable will result in a new error about a config.cfg file.

This file is required by the feed reader project, so you’ll need to make sure to include it in your build:

$ pyinstaller cli.py \
    --add-data venv/reader/lib/python3.6/site-packages/importlib_resources/version.txt:importlib_resources \
    --add-data reader/config.cfg:reader

Again, you’ll need to adjust the path to the file based on where you have the feed reader project.

At this point, you should have a working executable that can be given directly to users!

Debugging PyInstaller Executables

As you saw above, you might encounter problems when running your executable. Depending on the complexity of your project, the fixes could be as simple as including data files like the feed reader example. However, sometimes you need more debugging techniques.

Below are a few common strategies that are in no particular order. Often times one of these strategies or a combination will lead to a break-through in tough debugging sessions.

$ pyinstaller --log-level=DEBUG cli.py 2> build.txt

67 INFO: PyInstaller: 3.4
67 INFO: Python: 3.6.6
73 INFO: Platform: Darwin-18.2.0-x86_64-i386-64bit
74 INFO: wrote /Users/realpython/pyinstaller/reader/cli.spec
74 DEBUG: Testing for UPX ...
77 INFO: UPX is not available.
78 DEBUG: script: /Users/realptyhon/pyinstaller/reader/cli.py
78 INFO: Extending PYTHONPATH with paths
['/Users/realpython/pyinstaller/reader',
 '/Users/realpython/pyinstaller/reader']

Limitations

PyInstaller is incredibly powerful, but it does have some limitations. Some of the limitations were discussed previously: hidden imports and relative imports in entry-point scripts.

PyInstaller supports making executables for Windows, Linux, and macOS, but it cannot cross compile. Therefore, you cannot make an executable targeting one Operating System from another Operating System. So, to distribute executables for multiple types of OS, you’ll need a build machine for each supported OS.

Related to the cross compile limitation, it’s useful to know that PyInstaller does not technically bundle absolutely everything your application needs to run. Your executable is still dependent on the users’ glibc. Typically, you can work around the glibc limitation by building on the oldest version of each OS you intend to target.

For example, if you want to target a wide array of Linux machines, then you can build on an older version of CentOS. This will give you compatibility with most versions newer than the one you build on. This is the same strategy described in PEP 0513 and is what the PyPA recommends for building compatible wheels.

In fact, you might want to investigate using the PyPA’s manylinux docker image for your Linux build environment. You could start with the base image then install PyInstaller along with all your dependencies and have a build image that supports most variants of Linux.

Conclusion

PyInstaller can help make complicated installation documents unnecessary. Instead, your users can simply run your executable to get started as quickly as possible. The PyInstaller workflow can be summed up by doing the following:

1. Create an entry-point script that calls your main function.
2. Install PyInstaller.
3. Run PyInstaller on your entry-point.
4. Test your new executable.
5. Ship your resulting dist/ folder to users.

Your users don’t have to know what version of Python you used or that your application uses Python at all!