Let’s build a command-line utility for quickly generating a Flask boilerplate structure.
Modeled after the Flask-Skeleton project, this tool will automate a number of repetitive tasks so that you can quickly get a Flask project up and running with the structure, extensions, and configurations that you prefer, step by step:
- Set up the basic structure
- Add a custom config file
- Utilize Bower to manage front-end dependencies
- Create a virtualenv
- Initialize Git
Once done, you’ll have a powerful scaffolding script that you can (and should) customize to meet your own development needs.
Updates:
- 08/01/2016: Upgraded to Python version 3.5.1.
Quickstart
To start, we need a basic Flask application. For simplicity, we’ll use the Real Python boilerplate Flask structure, so just clone it down to set the base structure:
$ mkdir flask-scaffold
$ cd flask-scaffold
$ git clone https://github.com/realpython/flask-skeleton skeleton
$ rm -rf skeleton/.git
$ rm skeleton/.gitignore
$ mkdir templates
$ pyvenv-3.5 env
$ source env/bin/activate
This article utilizes Python 3.5; however, the final script is compatible with both Python 2 and 3.
1st Task – The Structure
Save a new Python file as flask_skeleton.py in the root directory. This file will be used to power the entire scaffolding utility. Open it up in you favorite text editor and add the following code:
# -*- coding: utf-8 -*-
import sys
import os
import argparse
import shutil
# Globals #
cwd = os.getcwd()
script_dir = os.path.dirname(os.path.realpath(__file__))
def main(argv):
# Arguments #
parser = argparse.ArgumentParser(description='Scaffold a Flask Skeleton.')
parser.add_argument('appname', help='The application name')
parser.add_argument('-s', '--skeleton', help='The skeleton folder to use.')
args = parser.parse_args()
# Variables #
appname = args.appname
fullpath = os.path.join(cwd, appname)
skeleton_dir = args.skeleton
# Tasks #
# Copy files and folders
shutil.copytree(os.path.join(script_dir, skeleton_dir), fullpath)
if __name__ == '__main__':
main(sys.argv)
Here, we’re using argparse to obtain an appname for the new project and then copying the skeleton directory (via shutil), which contains the project boilerplate, to quickly recreate the project structure.
The shutil.copytree() method (source) is used to recursively copy a source directory to a destination directory (as long as the destination directory does not already exist).
Test it out:
$ python flask_skeleton.py new_project -s skeleton
This should make a copy of the Real Python boilerplate Flask structure (source) to a new directory called “new_project” (destination). Did it work? If so, remove the new project since there’s still much work to be done:
$ rm -rf new_project
Handling Multiple Skeletons
What if you need an app with a MongoDB database or a payments blueprint? All apps have specific needs and you obviously can’t create a skeleton for them all, but maybe there are certain features that are needed a majority of the time. Maybe you need a NoSQL database about fifty percent of the time, for example. You can add a new skeleton to the root to accomplish this. Then when you run the scaffold command, simply specify the name of the directory containing the skeleton app you wish to make a copy of.
2nd Task – Configuration
We now need to generate a custom config.py file for each skeleton. This script is going to do just that for us; let the code do the repetitive work! First, add a file called config.jinja2 in the templates folder:
# config.jinja2
import os
basedir = os.path.abspath(os.path.dirname(__file__))
class BaseConfig(object):
"""Base configuration."""
SECRET_KEY = '{{ secret_key }}'
DEBUG = False
BCRYPT_LOG_ROUNDS = 13
WTF_CSRF_ENABLED = True
DEBUG_TB_ENABLED = False
DEBUG_TB_INTERCEPT_REDIRECTS = False
class DevelopmentConfig(BaseConfig):
"""Development configuration."""
DEBUG = True
BCRYPT_LOG_ROUNDS = 13
WTF_CSRF_ENABLED = False
SQLALCHEMY_DATABASE_URI = 'sqlite:///' + os.path.join(basedir, 'dev.sqlite')
DEBUG_TB_ENABLED = True
class TestingConfig(BaseConfig):
"""Testing configuration."""
DEBUG = True
TESTING = True
BCRYPT_LOG_ROUNDS = 13
WTF_CSRF_ENABLED = False
SQLALCHEMY_DATABASE_URI = 'sqlite:///'
DEBUG_TB_ENABLED = False
class ProductionConfig(BaseConfig):
"""Production configuration."""
SECRET_KEY = '{{ secret_key }}'
DEBUG = False
SQLALCHEMY_DATABASE_URI = 'postgresql://localhost/example'
DEBUG_TB_ENABLED = False
At the start of the scaffold script, flask_skeleton.py, just before the main() function, we need to initialize Jinja2 in order to render the config correctly.
# Jinja2 environment
template_loader = jinja2.FileSystemLoader(searchpath=os.path.join(script_dir, "templates"))
template_env = jinja2.Environment(loader=template_loader)
Make sure to add the import as well:
import jinja2
Install:
$ pip install jinja2
$ pip freeze > requirements.txt
Looking back at the template, config.jinja2, we have one variable that needs to be defined - {{ secret_key }}.To do this, we can use the codecs module.
To the imports of flask_skeleton.py add:
import codecs
Add the following code to the bottom of the main() function:
# Create config.py
secret_key = codecs.encode(os.urandom(32), 'hex').decode('utf-8')
template = template_env.get_template('config.jinja2')
template_var = {
'secret_key': secret_key,
}
with open(os.path.join(fullpath, 'project', 'config.py'), 'w') as fd:
fd.write(template.render(template_var))
What if you manage several skeletons and need several configuration templates?
Simple: You just have to check which skeleton is passed as an argument and use the appropriate config template. Keep in mind that os.path.join(fullpath, 'project', 'config.py') must represent the path where your configuration should be stored in your skeleton. If this is different for each skeleton, then you should specify the folder in which the config file is stored in as an additional argparse argument.
Ready to test?
$ python flask_skeleton.py new_project -s skeleton
Make sure the config.py file is present in the “new_project/project” folder and then remove the new project: rm -rf new_project
3rd Task – Bower
That’s right: We’ll be using bower to download and manage static libraries. To add bower support to the scaffold script, start with adding another argument:
parser.add_argument('-b', '--bower', help='Install dependencies via bower')
And to handle the running of bower, add the following code right below the config section of the scaffold script:
# Add bower dependencies
if args.bower:
bower = args.bower.split(',')
bower_exe = which('bower')
if bower_exe:
os.chdir(os.path.join(fullpath, 'project', 'client', 'static'))
for dependency in bower:
output, error = subprocess.Popen(
[bower_exe, 'install', dependency],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE
).communicate()
# print(output)
if error:
print("An error occurred with Bower")
print(error)
else:
print("Could not find bower. Ignoring.")
Don’t forget to add the subprocess module to the import section of flask_skeleton.py - import subprocess.
Did you notice the which() method (source)? This essentially uses the unix/linux which tool in order to indicate where an executable is installed in the filesystem. So, in the above code, we’re checking to see if bower is installed. If you’re curious to know how this works, test it out in the Python 3 interpreter:
>>> import shutil
>>> shutil.which('bower')
'/usr/local/bin/bower'
Unfortunately, this method, which(), is new to Python 3.3, so, if you’re using Python 2, then you need to install a separate package - shutilwhich:
$ pip install shutilwhich
$ pip freeze > requirements.txt
Update the imports:
if sys.version_info < (3, 0):
from shutilwhich import which
else:
from shutil import which
Finally, turn you attention to the following lines of code:
output, error = subprocess.Popen(
[bower_exe, 'install', dependency],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE
).communicate()
# print(output)
if error:
print("An error occurred with Bower")
print(error)
Start by looking at the official subprocess documentation. Put simply, it’s used for invoking external shell commands. In the above code we are simply capturing the output from stdout and stderr.
If you’re curious to see what the output is, uncomment the print statement, # print(output), and then run your code…
Before testing, this code assumes that in your skeleton folder(s), you have a “project” folder that contains a “static” folder. Classical Flask application. On the command line, you can now install multiple dependencies like so:
$ python flask_skeleton.py new_project -s skeleton -b 'angular, jquery, bootstrap'
4th Task – virtualenv
Since the virtual environment is one of the most important parts of any Flask (err, Python) application, creating the virtualenv using the scaffold script will be really useful. As usual, start by adding the argument:
parser.add_argument('-v', '--virtualenv', action='store_true')
And then add the following code below the bower section:
# Add a virtualenv
virtualenv = args.virtualenv
if virtualenv:
virtualenv_exe = which('pyvenv')
if virtualenv_exe:
output, error = subprocess.Popen(
[virtualenv_exe, os.path.join(fullpath, 'env')],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE
).communicate()
if error:
with open('virtualenv_error.log', 'w') as fd:
fd.write(error.decode('utf-8'))
print("An error occurred with virtualenv")
sys.exit(2)
venv_bin = os.path.join(fullpath, 'env/bin')
output, error = subprocess.Popen(
[
os.path.join(venv_bin, 'pip'),
'install',
'-r',
os.path.join(fullpath, 'requirements.txt')
],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE
).communicate()
if error:
with open('pip_error.log', 'w') as fd:
fd.write(error.decode('utf-8'))
sys.exit(2)
else:
print("Could not find virtualenv executable. Ignoring")
This snippet assumes that there is a requirements.txt file in your “skeleton” folder in the root directory. If so, it will create a virtualenv and then install the dependencies.
5th Task – Git Init
Notice a pattern yet? Add the argument:
parser.add_argument('-g', '--git', action='store_true')
Then add the code under the task for the virtualenv:
# Git init
if args.git:
output, error = subprocess.Popen(
['git', 'init', fullpath],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE
).communicate()
if error:
with open('git_error.log', 'w') as fd:
fd.write(error.decode('utf-8'))
print("Error with git init")
sys.exit(2)
shutil.copyfile(
os.path.join(script_dir, 'templates', '.gitignore'),
os.path.join(fullpath, '.gitignore')
)
Now within the templates folder add a .gitignore file, and then add the files and folders that you’d like to ignore. Grab the example from Github, if needed. Test again.
Sum and Confirm
Finally, let’s add a nice summary before the application is created and then ask for user confirmation before executing the script…
Summary
Add a file called brief.jinja2 to the “templates” folder:
Welcome! The following settings will be used to create your application:
Python Version: {{ pyversion }}
Project Name: {{ appname }}
Project Path: {{ path }}
Virtualenv: {% if virtualenv %}Enabled{% else %}Disabled{% endif %}
Skeleton: {{ skeleton }}
Git: {% if git %}Yes{% else %}{{ disabled }}No{% endif %}
Bower: {% if bower %}Enabled{% else %}Disabled{% endif %}
{% if bower %}Bower Dependencies: {% for dependency in bower %}{{ dependency }}{% endfor %}{% endif %}
Now we just need to catch every user-supplied argument and then render the template. First, add the import - import platform - to the import section and then the following code just below the “variables” section in the flask_skeleton.py script:
# Summary #
def generate_brief(template_var):
template = template_env.get_template('brief.jinja2')
return template.render(template_var)
template_var = {
'pyversion': platform.python_version(),
'appname': appname,
'bower': args.bower,
'virtualenv': args.virtualenv,
'skeleton': args.skeleton,
'path': fullpath,
'git': args.git
}
print(generate_brief(template_var))
Test this out:
$ python flask_skeleton.py new_project -s skeleton -b 'angular, jquery, bootstrap' -g -v
You should see something like:
Welcome! The following settings will be used to create your application:
Python Version: 3.5.1
Project Name: new_project
Project Path: /Users/michael/repos/realpython/flask-scaffold/new_project
Virtualenv: Enabled
Skeleton: skeleton
Git: Yes
Bower: Enabled
Bower Dependencies: angular, jquery, bootstrap
Nice!
Refactor
Now we need to refactor the script a bit to check for errors first. I suggest grabbing the code from the refactor tag and then comparing the diff since there are a number of small updates.
Make sure you use the updated script from the refactor tag before moving on.
Confirm
Now let’s add the user confirmation functionality by updating if __name__ == '__main__'::
if __name__ == '__main__':
arguments = get_arguments(sys.argv)
print(generate_brief(arguments))
if sys.version_info < (3, 0):
input = raw_input
proceed = input("\nProceed (yes/no)? ")
valid = ["yes", "y", "no", "n"]
while True:
if proceed.lower() in valid:
if proceed.lower() == "yes" or proceed.lower() == "y":
main(arguments)
print("Done!")
break
else:
print("Goodbye!")
break
else:
print("Please respond with 'yes' or 'no' (or 'y' or 'n').")
proceed = input("\nProceed (yes/no)? ")
This should be fairly straightforward.
Run!
If you use Linux or Mac you can make this script easier to run. Simply add the following alias to either .bashrc or .zshrc, customizing it to match your directory structure:
alias flaskcli="python /Users/michael/repos/realpython/flask-scaffold/flask_skeleton.py"
NOTE: If you have both Python 2.7 and Python 3.5 installed you will have to specify the version you want to use - either
pythonorpython3.
Remove the new project (if necessary) - rm -rf new_project - and then test out the script one last time to confirm:
$ flaskcli new_project -s skeleton -b 'angular, jquery, bootstrap' -g -v
Conclusion
What do you think? Did we miss anything? What other arguments would you add to argparse in order to customize your scaffold script even further? Comment below!
Grab the final code from the repo.
This is a collaboration piece between Depado and the folks at Real Python. Edits made by Derrick Kearney.
