Please note: This is a collaboration piece between Michael Herman, from Real Python, and Sean Vieira, a Python developer from De Deo Designs.
Articles in this series:
- Part I: Application setup <— CURRENT ARTICLE
- Part II: Setup user accounts, Templates, Static files
- Part III: Testing (unit and integration), Debugging, and Error handling
Updated: November 17th, 2013
Recently, thanks to an awesome gig, I re-introduced myself to Flask, which is “a microframework for Python”. It is a great little framework; unfortunately, like Sinatra (which inspired it), making the transition from small “self-contained” applications to larger applications is difficult. There are a number of boilerplates out there (including my own Flask-Boilerplate) to help make the transition easier. However, using a boilerplate doesn’t help us understand the “why” of the boilerplate’s conventions.
After reviewing the available tutorials out there I thought, “there must be a better way” – then I saw Miguel Grinberg’s mega-tutorial. The focus that building an application rather than a boilerplate gave to his articles impressed me. So in this tutorial series we are going to be building a mid-sized application of our own and as a side-effect generating a boilerplate. The hope is that when we are finished we will have a better appreciation for what Flask provides and what we can build around it.
What we are building
This series will be following the development of a web analytics solution called Flask-Tracking. The goal of this series is to have a working application that will enable users to:
- Register with the application.
- Add sites to their account.
- Install tracking codes on their site(s) to track various events as the events occur.
- View reports on their site(s)‘ event activity.
At the end of the day, our goal (over and above having a working application that conforms to the above napkin spec) is to:
- Learn how to develop a mid-sized application using Flask.
- Get practical experience with refactoring and testing.
- Gain a deeper appreciation of building composable systems.
- Develop a boilerplate that we can re-use for future projects.
This series assumes that you have a passing familiarity with Python and Flask already. You should already be comfortable with everything in Python’s tutorial, be familiar with the command line (or, at the very least, with
pip), and you should have read Flask’s Quickstart and Tutorial. If you want more practice, take it a step further and read my beginning tutorial on Flask here. That being said, if you are a newcomer to Python and Flask you should still be able to follow along.
Our goal today is to implement the core of our application – tracking visits to a site. We will allow multiple sites, but we will not worry about users or access control yet.
Before we dive in I would like to touch on some of the principles that will guide our development – we will not be delving too deeply into the “whys” of some of these best practices, but where possible will provide links to articles that explain what these practices provide us. Please consider all of the opinions in this series to be strong opinions, weakly held.
The Zen of Python
If you did not already know, Python has a “development philosophy” called the Zen of Python (also known as PEP 20). To give it a read, simply open a python interpreter and type:
and then you’ll see:
The Zen of Python, by Tim Peters
Beautiful is better than ugly. Explicit is better than implicit. Simple is better than complex. Complex is better than complicated. Flat is better than nested. Sparse is better than dense. Readability counts. Special cases aren’t special enough to break the rules. Although practicality beats purity. Errors should never pass silently. Unless explicitly silenced. In the face of ambiguity, refuse the temptation to guess. There should be one— and preferably only one —obvious way to do it. Although that way may not be obvious at first unless you’re Dutch. Now is better than never. Although never is often better than right now. If the implementation is hard to explain, it’s a bad idea. If the implementation is easy to explain, it may be a good idea. Namespaces are one honking great idea — let’s do more of those!
These are words to work by. Read Daniel Greenfeld’s notes on Richard Jones’ 19 Pythonic Theses talk for a more detailed explanation.
We will also conform our development practices to the following principles:
- While there are many tools we could build to make our lives easier, we are going to confine ourselves to building what is necessary to make our application work, remembering that in many cases YAGNI (You Ain’t Gonna Need It).
- That being said, when we come across a pattern of code that we are repeating across the application we will be refactoring our code to keep it DRY.
- Our trigger to cross over from YAGNI to DRY will be the three strikes and you refactor principle. Using a pattern in three places will qualify it for extraction. Two uses will still fall under YAGNI. (In large projects many suggest a modified version of this rule – when you need to reuse it, refactor it.)
A note on the repository structure
Throughout this series you will be able to find the finished code for these exercises in the
flask-tracking repository on Github. Each part of this tutorial has a branch in the repository and a release. The code for this section is in the part-1 branch. If you choose to check out the repository, you can work with the code for any article in this series by simply running:
N is the number of the article the code is for (so for this article use
git checkout part-1).
Starting the project
With that front-matter out of the way let’s start by creating a new folder to hold our project and activating a virtual environment (if you are unsure how to setup a virtual environment, take a moment and review the guide in Flask’s quickstart):
1 2 3 4
Like Alice in Wonderland we want to get our dependencies “just right”. Too many dependencies and we will not be able to work on the codebase after leaving it for a month without spending a week reviewing documentation. Too few dependencies and we will spend our time developing everything except our application. To ensure that we keep our dependencies “just right” we will use the following (imperfect) rule – each dependency must solve at least one difficult problem well. We will start with dependencies on three libraries – Flask itself for managing the request / response cycle, Flask-WTF for its CSRF protection and data validation and finally Flask-SQLAlchemy for database connection pooling and object / relational mapper. Here is our requirements.txt file:
1 2 3
Now we can run
pip install -r requirements.txt to install our requirements into our virtual environment.
Most Flask applications start out small and then are refactored as the scope of the project grows. For those of us who have not written and then refactored a small Flask application we have a single module version of our tracking application in the
part-0 branch (it weighs in at 145 lines total). Feel free to skip to the next section if you have already written a single module application in Flask.
For those of you who remain, please
git checkout part-0 or download Part 0 from the releases section of the repository. There should be a single module inside the application,
tracking.py, and the overal structure should look like this:
1 2 3 4 5 6 7 8 9 10 11 12
If you crack it open, the first thing you see after the imports is:
1 2 3 4 5 6 7 8 9 10 11 12
We set up some basic config settings – the
SECRET_KEY is used to sign Flask’s sessions, the
SQLALCHEMY_DATABASE_URI is the path to our database (we are using SQLite for now), and the
WTF_CSRF_SECRET_KEY is used to sign WTForms’ CSRF tokens. We initialize a new Flask application and tell it to configure itself (with our
app.config.from_object call) with all of the
ALL_CAPS symbols in the current module. Then we initialize our Flask-SQLAlchemy extension with our application.
From there, it is pretty much a straight shot – we set up our models:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
1 2 3 4 5 6
and routes that utilize these models and forms:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
There are also a couple of helper functions to turn SQLAlchemy query objects into lists of lists of data for our templates and then at the bottom of the module we set up a
main block to create our tables if they do not exist and run the application in debug mode:
1 2 3 4
If you run
python tracking.py and then navigate to localhost:5000 in your browser you will see the extremely bare-bones application. It is functional (you can create sites and add visits), but it has no users and no access control – everyone sees everything.
However, as you can see,
tracking.py is already quite a smörgåsbord of functionality – controllers, models, helpers, and command line setup all jammed into one file. Breaking it up into separate areas of functionality will make it easier to maintain. In addition, re-packaging this application will make it clearer where to add all of the other features that we want (users, access control, etc.). When you are done playing around with the Part 0 code just run
git checkout part-1 and move on to the next section.
A place for everything
1 2 3 4 5
Keeping in mind that our goal today is simply to track visits to a site, we will avoid creating a sub-package for our tracking code … yet (remember, YAGNI until you need it). Let’s go ahead and add separate
views modules to hold our domain models, our data translation layer, and our view code respectively. Let’s also create a
templates subdirectory to hold the resources we will use to render the site. Finally, we will add a configuration file and a script to run the application. Now we should have a directory structure that looks like this:
1 2 3 4 5 6 7 8 9 10 11
Our domain models are simple – a site has a root URL and a visit has metadata about the visitor (browser, IP address, URL, etc.). We include
__repr__ methods to give us better details on the command line (
<Visit for site ID 1: / - 2013-11-09 14:10:11> is more useful than
<package.module.Visit object at 0x12345>).
Site includes a
__str__ method to control what we display when we display a list of sites in a drop down menu.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
Note that we have not initialized our
flask.ext.sqlalchemy object with any application so these models are not bound to this particular application (this flexibility comes with some small costs, which we will encounter shortly).
Data translation layer
forms code is standard WTForms, with one exception. If you look at
1 2 3 4 5 6 7 8 9 10 11
you will note that we need to wrap
Site.query.all in a
lambda rather than passing it in as-is. Since our
db is not bound to an application when
VisitForm is being constructed we cannot access
Site.query. Creating a function that will call
Site.query only when
VisitForm is instantiated (e. g. in our views we call
form = VisitForm()) ensures that we will only access
Site.query when we will have access to a Flask application instance.
Our views are the most complex part of our application (witness how many dependencies we are importing):
1 2 3 4
We start out by creating a blueprint (we could also
from flask_tracking import app and use the app, but I prefer to switch to blueprints once my application has grown beyond a single file)
We then map our views to routes using the normal decorator syntax – I have added comments around some of the functionality where what we are doing may not be perfectly clear (or where we are repeating ourselves and we will want to refactor later):
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84
This gives us an application that will let us add a site or a visit on the home page, view a list of sites, and view each site’s visits. We have not actually registered the blueprint with an application yet, so …
… onwards to
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
We create an application, configure it, register our Flask-SQLAlchemy instance on our application, and finally register our blueprint. Now Flask knows how to handle our routes.
Configuration and command line runner
One level up our configuration enables Flask’s sessions and sets up our SQLite database:
1 2 3 4 5 6 7 8
And our application runner creates the database tables if they do not exist and runs the application in debug mode:
1 2 3 4 5 6 7 8 9 10 11 12
See Flask-SQLAlchemy’s Introduction to Contexts if you want to get a better sense of why we need to pass our
app into the
And everything in its place
Now we should be able to run
python run.py and see our application start up. Go to localhost:5000 and create a site to test things out. Then, to verify that others can add visits to the site, try running:
from the command line. When you go back to the application, click on “Sites” and then click on the “1” for your site. A single visit should be displayed on the page:
That’s it for this post. We now have a working application where sites can be added and visits recorded against them. We still need to add user accounts, an easy to use client-side API for tracking, and reports.
In Part II we will add users, access control, and enable users to add visits from their own websites. We will explore more best practices for writing templates, keeping our models and forms in sync, and handling static files.
In Part III we’ll explore writing tests for our application, logging, and debugging errors.
In Part IV we’ll do some Test Driven Development to enable our application to accept payments and display simple reports.
In Part V we will write a RESTful JSON API for others to consume.
In Part VI we will cover automating deployments (on Heroku) with Fabric and basic A/B Feature Testing.
Finally, in Part VII we will cover preserving your application for the future with documentation, code coverage and quality metric tools.
Thanks for reading and tune in next time!