Articles in this series:
- Part 1: Asynchronous Testing with Django and PyVows
- Part 2: Unit Testing with pyVows and Django
- Part 3: Integration testing with pyVows and Django (current article)
If you’ve been following this series – part 1 and part 2 – you should now have a basic understanding of what pyVows is all about and how it can be used for asynchronous test execution for a Django project. We will continue in this final article of the three part series to explore how pyVows can be used for integration testing django and testing of django models.
There is much debate across the web about unit testing as it pertains to models. According to the most rigid definition a unit test shouldn’t rely on a database being around as that means you are technically doing an integration test. I personally am not so rigid on the definition and tend to be more pragmatic. In other words, if it makes it easier / quicker to write tests with a database back end and you can still achieve stable, reusable tests with good coverage than I’m all for it.
One of the biggest (and most valid) arguments against using an actual database in unit tests is that they are slow. While that is true, you will see in this article that running your tests asynchronously against a multi-threaded database can help to improve performance of the unit test runs. While it’s true testing against a live database backend will probably never be as fast as testing without the database we can make it reasonably fast with asynchronous testing.
Furthermore, by including the database back-end we can actually improve test coverage by validating that our database queries are actually performed successfully. Because when we hit the database we can test things like database constraints or that we are not trying to put a VARCHAR in a NUMBER field. And when we don’t hit the database with a “pure” unit test, we can’t verify these things. Anyways, lets get started.
A “Pure” unit test for a Django Model
Starting with the sample login application we have been using in this series, lets write a simple model to take us through some examples. Add the following code to
1 2 3
Now let’s write a simple test for it in
1 2 3 4 5 6 7 8
The setup is very similar to all the other tests we have done previously with pyVows. The only difference here is we are using the
DjangoHTTPContext.model function, which just returns a
django_pyvows.assertions.Model class to ensure the object being passed inherits from
Next we use the
to_have_field function of the
django_pyvows.assertion.Model class to verify that we have the field setup correctly.
Once we know the field is setup correctly we can use a few more assertions in the Model class to further verify the Model under test.
Not quite a unit test, but oh so helpful
Here is where we start to blur the line between pure unit tests and integration tests. The function
django_pyvows.assertions.Model.to_be_cruddable will try to store the model in the database, with some dummy data, it will then do an update and finally delete the data. All of that in one line of code. Awesome!
This is a classic example of a test that not only verifies our
Account class is setup correctly but the backend database and tables are setup correctly and that we can successfully perform CRUD operations against the database. This is great because it make sure we don’t have any sort of key constraints, user permissions or table design issues that get in the way of regular CRUD operations.
But there is some setup involved. In particular if we were to run this test right now we would get the following failure. Go ahead and try
1 2 3
See for yourself:
As you can see from the error message this test is trying to create and execute a query against a “live” database, which isn’t there. We can make sure it’s always there by just calling syncdb from our setup function.
1 2 3 4
This is the same thing as running
syncdb from the command line: it will create all the default django tables as well as tables for the models that you have defined. The second line will clear out any data you have loaded in those tables. For smaller systems the syncdb will run pretty quickly so as not to harm performance too much, but for larger systems it can take a while to run the syncdb command, which is probably something that we don’t want for our unit tests.
BUT WAIT, pyvows is asynchronous right? That’s right, and if you remember from part 1 sibling context’s run in parallel, so you can be running all your other unit tests while the database is syncing. Imagine a test structure like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29
In the above example the
flush commands would run before any model tests were run. But, concurrently to the database setup commands being run, unit tests in the
ControllerVows context would run. So in other words, you’re not really wasting time initializing the database as your other unit tests are running at the same time.
Note: Concurrency isn’t exactly parallesim especially if you’re running on CPython due to limitations of the GIL. So what is described above isn’t parallel execution – it’s just concurrent. Which will ultimately be faster than synchronous execution but perhaps not as fast as it could be. Check out this Stackoverflow question for a better understanding of this. It’s a great place to get an overview of what is actually going on behind the scenes. Also checkout the links in the accepted answer. I found them very informative.
Full Blown Integration Tests
Now that we have basic tests for our models running pretty quickly lets finish the login example so that the login page actually checks the database backend to ensure a valid password. In a production system you would probably want to just use
django.contrib.auth. But bear with me as this example makes the point very clear (I hope :) ). Here is what the code would look like for
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
Basically we have added in the functionality to execute a query on POST to see if the username / password combination inputted by the user exists in the database. If it does, we will display the Login Successful screen. If it doesn’t, we will display the login screen with the heading ‘Incorrect login, please try again.’ So let’s add some tests to our existing suite to ensure that this functionality works.
Since we already covered the basic functionality in our previous test from part 2 let’s create the test to ensure an invalid login attempt returns the correct page.
1 2 3 4 5 6 7 8 9
This test is truly an integration test because it starts with sending the request to the server by using the
Let’s add one more test to ensure a valid login. In order to do that (since this will be a live integration test) we will need to add a user to the database. So let’s put that all together now.
1 2 3 4 5 6 7 8 9 10 11 12
The only difference in the ValidLoginTest is we first use the
setup function (which pyVows ensures to be the first function executed in the class) to create the user that we want in the database. This way we can verify that our login page is correctly querying the database. Also notice that since we are adding data to the database we would want to make sure that our syncdb function had previously been called. In other words we would want the
PostValidLogin context to be a child of whatever context initialized the database, which in our case is the
AccountVows context. So the structure would look like this.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
This structure ensures that our database is always setup and cleared out before we run our
should_return_valid_login test. That way we can be sure that the database is setup correctly for our test to run.
As a final note it can be said that the
PostValidLogin context is a bit sloppy because its not cleaning up after itself. For our example we don’t really need to as we are creating a separate database for testing and we are clearing it at the start of the test run. But for the sake of completeness we could add a teardown function to our
PostValidLogin context and clear out the newly created row. Doing so would make the context look like this.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Finally, run the tests:
And you should see the following results:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32
And with the teardown of our last test context its time for the teardown of this article and the Django Testing with pyVows series. I hope this article has shown you some techniques to speed up not only the execution but also the creation of your unit and integration test. I don’t like to be to strict on the separation between unit and integration tests as there is often a use for both. We have completely skipped the topic of mocking because with django-pyvows setting up and executing the integration tests is so fast and easy that mocks often aren’t needed. At least not for this simple case. (But who knows maybe I’ll get around to another article on mocks and when they are useful).
The main point to come away with here is that there are many methods and frameworks to test Django applications each with it’s own unique set of advantages and disadvantages. By digging into django-pyVows throughout this series I hope you have at least seen some alternative approaches to testing. Even if you don’t ultimately end up using django-pyVows I hope that the approaches and techniques you have learned can help your testing efforts in the future. Again, grab the code from the repo.
Hit me up in the comments and let me know what you think of the series.