Validating Results

00:00 In the previous lesson, I showed you the basic structure of a test class. In this lesson, I’ll show you how to test the results of your code.

00:08 You’ll recall that your test class inherits from the TestCase base class. That base class includes a whole whack of methods to help you validate results from your code.

00:18 Each of them does a different kind of comparison against an expected value, and they raise an exception if the comparison fails. There’s a family of these methods, and they all begin with the word assert.

00:29 Two things to note here. First, although they share the name assert with the Python keyword, these are just methods on the base class. They aren’t actually the assert call.

00:39 They use the keyword inside of the method, but it is still just a method. The second thing to note is that the unittest module has been around for a long time, before Python was consistent about naming conventions.

00:51 All of the assert methods are in camelCase, also known as mixed case, unlike the current naming convention of using snake_case, where everything is small letters and separated by underscores.

01:02 This makes my OCD a little cranky, but so be it. Since these assertion methods call assert, and since asserts that fail raise an AssertionError, if the comparison fails, the method raises an exception.

01:15 Tests that raise exceptions are considered failed by the framework. There isn’t anything magical about assertion errors. If your code blows up and raises an exception for some other reason, that’ll also fail the test.

01:27 Let’s look at an example that’s a little more realistic than the previous lesson’s Hello, World.

01:33 I’m inside of the check_values directory and looking at ages.py. This file has a single function called categorize() that takes an integer and returns a string based on the age of the argument.

01:45 If the person is between 0 and 9 years old, inclusive, the word child gets returned. 10 to 18 are teens, 19 to 150 is adult, and everything else is considered an invalid age.

02:00 Before I show you the test code, think about what you’d do to test this function. You’d want to test each of the possible conditions. So at bare minimum, you’d call the code four times.

02:10 But what if I messed up and accidentally made one of those greater than equals just a greater than, or if I missed an age in the ranges? To properly test, you should consider calling the function on both edges of every range to make sure you didn’t mess anything up.

02:25 That can mean a lot of testing code. In fact, it’s going to take 40-odd lines of test code to test this 11-line function. My tests are a little overly verbose because I’m teaching them to you, but still, I go back to that rule of thumb I mentioned before.

02:39 It often takes about the same number of lines to test as it does to write the code itself. Let’s look at the test.

02:48 This is test_age.py. As before, I’m importing TestCase. And similar to the last time, I’m importing the thing that I’m going to test.

02:57 My test class has several methods. The first one tests values that return child. You can organize your tests however you like. Here, I’ve grouped them by output.

03:06 I’ll babble a little more about that in a minute. But for now, let’s look at this test. Your test method can optionally take a docstring. If you do, that information will be printed to the screen when the framework runs.

03:19 I don’t normally bother myself, as the name of the test method is often good enough, but it’s there if you want it. When you test something, you need three things.

03:29 First, you need to know what you’re testing.

03:31 Second, you need to know what you’re changing. And third, you need to verify the result. If you’ve ever heard of given-when-then testing, which is popular in the behavioral-driven development world, that’s what it’s describing.

03:43 Given I’m testing the categorize() function, when I call it with zero, then I get back the word child.

03:51 In the test code here, I’m explicitly calling out those three things, but in a different order. I start by defining a variable that holds my expected value.

04:00 That’s the thing I hope my code outputs. This is the then part of given-when-then.

04:06 Next, I call the function with zero as an argument and store the result. This is the given and when parts. Then finally, I finish the then part by calling an assert method to validate that my expected value and returned results match.

04:21 The .assertEqual() method does what its name implies. It asserts that the two values passed in are equal. So if expected and result match, the assert passes and no exception gets raised.

04:33 The .assertEqual() method doesn’t care about the order of the arguments, but I highly recommend that you be consistent. If the assertion fails, the framework will print out a comparison for you.

04:44 Consistently putting your arguments in the same order keeps the error output in that order, making it easier to remember and understand. I’m not sure where I picked this convention up, so I don’t know if everyone else does it this way, but I always put the expected value first.

04:58 For other kinds of assertions, the order is important, but I’ll cover those in the next lesson. So far, I’ve only tested a single value. These two lines call our function again, this time with an argument of 9.

05:12 The expected value is the same, so it can be reused. And like before, you compare the expected and resulting values. I’ve already commented that I’m being verbose here.

05:22 There’s nothing wrong with doing all of this on a single line, hardcoding child as the first argument to .assertEqual(), and then calling the function as the second argument.

05:31 Personally, I tend to separate it out because it’s easier to scan the code and see what is being tested, but I could see how other programmers might think that this was way too many lines of code and do something a little simpler.

05:44 I have a different test for each of our four possible results. This one is for the teen case. It does the same things the child one does, but with a different range of values.

05:55 Then another for adult. And another for error handling.

06:01 Error handling can seem a little strange at first, as you’re trying to make the code behave in a non-typical way. But if your code is supposed to gracefully handle a problem, then you should test that it’s doing that.

06:12 Our categorize() function returns a string to indicate that the age passed in wasn’t in the valid range, so this function makes sure that it does what it’s supposed to.

06:22 The expected value is a little different this time, because the string is supposed to include the age inside of it. Here, I’m starting with the first part of the string. Then, in the .assertEqual(), I tack on the minus one that’s supposed to be included in the result.

06:36 Same goes here for a large value, tacking on 151 for this case. Let me open a terminal, and I’ll run these tests. I’ll start by invoking unittest with -v like I did before.

06:52 Let me scroll back up here. Each of the test methods in the class gets called. You can see the name of the method, the docstring from the method, then the colorized ok to indicate that the test passed.

07:03 And down here at the bottom, you can see the summary. Four tests got run, and they all passed. So far, I’ve been using the verbose flag. Let’s try it again without it.

07:18 When you’re not in verbose mode, each test prints a dot to the screen. Failed tests show an E or F. Here, the four dots mean four passed tests. Most of the time when I run tests, I don’t use verbose. I typically only go into that mode when I’m trying to debug a test or something weird is going on.

07:37 For example, if you misspell test in your test method, it won’t get run. So if you were expecting five tests and only got four, turning -v on will show you exactly what happened.

07:47 If you have a lot of tests and don’t want to run them all, you can pass a string to unittest in order to be specific about a single test or class.

08:04 The string is in the same format

08:09 as the fully qualified name shown in verbose mode,

08:11 starting with the directory if there is one. Then the file name, then the class name, and if you want to run a specific method, then that as well. Without the method, you’d get all of the tests in the class.

08:21 Here, I’ve invoked the test_adult() method only, and not surprisingly, it passes. Note, the summary shows that only a single test got run this time.

08:33 Although .assertEqual() is probably the most common assertion method to use, it’s only one of very many. In the next lesson, you’ll see the very many.