Asserting Yourself

00:00 In the previous lesson, I showed you how to validate the results of the function you were testing through the use of .assertEqual(). In this lesson, I’ll cover all the other things you can assert.

00:10 So far, you’ve only seen one assertion method, but TestCase has a bunch. In theory, you could get away with a single assertion method, .assertTrue(), then pass in a comparison as an argument. But that would tend to mean more typing.

00:24 And if something fails, the corresponding error message would just say the comparison didn’t evaluate to True. Whereas when you use the specialty methods, they give you more information.

00:34 For example, when comparing two strings, you’ll see a diff on the screen to show where they don’t match. Let’s go look at some of the more common assert methods.

00:44 This time around, I don’t actually have any code to test. I’ve just built a test case with a bunch of assert methods in it so you can see the variety. You saw this one in the previous lesson. .assertEqual() compares two values and passes if they’re the same.

00:58 If you’ve been kicking around Python for a long time, you might remember assertEquals with an S on the end. That got deprecated a while back, so no S for you.

01:09 Almost every assert method has a companion that is the negative.

01:13 .assertNotEqual() asserts that two values are different.

01:17 .assertTrue() asserts the value passed in is True, while .assertFalse() does the opposite.

01:23 .assertEqual() asserts that two values compare as equal. Python differentiates between two references being the same object versus them containing the same content.

01:33 Equal only says their content has to be the same. The .assertIs() test checks whether they’re the same object. To demonstrate this, I’ve created a list and then a new variable that is a reference to that list.

01:47 .assertIs() will pass because ref and nums are the same object. Whereas if I create a new list, you can see that nums and more_nums are considered equal, but they are not the same object, so the negative of .assertIs(), .assertIsNot(), will also pass.

02:04 You can also assert that something is None or not None, and assert that a value is in a sequence or not, as the case may be.

02:16 If you need to check class types, .assertIsInstance() asserts that the first thing is a type of the second. This is one of the few cases where the order of the arguments is important.

02:25 This assertion uses the same argument order as the corresponding isinstance() call. And what’s more, you can do the negative as well. You can also assert class inheritance.

02:37 To test that, I’ve created a subclass of string, and as you might expect, our new class is a subclass of string, but not a subclass of list.

02:48 Up until now, I’ve only shown you passing tests, so while I’m blasting away at assertion methods, I thought I’d create one that doesn’t pass so you can see what happens.

02:57 Here, I’m asserting that 42 and 13 are equal. They’re not, in case you needed that. Most of the assertion methods also support a message argument. For .assertEqual(), it’s the third argument.

03:11 If you pass in a message and the assertion fails, your message will get included in the output. This can be handy if you want to give yourself extra info or context about the validation.

03:23 When testing your error handling, you may want to check that your code raises a specific exception. As tests that raise exceptions are failed tests, you have to do a little extra work for that.

03:35 You could use a try-except block mixed with another assertion, but instead, you can use .assertRaises() as a context manager. How this works is you create a context block with .assertRaises(), passing in the class of the exception that you’re expecting.

03:50 Then when the code inside the block raises an exception, the framework catches it and squashes it. It also makes sure that the exception got raised. If the context block exits without raising the expected exception, then the .assertRaises() would fail and raise its own AssertionError.

04:08 To show how that works, I’ve done it twice. This second time, I’m still checking for a ValueError, but in the block I raise an AttributeError instead, which will result in a failed test.

04:19 Let’s run this puppy. Remember, there are a couple of methods that are going to fail here, so the output’s going to be noisy.

04:26 Here I go.

04:30 I didn’t use verbose mode, so I get a dot or letter for each test run. Note that it’s for each test, not for each assertion. Most of my assertions were in a single method, and that method is one of the ones that passed, so most of the work in our suite will be represented by a single dot.

04:47 The F and E indicate two different kinds of problems. F means fail, as in an assertion didn’t pass. E means error, as in something threw an exception that wasn’t an AssertionError.

05:01 That typically means your code did something wrong, but it could mean you forgot to handle a valid error condition. Our first problem is an error. This is the test method that used .assertRaises() expecting a ValueError.

05:14 But I raised an AttributeError instead. Since .assertRaises() wasn’t expecting this kind of exception, it’s treated as an error.

05:22 Note that you get a full traceback showing where the problem occurred. The traceback is shallow here because I caused the problem in the test code itself, but in real life you’d see the point of test at the bottom, then stack corresponding to the code you were testing, giving you a way to figure out where the problem occurred.

05:39 The next problem is an F. This says that the test method named test_fail() failed an assertion. An AssertionError got raised, and since it was an .assertEqual(), it gives you information about the arguments, telling you that in this case 42 and 13 are not equal.

05:56 That might seem really obvious, but if these were variables instead, this would be showing you the actual contents. You’ll recall I suggested you be consistent about the order of your arguments.

06:06 This is why. If you always put expected first, then you’ll know that the left side of this error message was your expected value and the right side your result.

06:17 The next failure is similar to the previous one, but this time I included the message argument to .assertEqual(), which the framework helpfully tacks to the end of the output.

06:27 At the bottom here you get a summary telling you how many tests got run. I finally had enough that it no longer rounded to zero. Waiting 0.001 seconds was hard, but I managed to be patient.

06:39 Since the tests had problems, you get more information. It tells you some tests had problems and that two of them were failures and the other an error. The assertion methods I’ve shown you so far are the most common.

06:52 There are others though. All the ones on this slide are context manager based, like the .assertRaises() you just saw. .assertRaisesRegex() is like .assertRaises(), but you can also provide a regular expression that validates the error message in the exception that got raised.

07:08 If the expression doesn’t match, then the test fails.

07:12 .assertWarns() is for checking that Python has issued a warning. These don’t get used a lot, but one place you may have seen them is a deprecation warning.

07:20 On screen it outputs like an exception does, but it doesn’t stop the code from running. There’s also an .assertWarnsRegex() to go with it. .assertLogs() ensures that something got logged within the context block. It has a corresponding negative method as well.

07:36 Everything on this screen is for value comparisons.

07:40 .assertAlmostEqual() is useful when comparing floats. It takes an extra argument that defines the precision. If the numbers are within the precision difference, they’re considered equal.

07:50 .assertGreater(), .assertGreaterEqual(), .assertLess(), and .assertLessEqual() are for numeric comparisons.

07:56 .assertRegex() is to check a string matches a regular expression. .assertCountEqual() checks the length of a sequence. And .assertStartsWith() and .assertEndsWith() are similar to the string methods of the same names.

08:09 Lastly, .assertHasAttr() checks if an object has the specified attribute. This last group of assertion methods are a little weird. I didn’t even know they existed until I wrote this slide.

08:21 These are specific types of equals comparisons. The reason you don’t see them often is that the regular old .assertEqual() automatically detects what is being compared and fires these for you.

08:32 So if you compare two lists, it actually triggers .assertListEqual(). I’m not entirely sure why you’d want to call them directly, but they’re there, so you can.

08:43 In the next lesson, I’ll show you how to make a test conditional or disable it with a decorator.