Coding the Missing Pieces

00:00 Coding the Missing Pieces. In the previous section, you saw the creation of the main glue code for the application. In this part of the course, you’ll see the missing functions created, completing the initial functionality of the checker application.

00:17 The site connectivity checker app will be able to check multiple URLs in every execution. Users will feed URLs into the app by listing them at the command line, providing them in a text file, or both.

00:30 To create the internal list of target URLs, the app will first process URLs provided at the command line. Then it will add additional URLs from a file, if any.

00:42 Here’s the code that accomplishes these tasks and returns a list of target URLs that combines both sources, the command line and the optional text file.

00:51 First pathlib is imported to manage the path to the optional URLs file. Next, a helper function is added with the following code. This defines urls, which initially stores the list of URLs provided at the command line.

01:08 Note that if the user doesn’t supply any URLs, then this will be an empty list. Here, you define a conditional that checks if the user has provided a URLs file. If so, then the if block augments the list of target URLs resulting from calling _read_urls_from_file() with the file provided in the user_args.input_file command-line argument. Finally, the list of URLs is returned.

01:38 _read_urls_from_file() runs the following actions. This turns a file argument into a pathlib.Path object to facilitate further processing.

01:51 This line defines a conditional statement that checks if the current file is an actual file in the local file system. To perform the check, the conditional calls the .is_file() method on the Path object.

02:05 Then the if block opens the file and reads its content using a list comprehension. The list comprehension strips any leading and trailing whitespace from every line to prevent processing errors later on. This conditional checks if any URL has been gathered.

02:23 If so, this line returns a resulting list of URLs. Otherwise, this line prints an error message to inform the user that the input file is empty.

02:39 The else clause prints an error message to point out that the input file doesn’t exist. If the function runs without returning a valid list of URLs, it returns an empty list.

02:49 That’s a reasonably complex program flow, but now you can continue with the final part of __main__.py. To run connectivity checks over multiple websites, you need to iterate through the list of target URLs, do the checks, and display the corresponding results.

03:09 That’s what the _synchronous_check() function seen next on-screen does.

03:19 Firstly, you update your imports by adding site_is_online() and display_check_result().

03:34 Next, the _synchronous_check() function is defined, which takes a list of URLs as arguments.

03:42 This line starts a for loop that iterates over the target URLs. Here, you define and initialize an error which will hold the message that will be displayed if the app doesn’t get a response from the target website.

03:57 Here, you define a try … except statement that catches any exception that may occur during the connectivity checks. These checks call site_is_online() with the target URL as an argument.

04:11 These lines update the result and error variables if a connection problem happens.

04:21 This line calls display_check_result() with appropriate arguments to display the connectivity check result to the screen. To wrap up this file, you add the typical if __name__ == "__main__": boilerplate code.

04:38 This calls main() when the module is run as a script or executable program and stops the code being run if it’s ever imported into another script.

04:50 You’ve written a lot of code without seeing it in action. You’ve coded the site connectivity checker CLI and the entry-point script. Now it’s time to give the application a try. Before doing that, make sure you’ve downloaded the included sample-urls.txt file from the course materials and placed it in the working directory.

05:12 Now go back to the command line and execute the command seen on-screen. Firstly, you run rpchecker with the -h flag, which gives a message explaining how to use the app.

05:24 Next, you can see the contents of the sample-urls.txt file. Finally, rpchecker is run with the f flag and the sample-urls file, which runs a check on several important websites.

05:39 If an error occurs during the check, then you get a message on-screen with the information about what’s causing the error. Go ahead and try some other URLs and features. For example, try to combine URLs at the command line with URLs from a text file using the -u and -f switches. Additionally, check what happens when you provide a URLs file that’s empty or nonexistent.

06:05 The connectivity checker works well, but it has a hidden issue. The execution time can be considerable if you run it with a long list of target URLs. This is because all of the checks run synchronously, waiting for site one to respond before starting the check on site two, and so on. With a long list of URLs and slow site responses, this can add up quickly.

06:28 The solution to this problem is to run checks asynchronously, running multiple checks at the same time. This is possible because the majority of the program’s execution time is waiting for responses from the sites being checked.

06:44 In the next section of the course, you’ll see how to implement asynchronous checks.