Creating a Custom Logger

Logging Inside Python Austin Cepalia 06:24

In this lesson, you’re going to get some hands-on practice. You’re going to create a custom logger as well as some handlers that will represent output destinations.

Then, you’ll attach formatters to those handlers so you can independently format your logging output based on destination. After all that, you’ll attach those handlers to the logger so that you can use it. Basically, you’re going to create a bunch of objects and then wire them together to form a logging system.

00:00 I figured I’d move away from the slide deck for this module, since these classes are better understood when you see them being used in action. In this video, I’m going to do a couple of things.

00:12 I’m going to create a custom logger; create some handlers, which will represent output destinations; attach formatters to those handlers so we can independently format our logging output based on destination; and then finally, I’ll attach those handlers to the logger so that we can use it.

00:31 Basically, I’m just creating a bunch of objects and wiring them all together to form a logging system. As always, I’m here in Visual Studio Code. I’m working in a directory called logging, which is important because this program will output a log file and I want to make sure it’s contained in this directory so I can easily find it. I’ll start by importing the logging module so that we can use it, and now I’ll create a custom logger.

00:59 I’ll call the object logger and I’ll get the new Logger with logging.getLogger(). And now we need to give it a name. You could name this whatever you want, so long as you pass in a string, but I’m going to pass in dunder __name__.

01:14 This special __name__ variable represents the name of the current module, which is helpful so that if later on I import this module into another one, we’ll know that the data this logger is outputting is coming from this module.

01:29 Now we need to create our handlers. First, I’ll make a console handler that will output to stdout. I’ll call it c_handler, and I’ll use the StreamHandler() method to get a new StreamHandler object capable of outputting to stdout.

01:47 Let’s do the same thing for the file handler, except I’ll call this one f_handler and I’ll use the FileHandler() method, passing in 'file.log'.

01:59 This will be the file that this handler sends its LogRecords to.

02:03 Now we need to set the level of each of these handlers. Remember, setting the level will tell the handler to redirect the LogRecords only if the level is equal or more severe than we set here.

02:17 So if I say c_handler.setLevel(logging.WARNING), just like this, then only events that are WARNING level or more severe will be logged to the console. I’ll do the same thing down here for the file handler, but I’ll use logging.ERROR as the severity level.

02:40 Now only events, a.k.a LogRecords, that are marked as ERROR or more severe will be logged to the external file. We’ve now got a system that lets us see warnings and greater logged to the console, but only errors and greater logged to the file.

02:57 But I don’t think we want the console-outputted logs to look exactly like the file ones. Let’s create a Formatter for each handler, specifying exactly what our logging output should look like for each one.

03:11 I’ll say c_format = logging.Formatter(), which will return a Formatter object. And I’ll pass in this familiar string right here.

03:23 This will display the logger name, the severity level, and the message separated by dashes.

03:30 Now I’ll do the exact same thing for the file formatter. I’ll say f_format = logging.Formatter(), passing in asctime first, which will log the exact time the event is created, and then everything else like the name, the levelname, and the message.

03:52 This means that the format of the console log and the file log will be the exact same, except the file log will also contain the time stored in the LogRecord for the recorded event.

04:05 Now, we have to actually attach the formatting objects to each of our handlers. So I’ll say c_handler.setFormatter() and I’ll pass in the c_format object we just created.

04:20 And as you might’ve guessed, I’ll do the same exact thing for the file handler, passing in the file formatter.

04:27 Now we’ve got two handlers, each with a formatter, but they aren’t yet linked to our custom logger object. In order to do that, we have to call the .addHandler() method on the logger object, passing in the handlers. That will look something like this.

04:46 Our logger is finally complete, but we need to actually log some events in order to see it in action. We can log an event by calling the severity level function right on that logger. So I’ll say logger.warning(), and I’ll pass in a message saying 'This is a warning', and now I’ll log an error saying 'This is an error'.

05:11 Once you finish typing, pause the video and see if you can guess what will be logged to the console and what will be logged to the file. Remember, our handlers for each destination define a severity level that must be met or exceeded in order for that event to be logged to that destination.

05:30 So, I will right-click here and select Run Code, and now we see that we get two lines of output in the console. Both the warning and the error logged because the handler representing the console was set to WARNING, and so both the WARNING and the more severe ERROR level events were logged.

05:49 We also notice a new file was created, as you see onscreen here. Only the error event logs into here. This is because the handler for the file had a severity level of ERROR, and because WARNING is less severe than ERROR, it wasn’t logged.

06:07 We also notice that the logs in our external file here include the date and the time from the LogRecord object associated with the event, which we obtained by including asctime in the file handler’s Formatter.

Najmeh on Aug. 15, 2019

Thanks for the great video and explanation.

Fahim on Nov. 7, 2019

Hi,

I am getting below error for mentioned code.

Traceback (most recent call last):
  File "/usr/lib/python2.7/logging/__init__.py", line 861, in emit
    msg = self.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 734, in format
    return fmt.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 469, in format
    s = self._fmt % record.__dict__
KeyError: 'level'
Logged from file python-logging.py, line 20
Traceback (most recent call last):
  File "python-logging.py", line 20, in <module>
    logger.warning("This is warning")
  File "/usr/lib/python2.7/logging/__init__.py", line 1179, in warning
    self._log(WARNING, msg, args, **kwargs)
  File "/usr/lib/python2.7/logging/__init__.py", line 1286, in _log
    self.handle(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 1296, in handle
    self.callHandlers(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 1335, in callHandlers
    if record.levelno >= hdlr.level:
AttributeError: 'Formatter' object has no attribute 'level'

Here is my complete code

import logging

logger = logging.getLogger(__name__)


c_handler = logging.StreamHandler()
f_handler = logging.FileHandler("File.log")
c_handler.setLevel(logging.WARNING)
f_handler.setLevel(logging.ERROR)

c_format = logging.Formatter('%(name)s - %(level)s - %(messeage)s')
f_format = logging.Formatter('%(asctime)s - %(name)s - %(level)s - %(messeage)s')

c_handler.setFormatter(c_format)
f_handler.setFormatter(f_format)

logger.addHandler(c_handler)
logger.addHandler(f_format)

logger.warning("This is warning")
logger.error("This is error")

Fahim on Nov. 7, 2019

Hi,

I am getting below error for mentioned code.

Traceback (most recent call last):
  File "/usr/lib/python2.7/logging/__init__.py", line 861, in emit
    msg = self.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 734, in format
    return fmt.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 469, in format
    s = self._fmt % record.__dict__
KeyError: 'level'
Logged from file python-logging.py, line 20
Traceback (most recent call last):
  File "/usr/lib/python2.7/logging/__init__.py", line 861, in emit
    msg = self.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 734, in format
    return fmt.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 469, in format
    s = self._fmt % record.__dict__
KeyError: 'level'
Logged from file python-logging.py, line 21
Traceback (most recent call last):
  File "/usr/lib/python2.7/logging/__init__.py", line 861, in emit
    msg = self.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 734, in format
    return fmt.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 469, in format
    s = self._fmt % record.__dict__
KeyError: 'level'
Logged from file python-logging.py, line 21```

Here is my complete code

import logging

logger = logging.getLogger(name)

c_handler = logging.StreamHandler() f_handler = logging.FileHandler(“File.log”) c_handler.setLevel(logging.WARNING) f_handler.setLevel(logging.ERROR)

c_format = logging.Formatter(‘%(name)s - %(level)s - %(messeage)s’) f_format = logging.Formatter(‘%(asctime)s - %(name)s - %(level)s - %(messeage)s’)

c_handler.setFormatter(c_format) f_handler.setFormatter(f_format)

logger.addHandler(c_handler) logger.addHandler(f_handler)

logger.warning(“This is warning”) logger.error(“This is error”)

```

Fahim on Nov. 7, 2019

I figure it out. There was some silly spelling mistakes.

amern2k16 on Dec. 6, 2019

c_handler = logging.StreamHandler() The video says StreamHandler() is a method. I believe it’s a class? Aren’t we instantiating object in above statement?

techdiverdown on April 16, 2020

The logger by default is at level INFO, so if a handler has a lower level like DEBUG, the message will not appear as it will be filtered. The solution is to set a low level in the logger and then the handler can set whatever other levels and it will work as intended.

logger = logging.getLogger(‘foo’)

Note set to debug on the logger then each handler can have another level

logger.setLevel(logging.DEBUG)

ch = logging.StreamHandler() ch.setLevel(logging.DEBUG) formatter = logging.Formatter(‘%(asctime)s - %(name)s - %(levelname)s - %(message)s’) ch.setFormatter(formatter) logger.addHandler(ch)

only write errors to file

fh = logging.FileHandler(‘log.txt’) fh.setLevel(logging.ERROR) fh.setFormatter(formatter) logger.addHandler(fh)

logger.debug(“debug”) logger.info(“info”) logger.warning(“warning”) logger.error(“error”) logger.critical(“critical”)

Patrick Prince on April 26, 2020

Hi,

I’m getting the following error -> Traceback (most recent call last): File “H:\python\samples\logging.py”, line 8, in <module> import logging File “H:\python\samples\logging.py”, line 11, in <module> logger = logging.getLogger(‘fdf2pdf_v6.py’) AttributeError: module ‘logging’ has no attribute ‘getLogger’

Source Code was copied from Course Materials ->

import logging

Create a custom logger

logger = logging.getLogger(‘fdf2pdf_v6.py’)

Patrick Prince on April 26, 2020

Regarding my comment above, oddly, the code runs without errors in ipython.

singh07 on June 29, 2020

I am getting below output:

__main__ - WARNING - This is a warning.
ERROR:root:This is a error.

and nothing in File.log. Here is my file. Could you please guide me what is wrong here.

import logging

logger = logging.getLogger(__name__)

c_handler = logging.StreamHandler()
f_handler = logging.FileHandler('File.log')
c_handler.setLevel(logging.WARNING)
f_handler.setLevel(logging.ERROR)

c_format = logging.Formatter('%(name)s - %(levelname)s - %(message)s')
f_format = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')

c_handler.setFormatter(c_format)
f_handler.setFormatter(f_format)

logger.addHandler(c_handler)
logger.addHandler(f_handler)

logger.warning('This is a warning.')
logging.error('This is a error.')

ed6 on Aug. 11, 2020

There is a an error in the code. This does NOT work if you set logger.info or logger.debug. Nothing will show even if you change the levels. You have to do as techdiverdown says and set the root logger to DEBUG and then the file handlers levels can then be set from there. Otherwise, you will only be able to see the WARNING level and above messages.

Become a Member to join the conversation.