Logging in Python

We know that Python is a user-friendly programming language. This property not only applies to the coding part but also the debugging part.

Do you know that Python provides a module for fixing the errors and other problems in the program? We are going to cover this module named logging in the article. So, let us begin with its introduction.

What is Logging in Python?

Logging is the way of tracking the events going on when we run our program. It is one of the important tools used by software developers for running and debugging purposes. There are many situations when we have to make some code changes either because we got an error or an unexpected result.

Logging a friend in such situations. Using this we can find the cause of the problem in less time and with less work. The word ‘log’ is used here because we get to know the information via log messages.

You would be getting a doubt why we cannot use the print statement instead, which would be simpler than importing a module and using its methods? The reason is that with printing, yes, we can find the problem. But this works well with simple problems. When we have to debug the complex problem, the printing approach fails.

In addition, logging supports both displaying the information on the console and storing the status in a file.

Logging in Python

Logging is a standard Python module used to track when the programming is running. It can be used in Python version 2.3 and above. To use it we can import the module using the below statement

import logging

As a developer, you can add logging calls to any part of the code to find the occurrence of certain events. Logging is mainly used for the following two purposes in Python-

1. Diagnostic Logging – To record the events going on while the application is operating.
2. Audit Logging – To record the events for analysis purposes in business

Example of logging in Python:

import logging
logging.warning ("This is a warning message")

Output:

Since we used the warning() function, we can see that we got a WARNING in the output.

Levels of Logging in Python

In the above example, we used the warning() method which gave a warning message. Similarly, there are other levels and these are listed below:

1. Debug: It is used when one needs detailed information, especially when one is dealing with diagnosing problems.

2. Info : It is used to confirm that the program is working as expected

3. Warning: It is used to indicate that something unexpected happened, or might happen in the future

4. Error: It is used to get to know about any serious problem when the software has not been able to execute some function

5. Critical: It indicates a very serious issue, when the program itself may be unable to continue to execute.

These levels are sufficient to handle any problem we might face. A developer is also allowed to create more levels. The above levels are given a numerical value. The below tables show them.

Basic Configuration

As said previously, we can store the information obtained by logging into a file rather than displaying it on the console. For this purpose, we use the basicconfig(**kwargs) function. The commonly given arguments to this function are:

1. level – This specifies the severity level is set by the root level.

2. filename – It specifies a file in which the information is to be stored

3. filemode – It specifies the mode in which the file should be opened. The default mode is ‘a’, which indicates we can append the content to the file.

4. format – This defines the format in which the log message is entered into the file.

When we want to display on the console, we don’t need to specify the file name and other information related to the file. For example,

Example of logging to the console:

import logging

logging.basicConfig(level=logging.ERROR)  # giving the level using the level keyword argument 
logging.error('This is an error message')  #displaying the error message

Output:

Similarly, to add it to a file named ‘msg_file.log’, we can write the below code.

Example of logging to the file:

import logging

"""
Storing the logging information into a file named msg.file
We are giving the mode as write(w)
And the format in which the message is stored in the name of the root, followed by the level of the message and the message obtained
"""
logging.basicConfig(filename='msg.log',filemode='w', format='%(levelname)s:%(name)s: %(message)s')  
                 
logging.error('This is an error message') 
logging.warning('This is warning!') 

Output:

Python Logging Methods

We saw some of the methods above like error, warning, etc. Let us see some more along with these.

1. Logger.info(msg) : This gives a log message of INFO level on this logger.

2. Logger.warning(msg) : This gives a log message of WARNING level on this logger.

3. Logger.error(msg) : This gives a log message of ERROR level on this logger.

4. Logger.critical(msg) : This gives a log message of CRITICAL level on this logger.

5. Logger.log(lvl,msg) : This gives a log message of integer level lvl on this logger.

6. Logger.exception(msg) : This gives a log message of ERROR level on this logger.

7. Logger.setLevel(lvl) : This is used to set the beginning of the logger to the level lvl and to ignore all the messages written below.

8. Logger.addFilter(filt) : This is used to add the specified filter ‘filt’ to the logger.

9. Logger.removeFilter(filt) : It is used to remove the specified filter ‘filt’ to the logger.

10. Logger.filter(record) : This function applied the filter on the specified record. And then it returns True if the record could be accessed and  processed. Or else, it returns False.

11. Logger.addHandler(hdlr) : This adds the specified handler ‘hdlr’ to the logger.

12. Logger.removeHandler(hdlr) : This removes the specified handler ‘hdlr’ to the logger.

13. Logger.hasHandlers() : This is used to check if the logger has any configured handler or not.

Classes and Functions in Python Logging

This module also has classes and related functions for handling different situations. Here are the common ones:

1. Formatters – These define the structure of the output. We can use the concept of string formatting to set the format of the log messages.

2. Logger – This is an object of the Logger class that can be used to call the logging functions directly.

3. LogRecord – This function creates an automatic log record file consisting of the information about all events being logged like the logger’s name, the function, the line number, the message, etc.

4. Handler – The handlers are used to manage the log messages and they are responsible for dispatching the correct message to the required destination. The FileHandler, StreamHandler, HTTPHandler, SMTTPHandler are the subclasses of a Handler.

Formatting the Message

In the above example, we gave the format as a name followed by level and message. Like this, we can give any format of our choice.

Formatting Date and Time

For example, let us see how to get time and date along with the message.
Example of formatting the output:

import logging

logging.basicConfig(format='%(asctime)s %(message)s', level=logging.DEBUG)  
                 
logging.error('At this time we got an error') 

Output:

In addition, we can customize more by adding the format in which date and time should be displayed by using the datefmt attribute.

Example of formatting the output:

import logging

logging.basicConfig(format='%(asctime)s %(message)s', level=logging.DEBUG,datefmt='%d-%b-%Y %H:%M:%S %p')  
                 
logging.error('At this time we got an error')  

Output:

Example of formatting the output:

import logging

logging.basicConfig(format='%(asctime)s %(message)s', level=logging.DEBUG,datefmt='%d-%m-%Y %H:%M:%S %p')  
                 
logging.error('At this time we got an error') 

Output:

Logging the Variables in Python

Besides just giving a constant string as the message, we can also give a variable by formatting the string.

Example of formatting the output:

import logging

var = 'Python'  
  
logging.warning('%s raised a warning', var)  

Output:

Stack Traces

We can also capture the stack traces using the logging method. And to get the error information or execution details, we can use the exc_info attribute and set it to boolean True.

For example, let us raise a log message when we try to add an integer and a string.

Example of getting stack information:

import logging

var1=3
var2='6'
  
try:  
    res=var1+var2
except Exception as e:  
    logging.error("Exception occurred", exc_info=True) 

Output:

Logging also provides an exception() function. Let see the output obtained when this function is used instead of the error().

Example of getting stack information:

import logging

var1=3
var2='6'
  
try:  
    res=var1+var2
except Exception as e:  
    logging.exception("Exception occurred", exc_info=True)   

Output:

Logger Object in Python

Till now we used the default logger named root for the logging purpose. We can create our own logger object that belongs to the Logger() class. We can use the getlogger() method to create the instance and use this to call the other logging functions.

Example of Python logger object:

import logging  
  
lgr = logging.getLogger()  #creating a logger object
lgr.error('This is an error message') #using the object to call error function   

Output:

We can also specify the name of our logger as arguments to the getLogger() function as shown below.

Example of logger object:

import logging  
  
logging.basicConfig(format='%(name)s:%(message)s')
                    
lgr = logging.getLogger('my_logger')  
lgr.error('This is an error message')

Output:

Handlers in Logging

Handlers in logging are used to configure the logger and send the log messages to multiple destinations like output stream, a file, or HTTP simultaneously. For example,

Example of handlers:

import logging  
  
# Create an logger object lgr 
lgr = logging.getLogger('my_logger')  
  
# Create the handlers, one to console and other to file
handler1 = logging.StreamHandler()  
handler2 = logging.FileHandler('myfile.log')  
handler1.setLevel(logging.WARNING)  
handler2.setLevel(logging.ERROR)  
  
# Create formatters and add it to the handlers  
frmt1 = logging.Formatter('%(name)s - %(levelname)s - %(message)s')  
frmt2 = logging.Formatter('%(asctime)s- %(message)s')  
handler1.setFormatter(frmt1)  
handler2.setFormatter(frmt2)  
  
# Add handlers to the lgr 
lgr.addHandler(handler1)  
lgr.addHandler(handler2)  
  
lgr.warning('Warning message!')  
lgr.error('Error message!') 

Output:

Interview Questions on Logging in Python

1. Write a program to create a file with mode ‘w’ to write an error message.
Ans1. Below is the example of saving message to file:

import logging  
  
logging.basicConfig(filename='msg.log', filemode='w')  
logging.error('This is an error')  

Output:

2. Write a program to create a logger and format the message with the date and the main message content.

Ans2. Below is the example of formatting output:

import logging  
  
logging.basicConfig( format='%(name)s:%(message)s')  
lgr = logging.getLogger('logger') 
lgr.warning('Warning!')   

Output:

3. Write a program to format the message with the logger name and the level.

Ans3. Below is the example of formatting output in Python:

import logging  
  
logging.basicConfig( format='%(name)s:%(levelname)s')  

logging.error('Error!')  

Output:

Q4. Write a program to print the value of a variable in the log message.
Ans4. Below is the example of formatting output with the variable name:

import logging  
  
var=10
logging.basicConfig(format='%(message)s')

logging.warning(f'var={var}')  

Output:

Q5. Write a program to raise an exception message if the denominator of the division operation is zero.
Ans5. Below is the example of stacking traces in Python:

import logging  
  
n=8
m=0
try:
    n/m
except Exception as e:
    logging.exception('',exc_info=True)

Output:

Quiz on Logging in Python

Conclusion

We are at the end of the article! We covered different topics on logging. Starting with the introduction, then we discussed the module, logging its functions, methods, and classes.

In the end, we saw some interview questions. Hope this article could give some new information about logging. Happy learning!