This is a simple guide to Python core
logging package basics.
logging package is very powerful and widely used.
For example, Django uses Python's built-in logging package.
For the most in-depth and up-to-date information, always refer to the official Python logging documentation. This guide will walk through a summary of the things I think are most important and useful in my regular work.
Here is a simple example of using the Python
# hello_logging.py import logging print('Printing to STDOUT') logging.error('Logging goes to STDERR')
By default, the logs to go STDERR not STDOUT. This means text will typically go to your terminal output and look like normal STDOUT print statements, unless you specifically pipe the STDERR stream somewher else. For example, to redirect the STDERR output to a file, you can run the application like this:
# Both outputs will go to terminal by default python hello_logging.py # Or you can redirect STDERR (file id 2 to the OS) # to a file like `debug.log` # This will work in Windows, Mac, and Linux python hello_logging.py 2>debug.log
You could also redirect one stream to another, for example, redirecting STDERR to STDOUT before redirecting STDOUT to a file. This way both outputs will go to a single source. Learn more in my STDOUT, STDERR, Piping, and Redirecting Tutorial.
Different log levels
Let's say you want to log some information, but it's not an error.
Then you wouldn't want to call
logger.error() because it sets
the log level too high.
To demonstrate the concept of log levels,
try a similar example to the first one, but this time, instead of calling
logger.error() also call
They are called in order of their log level, with debug being the lowest
and critical being the highest.
import logging # Default log level is WARNING logging.debug('This is a debug log') # Won't show logging.info('This is an informational log') # Won't show logging.warning('This is a warn log') logging.error('This is a error log') logging.critical('This is a critical log')
In order to get the debug level messages to print, you need to configure
the logger to have a different log level.
To configure the global logger, use
basicConfig() like this:
import logging # Configure global logger to output DEBUG and higher level logging.basicConfig(level=logging.DEBUG) logging.debug('This is a debug log') logging.info('This is an informational log') logging.warning('This is a warn log') logging.error('This is a error log') logging.critical('This is a critical log')
In the previous example we looked at how to set the log level using
basicConfig() method takes many other arguments
to configure the logger.
For example, you can configure:
- Log level
- Text and date formatting
- File name (optional) and mode (append vs create)
- Output stream
Note that you can only choose one of the output methods: file, output stream, or handlers. You can't have file and stream output at the same time, and you can't use handlers with any others. So the real important settings are:
- Log level
- Where you output the message (file, stream, or handler)
As demonstrated in the previous example, log level can be set using the
basicConfig() method on the logger. Specify one of the levels:
import logging logging.basicConfig(level=logging.DEBUG)
Customize log formatting
There are two strings you can format with the logger, the output message and the date. The default format does not include the time, but outputs the log level, the logger name, and the log message itself. You might want to exclude the logger name if you only have one global logger, and you may want to include the time or other attributes.
basicConfig() you can specify a format string
for both the message format and the date format.
You can review the fill list of log formatting variables, but there are some important ones that are worth noting:
asctime- a string with the log message time
levelname- a string with log level name
message- the actual log message string
module- the string module name where the log message originated
filename- the string file name where the log message originated
lineno- the integer line number where log message originated
A format string would look something like this:
%(asctime)s %(lineno)d %(levelname)s %(message)s
You can use it in the configuration like this:
import logging logging.basicConfig(format='%(asctime)s %(levelname)s %(message)s') logging.error('Test error message')
To alter the date formatting, also pass in a
datefmt option like this:
import logging logging.basicConfig(format='%(asctime)s %(message)s', datefmt='%Y-%m-%d %H:%M:%S') logging.error('Test error message')
Refer to the strftime() documentation for a list of the available variables for date formatting.
Output to file
To output to a file, you can use
basicConfig() to specify a file like this:
import logging logging.basicConfig(filename='debug.log')
By default, it will open the file in append mode (
You can specify the open mode, for example to create
a new file each time:
import logging logging.basicConfig(filename='debug.log', filemode='w')
Note that you cannot use file and stream or handler output at the same time. You must pick one or the other.
Output to stream
You can output all log messages to an in-memory file stream if desired. For more information on using the streams, check out my tutorial Python Use StringIO to Capture STDOUT and STDERR.
This example will output the log information to a text stream that you can read from later.
import logging from io import StringIO text_stream = StringIO() logging.basicConfig(stream=text_stream, level=logging.DEBUG) logging.debug('Testing output') logging.debug('to stream') print("Log contents:") print(text_stream.getvalue())
Note that you cannot use file and stream or handler output at the same time. You must pick one.
The previous examples used either a stream or a file output, and you had to pick only one. Starting in Python 3.3 they introduces a new option to use handlers. You can't use handlers with file or stream output so when you want to output to multiple things, handlers are the only way to go.
There are over 15 built-in handlers that you can use. Check out the full list of useful handlers. Some noteworthy handlers are:
As you can see, you have the basic stream and file handlers like were used in the previous examples, but you also have more advanced handlers like a file handler that includes log rotation, a syslog handler to send syslog events, and SMTP and HTTP handlers to send log events over email or HTTP. You can also create custom handlers.
You pass an iteratable of handler objects to the configuration.
import logging from logging import FileHandler, StreamHandler from logging.handlers import RotatingFileHandler handlers = [ FileHandler('log.txt'), # Default mode='a', encoding=None RotatingFileHandler('log-rotating.txt', maxBytes=10000, backupCount=3), StreamHandler(), # Default stream=sys.stderr ] logging.basicConfig(handlers=handlers) logging.error('This goes to stderr, log.txt, and the rotating log')
You can also add handlers by calling
.addHandler() on the logger object.
Create multiple loggers
In all of the previous examples we directly called and configured the global logger
You may want to configure separate or multiple loggers.
You can get the root (global) logger by calling
logging.getLogger() with no arguments.
You can create or fetch unique loggers by providing the name of the logger you want.
A common convention is to use
__name__, which is the name of the Python file, as the logger name.
You can also choose to use a string like
"global" or any other arbitrary name.
import logging logger = logging.getLogger(__name__) # Configure or alter the logger if neeeded # logger.basicConfig() # logger.addHandler() logger.error('This is a simple error log')
After reading this guide,
you should know how to use Python's default
to do simple logging tasks like outputting to STDERR,
controlling log levels, outputting to a file,
and customizing the log formatting.
You should also know how to use handlers to log via email, syslog, http, and rotate log files.