Django is a high-level web framework based on Python, and it encourages the rapid development of secure, scalable, and maintainable websites. It simplifies the creation of database-driven websites by emphasizing the reusability of components, low coupling, rapid development, and less code. Furthermore, it’s free and open-source.
The Python logging module is responsible for performing system logging in Django. A Python logging configuration components are loggers, handlers, filters, and formatters. These provide a richly formatted log that can redirect logs to other sources, such as writing to files or email, and a flexible filter.
This logger handles the insertion of variables into the log messages when passed as arguments in the logging functions. The implementation of best logging practices reduces the risk of project failure.
Additionally, early detection of problems improves efficiency. Moreover, issues developers would have taken a lot of time to solve are prevented. Best logging practices establish baseline standards for competency and efficiency that take nothing for granted.
Poor coding practices result in adverse performance issues, such as when the user interacts with the system code flow. Implementing coding standards overcomes or minimizes performance problems.
In this blog, I discuss the core logging concept in Python, logging in a Django project, the best practices for logging, and finally, a tool called SolarWinds® Papertrail™ designed to make logging in Django easy.
Core Logging Concepts in Python
Python uses a built-in logging module with four major components: loggers, handlers, filters, and formatters.
This module allows for writing status messages to a file or any other output stream using one of the debug, info, warning, critical, or error functions. Python’s logging package provides a flexible yet robust way of setting application logging. Log messages contain a lot of useful information.
Loggers
Loggers provide initial entry points to group log messages. They can generate multiple levels of responses. These levels indicate log severity and priority.
CRITICAL=50, ERROR=40, WARN=30, INFO=20, DEBUG=10 and NOTSET=0
Developers have the option to create more levels.
A Python logger’s main fields are:
- Propagate. Decides whether to propagate to the logger’s parent.
- A level. Filters logs based on their log levels to remove the “less important” logs.
- Handlers. Lists where a log will be sent once it arrives at a logger, such as a file log handler that writes to a file all DEBUG logs and an email log handler that will send all CRITICAL logs via email.
Log Handlers
This component is responsible for writing logs to flat files with the FileHandler and displaying them in the server’s console via StreamHandler by sending an email via SMTPHandler or SMS messages. They simply redirect log messages to destinations. It’s possible to use a single handler in multiple loggers.
Filters
As the name implies, these filter log messages based on various variables, such as log levels, as not all messages need to be stored or transmitted. Filters can be applied to both loggers and handlers.
Formatters
By default, logs are in a log record format (predefined class by a logging framework) and need to be converted before being sent to anyone. There are different formatters for varying handles. Log formatters convert log records into a string by default.
Logging in a Django Project
Configurations for logging in Django projects are defined in the logging variable within the settings.py file. All Django logging uses Python logging modules. Loggings are propagated to parents whereby logs in a subtree of a hierarchy are also captured by all its parent loggers.
The basic Django logging is based on the standard Python logging dictConfig() method that configures logging and creates loggers, handlers, filters, and formatters. This method also disables all existing loggers. The blocks for this method are formatters, filters, and handlers.
Formatters
Set to key-value pairs where the key is a formatter ID and the value is a dict. These describe how the corresponding formatter instance is configured. Key formatters hold all possible formats of a log message, and any of them can be assigned to the logger being defined.
Filters
Each key is a filter. Additionally, the corresponding value is a dictionary that describes the configuration of the corresponding filter instance.
Handler
Each key is a handler, and the corresponding value is a dict. Describing the configuration of the corresponding handler instance. Each handler contains a class level of the handler, the ID of its formatter, and a list of IDs of its filters.
Best Practices for Logging
During log creation, ensure logs are written asynchronously. Buffer or queue to ensure the logs don’t block the application. Organize logging in such a way that, should the need arise, it will be easy to make a change.
Achieve this by protecting the application from third-party tools by using a wrapper. Make logs human readable by using standard date and time formats, adding timestamps, employing log levels correctly, and including stack trace when logging the exception. In a multi-threaded application, include the thread’s name.
Format logs in the correct way, so they can be parsed by the machine. This is very useful in an event where reading the logs manually isn’t enough, such as auditing or alerting. Don’t log sensitive information such as passwords, authorization tokens, personally identifiable information, credit card numbers, and session identifiers information opted by the user. Don’t log too much, as it becomes hard to get value from it.
Some logs will increase the risk of being unable to troubleshoot problems. Achieve a balance during production by analyzing produced logs and increasing or reducing logging statements according to the problems found.
While transmitting logs, fault-tolerant protocols should be used to ensure packets aren’t lost. Secure log data through encryption and scrubbing any sensitive data before transmitting it. Write meaningful log messages, so you can easily understand what happened from the log file.
Clarify log messages by adding remediation information to the log. Don’t make log messages dependent on a previous message’s content because the previous messages might not appear if they’re logged on different levels.
SolarWinds Papertrail
SolarWinds Papertrail makes logging in Django easy. The event viewer/log viewer is a core part of Papertrail. You can see events or logs in real-time and View older logs from the log viewer.
Integration occurs with searching modules for a specific log. These searches are saved to be accessible, receivable in an email, or retainable. It has a velocity viewer to help you identify anomalies, reducing the time needed to troubleshoot errors or spot trends.
Time seek in the log viewer is used for seeking directly to any date or time in the searchable history. Papertrail can optionally filter logs. When a new log matches given searches or important services, Papertrail can notify external services.
These alerts are sent to various alert services, such as chat (Papertrail sends a chat message with logs and a link), monitoring and notifications (send emails to the specified addresses), Pushover (sends IOS or android push notifications), and VictorOps (notifies staff when critical events occur).
Graphing and metrics graph the number of occurrences of a log event, e.g., CloudWatch and Datadog. Papertrail webhooks notify the chosen HTTP URL. Papertrail uses Zapier to send feed logs to services like Google Sheets and Twitter.
With all these options, give Papertrail a try to see how it can make logging in Django easy.
This post was written by Daniel de Oliveira. Daniel is a full stack developer and analyst, experienced with Java, Angular, and Flutter.