Python JSON Logging

Logging allows us to keep track of events that happen while certain software is functioning. When the software fails and we don’t have any logging records, then there are very few possibilities that we would figure out the root of the issue. But with the logging records, we can maintain all the failures. Although, when a system has to parse the logs that don’t follow a predefined pattern or module, things get complicated. Python, therefore, contains JSON which organizes the data in a formatted manner using objects that adhere to a specified pattern. The Python-json-logger is the standard Python logging module that displays the log events as a JSON object. The JSON logger module improves the machine readability of our logs and prevents creating the unique parsers for entries of the syslog type.

How to Use the Python JSON Logging?

The “python_json_logger” library is required for the logs in the JSON format. We can install this Python library with the pip command. The following is the command to install it. When it is installed, the prompt also displays the confirmation message with the library’s recent version.

Example 1:

The logging module’s basic usage is relatively straightforward and it can be used to log the events into files. Here, we first provide the simple implementation of Python logging. We create the log and configure it using the basicConfig() method. This method takes the name of the file where the logs are created because the given file is open in the write “w” mode. The format is also set for the logs. Then, we declare the “logger” object where with the getLogger() method. We return the logger with the assigned name.

Next, the logger’s level is adjusted through the setLevel() method to serve as the tracking threshold for debugging which is based on the numerical values that are allocated to each level. After that, we deploy some logger class methods including the particular message as a parameter. The debug() method prints the debugging message. The info() method generates a message that is logged on to this logger with the INFO level. Then, the critical() method shows the CRITICAL level of the given logger. The warning() and the error() methods show the warning for the logger and any exception that is raised by the logger.

The program creates a file with the given name. When we open the file, it contains the information against each specified logger method which is listed in the following:

Example 2:

In the aforementioned program, we have the Python logging functionality. Now, we have a program where the Python JSON logger library is utilized to store the logs in the JSON format using the built-in logging methods. Here, we begin the Python JSON logging code by inserting the logging module and the jsonlogger module for the pythonjsonlogger class. After this, we declare the “formatter” object which is set with the JsonFormatter() method to get the results in JSON format. Then, we define the “json_handler” object to invoke the filehandler() method which gives the provided JSON file.

Next, we set the format for the JSON file using the setFormatter() method where the formatter object is passed as input. Then, we get the object of the logger from the getLogger() method. The “json_handler” is added through the addHandler() method. In the end, the logger.error() method which encounters any error from the logging in JSON format is raised here.

The following JSON log is created in the “NewLog.json” file when we open it from the jsonlogger module of Python:

Example 3:

The basic JSON log format is given in the previous example. Now, we demonstrate the exception and traceback of formatting of the log into JSON using the jsonlogger module. Tracebacks give specific information about the failure of the program, therefore logging them is frequently valuable. We can use the logging.exception method which is a built-in Python logging module to log the errors and add our custom error messages.

Furthermore, Logging.exception() method has an ERROR default severity. First, we use the import statement of loading the logging module and the jsonlogger module for JSON logs. After this, we create the instance of the log using the getlogger() method to access the logger. The log exception and traceback are handled and generated in the “BadCalculationDemo.json” file.

Next, we provide the jsonFormatter() method in the “formatter” object to format the logs into JSON structure and set the provided formatter with the setFormatter. Then, we add the created “logHandlerJson” handler to the addHandler() method. The try-except block is employed to catch the logs from the following traceback. Here, we have a try statement where the integer is divided by the string value which is a bad calculation practice. The except statement catches this log exception in the logging.exception() which prints the provided message along with a traceback report in the JSON format to the BadCalculationDemo.json file.

When the script is executed, a stringified JSON object appears in the BadCalculationDemo.json file’s contents. The following is the image of the BadMathDemo.json file which displays the error message with the traceback information:

Example 4:

The attributes with JSON-specific characters could cause the JSON object to become corrupt. We can customize the unique logging system using the formatter class and allow the JSON module to appropriately escape all attributes. Here, we provide the custom JSON logging which provides the additional attributes for the events of the logs. We insert the JSON, logging, and sys modules to create the log events of our system. Then, we establish the “JsonFormatter” class which takes the input logging and the formatter as parameters. After this, we create the “self” and “records” attributes for this class.

Then, we generate the “extra” object where the getattr() method is called to access the logging.LogRecord attributes in the JSON format. The JSON record is used to get the time, level name, file name, line number, and the provided message for the log of the system.

Moreover, we also extract additional details regarding the system logs. Then, we use the json.dump() method where the Python “json_record” object is passed for conversion into the JSON object. The JSON attributes from Python are configured from the jsonConfig() method definition. The if-else technique is used to create the custom handler which gives the logs in the specified file or the form of the output stream to the console. The root logger is configured with the customized handler. Then, we invoke the jsonConfig() method and the method of logging which give the level of the log messages in the form of JSON.

The Python JSON logging script generates the output of the following JSON dictionaries:

Conclusion

The best way to organize the logs and centralize their visualization is with Python JSON logging and a log management program. To format and get the most relevant details, we should also look at the Python logging procedures. It is important to get informed of our Python JSON log restrictions because they could prevent us from making a significant contribution. The logging event in the form of JSON objects is far better than the simple text. Using a JSON-based query language, it is simple to add the records to databases using the JSON format.