Parse Lighttpd Access.log

System administrators require detailed information to diagnose and fix problems that may be encountered in a system. One source of such information is logs. Logs are a series of records that describe how a system or resources in the system are utilized.

A web server such as Lighttpd allows you to configure the level of details of the output information by the use of logs. It uses the Common Log Format by default. However, the logging method is highly configurable for various needs and scenarios.

In this tutorial, you will learn how to enable logging in Lighttpd server and understand the formatting of the logs from the access.log file.

Enable Access Logging

Before parsing the file and learning how the file is formatted, we need to enable access logging on the webserver.

Access logs contain log information about the request for files and resources from the web application hosted by the webserver. Typical information stored in access logs includes the IP address, user agent, the files access, and more.

To enable access logging on the Lighttpd server, allow the access log module using the command:

The above command will load the access log module allowing the server to record the access logs to the files and resources on the hosted web app.

If you want to modify the options of the accessLog modules, edit the file located in:

Modifying the access log filename

By default, the access logs are stored in the access.log file as defined in the accessLog.filename option.

To set a custom name, edit the value of the entry. For example, to set the access logs in a file server.log:

Access Log options.

Other options you can modify to configure access logs in the Lighttpd server include:

1. accesslog.use-syslog – This option will send the accesslogs to syslog. This option is disabled by default.
2. accesslog.syslog-level – defines the logging level (severity) for syslog. By default, the level is set to INFO. Other options include:
1. 0 – Emergency
2. 1 – Alert
3. 2 – Critical
4. 3 – Error
5. 4 – Warning
6. 5 – Notice
7. 6 – Info
8. 7 – Debug

You can learn more about syslog levels in the resource below:

https://en.wikipedia.org/wiki/Syslog#Severity_level

1. accesslog.format – this option describes the structure of the logs in the access.log file.

Understanding the Log format

Let us dissect the format of the access logs in the Lighttpd server. It is good to note that we will be using the default log format for simplicity.

To view the contents of the log file, you can use the cat command as:

An example output is as shown below:

Let’s take a single entry as shown in the example below:

• The first block of the log entry contains the IP address of the remote host requesting the specified resource. In this example, the source IP address is localhost.
• The second part contains the hostname of the remote host.
• Next is the timestamp of the end-time of the HTTP request.
• Following that is the HTTP request method such as GET, POST,…
• The next part contains the requested URL. In our example above, the requested URL is index hence /
• The next part is the HTTP request protocol, such as HTTP/1.1
• Next is the HTTP status code for the requested resource or URL such as 200 OK, 404 not found, etc.
• Next comes the number of bytes sent in the request body.
• The final block contains the User-Agent for the request.

The following format describes all the blocks stated above:

NOTE: This is the default log format for Lighttpd version 1.4.13 and above.

Custom log format.

You can create a customized log format using the options provided in the Lighttpd log format documentation.

https://redmine.lighttpd.net/projects/1/wiki/Docs_ModAccesslog

Let us use the provided options to create a minimalistic log format for our server. In our example, we will include the source IP address, the remote hostname, request method, status code, requested URL, and the number of bytes in the request body.

1. %h – represents the remote IP address.
2. %l – represents the remote hostname.
3. %m – for the request method.
4. %s – status code.
5. %U – requested URL.
6. %b – number of bytes in the request body.

Edit the access log file:

Set the log format as shown below:

Save and close the file. Restart the service as:

Once the service has restarted, navigate the index web page. You should now see the log format as shown:

To learn how to create a customized log format, check the documentation.

Conclusion

In this guide, you learned how to set up access logging in the Lighttpd server and format the logs. You also learned how to create a customized log format using various options.

Thank you for reading, and stay tuned for more tutorials.