Adding the log Module to Nginx on Debian Cloud


Server administration is not about the initial configuration of services. It also consists of overseeing those services and ensuring they are running properly. One of the most required sources of knowledge for administration is log files, that contain data concerning system events.


In terms of the web server such as Nginx, logs will consist of valuable data related to ever attempt to access resources through the web server. Each and every site visitor, the image is seen or the file downloaded is thoroughly registered in the logs. Once the errors occur, they're going to be saved in the logs too. It is quite easier to work with the log files which are well-structured.


In this article, you will be guided how to utilize Nginx's logging module. You need to set up unique log files for various server blocks and later customize the logging output. You can also add additional data regarding the requests to the access log beyond what Nginx includes by default.


1) Creating the test files


You can create a 2-megabyte file named as  2mb.test in the default Nginx directory by making use of truncate.


sudo truncate -s 2M /var/www/html/2mb.test


Now you need to create two more files of various sizes, first 20 and then 200 megabytes, naming them as required.


sudo truncate -s 20M /var/www/html/20mb.test

sudo truncate -s 200M /var/www/html/200mb.test


At last create an empty file:


sudo touch /var/www/html/empty.test


2) Understanding the Default Configuration


On the new installation, Nginx logs will request to 2 different files like the access log and the error log. The access log is situated in /var/log/nginx/access.log, is used more frequently. And the error log is situated in /var/log/nginx/error.log, stores details about unusual server errors, or errors during the time of processing the request.


Now request the empty file that has been created in Step 1 from Nginx so the log file will not be empty.


curl - I http://localhost/empty.test


In response, you will be able to see numerous HTTP response headers:


Nginx response headers:


HTTP/1.1 200 OK

Server: nginx/1.9.5

Date: Tue, 20 Feb 2017 13:22:05 GMT

Content-Type: application/octet-stream

Content-Length: 0

Last-Modified: Tue, 20 Feb 2017 13:22:05 GMT

Connection: keep-alive

ETag: "584b38a9-0"

Accept-Ranges: bytes


From this response, you will be able to learn numerous things:


HTTP/1.1 200 OK will tell you that Nginx responded with 200 OK status code and it will mention that there is no error.


Content-Length: 0 that means the returned document will be zero-length.


The request has been processed on Tue, 20 Feb 2017 13:22:05 GMT.


Now you need to check whether it matches what Nginx has been stored in its access log. The log files will be readable only by administrative users, so sudo will be used to access them.


sudo tail /var/log/nginx/access.log


The log will consist of the line as mentioned below


000.0.0.0 - - [20/Mar/2015:04:02:10 +0000] "GET /empty.test HTTP/1.1" 200 0 "-" "curl/7.38.0"


Each and every log entry in the access log consists of numerous of valuable information about the request. Moreover, there's one necessary bit of info missing. As you have requested the right location of http://localhost/empty.test, only the path to the /empty.test file will be in the log entry and the details about the hostname is been lost.


3) Configuring the Separate Access Log


For changing the default Nginx server block configuration, you need to open the server block Nginx configuration file in nano or text editor of your choice.


sudo nano /etc/nginx/sites-available/default


You need to find the server configuration block, which will look like the below-mentioned query:


/etc/nginx/sites-available/default

. . .

# Default server configuration

#


server {

   listen 100 default_server;

   listen [::]:100 default_server;


. . .


and include the two lines marked in the red for the configuration:


. . .

# Default server configuration

#


server {

   listen 80 default_server;

   listen [::]:80 default_server;


   access_log /var/log/nginx/default-access.log;

   error_log /var/log/nginx/default-error.log;

. . .


The access_log directive sets the path to file where the access logs are stored, and error_log will do the same for the error log. You can use the same directory as the default Nginx logs (/var/log/nginx), however with completely different filenames. If you've got multiple server blocks, it's a right plan to name log files in a consistent and significant manner, utilizing the domain name in the filename.


You need to Save and shut the file to exit.


To enable your new configuration, restart the Nginx.


sudo systemctl restart nginx.service


To test your new configuration, you need to execute the same request for the empty test file.


curl -i http://localhost/empty.test


You need to check that the logline identical to the one you have seen before is written to the separate file that has been configured.


sudo tail /var/log/nginx/default-access.log


4) Configuring the Custom Log Format


To define the new logging format, you need to create a new configuration file known as timed-log-format.conf in the Nginx additional configuration directory.


sudo nano /etc/nginx/conf.d/timed-log-format.conf


Add the below contents:


/etc/nginx/conf.d/timed-log-format.conf

log_format timed '$remote_addr - $remote_user [$time_local] '

                '"$request" $status $body_bytes_sent '

                '"$http_referer" "$http_user_agent" $request_time';


You need to save and close the file to exit.


The log_format setting directive will define the new log format. The next element will be the unique identifier of this format, eg: abcd but you can use any name of your choice.


The custom log format is been named as abcd which is defined in the Nginx configuration, but the default server block will not use this format. Next, you need to open the server block Nginx configuration file.


sudo nano /etc/nginx/sites-available/default


You need to find the server configuration block which has been modified earlier and add the abcd log format name to the access_log setting as mentioned below:


/etc/nginx/sites-available/default


. . .

# Default server configuration

#


server {

   listen 100 default_server;

   listen [::]:100 default_server;


   access_log /var/log/nginx/default-access.log abcd;

   error_log /var/log/nginx/default-error.log;

. . .


Just save and close the file to exit.


To enable the new configuration, you need to restart the Nginx.


sudo systemctl restart nginx.service


5) Verification of the New Configuration


You will be able to check the new configuration by invoking few requests to Nginx with curl. You can use the sample files.


curl -i http://localhost/empty.test

curl -i http://localhost/4mb.test

curl -i http://localhost/120mb.test

curl -i http://localhost/250mb.test


You need to display the access log after executing the requests.


sudo tail /var/log/nginx/default-access.log


The log will now consist of more lines, but the last four lines will correspond to the test requests.


Access log entries


000.0.0.0 - - [20/Mar/2015:04:02:10 +0000] "GET /empty.test HTTP/1.1" 200 0 "-" "curl/7.38.0"

000.0.0.0  - - [20/Mar/2015:04:02:10 +0000] "GET /empty.test HTTP/1.1" 200 0 "-" "curl/7.38.0" 0.000

000.0.0.0  - - [20/Mar/2015:04:02:10 +0000] "GET /1mb.test HTTP/1.1" 200 1048576 "-" "curl/7.38.0" 0.000

000.0.0.0 - - [20/Mar/2015:04:02:10 +0000] "GET /10mb.test HTTP/1.1" 200 10485760 "-" "curl/7.38.0" 0.302

000.0.0.0  - - [20/Mar/2015:04:02:10 +0000] "GET /100mb.test HTTP/1.1" 200 68516844 "-" "curl/7.38.0" 7.938


Now you can notice that the paths are different each time, and show the correct filename, and the request size increases every time. One of the important part is the last number is the request processing time in milliseconds. Which was configured in the custom log format.


Finally, you have configured the custom log format in Nginx successfully.