Guys, have you ever encountered a situation when your application crashed in production? Of course you have. All you gonna do is to look for the cause through your log files. Log file records every events that occur in your application. In this case, error log is your log file. But, in order see the log file, we need to login into the server and read the log manually. That’s really not worthy.
To make this kind of job easier, we need to create a log monitoring system for our apps. So you don’t need to read your log file in each server manually to find the root cause. Why we need log monitoring? By using log monitoring, we can see all log file from all of our systems in a centralized place and in a convenient way. Also you can do log analysis and reporting though this system.
This tutorial will guide you how to implement simple log monitoring system. In tutorial, I will use Log.io as a monitoring tool for laravel project. Log.io in a monitoring tool which is backed by node.js and socket.io. Log.io consists of two service, i.e. log.io server and harvester. Harvester watch for every changes that happened in log files listed in harvester configuration file and send them to log.io server. Then log.io server will listen any messages sent from harvester and broadcast them to client/browser. Harvester send the messages via socket.io TCP protocol to log.io server.
These are the requirements that you will need in this tutorial:
- NVM (Node Version Manager, optional)
- js (I use node.js version 4.4.3)
- Laravel project (I use laravel 5.2)
- io (as a node.js package)
sudo npm install -g log.io –user “alvin”
-g means install log.io as a global package and –user “alvin” as a specific user (alvin is a current user). Now move to log.io configuration directory and you will see there are three configuration files.
Now open your harvester.conf file and edit the file according to your applications which you are going to monitor their log.
Put your application log path in logStream section so the harvester will watch every log files listed there. You can list as many project & log file as you want. After you edit the file, you can run these two services in different terminal:
Browse to http://localhost:28778 to open log.io web interface. Now I will simulate how log.io works. I will create a page which error occurs and open it.
And all information from the error log should been seen in log.io web interface.
Monitor laravel daily log file
Laravel allow you split your log into daily log files instead of single log file. The advantages of using daily log files are log easy to manage and it can avoid single huge log file. To configure laravel into daily log file, open config/app.php file and change log section into daily log. You can also add log configuration via .env file.
After your configuration has been changed, everytime an error occurs, laravel will insert log into daily log file based on this pattern laravel-yyyy-mm-dd.log.
After that, you need to restart harvester service to enable this changes. Now, every laravel log with current date format will be read by log.io harvester.
Note: you need to restart your harvester service every midnight in order to read current date (probably every 00:00 AM).
Modify log.io to read only some lines of laravel log messages
If you look in detail, once an error occurs happened, laravel would produce a lot of error log messages which is about 40 lines. Unfortunately, not all of those messages are important for developer to trace an error. Some programmers only need several lines of message in order find error and fix it (I personally only look up to first 4 or 6 lines of error messages).
So, how to make log.io to show only some lines of error messages? To do that, we need to make some changes to log.io main code.
Note: you may only do this if you use log.io only for monitoring one type of application (ex: only for laravel application). Different type of application will produce different log format. Use different log.io if apply this step for different application.
First, you need to locate harvester.js which is main log.io library file for harvesting log message process. Since I’m using NVM and node.js version 4.4.3, this file is located in ~/.nvm/versions/node/v4.4.3/lib/node_modules/log.io/lib/harvester.js. If you are not using NVM, this file should be located in your main node.js library directory. Open the file and find a piece of code like this.
This piece of code is responsible to read every log messages that are written to log file. We will modify this code. We need to declare how much lines will be read and put it into for loop code. Now, let’s modify the code into this one.
You can specify how many lines will be read by changing the value of numOfLines variable. Now restart your log.io server and harvester to enable this changes. After that, the log messages which is displayed in browser will be as many as you specified in harvester.js file.
Although log.io is not a new log monitoring system, it’s still a good tool to monitor your log file in a real time, so you don’t need to check to every server just to look for the root cause whenever your apps werte crashed in production. For more information about log.io, you can visit this website https://github.com/NarrativeScience/Log.io.