Logs are important to application developers. They give us detailed error information about our applications and the systems supporting them. They also can give us clues about what may go wrong in the future.
Where To Find Them
Logs are almost always found in
/var/log. Usually they are owned by a privileged user and group, such as
adm. Even if you don't have access to a privileged user (a user who can use
sudo), you can often - but not always - read these files.
/var/log directory, you'll find system and software logs. For example, in Debian/Ubuntu systems, you'll find your Apache access and errors logs in
/var/log/apache2. In other OS distributions, this may be
/var/log/httpd. Nginx logs are usually found in
Not all log files are in a subdirectory. For example, if you're using PHP-FPM, you might find logs in
/var/log/php5-fpm.log. MySQL is often similarly found at
Lastly, logs are often split between access (or events) and error logs. Apache, Nginx and MySQL all have separate error and access/use logs.
Some web applications create their own logs. You should consult your framework's documentation to find out where they might be. The Laravel PHP framework, for example, will create logs in the
app/storage/logs directory. To avoid getting a WSOD upon an error, you should make sure that the log directory can be written to.
Log information on other popular frameworks:
- laravel (PHP)
- Uses Monolog under the hood
- sailsJS (Node)
- Uses Winston under the hood
- django (Python)
- flask (Python)
- RoR (Ruby)
Watching Logs With Tail
Since logs may give you more error information than the browser when developing a web application, you may find it appropriate to monitor a log file. This is especially true in production environments where error reporting is turned off - they will only be displayed within log files.
To monitor a log file, we can "tail" it. This will show the last n lines of the file as they are written to it.
If I wanted to watch the last 50 lines of my apache error log for errors, I would run:
tail -n 50 -f /var/log/apache2/error.log
Virtual Hosts and Logs
If you read the previous newsletter on Apache virtual hosts, you may have noticed that I setup a unique access and error log for each virtual host. This lets you track the logs of just one website, rather than having multiple website events combined into one log.
For a site example.com, the log files might be
Analyzing Log Files
Here are some simple examples of examining some Apache (or Nginx!) log files.
Some of the command line tools available for things like are
head and others. For example here's how to count 404's per request in a group of access logs. This works for both regular and compressed (gzipped) access logs.
For more command-line tricks:
More Advanced Analysis and Tooling
As you might imagine, there are LOTS of tools available to analyze and store logs. This becomes especially important if you have a multiple-server environment, each of which may be generating it's own collection of logs.
In situations like this, a common strategy is to combine log files to a central storage. This usually comes with a bonus of being both searchable and more easily analyzed.
Here's a list of some available logging tools:
- Logstash - An easy to use Open Source log and event manager. Numerous people pointed out that I missed this at first. It's one of the easiest (and backed by Elasticsearch) open-source loggers!
- Webalizer - Good for single servers, and often found in hosting accounts, this is good for basic analyzing of web traffic from Apache logs
- Fluentd - Open source tool to collect events and logs (in JSON)
- Graylog2 - This both collects logs AND analyzes them. It's a powerful tool, often used in conjunction with Graphite to create meaningful graphs based on log analysis.
- Some information on Graylog2 and PHP Monolog
- Splunk - Enterprisey
Here's an interesting idea on logging: Fingers Crossed logging is the idea of only saving logs which are near errors. If logs do not appear near errors, they are (eventually) discarded.
For AWS users, Elastic Load Balancers will finally have Access logs and other data available!
- Log Resources for Distributed Environments (scroll to the bottom)