System Logs

• There are three main reasons for logging system activities:

1. to monitor for security intrusions or attempts,
2. to tweak performance based on historical system use,
3. and to diagnose problems by checking the system state just before the problem occurred.

• Most Linux log files will be stored in /var/log. Certain programs may store logs elsewhere, but can typically be configured to use /var/log as well.
• The log files are all written with the same fields: date and time, host name, program name, and the message itself.
• Logs can also be stored on remote machines. This is useful for the following reasons: consolidates all of the logs into one place for easy monitoring, safeguard log entries from fatal disk failures on the local machine, and add an additional impediment to crackers who will typically want to change the log files to cover their tracks.
• For most programs, logging is managed by two daemons: klogd (kernel log daemon) and syslogd (system log daemon). These two daemons accept log messages from the kernel, and all other programs, respectively, and write them to the appropriate log file/device/remote machine, as directed by the /etc/syslog.conf configuration file.
• The kernel also logs boot-time messages. Because these messages are generated before the rc scripts have a chance to start either the klogd or syslogd daemons, the kernel writes these messages to the "kernel ring buffer". The kernel ring buffer is usally about 8K and size, and overwrites old messages as new ones are added. The dmesg command displays the contents of the kernel ring buffer. The Red Hat distribution also stores the contents of the kernel ring buffer as it exists immediately after system startup in the /var/log/dmesg file. This is so that an administrator can diagnose any boot-time problems even if the kernel ring buffer has long since overwritten all of those messages.
• The /etc/syslog.conf file has its own particular syntax. Each logging decision is phrased as "selector action" where the selector is made up of "facility.priority". The facility and priority codes are described in the syslog Facilities and Priorities handout. Multiple selectors can be placed on a single line, separated by semicolons. The action is either a device file name, a log file name, a remote machine designation, or the * character which means display the message on all user terminals.
• Either the facility code or the priority code can be replaced with a * which means match any facility or priority, respectively.
• The priority may also be "none" which means that no messages for the specified facility will be logged. This syntax is typically used to filter out certain facilities from an encompassing selector on the same line.
• When a priority is specified, it means that all priorities of that level and higher will be logged.
• A comma may be used to list multiple facilities for a single priority. This is just a notational shorthand.
• There are other ways to narrow down the exact priority level, but this won't be covered in the course.

syslog Facilities

A facility is a process category. The syslogd and klogd daemons may take different logging actions based on the facility of the process attempting to write a log entry.

Facility

Description

auth, security

User authentication messages.

auth-priv

Private user authentication messages.

cron

crond daemon messages.

daemon

Generic daemon messages.

kern

Kernel messages.

lpr

Printer daemon messages.

mail

Mail daemon messages.

news

News daemon messages.

syslog

System logging daemon messages.

user

User-process messages.

uucp

Unix-to-Unix-Copy networking messages.

localN

Special-purpose messages defined by certain applications or Linux distributions. N can be between 0 and 7.

syslog Priorities

Each log message is assigned a priority. The following table describes all of the defined priority, in order from lowest to highest (most important).

Priority

Description

debug

Debugging messages used by programmers during software development.

info

Informational messages written during normal program execution.

notice

Noteworthy information messages.

warning, warn

Warnings about potential problems.

err, error

Notices of errors.

crit

Notices of errors that are serious enough to abnormally terminate the program.

alert

Notices of errors that may cause other processes to terminate, in addition to the process logging the message.

emerg, panic

Notices of errors that could potentially crash the kernel.