Tuesday, June 15, 2010

syslog-ng tips and tricks

Although I've been contemplating using scribe for our logging needs, for now I'm using syslog-ng. It's been doing the job well so far. Here are a couple of configuration tips:

1) Sending log messages for a given log facility to a given log file

Let's say you want to send all haproxy log messages to a file called /var/log/haproxy.log. In haproxy.cfg you can say:

global
 log 127.0.0.1 local7 info

...which means -- log all messages to localhost, to log facility local7 and with a log level of info.

To direct these messages to a file called /var/log/haproxy.log, you need to define the following in /etc/syslog-ng/syslog-ng.conf:

i) a destination:

destination df_haproxy { file("/var/log/haproxy.log"); };

ii) a filter:

filter f_haproxy { facility(local7); };

iii) a log (which ties the destination to the filter):

log {
source(s_all);
filter(f_haproxy);
destination(df_haproxy);
};

You also need to configure syslog-ng to allow log messages sent via UPD from localhost. Add this line to the source s_all element:

udp(ip(127.0.0.1) port(514));

Important note: since you're sending haproxy log messages to the local7 facility, this means that they'll also be captured by /var/log/syslog and /var/log/messages, since they are configured in syslog-ng.conf as destinations for the filters f_syslog and f_messages, which by default catch the local7 facility. As a result, you'll have triple logging of your haproxy messages. The solution? Add local7 to the list of facilities excluded from the f_syslog and f_messages filters.

2) Sending log messages to a remote log host

Assume you want to centralize log messages for a given service by sending them to a remote log host. Let's assume that the service logs via the local0 facility. The same procedure applies, with the creation of the following elements in syslog-ng.conf:

i) a destination


destination df_remote_log {
  udp("remote_loghost" port (5000));
};


ii) a filter:


filter f_myservice { facility(local0); };

iii) a log:

log {
        source(s_all);
        filter(f_myservice);
        destination(df_remote_log);
};

Note that you can also send messages for this particular filter (corresponding to local0) to a local file, by creating a destination poining to that file and a log element tying the filter with that destination, like this:

destination df_local_log { file("/var/log/myservice.log"); };
log {
        source(s_all);
        filter(f_myservice);
        destination(df_local_log);
};

Finally, to finish the remote logging bit, you need to configure syslog-ng on the remote host to allow messages on UDP port 5000, and to log them to a local file. Here's my configuration on host "remote_loghost":

i) a new source allowing messages on port 5000:

source s_remote_logging {
    udp(ip(0.0.0.0) port(5000));
};

ii) a destination pointing to a local file:

destination df_common_log { file ("/var/log/myservice_common.log"); };

iii) a log combining the source and the destination above; I am using the predefined f_syslog filter here, because I don't need to select messages based on a given facility anymore:

log {
        source(s_remote_logging);
        filter(f_syslog);
        destination(df_common_log);
};




1 comment:

Modifying EC2 security groups via AWS Lambda functions

One task that comes up again and again is adding, removing or updating source CIDR blocks in various security groups in an EC2 infrastructur...