Rsyslog – Remote system logs

So next on the list – rsyslog. A fairly simple and short topic, but one i wanted to cover anyway – more for my own revision purposes than anything.

What is it?

Simply put – it is a way of sending system logs from lots of servers to a single point, a “syslog server” if you like. This allows them to be analysed en-masse, etc.

It works by simply setting up one server, the syslog server, to accept incoming syslogs from other devices, and by setting up the other servers to send syslogs to the syslog server. This is done on both types (recipient/server) in the /etc/rsyslog.conf file which comes with RHEL.

Configuration

On the server side (the device receiving logs), open up /etc/rsyslog.conf and edit it as below:

# rsyslog v5 configuration file
# For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html
# If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html
#### MODULES ####
$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imklog # provides kernel logging support (previously done by rklogd)
#$ModLoad immark # provides --MARK-- message capability
# Provides UDP syslog reception
#$ModLoad imudp
#$UDPServerRun 514
# Provides TCP syslog reception
$ModLoad imtcp
$InputTCPServerRun 514

Here we have uncommented the 2 lines above, to accept inbound rsyslog traffic on TCP 514.

Next, iptables – lets open up said port with a simple firewall rule:

iptables -i INPUT 4 -p tcp --dport 514 -j ACCEPT

And thats the server side configuration done. Next, the clients!

On our clients – again edit the file /etc/rsyslog.conf, this time look at the bottom of the file:

# remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional
#*.* @@remote-host:514
# ### end of the forwarding rule ###
*.* @@rsyslog-remote-server.example.com

As you can see, we’ve added a new rule to send all our syslogs (*.*) to our remote rsyslog server.

Finally, just restart rsyslog (/etc/init.d/rsyslog restart) and then create some events!

[root@ip-10-224-74-103 ec2-user]# logger FAIL
[root@ip-10-224-74-103 ec2-user]# logger FAIL2
[root@ip-10-224-74-103 ec2-user]# logger FAIL2
[root@ip-10-224-74-103 ec2-user]# logger FAIL24

On our rsyslog server, we can cat /var/log/messages, and see:

Jul 11 11:39:44 ip-10-224-74-103 ec2-user: FAIL
Jul 11 11:39:45 ip-10-224-74-103 ec2-user: FAIL2
Jul 11 11:39:45 ip-10-224-74-103 ec2-user: FAIL2
Jul 11 11:39:47 ip-10-224-74-103 ec2-user: FAIL24

Voila! We are receiving the log files from the remote server. Easy.

Closing thoughts

Rsyslog servers are fairly simple on the face of it, you just configure the one file /etc/rsyslog.conf, and then set one to receive, one to send, open iptables, and your done.

Theres plenty to do with it if you want to get deep in the weeds, changing *.* to *.info, etc.