Setting up a central syslog server for VMware vSphere

Currently I’m working on a project that involves multiple vCenter servers and vSphere servers on multiple sites. When you install ESXi on a SD card you get a warning on you ESXi host that log file are stored on non-persistent storage. You can either store the logfiles on local storage or and a syslog server.

With a syslog server you have 2 options:

1. Log to the vCenter Appliance who has a build in log server
2. Setup your own syslog server.

Option 1 is sufficient when you have 1 vCenter and only want to log ESXi log file because of the warning you get when you don’t provide a syslog server.

If you have more vCenter server, more devices that want to make use of syslog, it’s best to setup your own syslog server (option 2).

Setup syslog server

Setting up your own syslog server is not difficult. Because my roots are with Novell, I’m quite experience with Suse linux I use Suse Linux Enterprise server 11 for my setup with the following specifications:

• 1 CPU
• 2 GB Memory
• 1 NIC
• 2 Disks (1 of 16 GB for the OS, 1 of 50 GB for logging)

As you can see in the specifications I use 2 disk, 1 for the OS and 1 for logging. The reason for this is that the when log fills up the disk, it won’t fill up the OS disk. Leaving linux running for troubleshouting.

I won’t describe the installation of SLES because this is straight forward.

After the installation of SLES you want to configure syslog. SLES uses syslog-ng for the logging. There are 4 things you have to configure for syslog:

1. Make syslog accept log messages from the network
2. Create a log filter
3. Create a log destination
4. Combine the log filter and destination

Step 1: Make syslog accept log messages from the network

• Open the file /etc/syslog-ng/syslog-ng.conf
• Goto the second where it say’s: “uncomment to process log messages from network” and remove the # before the udp rule. It should look something like this.

Step 2: Create a log filter

• Goto the filter section and add a line:filter f_vmware { match(‘^Vpxa:’) or match(‘^Hostd:’); };
• This section say’s to syslog that every message that contains Vpxa: or Hostd: is a VMware message.

Step 3: Create a log destination

• Goto the bottom of the file and add the following line:destination vmware {
  file(“/var/log/vmware/$HOST_FROM.log”
  create_dirs(yes) );
  };
• The path /var/log/vmware can be changed to whatever you want.

Step 4: Combine the log filter and destination

• At the bottom of the file add the following line:log { source(src); filter(f_vmware); destination(vmware); };

Restart the syslog-ng deamon by typing “rcsyslogd restart” and your good to go.

By the way, I don’t have to mention you have to disable or open a port (udp 514) on the firewall 🙂 ?

Last thing is to configure your ESXi host to make use of the new syslog server. VMware wrote a nice KB article about that.

Update 29/11/2013
The filter for VMware vSphere logging is not complete. This had to be:

filter f_vmware { match(‘^Vpxa:’) or match(‘^Hostd:’) ; or match(‘^Rhttpproxt:’) or match(‘^Fdm:’) or match(‘^hostd-probe:’)  or match(‘^vmkernel:’); };

Als alter the filter for /var/log/messages not to include VMware vSphere logging. Otherwise you log everything twice.

filter f_messages { nog facility(new, mail) and not filter (f_iptables) and not filter (f_vmware); };