vExpert 2014

I’m honored to announce that this year for the 4th time I’m awarded with the vExpert 2014 title.

vExpert is title who you can ear for a year if you have demonstrated significant contribution to the community and a willingness to share their expertise with others.

I’ve got mine for my grey VMware courses that I’ve developed and of course this blog

The complete list of all vExpert 2014 can be found here.

How to add a new node to a Nutanix cluster using CLI

If you want to expand you Nutanix cluster with a new node you have the option to do this from the GUI. But sometime you want to do it from the command line using the CLI.

Before you start check a couple of things:

Is IPV6 working between the Nutanix hosts?

1. Get the IPV6 address of the node you want to add by entering the command:
   ifconfig etho | grep inet6

   You will get a output something like:
   inet6 addr: fe80::20c::29ff:feb7:a9c7/64 Scope:Link

2. Try to ping the node you want to add from a cluster node by entering the command:
   ping6 -I eth0 fe80::20c::29ff:feb7:a9c7

   You should get a reply

If IPV6 is working you can add the new node to the cluster.

1. Enter the NCLI interface by entering ncli
2. Check if the node you want to add is discovered by entering:
   cluster discover-nodes

   You should get a output something like:
   ncli> cluster discover-nodes
   Cluster Id                           :
   Hypervisor Address         : xx.xx.x.xx
   Ip                                         : fe80::20c:29ff:feb7:a9c7%eth0
   Ipmi Address                    : xx.xx.x.xx
   Node Position                   : A
   Node Serial                        : ZMxxxxxxxxxx
   Node Uuid                         : d1b9d9c3-7f6f-4cb8-bf16-f379fd73e342
   Rackable Unit Model       : NX-3051
   Rackable Unit Serial        : 13SMxxxxxxxx
   Service Vm Address         : xx.xx.xx.xx
   Svm Id                                :

   Notice the Node Uuid

3. Add the node to the cluster by entering the command:
   cluster add-node node-uuid=[node uuid from step 4]

After this the node will be add to your cluster

How to check your version of Nutanix NOS

On the CVM controller issue the command:

cat /etc/nutanix/svm-version

My first thoughts on Nutanix

Currently I’m working on a project designing a 3000 VMware Horizon View environment for a local government in the west of the Netherlands. When I started this job the hardware was already purchased by the project manager. Not the correct way (according to Prince2) but it’s a fact a have to deal with. Honestly I was very pleased to hear that the vendor chose was Nutanix. On the first day I got a tour in the datacenter and 1 19″ rack was almost fully filled with NX-3050  (8 blocks with 4 servers connect with 10Gb on a Cisco Nexus 5000).

I’m quite experienced with VMware vSphere an d View but not with Nutanix. So the first thing I did was Google for a Nutanix. I didn’t want to search the Nutanix website itself but a independent people who give there thoughts on Nutanix. One of the first website I found was was IT Blood Pressure and the website from Stevan Poitras. Stevan website contains a section called the Nutanix Bible. Really interesting stuff especially the section witch describes in detail how the controller VM handels the storage workloads. You can see that some really smart people developed this system.Looking at the profiles of both authors I discovered that the both work for Nutanix. Oke, that’s by the articles are so in depth and detailed. So maybe not as independent that I was looking for, but nevertheless really interesting stuff.

After reading a lot about Nutanix I posted a Tweet that I was working with Nutanix. The same day I was contacted by Raymon Epping (@repping) that if I have any questions I can contact him. This really enjoyed me. Although Nutanix isn’t a small company the aren’t that spoiled  that talk directly whit engineers. Of course Nutanix has a benefit if this project succeeds (and this is when the user is satisfied) the could think that the sold the hardware and it isn’t there concern anymore. That if I need more information I can go to a training a read the manual. Nutanix even went so far that I had Raymon Epping for a couple of hours for myself talking about how to design a Nutanix cluster. Great!

As far as I now can overlook Nutanix isn’t so difficult to setup. The manuals are really clear and on the Internet you can Google for a of information. But as whit most (large) implementation there are 2 way’s to implement/desing a environment. The way that it runs, and the way that it runs on it’s best and it’s manageable. In my course I often say that 60% till 70% is thinking about how to setup your environment. Think in what if scenarios. The last 40%  till 30% is just clicking your environment together. You cannot design a Nutanix cluster the same way as you would design a traditional cluster. Some stuff you have to think about is:

• Is it wise to run vCenter on the same hardware because of the dependency of the controller VM?
• How to size your storage cluster, 1 big one or multiple smaller ones because you have different workloads
• What is the impact of the controller VM and what if the controller VM isn’t available (backup?)?
• How do you setup your network (VSS versus VDS)?
• How to setup time synchronization (because of the database Nutanix uses) and what if time ain’t available.

For sure I’m going to contact Raymon Epping about designing issues.

One thing I learned in a early stage. RTFM! 🙂

Setting up a central syslog server for VMware vSphere

Currently I’m working on a project that involves multiple vCenter servers and vSphere servers on multiple sites. When you install ESXi on a SD card you get a warning on you ESXi host that log file are stored on non-persistent storage. You can either store the logfiles on local storage or and a syslog server.

With a syslog server you have 2 options:

1. Log to the vCenter Appliance who has a build in log server
2. Setup your own syslog server.

Option 1 is sufficient when you have 1 vCenter and only want to log ESXi log file because of the warning you get when you don’t provide a syslog server.

If you have more vCenter server, more devices that want to make use of syslog, it’s best to setup your own syslog server (option 2).

Setup syslog server

Setting up your own syslog server is not difficult. Because my roots are with Novell, I’m quite experience with Suse linux I use Suse Linux Enterprise server 11 for my setup with the following specifications:

• 1 CPU
• 2 GB Memory
• 1 NIC
• 2 Disks (1 of 16 GB for the OS, 1 of 50 GB for logging)

As you can see in the specifications I use 2 disk, 1 for the OS and 1 for logging. The reason for this is that the when log fills up the disk, it won’t fill up the OS disk. Leaving linux running for troubleshouting.

I won’t describe the installation of SLES because this is straight forward.

After the installation of SLES you want to configure syslog. SLES uses syslog-ng for the logging. There are 4 things you have to configure for syslog:

1. Make syslog accept log messages from the network
2. Create a log filter
3. Create a log destination
4. Combine the log filter and destination

Step 1: Make syslog accept log messages from the network

• Open the file /etc/syslog-ng/syslog-ng.conf
• Goto the second where it say’s: “uncomment to process log messages from network” and remove the # before the udp rule. It should look something like this.

Step 2: Create a log filter

• Goto the filter section and add a line:filter f_vmware { match(‘^Vpxa:’) or match(‘^Hostd:’); };
• This section say’s to syslog that every message that contains Vpxa: or Hostd: is a VMware message.

Step 3: Create a log destination

• Goto the bottom of the file and add the following line:destination vmware {
  file(“/var/log/vmware/$HOST_FROM.log”
  create_dirs(yes) );
  };
• The path /var/log/vmware can be changed to whatever you want.

Step 4: Combine the log filter and destination

• At the bottom of the file add the following line:log { source(src); filter(f_vmware); destination(vmware); };

Restart the syslog-ng deamon by typing “rcsyslogd restart” and your good to go.

By the way, I don’t have to mention you have to disable or open a port (udp 514) on the firewall 🙂 ?

Last thing is to configure your ESXi host to make use of the new syslog server. VMware wrote a nice KB article about that.

Update 29/11/2013
The filter for VMware vSphere logging is not complete. This had to be:

filter f_vmware { match(‘^Vpxa:’) or match(‘^Hostd:’) ; or match(‘^Rhttpproxt:’) or match(‘^Fdm:’) or match(‘^hostd-probe:’)  or match(‘^vmkernel:’); };

Als alter the filter for /var/log/messages not to include VMware vSphere logging. Otherwise you log everything twice.

filter f_messages { nog facility(new, mail) and not filter (f_iptables) and not filter (f_vmware); };