Alignment of a NSS volume in Vmware ESX
As you all probely know, It’s wise to align a filesystem partion in Vmware NSS. If you look at the figures below, you can see that it’s possible that if you don’t align a filesystem ESX has to read 3 block to get 1 block of data.
Here a screenshot of the ESX_partition_align.pdf from Vmware where the filesystem is not aligned
And here the screenshot where the filesystem is aligned:
I guess you can imaging that when the filesystem is aligned the performance will benefit from this. The pdf from Vmware describes how to align a Windows or a Linux filesystem but not Novell NSS.
It took me some work but I finally figured it out. It pretty much the same a aligning a Linux filesystem. The difference is that with Linux you create a empty linux partition, you move the beginning of the partition to a certain block and than you create a filesystem in that empty partition.
With NSS we cannot do this. We cannot create a empty NSS partition and after that create the NSS Pool and Volume.
The trick is to create a NSS Pool without a volume. Deactivate the pool and then follow the procedure as described in the PDF file from Vmware for Linux filesystems.
After the alignment, activate the NSS Pool and create a volume.
Just to be sure, here is the procedure. Is removed the commands for creating a Linux partition. So this is just for NSS.
1. Enter fdisk /dev/sd<x>
2. Determine the NSS partition (Type 65, Novell Netware 386)
3. Type x to go into expert mode.
4. Type b to adjust the start block number.
5. Type 1 to choose partition 1.
6. Type 128 to set it to 128.
7. Type w to write label and partition information to disk.
The performance is better when the partition is algined but not as fast as a RDM. Here little overview I create with the tool Bonnie.
| Write Char | Write block | Rewrite | Read Char | ||||||||||
| K/Sec | %CPU | K/Sec | %CPU | K/Sec | %CPU | K/Sec | %CPU | K/SEC | %CPU | /sec | %CPU | ||
| rdm iscsi algined from 128 | 1100 * 2 | 21863 | 67.7 | 31413 | 23.7 | 16188 | 21.1 | 19150 | 76.4 | 34120 | 44.3 | 382.3 | 7.7 |
| rdm iscsi aligned from 32 | 1100 * 2 | 20855 | 63.9 | 30979 | 23.6 | 15996 | 20.8 | 19538 | 77.5 | 33439 | 40.5 | 403.9 | 7.7 |
| vmfs iscsi aligned from 128 | 1100 * 2 | 22100 | 69.5 | 31416 | 23.7 | 11529 | 14.2 | 15164 | 61.3 | 18316 | 21.0 | 386.1 | 6.9 |
| vmfs iscsi aligned from 32 | 1100 * 2 | 12478 | 52.7 | 20077 | 35.6 | 12105 | 14.8 | 15237 | 62.3 | 19156 | 22.7 | 380.1 | 7.9 |
As you can see the write performance (per char or per block) on a aligned filesystem is much better. On rewrite a RDM still beats VMFS.
Michael Wilmsen is a experienced VMware Architect with more than 20 years in the IT industry. Main focus is VMware vSphere, Horizon View and Hyper Converged with a deep interest into performance and architecture.
Michael is VCDX 210 certified, has been rewarded with the vExpert title from 2011, Nutanix Tech Champion and a Nutanix Platform Professional.
By Willem, February 18, 2009 @ 10:53
Volg deze vmfs/rdm blogs met grote interesse! Volgende week heb ik weer een OES2 installatie met iSCSI binnen VMWare.. Daar heb ik al de keuze gemaakt voor RDM, daar levert volgens je tests de alignment duidelijk ook winst op (dacht altijd dat het alleen betrekking had op VMFS).
Groeten,
Willem
By Willem, February 18, 2009 @ 10:53
Volg deze vmfs/rdm blogs met grote interesse! Volgende week heb ik weer een OES2 installatie met iSCSI binnen VMWare.. Daar heb ik al de keuze gemaakt voor RDM, daar levert volgens je tests de alignment duidelijk ook winst op (dacht altijd dat het alleen betrekking had op VMFS).
Groeten,
Willem
By Mike, February 18, 2009 @ 15:18
Hi Willem,
I post in English so everybody can read it.
Good luck with your OES2 implementation next week. Are you going to implement a VMI kernel also?
By Mike, February 18, 2009 @ 15:18
Hi Willem,
I post in English so everybody can read it.
Good luck with your OES2 implementation next week. Are you going to implement a VMI kernel also?
By Willem, February 21, 2009 @ 09:45
Absolutely, have been using the VMI kernel with SLES as soon as it was available (and I got to know about it). The performance difference to me is very evident in how the systems handle and I haven’t had issues with the VMI kernel so far. The downside is that it’s only for 32bit guests, but then again the OES2 & SLES servers I’m running purrr along just fine with 2 tot 4 GB’s of memory.
Setting the start block to 128, is that always the optimal value here?
Cheers,
Wj
By Willem, February 21, 2009 @ 09:45
Absolutely, have been using the VMI kernel with SLES as soon as it was available (and I got to know about it). The performance difference to me is very evident in how the systems handle and I haven’t had issues with the VMI kernel so far. The downside is that it’s only for 32bit guests, but then again the OES2 & SLES servers I’m running purrr along just fine with 2 tot 4 GB’s of memory.
Setting the start block to 128, is that always the optimal value here?
Cheers,
Wj
By Mike, February 22, 2009 @ 14:02
There is also a VMIPAE kernel. Here you can address more than 4 GB of memory.
For most SANs 128 is the standard. So yes most system it’s ok. But check the company website to make sure. I know its the optimal value for Netapp, Equallogic and EMC.
By Mike, February 22, 2009 @ 14:02
There is also a VMIPAE kernel. Here you can address more than 4 GB of memory.
For most SANs 128 is the standard. So yes most system it’s ok. But check the company website to make sure. I know its the optimal value for Netapp, Equallogic and EMC.
By Wouter Mignon, February 27, 2009 @ 16:54
How do you use iSCSI in combination with a VMI enabled kernel. I can’t get iSCSI initiator working when using this kernel. I also tried using RDM and let ESX be the iSCSI initiator but them i run into a different problem: our datastore is running on NFS and RDM is not supported by ESX 3.5 on NFS.
Any Ideas here?
By Wouter Mignon, February 27, 2009 @ 16:54
How do you use iSCSI in combination with a VMI enabled kernel. I can’t get iSCSI initiator working when using this kernel. I also tried using RDM and let ESX be the iSCSI initiator but them i run into a different problem: our datastore is running on NFS and RDM is not supported by ESX 3.5 on NFS.
Any Ideas here?
By Mike, February 27, 2009 @ 18:09
Strange, So if you don’t use a standard (or Bigsmp) kernel it’s no problem? Which Operating system do you use? SLES10 and which Service Pack, than I will try it at home here.
You are right that you cannot use a RDM because you have a NFS Datastore. What kind of SAN/NAS do you use?
By Mike, February 27, 2009 @ 18:09
Strange, So if you don’t use a standard (or Bigsmp) kernel it’s no problem? Which Operating system do you use? SLES10 and which Service Pack, than I will try it at home here.
You are right that you cannot use a RDM because you have a NFS Datastore. What kind of SAN/NAS do you use?
By Wouter Mignon, March 3, 2009 @ 18:50
No, iSCSI works just fine with de default kernel. I get “FATAL: Module iscsi_tcp not found” when trying to start iSCSI with the VMI enabled kernel. (http://forums.novell.com/novell-product-support-forums/suse-linux-enterprise-server-sles/sles-install-boot/360579-sles-10-sp2-vmi-kernel-iscsi.html)
Please let me know if you can reproduce!
We are using NFS on a NetApp FAS2050.
By Wouter Mignon, March 3, 2009 @ 18:50
No, iSCSI works just fine with de default kernel. I get “FATAL: Module iscsi_tcp not found” when trying to start iSCSI with the VMI enabled kernel. (http://forums.novell.com/novell-product-support-forums/suse-linux-enterprise-server-sles/sles-install-boot/360579-sles-10-sp2-vmi-kernel-iscsi.html)
Please let me know if you can reproduce!
We are using NFS on a NetApp FAS2050.
By Mike, March 10, 2009 @ 14:45
Hi,
I have no problem connecting a iscsi initiator from SLES to my freenas. Have you run the latest patches from Novell? I Guess so. It’s strange that the give a error on tcp?
You can connect using tcp not?
By Mike, March 10, 2009 @ 14:45
Hi,
I have no problem connecting a iscsi initiator from SLES to my freenas. Have you run the latest patches from Novell? I Guess so. It’s strange that the give a error on tcp?
You can connect using tcp not?
By jackfrost, May 13, 2010 @ 04:26
Unfortunately if you move the starting block to 128 after rebooting, the oes2 server won’t detect your pools, much less your volumes. So your data will be wiped out. This is not good advice.
I did not notice a difference (before rebooting) after adjusting to 128. Thus far I’m finding all this talk about partition alignment (at least for the guest partition) is overblown.
By Mike, May 17, 2010 @ 10:27
Hi Jack,
Your right, this worked on OES2 sp0 but as off sp1 this trick doesn’t work any more. I’m currently working on a new presentation voor the Novellcongres in the Netherlands how to allign nss on OES2 sp2. If you want I can send you a copy of the presentation.
By jackfrost, May 13, 2010 @ 04:26
Unfortunately if you move the starting block to 128 after rebooting, the oes2 server won’t detect your pools, much less your volumes. So your data will be wiped out. This is not good advice.
I did not notice a difference (before rebooting) after adjusting to 128. Thus far I’m finding all this talk about partition alignment (at least for the guest partition) is overblown.
By Mike, May 17, 2010 @ 10:27
Hi Jack,
Your right, this worked on OES2 sp0 but as off sp1 this trick doesn’t work any more. I’m currently working on a new presentation voor the Novellcongres in the Netherlands how to allign nss on OES2 sp2. If you want I can send you a copy of the presentation.