Alignment of a NSS volume in Vmware ESX

As you all probely know, It’s wise to align a filesystem partion in Vmware NSS. If you look at the figures below, you can see that it’s possible that if you don’t align a filesystem ESX has to read 3 block to get 1 block of data.

Here a screenshot of the ESX_partition_align.pdf from Vmware where the filesystem is not aligned

screenshot1

And here the screenshot where the filesystem is aligned:

screenshot2I guess you can imaging that when the filesystem is aligned the performance will benefit from this. The pdf from Vmware describes how to align a Windows or a Linux filesystem but not Novell NSS.
It took me some work but I finally figured it out. It pretty much the same a aligning a Linux filesystem. The difference is that with Linux you create a empty linux partition, you move the beginning of the partition to a certain block and than you create a filesystem in that empty partition.

With NSS we cannot do this. We cannot create a empty NSS partition and after that create the NSS Pool and Volume.

The trick is to create a NSS Pool without a volume. Deactivate the pool and then follow the procedure as described in the PDF file from Vmware for Linux filesystems.
After the alignment, activate the NSS Pool and create a volume.

Just to be sure, here is the procedure. Is removed the commands for creating a Linux partition. So this is just for NSS.

1. Enter fdisk /dev/sd<x>
2. Determine the NSS partition (Type 65, Novell Netware 386)
3. Type x to go into expert mode.
4. Type b to adjust the start block number.
5. Type 1 to choose partition 1.
6. Type 128 to set it to 128.
7. Type w to write label and partition information to disk.

The performance is better when the partition is algined but not as fast as a RDM. Here little overview I create with the tool Bonnie.

Write Char Write block Rewrite Read Char
K/Sec %CPU K/Sec %CPU K/Sec %CPU K/Sec %CPU K/SEC %CPU /sec %CPU
rdm iscsi algined from 128 1100 * 2 21863 67.7 31413 23.7 16188 21.1 19150 76.4 34120 44.3 382.3 7.7
rdm iscsi aligned from 32 1100 * 2 20855 63.9 30979 23.6 15996 20.8 19538 77.5 33439 40.5 403.9 7.7
vmfs iscsi aligned from 128 1100 * 2 22100 69.5 31416 23.7 11529 14.2 15164 61.3 18316 21.0 386.1 6.9
vmfs iscsi aligned from 32 1100 * 2 12478 52.7 20077 35.6 12105 14.8 15237 62.3 19156 22.7 380.1 7.9

As you can see the write performance (per char or per block) on a aligned filesystem is much better.ย  On rewrite a RDM still beats VMFS.

About Michael
Michael Wilmsen is a VMware training/consultant (no specific order) with more than 15 years in the IT industry. Main focus is VMware vSphere, Horizon View and Site Recovery Manager with a deepdive to performance. Michael is VCDX 210, VCAP 4/5 certified, has been rewarded with the vExpert title from 2011, is a PernixPro, Nutanix Tech Champion and a Nutanix Platform Professional.

24 Comments to “Alignment of a NSS volume in Vmware ESX”

  1. By Willem, February 18, 2009 @ 10:53

    Volg deze vmfs/rdm blogs met grote interesse! Volgende week heb ik weer een OES2 installatie met iSCSI binnen VMWare.. Daar heb ik al de keuze gemaakt voor RDM, daar levert volgens je tests de alignment duidelijk ook winst op (dacht altijd dat het alleen betrekking had op VMFS).

    Groeten,
    Willem

  2. By Willem, February 18, 2009 @ 10:53

    Volg deze vmfs/rdm blogs met grote interesse! Volgende week heb ik weer een OES2 installatie met iSCSI binnen VMWare.. Daar heb ik al de keuze gemaakt voor RDM, daar levert volgens je tests de alignment duidelijk ook winst op (dacht altijd dat het alleen betrekking had op VMFS).

    Groeten,
    Willem

  3. By Mike, February 18, 2009 @ 15:18

    Hi Willem,

    I post in English so everybody can read it.
    Good luck with your OES2 implementation next week. Are you going to implement a VMI kernel also?

  4. By Mike, February 18, 2009 @ 15:18

    Hi Willem,

    I post in English so everybody can read it.
    Good luck with your OES2 implementation next week. Are you going to implement a VMI kernel also?

  5. By Willem, February 21, 2009 @ 09:45

    Absolutely, have been using the VMI kernel with SLES as soon as it was available (and I got to know about it). The performance difference to me is very evident in how the systems handle and I haven’t had issues with the VMI kernel so far. The downside is that it’s only for 32bit guests, but then again the OES2 & SLES servers I’m running purrr along just fine with 2 tot 4 GB’s of memory.

    Setting the start block to 128, is that always the optimal value here?

    Cheers,
    Wj

  6. By Willem, February 21, 2009 @ 09:45

    Absolutely, have been using the VMI kernel with SLES as soon as it was available (and I got to know about it). The performance difference to me is very evident in how the systems handle and I haven’t had issues with the VMI kernel so far. The downside is that it’s only for 32bit guests, but then again the OES2 & SLES servers I’m running purrr along just fine with 2 tot 4 GB’s of memory.

    Setting the start block to 128, is that always the optimal value here?

    Cheers,
    Wj

  7. By Mike, February 22, 2009 @ 14:02

    There is also a VMIPAE kernel. Here you can address more than 4 GB of memory.

    For most SANs 128 is the standard. So yes most system it’s ok. But check the company website to make sure. I know its the optimal value for Netapp, Equallogic and EMC.

  8. By Mike, February 22, 2009 @ 14:02

    There is also a VMIPAE kernel. Here you can address more than 4 GB of memory.

    For most SANs 128 is the standard. So yes most system it’s ok. But check the company website to make sure. I know its the optimal value for Netapp, Equallogic and EMC.

  9. By Wouter Mignon, February 27, 2009 @ 16:54

    How do you use iSCSI in combination with a VMI enabled kernel. I can’t get iSCSI initiator working when using this kernel. I also tried using RDM and let ESX be the iSCSI initiator but them i run into a different problem: our datastore is running on NFS and RDM is not supported by ESX 3.5 on NFS.

    Any Ideas here?

  10. By Wouter Mignon, February 27, 2009 @ 16:54

    How do you use iSCSI in combination with a VMI enabled kernel. I can’t get iSCSI initiator working when using this kernel. I also tried using RDM and let ESX be the iSCSI initiator but them i run into a different problem: our datastore is running on NFS and RDM is not supported by ESX 3.5 on NFS.

    Any Ideas here?

  11. By Mike, February 27, 2009 @ 18:09

    Strange, So if you don’t use a standard (or Bigsmp) kernel it’s no problem? Which Operating system do you use? SLES10 and which Service Pack, than I will try it at home here.

    You are right that you cannot use a RDM because you have a NFS Datastore. What kind of SAN/NAS do you use?

  12. By Mike, February 27, 2009 @ 18:09

    Strange, So if you don’t use a standard (or Bigsmp) kernel it’s no problem? Which Operating system do you use? SLES10 and which Service Pack, than I will try it at home here.

    You are right that you cannot use a RDM because you have a NFS Datastore. What kind of SAN/NAS do you use?

  13. By Wouter Mignon, March 3, 2009 @ 18:50

    No, iSCSI works just fine with de default kernel. I get “FATAL: Module iscsi_tcp not found” when trying to start iSCSI with the VMI enabled kernel. (http://forums.novell.com/novell-product-support-forums/suse-linux-enterprise-server-sles/sles-install-boot/360579-sles-10-sp2-vmi-kernel-iscsi.html)

    Please let me know if you can reproduce!

    We are using NFS on a NetApp FAS2050.

  14. By Wouter Mignon, March 3, 2009 @ 18:50

    No, iSCSI works just fine with de default kernel. I get “FATAL: Module iscsi_tcp not found” when trying to start iSCSI with the VMI enabled kernel. (http://forums.novell.com/novell-product-support-forums/suse-linux-enterprise-server-sles/sles-install-boot/360579-sles-10-sp2-vmi-kernel-iscsi.html)

    Please let me know if you can reproduce!

    We are using NFS on a NetApp FAS2050.

  15. By Mike, March 10, 2009 @ 14:45

    Hi,

    I have no problem connecting a iscsi initiator from SLES to my freenas. Have you run the latest patches from Novell? I Guess so. It’s strange that the give a error on tcp?
    You can connect using tcp not?

  16. By Mike, March 10, 2009 @ 14:45

    Hi,

    I have no problem connecting a iscsi initiator from SLES to my freenas. Have you run the latest patches from Novell? I Guess so. It’s strange that the give a error on tcp?
    You can connect using tcp not?

  17. By jackfrost, May 13, 2010 @ 04:26

    Unfortunately if you move the starting block to 128 after rebooting, the oes2 server won’t detect your pools, much less your volumes. So your data will be wiped out. This is not good advice.

    I did not notice a difference (before rebooting) after adjusting to 128. Thus far I’m finding all this talk about partition alignment (at least for the guest partition) is overblown.

    • By Mike, May 17, 2010 @ 10:27

      Hi Jack,

      Your right, this worked on OES2 sp0 but as off sp1 this trick doesn’t work any more. I’m currently working on a new presentation voor the Novellcongres in the Netherlands how to allign nss on OES2 sp2. If you want I can send you a copy of the presentation.

  18. By jackfrost, May 13, 2010 @ 04:26

    Unfortunately if you move the starting block to 128 after rebooting, the oes2 server won’t detect your pools, much less your volumes. So your data will be wiped out. This is not good advice.

    I did not notice a difference (before rebooting) after adjusting to 128. Thus far I’m finding all this talk about partition alignment (at least for the guest partition) is overblown.

    • By Mike, May 17, 2010 @ 10:27

      Hi Jack,

      Your right, this worked on OES2 sp0 but as off sp1 this trick doesn’t work any more. I’m currently working on a new presentation voor the Novellcongres in the Netherlands how to allign nss on OES2 sp2. If you want I can send you a copy of the presentation.

  19. By Andre, October 27, 2010 @ 09:35

    Hey Mike,

    I’m interested in that presentation ๐Ÿ™‚

  20. By Andre, October 27, 2010 @ 09:35

    Hey Mike,

    I’m interested in that presentation ๐Ÿ™‚

  21. By Rasmus, November 3, 2010 @ 09:47

    Hi Mike,
    I’m also very interested in that presentation on how to align nss on OES2 sp2 ๐Ÿ™‚

    Thanks in advance..

  22. By Rasmus, November 3, 2010 @ 09:47

    Hi Mike,
    I’m also very interested in that presentation on how to align nss on OES2 sp2 ๐Ÿ™‚

    Thanks in advance..

RSS feed for comments on this post. TrackBack URI

Leave a Reply

You must be logged in to post a comment.