Wednesday, 5 February 2014

XenServer 6.2 - installed on a USB drive and installed on an SD Card

In this post I want to talk about recent experiences I've had with XenServer 6.2 on USB drives and SD cards. In a particular situation, I was forced initially down this path due to specific hardware and lack of appropriate drives. The upshot is that it was entirely unsuccessful and here is why:

Firstly, it's important to be aware that XenServer isn't all that happy being installed on these devices. After you install, you must boot back up on the XenServer CD and then (using ALT-F2) drop to a command line. To get XenServer to boot on USB do the following:

  • From the command line do this: 
    • # cat /proc/partitions
      • this tells you what partitions there are that XenServer can see. Typically you will see /dev/sda and it's children - /dev/sda1, /dev/sda2 etc. We want /dev/sda1
    • # mkdir /target
      • create a temporary location so we can change our root directory
    • # mount -t ext3 /dev/sda1 /target
      • mount our existing /dev/sda1 partition to /target, giving us access to the files
    • # mount proc /target/proc -t proc
      • we need a proc filesystem!
    • # mount sysfs /target/sys -t sysfs
      • and a sysfs
    • # mount --bind /dev /target/dev
      • and of course somewhere for devices to be found
    • # chroot /target
      • change to the /target install of the system (chrooting means we are running now under the actual XenServer install, not the boot cd live filesystem)
    • # cd /boot
    • # ls –al
Now we need to actually tell the kernel of our installed XenServer to add USB support to it so it'll boot. Up until this point, the system will just hang, unable to find a filesystem.

  • # mv initrd-lots.of-numbersxen.img initrd-lots.of-numbersxen.img.old
    • this backs up the initrd ing
  • # mkinitrd --with-usb initrd-lots.of-numbersxen.img lots.of-numbersxen.img
    • make sure you replace lots.of-numbers with the actual kernel version. It could look like this: 2.6.18-53.1.13.el5.xs4.1.0.254.273
  • # exit
  • # sync
  • # reboot
At this stage we are finished and the server will reboot and actually get into XenServer. After patching the pool, or indeed, the server by itself, you may need to apply the initrd update again, but with the new kernel number.

While this was cumbersome to do, particularly because I had to do it six times - 3 for initial install, 3 again after SP1 was applied, it wasn't hard to do, or particularly time consuming. I found that I had to do this for both USB drives and on the SD Card. The SD Card was significantly slower than USB. The key difference between the two was that the USB, after some magical, indeterminate time, would lose it's filesystem! XenCenter would lose connectivity to it, and the console interface was non-functional - unable to find "root"! A reboot fixed this, but these are supposed to be on all the time with minimal service loss. That was when I turned to the SD Card idea. These servers had been running an awful install of VMware 4.1 on SD Cards so I thought they would just work. Sadly, it was not to be. Once again, I'd lose connectivity to the server, but this time, I was able to get the xsconsole to respond and reboot the machine. Totally unacceptable though. I don't have time to test with this system - it's supposed to be production! At any rate, I've purchased disks and installing them. If anyone has experienced this and knows the answer - I'd love to hear it!

12 comments:

  1. Hi ryv's,

    I have the same problem with you.. I did exactly your step.
    I've been looking for answers in the citrix forums and there is no solution for this problem.

    And finally, installing xen6.2 to hardisk is the final solution.

    I think the big problem was here :
    # mv initrd-lots.of-numbersxen.img initrd-lots.of-numbersxen.img.old
    # mkinitrd --with-usb initrd-lots.of-numbersxen.img lots.of-numbersxen.img

    Because when xenserver 6.2 installed on the hard drive, we don't have to do that thing. And xenserver goes well.

    Maybe you have an idea?

    Thanks,
    wallsdroid

    ReplyDelete
  2. Hey wallsdroid,

    see if you can disable any USB power timeouts or anything. I found that it was fairly random as to when the USB died on me, but it was usually during a quiet time. Unfortunately on the servers I was working on, there wasn't really anywhere to modify the USB power settings. I haven't had any issues with the hard drives so I've been disinclined to continue trying to figure it out :-)

    Good luck - hopefully someone else reading my blog has something to offer.

    ryv

    ReplyDelete
  3. Hi folks.

    This has helped me f***ing lots.

    I did went through an error, though I could fix it. Hope this helps future doubts:

    When recreating the initrd, the last bit of it does not use the ".img" part.

    # mkinitrd --with-usb initrd-lots.of-numbersxen.img lots.of-numbersxen.img << this .img shouldn't be here.

    It went all right with:

    # mkinitrd --with-usb initrd-lots.of-numbersxen.img lots.of-numbersxen

    Anyways, thank you so very much bro. Was struggling with this for some week now.

    Cheers!

    ReplyDelete
  4. Samuel SS - glad it helped out and you got it to work. Hope it's super stable. I should note, that on disks I've yet to have a failure at this particular site - over 4 months!

    ReplyDelete
  5. Hi,
    i am step by step is ibm x3650 m4 hypervisor usb on installation.
    Boot xen 6.2 bootable and F2 "shell" enter
    then
    # cat /proc/partitions
    # mkdir /target
    # mount -t ext3 /dev/sda1 /target
    # mount proc /target/proc -t proc
    # mount sysfs /target/sys -t sysfs
    # mount --bind /dev /target/dev
    # chroot /target
    # cd /boot
    # ls –al
    # mv initrd-2.6.32.12-0.7.1.xs5.6.100.323.170596xen.img initrd-2.6.32.12-0.7.1.xs5.6.100.323.170596xen.img.old
    # mkinitrd --with-usb initrd-2.6.32.12-0.7.1.xs5.6.100.323.170596xen.img 2.6.32.12-0.7.1.xs5.6.100.323.170596xen
    # exit
    # sync
    # reboot

    and machine reboot but erro hypervisor usb boot failed

    please help me.

    Thank you

    ReplyDelete
  6. Any idea if these steps will help with getting XenServer 6.2.0 working with a USB ethernet adapter?

    ReplyDelete
  7. Matt S - not sure. The issues I had with the USB device not responding were probably more linked to the fact that the data off the USB drive was loaded in to memory and wasn't being accessed off the drive for long periods. A USB to Ethernet adaptor would be more or less active all the time. If you have a crack at it let us know how you get on :-)

    ReplyDelete
  8. hi Ümit, your issue here doesn't lie in your install but the fact that your server will not accept to boot on a usb disk bigger than 4G, if you feel bold though you can very well trick it into installing a xenserver on a 4G drive.

    step by step, install on plain 16G+ key, taking care to use mbr partitionning and specifiy 3.5G for root drive size in your constants.py , normal install no local vm storage. Then, boot on a gparted CD, copy your /dev/sdx1 partition to your smaller key (4G) ignoring the backup partition. boot on your xen install cd with your usb key plugged in, use the mkinitrd procedure specified above, don't forget to edit extlinux.conf and update your boot sector. voilà! one xenserver install on less than 4G drives.

    ReplyDelete
  9. Hi,
    I've just found your page and tried the initrd --with-usb and it doesn't work for me. it said something like "unable to find module" ...
    any ideas?

    and when I reboor know it's all broken. I've the "boot: " , it's starting with:
    Loading /boot/xen.gz... ok
    Loading /boot/vmlinuz-2.6-xen... ok
    Loading /boot/initrd-2.6-xen.img... failed!

    did I miss something ??

    Thanks in advance for your help
    Regards
    Erix

    ReplyDelete
    Replies
    1. Hi Erix,

      have you got the full file path and name in the initrd changes you are making? There may well be a typo in your commands - check them very carefully and run it again. Tab complete is your friend :-)

      -ryv

      Delete
  10. Hello,
    thanks very much for your reply. I'm travelling at the moment and don't have access to my lab, but as soon I'm back, I'll look at this and I'll post/send you a reply. Do apologize for the delay.
    Really appreciate your help.
    Have a nice day.
    Regards
    Erix

    PS: Back on Monday with more news I hope

    ReplyDelete
  11. This looks interresting!
    Can i ask you if your still running xenserver on a USB? and for how long?

    ReplyDelete