Skip to main content

XenServer 6.2 update woes

Recently I was required to update a number of XenServers that I've taken over. They were running 6.2SP1 but were about 6 updates behind. No problem, I'll just queue the updates up and shoot them over. Had 2 Windows VM's (2008R2 and 2012 server) and 5 Ubuntu 12.04LTS Linux Servers. The updates were fairly straight forward - I followed my own guide :-)

After the final update was completed I booted up the VMs only to find the two Windows Servers came up no problems, and none of the Linux servers booted. None of them. They started up, black screen and white cursor in the top left hand corner and that was it. No boot no errors, just gone.

I ran through some basic repair work and had no joy. I changed the GrubConf.py from if arg.strip() == "${saved_entry}": to if arg.strip() == "${saved_entry}" or arg.strip() == "${next_entry}":. Still no luck. I re-installed Grub - no joy. I booted from the Ubuntu server CD and tried to run rescue the broken install. On several of the VM's I couldn't even get the system to run a shell in the / file system (/dev/xvda1 typically on these setups). It was incredibly frustrating and after three weeks of banging my head against this, I've still not managed to get this all going.

Luckily I had backups of most of the servers and so I haven't lost everything but I have lost some critical data. The message here, dear friends, is multiple:

  • test the updates one at a time to make sure everything starts up. Yes it will take longer and it's a bit more frustrating, but believe me, you want to know if things are going to shit themselves.
  • backups backups backups - are not important. RESTORES! are the important part. Believe me, I cried into my beer when I couldn't get one of these machines back up and going - it just wasn't going to happen. I lost a wiki with data that wasn't anywhere else and that's caused me no end of grief. The extra work it's caused is phenomenal and I feel very bad about it indeed.
I've reached out to the collective wisdom of SAGE-AU (www.sage-au.org.au) but sadly no luck yet. I'm open to suggestions!

Comments

Popular posts from this blog

Windows 10 Enterprise Eval - gotchas

After an annoying turn of events where my Windows 10 Enterprise USB drive failed, attempts to install Win10 onto a computer failed miserably. I turned to the net and managed to get my hands on Microsoft's Windows 10 Enterprise Evaluation. I have an enterprise key so I thought - cool! Here's the opportunity to get it going and to then upgrade the license later. Full install, patched etc and all is swell. Except when I try to upgrade. I straight up tried changing the licence key only to get a variety of errors, most of which are pertaining to the activation system being unavailable. The I try this: https://winaero.com/blog/upgrade-windows-10-evaluation-to-full-version-easily/ but it doesn't work either. Next I'll try this: h ttp://www.edugeek.net/forums/windows-10/174594-upgrading-windows-10-enterprise-90-evaluation-full.html And if all else fails, in goes the bootable USB I've now created. If only I'd had this in the first instance I would not be writing t

Fixing a black screen after doing a Kali Linux update

Kali Linux is a rolling Linux distribution designed for security and penetration work. You can find details on it here: www.kali.org . We run this excellent product for a range of different security work and it's been great. I built the image in VMplayer, then shared it to the team and we've all been at it since. A recent update broke it though - black screen, no network and completely unresponsive. There are lots of posts about similar things - mostly to do with graphics adaptors, however, we found that executing the following at a root prompt fixed it. But how to get to the root prompt from a blank screen? Linux has a number of terminals available to the user - most of us use the graphical one to do our day to day, but you can access a command line prompt without much trouble. Simply hold CTRL-ALT and then F2 or F3 down at the same time and it drops you to a command line login. BOOM. Time to fix it up. For me, and for the other fellas in the team, all it too was to

Plone - the open source Content Management System - a review

One of my clients, a non-profit, has a lot of files on it's clients. They need a way to digitally store these files, securely and with availability for certain people. They also need these files to expire and be deleted after a given length of time - usually about 7 years. These were the parameters I was given to search for a Document Management System (DMS) or more commonly a Content Management System (CMS). There are quite a lot of them, but most are designed for front facing information delivery - that is, to write something, put it up for review, have it reviewed and then published. We do not want this data published ever - and some CMS's make that a bit tricky to manage. So at the end of the day, I looked into several CMS systems that looked like they could be useful. The first one to be reviewed was OpenKM ( www.openkm.com ). It looked OK, was open source which is preferable and seemed to have solid security and publishing options. Backing up the database and upgradin