Pages

Thursday, January 8, 2015

XenServer 6.2 update woes

Recently I was required to update a number of XenServers that I've taken over. They were running 6.2SP1 but were about 6 updates behind. No problem, I'll just queue the updates up and shoot them over. Had 2 Windows VM's (2008R2 and 2012 server) and 5 Ubuntu 12.04LTS Linux Servers. The updates were fairly straight forward - I followed my own guide :-)

After the final update was completed I booted up the VMs only to find the two Windows Servers came up no problems, and none of the Linux servers booted. None of them. They started up, black screen and white cursor in the top left hand corner and that was it. No boot no errors, just gone.

I ran through some basic repair work and had no joy. I changed the GrubConf.py from if arg.strip() == "${saved_entry}": to if arg.strip() == "${saved_entry}" or arg.strip() == "${next_entry}":. Still no luck. I re-installed Grub - no joy. I booted from the Ubuntu server CD and tried to run rescue the broken install. On several of the VM's I couldn't even get the system to run a shell in the / file system (/dev/xvda1 typically on these setups). It was incredibly frustrating and after three weeks of banging my head against this, I've still not managed to get this all going.

Luckily I had backups of most of the servers and so I haven't lost everything but I have lost some critical data. The message here, dear friends, is multiple:

  • test the updates one at a time to make sure everything starts up. Yes it will take longer and it's a bit more frustrating, but believe me, you want to know if things are going to shit themselves.
  • backups backups backups - are not important. RESTORES! are the important part. Believe me, I cried into my beer when I couldn't get one of these machines back up and going - it just wasn't going to happen. I lost a wiki with data that wasn't anywhere else and that's caused me no end of grief. The extra work it's caused is phenomenal and I feel very bad about it indeed.
I've reached out to the collective wisdom of SAGE-AU (www.sage-au.org.au) but sadly no luck yet. I'm open to suggestions!

elementary OS Review

I came across elementary OS on LifeHacker I think and thought it looked pretty interesting. I've been looking for a lightweight operating system that's reliable and fairly full featured (I accept it won't be totally featured - that's the cost of lightweight) for use on various older laptops and the like.

elementary OS is a free download with donations as optional. I downloaded it and burned it to a CD. Alas my favourite test laptop has gone the way of the dodo and is never to return (much like the dodo). I found a HP Pavilion D6 floating around, replaced the hard disk (it was toast) and the RAM (also toast) with 6GB. It's an i3 which doesn't really qualify as a lower end machine, but what the hey.

elementary is based on Ubuntu linux and is quite heavily customised running a lightweight desktop called Pantheon. The current version of elementary is called Luna and it's nice to see they are in development of new versions and it seems like an active development environment. Midori is available for Internet access - I installed Chromium as that's my preferred web browser. There is a calendar app, Geary Mail (which I haven't used), Shotwell for photo organisation and Empathy to tie in with Jabber, Facebook etc.

The interface is slick and looks good and the laptop boots a rocket. All the apps are fast and updates etc are easy as pie. There is an application for updates which is simple and straightforward and the whole thing is well organised and designed. It's quite a joy to use actually. The application install package - Software Center - looks good and is straightforward to use, offering a large range of packages. All the stuff I like to use I was able to install and get on with the job. I have a pretty small requirement list though, in the interests of full disclosure. Chromium has to work, I have to be able to burn CDs and DVD's and I need terminal to be fully functional. Pretty, fast and stable are the other three musts for any OS and I find elementary to check all those boxed.

I now run Mint and elementary side by side on laptops and desktops - the speed difference is quite noticeably between the two, and with their ultra stable Ubuntu (Debian) ancestry it's lovely to use. I've enjoyed using elementary OS Luna and I'll be keeping it on my Pavilion for the future - well as long as the Pavilion holds together!

Sunday, November 2, 2014

Google Sheets - inserting a static variable

Ever had a spreadsheet with set variables and wanted to use them in a formula but not have to type them over and over - just use the drag down feature and have it work? For example, if I had a formula where I wanted A3 to be present as part of it, and when I try to apply that formula to multiple cells the A3 becomes A4, A5, A6 etc? That's very annoying and up until now I've gone through and manually corrected it.

Not any more! By putting a $ sign in front of the column ID and the row ID e.g. $A$3 - it won't change! It stays the same! OMGWTFBBQ!

How long have I looked for this - I found the answer here: http://www.gcflearnfree.org/googlespreadsheets/14.3 - read up for more info! Yay!

Wednesday, September 24, 2014

How to restore a file with StorageCraft ShadowProtect

I've installed ShadowProtect on most of my clients' servers - it's a great product and if you're not using it for backups, then seriously consider it. One of our sub-contractors emailed me with some issues on restoring files so I thought I'd add my reply to him here as a quick cheat sheet:

  • Log onto the server you need to restore the file from
  • Open up the share where your backups are going
  • Browse through the list of files and look for an .cd or .cw or .cm file around the correct date
    • -cd.spi is a consolidated daily
    • -cw.spi is a consolidated weekly
    • -cm.spi is a consolidated monthly
    • for a full listing see here: http://bit.ly/1mNLBps
  • once you've found the correct file, right click on it and choose ShadowProtect Mount
  • pick the defaults, except when it comes to the right date - the consolidated files have a list of possible days / weeks that you can choose - find the right one and click on it, then go Next
  • mount the file as read-only unless you need read-write access
  • the computer will mount the drive as a new drive letter
  • browse through that drive until you find the file you want to restore, then copy and paste it to the right location and that's it
  • unmount the backup image and all finished! 

It's important to note this is only one of two ways of doing it. You can use the wizard that is part of ShadowProtect and it's even easier. At the time I was in a hurry and had to find multiple awful files so I used this method, plus I find that I use this method for Granular Recovery for Exchange restores.

Friday, August 8, 2014

Restoring Windows Sharepoint Services 3.0 - from disaster to victory beer!

Recently during a server upgrade I applied SP3 to Windows Sharepoint Services 3.0. This particular server had seen no love in a long, long time and it needed an absolute slew of updates. Naturally, Sharepoint broke and the site loved by my client was unavailable, as were many other services.

The errors in the Eventlog were varied and painful, with lots of vague references to the apocalypse and the like. Naturally the logs get incredibly dense and I had another issue to contend with along the way - disk corruption. The ntfrs filesystem was reporting corruption and had taken out a chunk of the Sharepoint Wizard's configuration. That obviously had to be fixed first and was very worrisome - especially given I was working on a RAID1 disk.

Normally, because the database needs an upgrade when you apply SP3, if it doesn't start straight up you can run the SharePoint Products and Technologies Configuration Wizard to repair it. Failing that, you can disconnect the farm, fix the database issues and then re-run the wizard and connect back to the farm. With the disk issues and also with the failure of the systems admin to fully apply all the updates none of this was working - in fact the Wizard was failing spectacularly.

This is where things got to from my notes:
  • Ran the config wizard and told it to disconnect from the server farm per MS documentation
  • re-ran config wizard - it is now reporting that IIS is not working properly.
  • have looked in to this - suggestion is that a compatibility mode setting has not been applied. Unable to apply this in Windows Server 2003.
  • have run a repair on WSS 3.0 - this requires a reboot
  • many many ASP.NET 2.blah errors. All non-descriptive and very dense to understand without being a .NET programmer.
So we were right up that fabled creek without a paddle. I finished patching the system, which sorted out the issues with ASP.NET. I still had no connectivity to SharePoint so I ran through some more updates and managed to partially get the SharePoint Admin site up. I was still getting all sorts of errors and came across a post that suggested I change the ASP.NET version of the Admin site to 2.0.whatever. You can get to this via the IIS management tool, right click on the website, go to ASP.NET and edit the configuration, altering it to the version you want. I did this and it made no difference, but after restarting IIS the admin site came up. Awesome sauce. There were also a few permission changes I needed to make - the Network Service account had somehow lost access to the content database.

I had a backup of the all the WSS databases, and the databases themselves were actually running on the server still. What I didn't realise and what I hope you, gentle reader, can take from this, is that the restore was far easier than I thought. I removed SharePoint from IIS, and created a new web application. I also created a new site collection and new database. From here I went to Content Databases and added in the old content database but I still couldn't get the right site to come up. In fact, the old content DB and the new one conflicted and I had no access to anything. What I should have done was this (all through the WSS Central Administration Site)

  • create a web application
  • in Content Databases add the old content database - you may have to use the stsadm command to do it which is:
    • stsadm -o addcontentdb -url http://server -databasename WSS_Content (which is the default name)
  • Check under Site Collection List - you should see your old website application there
  • restart IIS and check the site.
Where I had a lot of pain was that I didn't realise the old site was held within the WSS_Content database and I didn't need to add a new site or create a new site collection. How remarkably painful is all I can say. I hope in future that it'll be a bit easier during upgrades.

Tuesday, July 15, 2014

Upgrading DragonFlyBSD

I always forget how to do this, so I'm documenting it here. The DragonFlyBSD website is quite good and this all comes from www.dragonflybsd.org/docs/newhandbook/Upgrading

Firstly, make sure the Makefile is present:

# cd /usr
# make src-create

and wait while it does it's thing.

Then we need to get the new source to build it:

# cd /usr/src
# git checkout DragonFlyBSD_RELEASE_3_8 (which is the current one)

To find out what the current one is:

# cd /usr/srv
# git pull
# git branch -r

Then the build and upgrade process:

# cd /usr/src
# make buildworld
# make buildkernel
# make installkernel
# make install world
# make upgrade
# reboot

And it should all be done.

Monday, June 9, 2014

Adventures with Crashplan for backups

Recently through the excellent SAGE-AU (www.sage-au.org.au) I read about Crashplan. Produced by Code42 and found here: http://www.code42.com/crashplan/ there were lots of positive comments about it. I've since deployed it in two separate locations - in the office and at home. I'm using the free implementation at the moment, which allows you a backup each day to a variety of places. They include a 30 day trial of their cloud backup solution - which is quite cheap for a home implementation - $165 / year for 2 - 10 computers. Check out the full pricing - but see what you can do with the free version:-

At the office we have a straight Microsoft Windows based environment - Windows 7, Server 2008 R2 and a wee Windows 8 here and there. I've set up a Crashplan account using a single email address and installed it on almost all our machines. I have a Windows 2012 Server running in our virtual environment and I'm using it as the base for all the backups to go to. I added a 2TB virtual disk to it, configured Crashplan and started pointing machines back to it. It's working brilliantly! As they say though, backups are optional - restores are mandatory. Since implementation I've had to run three separate restores, from all sorts of weird files to basic word documents and it's run flawlessly!

At home I've been messing with it too. I've installed it on my Linux Mint desktop which runs all the time, and has an NFS share back to my FreeNAS. I've set up Crashplan to use that location for backups and I have the wife's Windows 7 laptop, my MacBook Air and my Windows 8 PC all backing up to that location now. Totally cool! Crashplan has installed and worked on all the machines without any issues, complications or anything. It's excellent!

Emails are sent from Crashplan to notify you if machines are backing up properly or haven't backed up for a given amount of time and this is very handy. Our offsite techs are frequently away for days and as soon as they get back, their laptops start automatically backing up. It's the easiest implementation I've found so far.

Check it out http://www.code42.com/crashplan/ it's awesome!