ShadowCraft ShadowProtect - Employee of the Month

The last two weeks have been absolutely awful. The IOmega NAS I mentioned in a previous post had a large number of things go wrong with it all at the same time, resulting in complete failure and total data loss. This also meant a significant amount of down time for a client and ridiculous amounts of stress for me.

The good thing to come out of it was the performance of ShadowProtect. This backup solution far exceeded my expectations to the point where once the recoveries were completed we had only lost 10 minutes of work. Very little data was realistically lost, only time as we worked to recover it. The Recovery Environment was easy to use and the Hardware Independent Restore allowed me to recover virtual machine from VMware 4 ESX to VMware Player, a physical machine and also to XenServer 6.2 quite a remarkable feat really. I've had this client running in borrowed hardware for a week while we get new gear in.

The snapshot capability, ease of restore and simply replication of backup files makes ShadowProtect a real winner. Check it out! 

Experiences with the IOmega ix12 NAS

One of my clients has an IOmega ix12 - a rack mounted, 12x3.5" bay NAS. This one in question is running 2.6.5.something firmware and has (or should I say had) 8 x 1TB disks in it. Today it dropped another one. The NAS has been set up with a RAID10 with total storage of 2TB (1.5 or so in reality). Although it has this redundancy, I can assure you it's not all that it's cracked up to be. In the main storage pool is a number of iSCSI drives shared out with various ESX servers. Sadly, the largest and most important of these iSCSI drives has an unfortunate tendency to vanish whenever there is a disk issue.

Today one of the drives failed. No big deal right? RAID10 is tolerant of such faults usually. This one kept going except for the aforementioned large drive that disappeared like magic. After replacing the disk, impatiently waiting for it to rebuild and then attempting to reconnect the iSCSI drives I realised I still couldn't see it. The fix? Turn the NAS completely off, count to 30 and turn it back on again. Once it booted, just like magic everything reappeared. Be wary of this. My previous post was the precursor to this. Finding out the actual issue is still to be completed.

I'm concerned about upgrading the firmware because the backups, while reliable, are never straightforward to recover and I'd hate to have to recover so many disks. Its a job for next week I reckon.

VMware nightmares

After a great couple of days away I was called urgently to work - one of my client's networks was down and my colleague was stuck. The VMsphere client couldn't see the hosts, which couldn't see their datastores and the network had zero stability. Yay. Just wanted I wanted to come home to.

The servers are running ESXi 4.1 that needs a few updates but the networks stability has always been a real issue for us. We took it over from some other chaps, solving a long series of issues with the servers and particularly the IOmega NAS. Throw in a database issue on one of the guest servers that kept the disk IO at absolute peak all the time and things were pretty tricky. All those things have been resolved, more or less, and the network had been fully functional for some months. So why did it change? Several reasons and I hope you take away from this some ideas for yourself.

The NAS appeared to have dropped a disk and while it has a RAID10 set up, it paralysed the system (not at all like the FreeNAS I've talked about before). The whole thing fell over and we shut it down before reseating the disk and powering back up. The NAS detected the disk and began a rebuild. The VMware hosts couldn't see the datastore though and the server managing the virtual environment periodically disconnected from everything. Initially we thought it was a network issue and restarted everything. The hosts were slow to come up, with errors in the logs indicating an inability to see the iSCSI disks. We could connect to the NAS via it's web interface, ping it and it looked quite happy on the network so we couldn't understand what was happening. An added complication was we have a new HP NAS on the network and while we were able to migrate several of the hosts to it, we've had problems getting then started. Don't know why, but the host's cpu goes through the roof everytime we try to start them. I thought we might have an all paths down bug and most of the documentation suggests a call to VMware and let them sort it out. At 8pm at night this isn't so great a plan, and with the client losing production time and money we had to solve it.

So with all these errors and problems left and right I was at a loss. Eventually we took inspiration from The IT Crowd and turned it all off, counted to 10 and started it back up. Would you believe that it took 3 reboots of the management server before the VMsphere client would connect to anything - especially considering it was running locally! The iSCSI shares from the NAS became available finally - there was a service issue on the iOmega NAS that was failing silently. It was alleging that all was good and it really wasn't. Now those shares were available we were able to reconnect to the datastores and boot the guest machines. The management server was still disconnecting from the hosts constantly but we were able to at least get things going. There is still quite a bit to do there but the servers are finally running.

Going forward I think we'll use XenServer and NFS shares. Simpler, fully functional and quick. Easy to backup and expand disks. Adios VMware I reckon.

The final thought is "Have you turned it off and back on again?" Something in that for all of us :-)

The pure awesomeness of Google Forms

As a Google Apps reseller it never fails to surprise me how many people love Forms. I swear, I could nearly sell Google Apps to people just based on Forms. If you've never used it, here is a quick run down of what it can do for you:

• create a web form very quickly and easily, choosing from a range of different responses including:
• time (duration and date)
• multiple choice
• checkboxes
• create separate pages based on responses
• autofill a Google Spreadsheet
• automatically create a summary of responses with a range of different graphs and information instantly
• embed it into websites
• email notifications of changes and updates
• make you more popular with friends (OK maybe not this one!)

Some examples of how I use Google Forms:

• PC Audit
• capture make / model, RAM, CPU etc
• Job Audit
• bookings for jobs
• completion of job information
• Incident responses

Seriously they are awesome and if you're trying to do any type of statistics about stuff, they are a terrific place to start. Make sure though that your questions are good and that you choose the right type of responses to get the right Summary of Responses. It's cool though - you can change it even with live data in the spreadsheet. I have taught a friend how to work Google Forms in under 15 minutes - it is that straightforward. Enjoy!

HOW TO: POP3 on Windows 8

I have a few clients still clinging to POP3 mail accounts and the newest version of the Windows Mail client doesn't work with it. Instead you will need to install the desktop version of Windows Live Mail. This can be found here: http://download.live.com

When you grab it, only install the mail component and when that is done, go into Start, find the Windows Live Mail shortcut and set up your beloved POP3 account. Enjoy

ownCloud - your in house cloud file service

As usual, a client of mine requires something out to of the ordinary - they love Dropbox and Drive, using them extensively at home, and now want something for themselves at work. The catch is - the work they do requires their data to be housed in Australia. Some legal requirement I gather for their funding. After searching through the archives in the mailing lists for SAGE-AU (www.sage-au.org.au) for wisdom and nuggets of gold I stumbled across ownCloud. Several of their members expressed an admiration for this software so I thought I'd check it out.

You can find out a lot more about ownCloud on their website - www.owncloud.org but here is a brief overview of my experiences using this application.

Firstly I installed it on a spare PC I had, after first putting Ubuntu 12.04 LTS on there. Installation was straightforward, securing it a bit less so - there's a bit of work to be done there but it's reasonably well documented and then I installed the application on my Windows 7 laptop. It all just worked and worked quite well. The application on my Windows 7 box works almost exactly like Drive/Dropbox - copy files into there and they sync up to my ownCloud server. They are then available from the web page - secured with SSL - and available anywhere. I've punched a hole through the firewall for testing purposes to get this available. My clients love it.

The configuration options are quite straightforward and although I did get rid of a lot of the helpful folder customisations, I can see why these would be useful to others.

In terms of load on the server it's been quite minimal so far. The PC is running quite happily, only a low end dual core box with 2GB of RAM and a 160GB hard disk drive.

The production box with be either a Dell PowerEdge T110 II or a HP N54L - depending on funding. Requirements are quite low and depending on the volume of data to be stored, either of these servers will do a great job. Cloud in the office for under a $1000!

If you have half an hour (that's all it took for me to get it going) check out ownCloud :-)

Goodbye Cisco SRP527 and hello... what?

Recently I learned the SRP527 was being End Of Life'd. This is a sad moment - we started using the Netgear DG834's to provide a low cost VPN solution between two sites, then graduated to the double the price Cisco and now I'm on the hunt again.

The issue we have is so many of our clients only need a single VPN tunnel between ADSL enabled sites. Cisco 800 series routers are too expensive and provide far more bang than the client usually needs or wants.

Currently we are investigating the Draytek range of routers. I've ordered two and I'll review them in my next post.

The SRP527's have been quite a reasonable unit for us. I've found that the wireless connectivity, particularly to Apple devices is a bit hit and miss, but it's generally provided us with a stable connection that has been 1Mb/s faster than other ADSL2+ devices I've used.

Bring on the next challenger!