Andrew Breese

Musings of a professional geek

Tag Archives: Backup

Managing backup, cheaply

For years I’ve had no real backup strategy at home. Files saved into various cloud providers like DropBox, Google, etc are one thing which might appear to be backup-ish but they really are not a backup tool at all. Those are file sharing and synchronisation for the sake of easy access. In other cases the family was using a combination of removable hard disks and usb memory keys. In our house different files are in different “cloud” places, which makes finding them a mess and managing it difficult.

I decided that my holiday mini-project was to get a backup solution for the house.

Goals and considerations:

  • Applications should have a simple interface. Be usable by a non-geek to back-up their personal files.
  • The ideal solution should have multiple backups locations and hopefully different styles of backup. This means copying the same data into several different places, and consider using cloud or NAS or portable drives in combination.
  • Ideally be automatic, or at the very least able to be scheduled or controlled.
  • Be something my family can use, and something that I can maintain without too many headaches.
  • Due to the size of my data, a cloud solution is problematic. However some of my family might be able to use a cloud service, as they have a much small amount of data to backup. eg. My core set is around 180 gig with a fair amount of extra stuff I’d like to add to it, compared to my partner’s data which is around 12 gig.
  • Backup system should perform incremental backup, especially if this is to run over slower connections; like broadband or wifi. A very large part of my home’s data does not change often at all, and I want the software to alter the backups it has already made, not do the entire data block again.
  • USB memory drives are now hated and won’t be used as any type of backup. They are too easy to loose and fail too often.
  • Support for Windows is mandatory, anything else (Macs and Unix or blah blah phones) is a nice to have. I’m not a SysAdmin anymore so I no longer have a linux server chugging under my desk.

So in a bit more detail…

Simplicity is needed as I’ll most likely be configuring this, but in the event that I need to recover the data from a back-up I want something that I can walk somebody else through. I’m also likely to be called upon to “fix” a backup related issue on short notice and with a time deadline, so something that performs most of the work auto-magically (I hate that phrase but it suits) is important.

Multiple backup styles is important because any single style of backup might be corrupted or fail, so having several redundant techniques is a stronger/broader approach.

Initially I’ve chosen the CrashPlan application, as it allows a very simple to configure and run interface, which is suitable for non-IT literate users, and also has options that are reasonable for IT geeks. I’ll be testing this in a local setting (phase one below).

I’ve read there are solutions with easier interfaces and potentially slightly cheaper, but from my short scan of the featureset on the top 10 vendors, only CrashPlan has remote pc to pc backup which is independent of a cloud solution. I really like this idea when I can consider how a family might interlink their computer resources to backup each other’s data.

CrashPlan is also interesting because the basic backup tasks of saving a set of profile related data to a removable drive or other computer is really easy and free. Yup, free for the basic backups. There are a wide range of backup application vendors and especially cloud backup providers now (Backblaze, Carbonite, Crashplan, etc), so do a little digging to find the vendor that suits your needs.

Lastly before I get to the detail, a note about “the Cloud” and “cloud backup“. To be frank in Australia a typical USA cloud solution isn’t viable or fast. I love the cloud as a concept & tool, and liked the idea decades ago; before server centric computing was morphed by marketing jargon into calling it “the cloud“.

Downunder we are still not able to access cheap fast cloud services, and that kills what I’d really like to do – which is just pay a monthly fee and backup every damn file, quickly. I tried it with a few providers and the speed just isn’t good enough yet. Try moving 160gig up into a cloud provider and you’ll have a task measured in weeks! Time will change that eventually.

The rest of the article is about what I’ve got working and what is next.

Phase one is getting the solution working at our house, across the important computers. Not hard, a bit time consuming, and needs to be watched over a few weeks to know for sure.

Phase One – Get it working locally.

This means get the backup working locally to the home where the PCs are used. This was a simple task of installing the application on each PC and then pointing them at a backup location. e.g.

  • Laptop 1 is an older unit with very little free hard disk space, and also has a very small backup amount (12 gig). So that unit pushes data to Laptop 2 which has far more free space.
  • Laptop 2 needs to save about 180 gig of data somewhere, and has a local disk with 250+ gig free. So it pushes it’s data to a removable drive.
  • Ideally both computers would have a fair amount of free space and they could backup to each other, but such is life. When that older laptop is retired then I’ll use the new one in the same way.

This has been in place for a week now and seems to be working. I’m watching with interest how the different systems connect to each other, how much bandwidth they demand when running, and also how much cpu the solution churns when running and in background.

My hope is that the backup solution will “just queue a retry” when the appropriate destination isn’t present. It seems to, but time will tell. A month is reasonable shake down time.

Interesting and useful that CrashPlan sends a periodic email summary to me of where it is up to. I like that.

Phase Two – Get it working with a NAS, where the NAS is the destination for the backup.

Specifically, I want it to backup onto the NAS, and I’m not at all trying to backup a NAS. Backing up a NAS via a Windows program running on a separate PC, across a wifi network, … is a nightmare scenario with too many moving parts. Most of the articles linked below are addressing how to mount and backup a NAS drive to elsewhere using the CrashPlan client.

Instead I’m looking to use the NAS space as the place to drop the backups. CrashPlan does not do that easily either, but the work around for  cheating with a NAS pointed me at a working unsupported solution.

Aside on NAS backups – CrashPlan does not support backup to a NAS for Windows (macs and such work fine, as it is a limitation of the way Windows OS handles services running as users and the security permissions), so yes it can go between PCs, but those PCs have to have CrashPlan installed and my NAS is a more generic consumer media drive with no WindowsOS to use. This is a limitation that hurts the product for Windows systems.

There are unsupported known work-arounds though (have a read here), which is where this step comes in. I did not use the advice in the linked article, as it uses the “Net Use” command in a windows batch file, which I don’t see as a useful approach for Windows 7 and Windows 8. After too many years using batch files to bend Windows to my ever changing and unforgiving will, I now avoid it.

It certainly might be ok in WinXP, but thankfully I only need to worry about Win7/8.

Instead I’ve used an NTFS Symbolic Link (which is akin to a shortcut, but not), however the operating system sees the connection as present and working file or volume. Essentially the user will see a folder which looksalmost normal, and then when they open it the current user session credentials are passed through to the NAs and the NAs folder will show up.

This is useful as it disguises the network share as a directory local to the computer’s operating system, which can then be used by CrashPlan for backup. It also establishes the link as something which is persisted to all parts of the OS, which means that the authentication for the connection is no different than any other share. This is handy because the batch files would have required authentication details saved within them, which I conceptually detest.

Still to do -I need to double-check that the Windows indexing service and all the associated scanning services (anti virus, etc) are set to ignore that symbolic link. I don’t want the OS to manage that area at all, just point to it.

The approach is predicated on the NAS server being online and present when CrashPlan needs to run, but as do all the approaches (duh!); you’ll always need the location to be online.

The mklink command syntax needs to:

  • run the command as Admin, which on Win8 might be a UI challenge in itself. Find it in the GUI then right click to run with elevated permissions.
  • use the command switch to indicate it is a directory, using “/d”.

mklink /D C:\temp11111 \\server\share\foldername\

After adding the link and then configuring CrashPlan on my laptop to point to the NAS – it backed up.

That was an exciting nerdy moment for me as it meant that my files were now backed up in two places. A huge tick in the redundant locations requirement. This is in place now too, although I do need to make sure the local network isn’t being crushed by all the traffic. The backup app kicks off automatically, so it could gobble up the local network unexpectedly.

Still to do – Last trick I’ve yet to do is to investigate and configure the advanced setting to disallow it run during the “busy times”. I think I’ll set a schedule from midnight to 6am where it can run unhindered.

Phase Three – Get it working cross-site.

The last step is to get an additional redundant location which is also offsite.This phase is still very much in the playground stage.

The product’s cloud solution could do this, except that it initially said the data upload was going to take 4-6 weeks continuously running! Nope, sorry. I want a backup somewhere which isn’t in the house so that theft and fire can’t affect the backup and until the online speed improve I’ve reverted back to using portable hard disks.

Aside for investigation later – There might be a way to seed a backup set onto a portable drive, move it to an offsite location, then copy that backup to the NAS or some such. I’ll dig in to that later. Might be useful for both the CrashPlan cloud backup, or even backup across the VPN between houses.

This is the tricky and complex step, as it involves setting up a VPN into my home network and then configuring the laptops to recognise the NAS as a valid CrashPlan backup location. It has the flaw of affecting both my home bandwidth and the bandwidth of the other end, and also will chew up a lot of my family’s internet allowance each month. The usage is certainly something that I’m not sure about.

There are two options, either I use my account and backup between my computers and/or I setup an account for each user and cross link the machines. Hard choice, so I’m doing both.

e.g.

  • My laptop will use my NAS, and also a removable 2TB USB3/SATA drive. I might throw a 3rd local network location in too while I figure out an external location.
  • My partner’s laptop will use my laptop, and also the NAS.
  • My mother’s two laptops will use each other (locally) at her house, and also share files into my NAS via a VPN.
  • My brothers family of laptops (4x of them) will backup onto each other in whatever mesh makes sense according to their disk space and backup size, also onto his removable hard drive, and then onto my NAS via VPN.
  • I’ll consider a family Cloud backup plan at this point too, to see if the backup is worth it, but frankly the speed from Australia isn’t great.
  • I’ve love it if each of the three sites all had a NAS which then swapped backups, but that is a dream for a more enlightened time. Maybe next year.

Another interesting factor in the choices is which VPN type to configure. PTPP is quick and simple, but not terribly secure anymore. L2TP is better (far better I’m told) but requires setup of keys, then OpenVPn is great and I use it a lot personally but I’ve never set it up before from the “end” side.

I got the PTPP styel VPN working at home with my router as the end point in a few hours, opened a firewall port, and then closed it all up again after reading more about just how hackable PTPP is.

And this is where I am now, muddling around with reading about OpenVPN vs L2TP config, and how that might be performed on my router, or more likely through my router straight to a host (yikes!).

It’s nice to have a technical challenge hobby project again.

Last comments

I’m feeling far happier with just a basic backup being performed automatically regularly at home. Even without an off-site solution this is far better than nothing. The VPN questions will take a long time to get solved, and likely be a tech support challenge for the other users.

I wish I’d purchased far higher end gear for my media server and the hardware in the laptops, as it would have made this a little simpler, but my choice at the time was to limit cost and not be doing SysAdmin tasks at home. It means the next set of hardware will likely be mid range gear, and avoid the home user stuff.

…So if you are after a home backup solution consider CrashPlan’s free offering. All you need is either some diligence or an extra pc in the house to swap backups with; and the configuration is not hard.

%d bloggers like this: