Rolling backup on the cheap

Thu 28-Jan-2010 in english, geek, sysadmin | Please leave a comment!

Within, i’ll present a free and low-pain solution to implement a backup copy method for Windows using an external hard disk. The same method could also be used for backups over the network.

A user at a customer of mine needed a way to copy his documents to an external disk which is easy and cheap. While it would be possible to use Windows backup, it’s not the nicest of programs to work with (he’s on Windows XP, the backup software on Win7 is probably much nicer), so i decided against it.

My requirements were:

  • Simplicity – easy to use for the user
  • Unobtrusive – doesn’t require complex installs to the computer which may be against the company IT policy
  • Open – doesn’t lock out the user if the backup program fails or goes out of date
  • Maintainable – even if i went away, somebody else could update and maintain the system

So with some painful research, i ended up with the Toucan backup Portable App. In fact, i had done an installation like this before but with less elegance, which is to say that i will here spare you from some lack-of-elegance. Not bad.

The whole method is based on example code from the Toucan help files.

Step 0: A wee bit of theory (won’t hurt … much)

We’re going to create two backup routines. One will create a full backup of a source directory onto a target directory on a removable disk. The other one will create an archive containing all files that have changed since the last full backup. Both of these are created with Toucan’s differential backup. Five full backup files will be kept and automagically cleaned out when a full backup is performed. Everything is configurable and probably also schedule-able.

Step 1: Preparation

The first thing to do is to give the external hard disk a persistent mapping. With the external hard disk plugged in, right click My Computer, choose Manage, select the Disk management tool. Right click the external disk, choose Change Drive Letter and Paths and select a nice and backup-friendly letter, say Q.

Then, get the Toucan Portable App. Toucan portable is designed to run within the PortableApps framework but it’ll work nice by itself. By design, that means it will run without making any changes on your system, and we’ll use that to actually run Toucan from the external disk itself. If you want the PortableApps framework, go ahead. It won’t hurt. Much :)

Install Toucan on the external disk, Q:. Due to the PortableApps framework, it’ll install in some directory structure underneath the root of Q. Navigate to the Toucan executable and run it.

Step 2: Configure what to back up

The Toucan user interface is a bit scary, but don’t worry. I’ll keep you company until we’re ready to run. Click on the Backup tab. Click the big plus-sign button in the Job Name box to create a new Job. Give the job the name Full backup. In the Type box, select Differential (which may seem misleading but bear with me).

From the big area on the left, select one directory (or even one whole disk, but that’s going to be a lot to backup) you want backed up. I suggest you choose a reasonably small hierarchy to start with, otherwise the testing phase will take some time. Press the plus-sign button in the middle of the screen to have that directory added to your backup list. Unfortunately, Toucan doesn’t support differential backups on multiple source directories. If you want that, you’ll need to repeat this article multiple times. But there are worse pains than that.

In the Backup Location text box, enter @backupfolder@\ (we’ll get to that shortly – oh, and don’t miss that backslash \ at the end of @backupfolder@ as it’s probably important).

Press the Save button which is in the Job Name box.

Step 3: The automagic bits

Click the Variables tab. Click the plus-sign button to create a variable. Name it backupfolder. You’ll get two lines of text in the big box below, one being your computer’s name. Double click that one and enter Q:\backup (or @drive@\backup which would be the cooler and more portable notation). Click the save button.

Click the Script tab. Press the plus-sign button and name a script Backup-rotational. Paste the following into the edit window:

Delete "@backupfolder@\BaseFile-5.zip"
Rename "@backupfolder@\BaseFile-4.zip" "@backupfolder@\BaseFile-5.zip"
Rename "@backupfolder@\BaseFile-3.zip" "@backupfolder@\BaseFile-4.zip"
Rename "@backupfolder@\BaseFile-2.zip" "@backupfolder@\BaseFile-3.zip"
Rename "@backupfolder@\BaseFile-1.zip" "@backupfolder@\BaseFile-2.zip"
Rename "@backupfolder@\BaseFile.zip" "@backupfolder@\BaseFile-1.zip"
Backup "Full backup"

Press the save button.

Yeah, i know it’s ugly, but the Toucan scripting language is just about that developed. It does get worse though.

Anew, press the plus-sign button and create another script. Call it Diff-backup. The only code it will have is:

Backup "Full backup"

Press the save button.

Step 4: Intermediate testing

Still within the Script tab, select the Backup-rotational script and press Run. You should get a few warnings that there aren’t any BaseFile-n.zip files to delete or rename but the backup bit should work fine. The jolly magic here which we couldn’t really influence is that when Toucan runs a differential backup but there is no file to “different against”, it will save the full backup into the file BaseFile.zip.

A reasonably big hierarchy will backup in 15 minutes, a smaller one in a minute or so. If there were severe errors, check your code. If it matches mine, there must be a bug in my code, which you should remark about in the comments section below.

When the Backup-rotational script has run, choose the Diff-backup script and run that. If you want to, you can make some changes to the source hierarchy before running the Diff-backup to see some reality in the process.

Step 5: Enter Batman

You’ll still need two batch files to make the whole magic run. In the directory where Toucan.exe is installed, create the following two files with the contents below:

do-full-backup.cmd

del Q:\backup\20*.zip Toucan Script "Backup-rotational"

do-diff-backup.cmd

Toucan Script "Diff-backup"

The sad bit is that you need to delete the incremental files from the batch file, as Toucan doesn’t expand wildcards (caveat: this script only works in the 3rd millennium Gregorian time – if you’re reading this in another time zone, please edit your script to suite).

Run the two batch files. Watch the output and observe what happens in your backup directory.

Step 6: Shortcuts or schedules

Add shortcuts to your user’s desktop or set a schedule using your favourite cron replacement. Educate said user to run those shortcuts on a regular basis.

Step 7: Restoring files (this should never happen)

In case Bad Things happen, go to the backup directory of your external hard disk. Check out the BaseFile.zip (or an older BaseFile-n.zip if you realize the Bad Thingness only weeks later) or the relevant timestamp-named file if the Bad Thing just happened. Navigate and restore. Take a bow.

You’re done.

Tags:

The iPad is real (finally)

Wed 27-Jan-2010 in english, geek, irrelevant | Please leave a comment!

After much speculation and a lot of waiting, The Steve Jobs Magic Factory has released the iPad. After all, i did suggest – heck, request – the iPad already in December 2007. I’m sure Steve will want to deliver me a slate in person when he has one manufactured. You know, for my suggestion/request which must have been the source of his inspiration. And for the name i suggested. Right, Steve?

And i’m kinda buggered that i didn’t register ipad.com back then just in case he’s forgotten about me now :)

Tags: , , , ,

Meanwhile, at the posterity office…

Wed 6-Jan-2010 in english | Please leave a comment!

I created a brain dump at Posterous.

And i wrote this entry two weeks ago. Strange that i didn’t publish it then.

Tags:

Wanted: Networked ePaper photo frame

Wed 6-Jan-2010 in english, geek | 1 comment

I just realized what was wrong with digital photo frames. The fact that they shine, like monitors do. They emit light to display a picture.

If they would require light to show a picture, much like a printout, they would look a whole lot more natural. And the answer to that is to use e-paper. Colour e-paper to be specific. It doesn’t even have to be touch sensitive, though that would be a bonus. I’m just not sure if touch sensitive electronic paper is invented yet. Could be. Should be.

So if somebody out there just got a terrific business idea with this, the least you can do is send me a few networked epaper photo frames for making you stinking rich. Thank you.

Tags: , , , , ,

Networked photo frames are coming

Tue 5-Jan-2010 in english | Please leave a comment!

Digital photo frames in a home environment are … well, almost neat. Sure, they can be cool eye catchers in commercial environments but in my aestetic, they still are a wee bit tacky in homes. Maybe i’m just old fashioned, but i think that art is physical, photos are static and monitors sweeping and cross-fading are swooshy (in the bad sense). But most of all, i think they are inconvenient. The way to get photos on the frame is to stick some media onto them. The way to change pictures on them is to stick some other media into them. And the way to change pictures at the grandparents’ places is to remember to stick the new media into the frames when you visit them.

This is also the reason i love the Slickr screen saver, which loops photos from my Flickr contacts on my screen. That is the kind of digital photo frame i can appreciate. Not only because it doubles as the computer display i work on, but most of all because it’s my contacts who put their pictures on it. In real time. Without any extra effort from either them or me. Heck, most of them probably do not even realize that they feed my frame — it’s that easy.

For quite some time, i’ve been waiting for a networked photo frame, that’s nifty, affordable and grandparent-usable. Buy it, config it once (until they change their WLAN, but you’ll be there when that happens anyway) and plug it in. Presto, there be pictures. Sure you can do it by recycling a laptop (or PDA, or why not one of those tablets), but that will with most certainty fail in at least one of the three requirements specifications above.

But i see light in the end of the tunnel. A company called PF Digital has the gadget eStarling TouchConnect, a wireless photo frame with a touch interface. Currently the available update mechanisms are RSS, Flickr, Picasa, Twitter, Facebook, Google Calendar. Oh, and and email. Which just screams to be spammed by Viagra and pr0n ads (now that would be funny, granny). I haven’t read through the photo frame manual yet (yeah, photo frames come with manuals these days) but if you can activate many sources at the same time, we have something of a winner on our hands. One feed per grandchild’s parents in our case. And feeds to the calendars where you want the grandparents to see the grandkids.

The US$200 price tag is approximately twice the price i would want to cough up for a 10″ 800*480 pixel gadget but that’s the Early Adopters’ Tax for you, my friend. In a year from now, at least the specs will have come up. And at least the market has now been opened.

Tags: , , , , , ,

Enter Aylee

Mon 28-Dec-2009 in english, geek | Please leave a comment!

…or “Installing Debian and the Coherence UPnP media server on a Linksys NSLU2 NAS thing”.

My two Slugs Bun-bun and Kiki are getting a new companion, Aylee. Aylee is a shape-shifter by nature, which means she is running Debian.

Getting Debian on the Slug was surprisingly uncomplicated. I first booted the off-the-shelf Slug. Using ping -b 192.168.1.255 i figured out it was using the “standard Slug IP address” 192.168.1.77. Using its web interface (which still was running the old R24 firmware), i sent it the Debian installer and waited. A few minutes later, the installer was on the Slug, which then booted.

The next step was to ssh installer@192.168.1.77. The password is install. This will start the actual installing process, which will get all the freshest Debian files for the Slug from o’er the Internets. I chose all the easiest and blankest defaults with the only added spice that my Slug would also be a file server. This comes late in the process from Tasksel. The whole installation process takes a number of hours to complete, which was a reminder from the days of old when installations, well, took hours.

The installation process also asked which hard disk it should use and format. I had gone through the extra work of formatting it on the off-the-shelf NSLU2 interface, but this was unnecessary.

During the installation, i was recommended to install ntp or ntpdate. So when the Slug finally had done its installing magic and rebooted (which it does automagically after it “cannot stress enough” the importance about rebooting), i ran apt-get update and apt-get upgrade. Much to my surprise, my system was already up to date. Take that, Windows :) .

The next step was to apt-get install ntpdate. This installed ntpdate but didn’t seem to configure it, so i had to do some manual labour.

cd /etc/cron.hourly
cat > ntpdate
#!/bin/sh
ntpdate fi.pool.ntp.org > /dev/null
^D
chmod +x ntpdate
run-parts --verbose .

Replace fi in fi.pool.ntp.org to whichever country you’re in, or just leave the country bit out (i.e. just write ntpdate pool.ntp.org) for the automagia to do its thing. At ^D, press Control-D. run-parts --verbose . will run the scripts in the current directory (you saw the ., right?) and report how things went. It was this way i realized that the script ntpdate needs to start with the magic line #!/bin/sh and that it must be made executable with chmod +x. You can leave out the > /dev/null bit to begin with and if you get an hourly email from root that ntpdate has adjusted the clock with zero point zero something seconds, everything works as it should and you can add the > /dev/null bit which will silently keep your Slug in time and not give you more email.

My aim with Aylee is to have it as a photo server. As a challenge, i’m going to use the Coherence UPnP server for this. If all goes pear-shaped, i should still be able to re-flash and shape-shift Aylee back into something easier to handle :)

Coherence runs on Python and Python is already on the base Debian installation. There are a number of ways to install Coherence. One is using aptitude, but that will install an old version of Coherence. Another option is to manually install all the dependencies. Not fun. And the Simple way is to use EasyInstaller which in itself first must be installed. For that you need to get setuptools for your version of Python (say python --version to your slug to find out) and run the downloaded file as a script, i.e. sh setuptools-version-py2.x-egg.

At this time, i thought i would be installing Coherence (easy_install Coherence) but ran into a dependency problem. And i thought easy_install would take care of just those. Pfft. I was missing the packages Twisted and Twisted.Web and was suggested to install them. Not knowing exactly how, i said easy_install Twisted. This looked promising for a moment until i was informed the easy_installer was missing gcc, the GNU C compiler..

Duh. This was going to take some time. Compiling stuff on the Slug? Not my idea of fun.

After successfully installing Coherence 0.5.8 with aptitude instead (aptitude install python-coherence) i became a bit disappointed that the offered version was about one year old. Also, i could not get my photos to show on XBMC using UPnP though they showed okay on my Samsung telly. So it was back to the documentation. According to it, the dependencies can be installed with apt-get as well:

apt-get install python-twisted-core
apt-get install python-twisted-web
apt-get install python-configobj

You could also install the dependencies with easy_install, which i only read after having installed the above packages with apt-get. Not that it should matter much.

After that, i could  finally get the latest greatest Coherence installed using easy_install Coherence. About time, i say :)

There were a bunch of warnings during the installation, but a coherence --version at least confirmed that the software did install.

Finally, i installed rsync so i could copy the jpeg versions of my photos from my laptop to Aylee. Which it is doing currently. It’ll take a while. Unfortunately, the kids are now watching Moomins from the telly, so i can’t test my XBMC now. But i have hopes. And the hardware.

Tags: , , , , , ,

Cisco ASDM Upgrade from the Command Line

Mon 21-Dec-2009 in english, sysadmin | Please leave a comment!

After having locked myself out from the graphical user interface goodness of the Cisco ASA, i needed to Set Things Straight again. Rebooting (or reloading in Cisco lingo) the firewall would of course only reload the firmware into the upgraded 8.0.x version which won’t let me in. The Cisco upgrade example documentation happily mentions that one can use TFTP to do the upgrading bit if the GUI seems too easy. Or unavailable.

But there’s a huge chasm between “you could do this” and “here’s how you do it”. So here’s how i did it.

0. Get the ASA and ASDM images from Cisco

I could write a whole rant about this because it’s a nightmare getting the software updates from Cisco. Why can’t they be like other vendors and just distribute the updates to the customers who have bought their hardware? Anyway, if you managed to lock yourself out with an ASA update, you probably have the ASDM software handy as well.

Trivial FTP

TFTP is not FTP. TFTP is a simpler file transfer protocol, joyously dubbed Trivial File Transfer Protocol. Windows comes with a TFTP client, but no server. Linux comes with both. And if you’re on a Mac, you already know more than i.

In this posting, i will assume you have ASDM 6.2.3 handy, and that it is saved as asdm-623.bin

1. Locate a TFTP server software

I decided to get the Open TFTP server from Sourceforge. WinAgents has an Industry Strength TFTP server which has an installer package of 24 megs, which is a bit overkill for a one shot installation. Jounin.net has a nice and graphical tftp server i’ve used before, but i missed it when googling. The OpenTFTPd installer is only 173 kB which was nice.

Go get it.

2. Configure it

The Open TFTP Server installs in c:\Program Files (x86)\OpenTFTPServer by default, which is also where the configuration file OpenTFTPServerMT.ini is. To edit this, you need to have Admin privileges. I’m (still) on Vista, so i pushed the Windows button, wrote Command line, right clicked that on the Start menu and chose Run as Administrator. Yeah i know there was some meta-alt-shift-something to do the same thing but i’ve forgotten the chord. Now edit the ini file (notepad will do) and enter the directory where your ASDM image is under the section [HOME].

I went and #commented out all lines starting with an ‘apostrophe just to be sure there’d be no bugs.

2½. Know your IP address

Type ipconfig (and scroll up) to see what your IP address is. Mine is 10.10.42.4.

3. Run it

Run the RunAsStandAloneMT.bat file from the admin command line window. Running the .exe file from the command line will just ask you to unblock the TFTP service and exit. No fun.

4. Suck the image

If you haven’t got a terminal connection to the ASA, now’s the time. Connect that nice blue flat cable between your serial port and the ASA console. Use PuTTY (or whatever that terminal thingy that comes with Windows is called, if you must) and connect to COM1 with 9600 bps, 8-N-1.

Tap enter a few times, log on if you need, and enter the following magic words, remembering to breathe normally:


ena
tap your “enable password”
conf t
copy tftp://10.10.42.4/asdm-623.bin disk0:asdm-623.bin

At this stage (dumb) ASA will re-ask what all the parameters you just entered above were, and then proceed to…

Accessing tftp://10.10.42.4/asdm-623.bin…!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! (a lot of exclamation marks omitted here!)

Writing file disk0:/asdm-623.bin… !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! (yeah)

Finally, type


asdm image disk0:/asdm-623.bin
wr

…whereby the firewall answers something along the lines of

Building configuration…
Cryptochecksum: 058305fa 13371597 acdcafb8 gabagaba

5368 bytes copied in 1.440 secs (5368 bytes/sec)
[OK]

Take a deep breath.

Write a blog post.

reload

Tags: , ,

Cisco ASA Upgradefail

Mon 21-Dec-2009 in english, sysadmin | 1 comment

Grrrr. Sometimes you should just go by your hunch. I was in the process of updating a Cisco ASA 5505 firewall from software version 7.x to 8.0 according to the instructions from Cisco, using the ASA management (“ASDM”) software that came on the firewall.

Versioning?

To confuse the novice firewall administrator, the ASA has one series of version numbers which has absolutely nothing in common with the ASDM version numbers.

Anyway, my ASA was at 7.2.4 going to 8.0.5 and my ASDM was on 5.2.4 and was eventually going to be upgraded to 6.2.3.

I was really wondering if the old ASA management software (“ASDM”) would be able to manage the newer ASA software, but the instructions were in the order of first upgrading the ASA software, then reboot, then upgrade the ASDM. So i follow the instructions, upgrade, select the proper boot image, reload, fire up the (old) ASDM and…

Boom. I’m stranded.

Now i can either make a careful guess on how to get to the right boot image using the command line or try and upgrade the ASDM image using TFTP. I really don’t fancy either option….

Anyway, here’s my humble suggestion if you want to upgrade your ASA: start with the ASDM. It Just Might Work™.

Tags: , , , , ,

How to render a HP server useless (and back again)

Wed 16-Dec-2009 in english, sysadmin | Please leave a comment!

I had a very tense few hours with a customer’s server yesterday. The fact that it’s a Small Business Server and thus, the “Everything Server”, didn’t make things much better. I did two things, and both turned out to be bad. I also didn’t reboot between the two things, which also turned bad to be even worse.

One. I installed the new service pack, which is a Good Thing (generally), except when the computer hangs at “setting up, stage 3 of 3, 0% ready” and spins the little circle thingy for half an hour. At that stage the “please do not turn off your computer” becomes stressful to ignore. So i leaned on the power button, chose to restart in Safe mode and everything seemed okay. For a while.

Two. I changed the network adapter to traffic at 1 Gb/s full duplex. This turned out to be catastrophic. And i fully blame HP for this. After a reboot into normal mode, i had no network. At all. And i was not able to open the HP network interface control panel thingy, since the “management database” was locked. Not even netsh would help me this time.

After much stressful head scratching and beard tearing, i hypothesized that HP NIC management is grumpy because it was in fact plugged into a switch that only goes to 100 Mb/s. Yeah, i can appreciate that it can’t traffic with the wrong line speeds but that i can’t turn that setting off is criminal. If that indeed was the case. So i plugged the server’s NIC into a backline giga-Ether switch (yeah, you shouldn’t do that either) and rebooted. And hey presto, the “management database” was no longer locked.

Back to 100/full, plug the server where it belonged, and normality is restored. Just in time to go and fetch the kids. Sysadmin feat in true Hollywood style.

I just wonder what those HP engineers were thinking about.

Tags: , , , ,

Minimal bginfo

Mon 14-Dec-2009 in english, geek | Please leave a comment!

Update: The VBscript code i had was both long and buggy. The new code is short and sweet, and at least works no less than the previous code.

BGinfo is a nifty piece of software which can print out a whole lot of technical information on the desktop background of a Windows box. As an administrator for a bunch of client machines, BGinfo has proven Most Useful.

There are two issues, however. Sometimes the information i use on my backgrounds can be a bit over the top. And then there’s one little bit of info not included in the admittedly colossal BGinfo arsenal: whether the computer needs rebooting after having been updated. So here’s my fix.

Step Zero is to download BGinfo from the link above and save it anywhere that can be addressed over the Windows network during a logon procedure. I chose the domain controller’s Netlogon share, or \\%LOGONSERVER%\NETLOGON in the examples below. In reality, i used the real name of the logon server instead of %LOGONSERVER% but i suppose the variable name will work just as well. You might need to add %-signs for added magic.

I then created a minimal BGinfo template with just the hostname, IP address and a custom field i call Is Reboot Required. The template uses the user’s own default wallpaper and the BGinfo data is aligned to the top right of the window. Your mileage may vary. Save the template with the BGinfo executive. My path is \\%LOGONSERVER%\NETLOGON\bginfo-minimal.bgi

The custom field Is Reboot Required points to the output of a certain is-reboot-required Visual basic script, saved with above two files as is-reboot-required.vbs:


If CreateObject("Microsoft.Update.SystemInfo").RebootRequired Then
Echo "Reboot required"
End if

Old code. Don’t use:

function readFromRegistry (strRegistryKey, strDefault )

Dim WSHShell, value
On Error Resume Next
Set WSHShell = CreateObject("WScript.Shell")
value = WSHShell.RegRead( strRegistryKey )

if err.number <> 0 then
readFromRegistry= strDefault
else
readFromRegistry=value
end if

set WSHShell = nothing

end function

str = readFromRegistry( "HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\PendingFileRenameOperations", "no" )
if( isNull( str )) then
msg = ""
else
msg = "Reboot required"
end if

Echo msg

What the script does is check whether (Windows Update, usually) requires some files to be renamed during the next reboot cycle. This information is stored in the PendingFileRenameOperations registry key. If it’s non-empty,If our computer’s Microsoft Update client deems a reboot is required, we emit the administrator-friendly message “Reboot required”, otherwise we just shut up (having a “Reboot not required” message on the wallpaper isn’t what i call good usability).

Disclaimers: This script works when plugged in but not when run on the command line, oddly enough. And, i’m no VBS guru. The script was created by creative copy-pasting from other resources on the ‘Net.

To paste things together, i created the following one-liner batch file bginfo-minimal.cmd:


\\%LOGONSERVER%\NETLOGON\bginfo.exe \\%LOGONSERVER%\NETLOGON\bginfo-minimal.bgi /timer:0 /nolicprompt

Finally, i added \\%DOMAINCONTROLLER%\NETLOGON\bginfo-minimal.cmd in the startup scripts. Since this happened a week ago, i can’t remember if i did it through Group Policy or through the Administrator’s logon script or (ungh) through the Startup group in the Start menu but in any case it works. If i did it the Right Way (through Group Policy), that means i had to create a new Organizational Unit “Wizards”, add a custom group Admins, add Domain Administrators to it, create a new Group Policy to the Wizards, and apply the bginfo-minimal.cmd from the right path to that group, for that is the way of Windows Server 2003. But then again, i might just have been lazy.

Tags: , , ,

« Older entries § Newer entries »

Bad Behavior has blocked 747 access attempts in the last 7 days.

Bear