My Creations

Even with today’s ultra fast fibre optic data lines, the tangible throughput people actually get when transferring files is still functionally limited. This is compounded further by the fact that the majority of uses have low upload speeds compared to higher download speeds. For example a typical home internet connection would promise 4Mbit download while only 256Kbit would be allocated for upload. To make matters worse, today files are many times larger than their ancestors.  If your hair has greyed (or fallen off) in sufficient quantities, you probably recall a time when a 1.2MB floppy disk would hold all your documents as well as the word processing program itself with space to spare. This thousand word document is over 6 times larger than that floppy disk!

In this environment, when one discusses online backups, the critical issue is to try to limit as much as possible the amount of data that has to be uploaded from the person’s computer (the source) to that person’s archive vault (the destination).   Many people would consider online backups only if the lengthy wait periods are few and far between.

On a regularly backed up system, the state of the source compared to the destination can be as follows:

• New file on source that does not exist on the destination – the file must be copied over to the archive vault;
• File on source that no longer exists on the destination – the file must be removed from the archive vault;
• File on source and destination are identical – no need to do anything;
• File on source is different from that on destination.

On a system that is regularly backed up, one would normally find few files that are new or that have been deleted. The majority of files would either have not changed or would have been altered.  As I will demonstrate shortly, the Rsync algorithm for transferring data from source to destination is greatly suited for situations in which a file has been adjusted.  There are two ways a system can handle altered files, resending the adjusted files in its entirety or simply transferring the changed pieces. The Rsync algorithm does the latter.

The Rsync algorithm was developed by Andrew Tridgell and Paul Mackerras. When taking data backup using the Rsync method results in ultra fast and efficient backups. Imagine a database, worksheet or word processing document in which the author only changed one record, cell or paragraph.  Take your PIM database; on a daily basis you receive new emails, delete junk and old mail, setup appointments and have the system remove expired ones.  Rsync backups will transmit a few megabytes of changes rather than let you wait until all modified files (that probably run into gigabytes) are uploaded.

The Rsync algorithm

Hop over to http://www.backupmyhost.com/blog/remote-backups/rsync-%e2%80%93-a-top-view-introduction/ blog for the rest of this story.