Home Posts


duplicity with Amazon S3

Reality: Online storage generally isn’t cheap

That is, until Amazon’s S3 came onto the scene last year. I’d heard of duplicity and knew it had backend support for S3, so I figured that it couldn’t be too hard to setup. Unfortunately, I couldn’t find a place online that ties everything together, so I’ll try to do that here.

Background: Amazon’s S3 (Simple Storage Service) is cheap, available storage. How cheap? $0.15 each GB-month, and $0.20/GB transfer. duplicity gives you “encrypted, bandwidth-efficient backup using the rsync algorithm.” Well, good enough for me. Let’s start.

On my Ubuntu Linux 6.10 (Edgy Eft) machine, I first had to get librsync-dev:

sudo apt-get install librsync-dev

Then, I got duplicity-cvs and installed it:

cvs -z3 -d:pserver:anonymous@cvs.sv.gnu.org:/sources/duplicity \ co duplicity

cd duplicity dist/makedist 0.4.3 tar xzf duplicity-0.4.3.tar.gz cd duplicity-0.4.3 python setup.py install –prefix=/home/tkho</blockquote> If you install into your home folder, make sure to set the PYTHONPATH:

export PYTHONPATH=/home/tkho/lib/python2.4/site-packages
For duplicity to use the S3 backend, you need the BitBucket library. Once downloaded, install it:
tar xzf BitBucket-0.4a.tar.gz cd BitBucket-0.4a python setup.py install ??prefix=/home/tkho
Now, everything is installed so we can start a sync. On a side note, it took about half an hour for my S3 account to start working.
export PASSPHRASE=password export S3KEY=your_s3key export S3SECRET=your_s3secret duplicity /home/tkho/sync s3+http://duplicity_sync
That’s it! Add that to a daily or weekly cron and you have cheap, off-site backups. And, since you’re using duplicity, you get snapshots for free!

Lastly, don’t forget to grab Jets3t Cockpit–it’s a GUI file browser for your S3 data.

Thomas Kho - 25 Jan 2007



The opinions expressed on this site are mine and do not necessarily represent those of my employer.