tech stuff.

OSError with duplicity 0.6.19 on OpenBSD and OS X

leave a comment »

Sometime around duplicity 0.6.15 (ish) I started running into OSError exceptions that I just didn’t have time to track down. I’ve finally made time, though, and it wasn’t too hard to track down the culprit. I didn’t realize it at the time, but this only affects non-privileged users running duplicity. tl;dr choose a different TMPDIR.

First, a statement of the problem:
% duplicity -vw --no-print-statistics --full-if-older-than 1M --include-globbing-filelist /home/user/.backups.list --exclude /home/user /home/user file:///home/user/.backups
Traceback (most recent call last):
File "/usr/local/bin/duplicity", line 1394, in
File "/usr/local/bin/duplicity", line 1387, in with_tempdir
File "/usr/local/bin/duplicity", line 1263, in main
File "/usr/local/bin/duplicity", line 1060, in sync_archive
File "/usr/local/bin/duplicity", line 1009, in copy_to_local
File "/usr/local/lib/python2.7/site-packages/duplicity/", line 617, in move
File "/usr/local/lib/python2.7/site-packages/duplicity/", line 443, in copy
File "/usr/local/lib/python2.7/site-packages/duplicity/", line 448, in copy_attribs
util.maybe_ignore_errors(lambda: os.chown(, self.stat.st_uid, self.stat.st_gid))
File "/usr/local/lib/python2.7/site-packages/duplicity/", line 65, in maybe_ignore_errors
return fn()
File "/usr/local/lib/python2.7/site-packages/duplicity/", line 448, in
util.maybe_ignore_errors(lambda: os.chown(, self.stat.st_uid, self.stat.st_gid))
OSError: [Errno 1] Operation not permitted: '/home/user/.cache/duplicity/07adf1b6a7475a5bd7cc218f415304f8/duplicity-full-signatures.20121010T000001Z.sigtar.gz'

Why would a chown() be failing?  It turns out it’s because of the BSD semantics for new file creation in which a new file inherits the GID of the directory in which it was created. e.g.

[user@host]:~% cd /tmp
[user@host]:/tmp% ls -ld
drwxrwxrwt 7 root wheel 512 Nov 13 01:18 .
[user@host]:/tmp% id
uid=1002(user) gid=1000(group) groups=1000(group)
[user@host]:/tmp% touch look_at_my_gid && ls -lh look_at_my_gid
-rw-r--r-- 1 user wheel 0B Nov 13 01:19 look_at_my_gid

Duplicity attempts to explicitly copy file attributes when importing a file from the temporary directory to duplicity’s cache directory. I’m a little unclear of the utility of this feature unless the same code path is used to copy files being backed up and restored. There’s no way to disable that behavior, but a decent work around is to use a tmp directory with a GID matching that of the user running duplicity. Since duplicity is using tempfile this can be accomplished by modifying the environment, but you could also use the –tempdir argument. I’ve added the following to all of my duplicity scripts:

export TMPDIR=/home/user/.cache/tmp

Bug Report

Written by Lee Verberne

2012-11-21 at 12:17

Posted in BSD, OS X

Tagged with , , ,

Leave a Reply

Please log in using one of these methods to post your comment: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: