Front page

Cloning a obnam repo safely

80b02d8361594a7ea0b32ff2c3883334
MERIT PHONETIC BUTTON

From: "Robin H. Johnson" <robbat2@gentoo.org>
Date: Fri, 17 Jun 2016 08:45:38 +0000

   I'd like to be able to safely clone an entire obnam repo (~3TB), and I'm
   wondering how best to go about it such that it's still consistent, but
   without blocking too much new client traffic (ideally no need to stop
   clients even).
   
   BorgBackup has an append-only mode, wherein almost all content is
   written to new files only (the only file that's actually appended is the
   transaction log). This makes it comparatively easy to replicate: just
   keep running rsync (no --delete needed), optionally ignoring any files
   that are presently writing.
   
   Filesystem snapshots are only sufficient if it's possible to temporarily
   tell obnam to give a temporarily safe state for the snapshot to be
   taken.
   obnam take-global-lock && \
   take-file-system-snapshot && \
   obnam drop-global-lock
   
   Depending on your filesystem, it might be useful to temporarily deny new
   files from being opened entirely while the global lock is in progress.
   
   Relatedly to this, are all of obnam's writes into the repository done in
   an atomic manner (upload to temp filename && rename into place)?
From: Lars Wirzenius <liw@liw.fi>
Date: Fri, 17 Jun 2016 11:54:57 +0300

   On Fri, Jun 17, 2016 at 08:45:38AM +0000, Robin H. Johnson wrote:
   > Filesystem snapshots are only sufficient if it's possible to temporarily
   > tell obnam to give a temporarily safe state for the snapshot to be
   > taken.
   > obnam take-global-lock && \
   > take-file-system-snapshot && \
   > obnam drop-global-lock
   
   Obnam doesn't have that functionality currently. You could fake it by
   creating the necessary lock files manually. You could write a plugin
   to add the functionality. See the RepositoryInterface.lock_everything
   method.
   
   > Relatedly to this, are all of obnam's writes into the repository done in
   > an atomic manner (upload to temp filename && rename into place)?
   
   Should be. See the write_file method in LocalFS and SftpFs.