duplicity-team team mailing list archive

[Angus Gratton <gus@xxxxxxxxxxxxxx>]

Angus Gratton has proposed merging lp:~angusgr/duplicity/exclude-older-than into lp:duplicity.

Requested reviews:
  duplicity-team (duplicity-team)

For more details, see:
https://code.launchpad.net/~angusgr/duplicity/exclude-older-than/+merge/247922

Hi,

I have a small change that I've been using for over a year now for my own backups, and I realised I should probably submit it back. I'm not sure if you'll want to include it, but it's been useful for me.

The change is an "--exclude-older-than" command line option, that allows you to only back up files with a modification date newer than a particular threshold.

How I personally use it is for my off-site backups. Every couple of months I swap a physical hard disk off-site which contains a full disk backup via duplicity. However every hour or so I run an incremental backup to S3 covering my most used directories. The S3 backup has the option --exclude-older-than 90D passed to it.

Provided I rotate the "large" off-site backup more regularly than every 90 days, I can expect that between the two backups I have a complete set of the data that matters to me. If I backuped everything up to S3 then it'd be expensive and clog up my poor low-bandwidth upstream ADSL link, but a lot of it is data that hasn't changed in months.

I don't know if this kind of configuration will work for anyone apart from just me, but I thought you might find the option interesting anyhow.

Thanks very much for duplicity btw, it's a great tool that has saved me several times now!


Angus
-- 
Your team duplicity-team is requested to review the proposed merge of lp:~angusgr/duplicity/exclude-older-than into lp:duplicity.

=== modified file 'bin/duplicity.1'
--- bin/duplicity.1	2015-01-26 16:08:57 +0000
+++ bin/duplicity.1	2015-01-29 01:01:26 +0000
@@ -510,6 +510,15 @@
 come before any other include or exclude options.
 
 .TP
+.BR "--exclude-older-than " time
+Exclude any files whose modification date is earlier than the specified
+.IR time .
+This can be used to produce a partial backup that contains only
+recently changed files. See the
+.B TIME FORMATS
+section for more information.
+
+.TP
 .BI --exclude-other-filesystems
 Exclude files on file systems (identified by device number) other than
 the file system the root of the source directory is on.

=== modified file 'duplicity/commandline.py'
--- duplicity/commandline.py	2015-01-26 16:08:57 +0000
+++ duplicity/commandline.py	2015-01-29 01:01:26 +0000
@@ -316,6 +316,10 @@
     parser.add_option("--exclude-regexp", metavar=_("regular_expression"),
                       dest="", type="string", action="callback", callback=add_selection)
 
+    # Exclude any files with modification dates older than this from the backup
+    parser.add_option("--exclude-older-than", type="time", metavar=_("time"),
+                      dest="", action="callback", callback=add_selection)
+
     # Whether we should be particularly aggressive when cleaning up
     parser.add_option("--extra-clean", action="store_true")
 

=== modified file 'duplicity/selection.py'
--- duplicity/selection.py	2015-01-22 20:04:16 +0000
+++ duplicity/selection.py	2015-01-29 01:01:26 +0000
@@ -245,6 +245,8 @@
                     self.add_selection_func(self.other_filesystems_get_sf(0))
                 elif opt == "--exclude-regexp":
                     self.add_selection_func(self.regexp_get_sf(arg, 0))
+                elif opt == "--exclude-older-than":
+                    self.add_selection_func(self.exclude_older_get_sf(arg))
                 elif opt == "--include":
                     self.add_selection_func(self.glob_get_sf(arg, 1))
                 elif opt == "--include-filelist":
@@ -644,6 +646,24 @@
         else:
             return exclude_sel_func
 
+    def exclude_older_get_sf(self, date):
+        """Return selection function based on files older than modification date """
+
+        def sel_func(path):
+            if not path.isreg():
+                return None
+            try:
+                if os.path.getmtime(path.name) < date:
+                    return 0
+            except OSError, e:
+                pass # this is probably only on a race condition of file being deleted
+            return None
+
+        sel_func.exclude = True
+        sel_func.name = "Select older than %s" % (date,)
+        return sel_func
+
+
     def glob_get_prefix_res(self, glob_str):
         """Return list of regexps equivalent to prefixes of glob_str"""
         glob_parts = glob_str.split("/")