duplicity-team team mailing list archive
-
duplicity-team team
-
Mailing list archive
-
Message #03020
[Merge] lp:~jmwilson/duplicity/filecaps into lp:duplicity
James Wilson has proposed merging lp:~jmwilson/duplicity/filecaps into lp:duplicity.
Requested reviews:
duplicity-team (duplicity-team)
For more details, see:
https://code.launchpad.net/~jmwilson/duplicity/filecaps/+merge/266773
Continuing the earlier discussion on using file capabilities, this will use cx_Freeze to create a binary executable for duplicity, and if the setcap command is available during package installation, then it will set inherited capabilities cap_chown,cap_dac_read_search,cap_fowner on the executable. This means if the running user has inherited these capabilities (typically through a pam module), then when they run duplicity, they will be able to read any file, and use chown and chmod on any file. No grant of privileges applies for users who do not have these capabilities in their inherited set.
I've been running these changes to do full system backups on my machine for weeks without running as root, and I also used it to restore my home directory in an emergency. I installed libpam-cap, and then added a line "cap_dac_read_search backup" in /etc/security/capability.conf. In /etc/crontab, I run duplicity as the backup user, and it inherits the permissions needed to backup everything.
Most of the changes are in setup.py to run cx_Freeze as part of the build step. I'll highlight some of the code changes to duplicity with their rationale:
bin/duplicity: Use execvpe instead of execve since argv[0] is not guaranteed to be a full path. It seems the python interpreter makes argv[0] a full path, at least on some systems. The setuid(geteuid()) incantation doesn't really accomplish anything if geteuid() == 0.
duplicity/selection.py: os.access() won't take into account either EUID or capabilities: "The check is done using the calling process's real UID and GID, rather than the effective IDs as is done when actually attempting an operation (e.g., open(2)) on the file." If duplicity is run as a regular user with cap_dac_read_search effective permissions, it can read any file, but os.access may return false. The preferred way to see if you can read from a file is to attempt to open it for reading, so instead we just catch the exception later on when we try to use the path.
--
Your team duplicity-team is requested to review the proposed merge of lp:~jmwilson/duplicity/filecaps into lp:duplicity.
=== modified file 'bin/duplicity'
--- bin/duplicity 2015-05-08 12:28:47 +0000
+++ bin/duplicity 2015-08-03 18:32:04 +0000
@@ -1302,7 +1302,7 @@
log.Notice(_("RESTART: The first volume failed to upload before termination.\n"
" Restart is impossible...starting backup from beginning."))
self.last_backup.delete()
- os.execve(sys.argv[0], sys.argv, os.environ)
+ os.execvpe(sys.argv[0], sys.argv, os.environ)
elif mf_len - self.start_vol > 0:
# upload of N vols failed, fix manifest and restart
log.Notice(_("RESTART: Volumes %d to %d failed to upload before termination.\n"
@@ -1317,7 +1317,7 @@
" backup then restart the backup from the beginning.") %
(mf_len, self.start_vol))
self.last_backup.delete()
- os.execve(sys.argv[0], sys.argv, os.environ)
+ os.execvpe(sys.argv[0], sys.argv, os.environ)
def setLastSaved(self, mf):
vi = mf.volume_info_dict[self.start_vol]
@@ -1341,12 +1341,8 @@
See https://bugs.launchpad.net/duplicity/+bug/931175
"""), log.ErrorCode.pythonoptimize_set)
- # if python is run setuid, it's only partway set,
- # so make sure to run with euid/egid of root
if os.geteuid() == 0:
- # make sure uid/gid match euid/egid
- os.setuid(os.geteuid())
- os.setgid(os.getegid())
+ log.Warn(_("Running duplicity as root is not recommend."))
# set the current time strings (make it available for command line processing)
dup_time.setcurtime()
=== modified file 'debian/control'
--- debian/control 2014-10-27 14:15:52 +0000
+++ debian/control 2015-08-03 18:32:04 +0000
@@ -15,6 +15,7 @@
python-pexpect,
rdiff,
rsync,
+ cx-freeze,
Homepage: https://launchpad.net/duplicity
Standards-Version: 3.9.5
X-Python-Version: >= 2.6
@@ -27,9 +28,11 @@
gnupg,
python-lockfile,
python-pexpect,
+Recommends: libcap2-bin,
Suggests: ncftp,
python-boto,
python-paramiko,
+ libpam-cap,
Description: encrypted bandwidth-efficient backup
Duplicity backs directories by producing encrypted tar-format volumes
and uploading them to a remote or local file server. Because duplicity
=== added file 'debian/duplicity.docs'
--- debian/duplicity.docs 1970-01-01 00:00:00 +0000
+++ debian/duplicity.docs 2015-08-03 18:32:04 +0000
@@ -0,0 +1,2 @@
+README
+README-LOG
=== added file 'debian/duplicity.postinst'
--- debian/duplicity.postinst 1970-01-01 00:00:00 +0000
+++ debian/duplicity.postinst 2015-08-03 18:32:04 +0000
@@ -0,0 +1,6 @@
+#!/bin/sh -e
+
+PROGRAM=/usr/bin/duplicity
+
+[ -e "${PROGRAM}" -a -x "${PROGRAM}" ] || exit
+which setcap >/dev/null && setcap -q cap_chown,cap_dac_read_search,cap_fowner=ei "${PROGRAM}"
=== modified file 'debian/rules'
--- debian/rules 2014-10-31 20:33:27 +0000
+++ debian/rules 2015-08-03 18:32:04 +0000
@@ -12,14 +12,16 @@
override_dh_auto_install:
dh_auto_install
-
+
# Debian installs docs itself in /usr/share/doc/duplicity/
rm -r debian/duplicity/usr/share/doc/duplicity-*
-
+
# Modify upstream's version string into the right version
find debian/duplicity -name "*\$$version*" | xargs rename "s/\\\$$version/$(UPSTREAM_VERSION)/g"
find debian/duplicity -name "*_version*" | xargs rename "s/_version/$(UPSTREAM_VERSION)/g"
grep -Rl "\$$version" debian/duplicity | xargs sed -i "s/\$$version/$(UPSTREAM_VERSION)/g"
-override_dh_installdocs:
- dh_installdocs README README-LOG
+override_dh_strip:
+ # Don't strip the executable since it will remove the attached zip
+ # archive, and the base image from cx_Freeze is already stripped.
+ dh_strip --exclude=/usr/bin/duplicity
=== modified file 'duplicity/selection.py'
--- duplicity/selection.py 2015-07-31 08:22:31 +0000
+++ duplicity/selection.py 2015-08-03 18:32:04 +0000
@@ -151,16 +151,7 @@
for filename in robust.listpath(path):
new_path = robust.check_common_error(
error_handler, Path.append, (path, filename))
- # make sure file is read accessible
- if (new_path and new_path.type in ["reg", "dir"]
- and not os.access(new_path.name, os.R_OK)):
- log.Warn(_("Error accessing possibly locked file %s") % util.ufn(new_path.name),
- log.WarningCode.cannot_read,
- util.escape(new_path.name))
- if diffdir.stats:
- diffdir.stats.Errors += 1
- new_path = None
- elif new_path:
+ if new_path:
s = self.Select(new_path)
if s == 1:
yield (new_path, 0)
@@ -447,9 +438,12 @@
def exclude_sel_func(path):
# do not follow symbolic links when checking for file existence!
- if path.isdir() and path.append(filename).exists():
- return 0
- else:
+ try:
+ if path.isdir() and path.append(filename).exists():
+ return 0
+ else:
+ return None
+ except OSError:
return None
if include == 0:
=== modified file 'setup.py'
--- setup.py 2015-02-01 17:37:37 +0000
+++ setup.py 2015-08-03 18:32:04 +0000
@@ -22,10 +22,15 @@
import sys
import os
-from setuptools import setup, Extension
+import cx_Freeze
+from setuptools import setup, Command, Extension
+from setuptools import Distribution as _Distribution
from setuptools.command.test import test
from setuptools.command.install import install
from setuptools.command.sdist import sdist
+from distutils.command.clean import clean
+import distutils.dir_util
+import distutils.log
version_string = "$version"
@@ -77,6 +82,9 @@
# And make sure our scripts are ready
build_scripts_cmd = self.get_finalized_command("build_scripts")
build_scripts_cmd.run()
+ # And make sure our executables are ready
+ build_exe_cmd = self.get_finalized_command("build_exe")
+ build_exe_cmd.run()
# make symlinks for test data
if build_cmd.build_lib != top_dir:
@@ -88,14 +96,45 @@
except Exception:
pass
- os.environ['PATH'] = "%s:%s" % (
+ os.environ['PATH'] = "%s:%s:%s" % (
+ os.path.abspath(build_exe_cmd.build_exe),
os.path.abspath(build_scripts_cmd.build_dir),
os.environ.get('PATH'))
test.run(self)
+class Distribution(_Distribution):
+ def __init__(self, attrs):
+ self.executables = []
+ _Distribution.__init__(self, attrs)
+
+
class InstallCommand(install):
+ user_options = install.user_options + [
+ ('install-exe=', None, 'installation directory for executables'),
+ ]
+
+ def expand_dirs(self):
+ install.expand_dirs(self)
+ self._expand_attrs(['install_exe'])
+
+ def get_sub_commands(self):
+ subCommands = install.get_sub_commands(self)
+ if self.distribution.executables:
+ subCommands.append("install_exe")
+ return [s for s in subCommands if s != "install_egg_info"]
+
+ def initialize_options(self):
+ install.initialize_options(self)
+ self.install_exe = None
+
+ def finalize_options(self):
+ install.finalize_options(self)
+ self.convert_paths('exe')
+ if self.root is not None:
+ self.change_roots('exe')
+
def run(self):
# Normally, install will call build(). But we want to delete the
# testing dir between building and installing. So we manually build
@@ -110,6 +149,67 @@
install.run(self)
+ def select_scheme(self, name):
+ install.select_scheme(self, name)
+ if self.install_exe is None:
+ self.install_exe = self.install_scripts
+
+
+class InstallExeCommand(Command):
+ description = "install executables built from Python scripts"
+ user_options = [
+ ('install-dir=', 'd', 'directory to install executables to'),
+ ('build-dir=', 'b', 'build directory (where to install from)'),
+ ('force', 'f', 'force installation (overwrite existing files)'),
+ ('skip-build', None, 'skip the build steps'),
+ ]
+
+ def initialize_options(self):
+ self.install_dir = None
+ self.force = 0
+ self.build_dir = None
+ self.skip_build = None
+
+ def finalize_options(self):
+ self.set_undefined_options('build', ('build_exe', 'build_dir'))
+ self.set_undefined_options('install',
+ ('install_exe', 'install_dir'))
+ self.set_undefined_options('install',
+ ('force', 'force'),
+ ('skip_build', 'skip_build'))
+
+ def run(self):
+ if not self.skip_build:
+ self.run_command('build_exe')
+ self.outfiles = self.copy_tree(self.build_dir, self.install_dir)
+
+ def get_inputs(self):
+ return self.distribution.executables or []
+
+ def get_outputs(self):
+ return self.outfiles or []
+
+
+class CleanCommand(clean):
+ user_options = clean.user_options + [
+ ('build-exe=', None, 'build directory for executables'),
+ ]
+
+ def initialize_options(self):
+ clean.initialize_options(self)
+ self.build_exe = None
+
+ def finalize_options(self):
+ clean.finalize_options(self)
+ self.set_undefined_options('build_exe', ('build_exe', 'build_exe'))
+
+ def run(self):
+ if os.path.exists(self.build_exe):
+ distutils.dir_util.remove_tree(self.build_exe, dry_run=self.dry_run)
+ else:
+ distutils.log.warn("'%s' does not exist -- can't clean it", self.build_exe)
+ clean.run(self)
+
# TODO: move logic from dist/makedist inline
class SDistCommand(sdist):
@@ -145,11 +245,20 @@
include_dirs=incdir_list,
library_dirs=libdir_list,
libraries=["rsync"])],
- scripts=['bin/rdiffdir', 'bin/duplicity'],
+ scripts=['bin/rdiffdir'],
data_files=data_files,
tests_require=['lockfile', 'mock', 'pexpect'],
test_suite='testing',
cmdclass={'test': TestCommand,
'install': InstallCommand,
- 'sdist': SDistCommand},
+ 'install_exe': InstallExeCommand,
+ 'sdist': SDistCommand,
+ 'build': cx_Freeze.build,
+ 'build_exe': cx_Freeze.build_exe,
+ 'clean': CleanCommand},
+ distclass=Distribution,
+ executables=[cx_Freeze.Executable('bin/duplicity')],
+ options={"build_exe": {"copy_dependent_files": False,
+ "create_shared_zip": False,
+ "append_script_to_exe": True}},
)
Follow ups
