duplicity-team team mailing list archive

[Kenneth Loafman <kenneth@xxxxxxxxxxx>]

Getting it to work is the major issue.  Making it work the way it should is
kinda nice.

When you do an *'lsof -p NNNN | wc -l*', where NNNN is duplicity's pid,
you're getting a low count?

Unless something is broken, you should only have 2-3 gpg tasks, no more.

Where exactly did you break.  Did you check lsof then?

What is your command line?


On Mon, May 8, 2017 at 2:38 PM, Howard Kaye <
question631423@xxxxxxxxxxxxxxxxxxxxx> wrote:

> Question #631423 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/631423
>
>     Status: Answered => Open
>
> Howard Kaye is still having a problem:
> Increasing the number of open files by 2 orders of magnitude allowed this
> particular restore to proceed.
>
> In debugging the issue though, I tried catching the exception, doing an
> os.open() in the handler, and this succeeds, though a subsequent call to
> open() does not.
> The process does not have an excessive number of open files.
> So, the root cause of this does not appear to be actual open files, but
> perhaps a bug somewhere else.
>
> Rather than mask it by increasing the ulimit for open files, I'd prefer to
> find the root cause.
>
> On Mon, May 8, 2017 at 3:23 PM, Kenneth Loafman <
> question631423@xxxxxxxxxxxxxxxxxxxxx> wrote:
>
> > Your question #631423 on Duplicity changed:
> > https://answers.launchpad.net/duplicity/+question/631423
> >
> >     Status: Open => Answered
> >
> > Kenneth Loafman proposed the following answer:
> > That bug applies only to runs made with --no-encryption and
> > --no-compression.  He's got gpg processes, so he's not running that way.
> >
> > Also, that bug should only affect the memory use, some.
> >
> > Really should try my settings.
> >
> >
> > On Mon, May 8, 2017 at 9:58 AM, Howard Kaye <
> > question631423@xxxxxxxxxxxxxxxxxxxxx> wrote:
> >
> > > Question #631423 on Duplicity changed:
> > > https://answers.launchpad.net/duplicity/+question/631423
> > >
> > >     Status: Answered => Open
> > >
> > > Howard Kaye is still having a problem:
> > > That does not seem to be related to the open files.
> > >
> > > On Mon, May 8, 2017 at 6:43 AM, edso <question631423@answers.
> > launchpad.net
> > > >
> > > wrote:
> > >
> > > > Your question #631423 on Duplicity changed:
> > > > https://answers.launchpad.net/duplicity/+question/631423
> > > >
> > > > edso proposed the following answer:
> > > > On 05.05.2017 20:13, Howard Kaye wrote:
> > > > > New question #631423 on Duplicity:
> > > > > https://answers.launchpad.net/duplicity/+question/631423
> > > > >
> > > > > In testing a full restore, I am seeing an exception get raised for
> > "Too
> > > > many open files" from within Path.open:
> > > > >
> > > > > I am running on macOS 10.12.4, duplicity 0.7.10
> > > > >
> > > > > I have raised the ulimit for the number of files up to 12000.
> > > > > lsof shows there are only 89 open files
> > > > > sample shows a number of threads blocked on subprocesses (gpg
> > > processes)
> > > > > sysctl -a shows:
> > > > >   kern.maxfiles: 24576
> > > > >   kern.maxfilesperproc: 10240
> > > > >   kern.num_files: 5356
> > > > >
> > > > > There are currently only 17 incrementals following my latest full.
> > We
> > > > usually do monthly fulls.
> > > > >
> > > > > Is anyone else seeing this recently?
> > > > >
> > > >
> > > > there's going to be a fix in 0.7.13, replacing the use of
> os.system('cp
> > > > ...') with the native python routine, which should reduce the open
> > files
> > > > count dramatically
> > > >   http://duplicity.nongnu.org/CHANGELOG
> > > >
> > > > it's explained in more detail here
> > > >   https://bugs.launchpad.net/duplicity/+bug/1684312
> > > >
> > > > ..ede/duply.net
> > > >
> > > > --
> > > > If this answers your question, please go to the following page to let
> > us
> > > > know that it is solved:
> > > > https://answers.launchpad.net/duplicity/+question/631423/+
> > > > confirm?answer_id=4
> > > >
> > > > If you still need help, you can reply to this email or go to the
> > > > following page to enter your feedback:
> > > > https://answers.launchpad.net/duplicity/+question/631423
> > > >
> > > > You received this question notification because you asked the
> question.
> > > >
> > >
> > > --
> > > You received this question notification because your team
> duplicity-team
> > > is an answer contact for Duplicity.
> > >
> > > _______________________________________________
> > > Mailing list: https://launchpad.net/~duplicity-team
> > > Post to     : duplicity-team@xxxxxxxxxxxxxxxxxxx
> > > Unsubscribe : https://launchpad.net/~duplicity-team
> > > More help   : https://help.launchpad.net/ListHelp
> > >
> >
> > --
> > If this answers your question, please go to the following page to let us
> > know that it is solved:
> > https://answers.launchpad.net/duplicity/+question/631423/+
> > confirm?answer_id=6
> >
> > If you still need help, you can reply to this email or go to the
> > following page to enter your feedback:
> > https://answers.launchpad.net/duplicity/+question/631423
> >
> > You received this question notification because you asked the question.
> >
>
> --
> You received this question notification because your team duplicity-team
> is an answer contact for Duplicity.
>
> _______________________________________________
> Mailing list: https://launchpad.net/~duplicity-team
> Post to     : duplicity-team@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~duplicity-team
> More help   : https://help.launchpad.net/ListHelp
>