duplicity-team team mailing list archive

[Howard Kaye <question631423@xxxxxxxxxxxxxxxxxxxxx>]

Question #631423 on Duplicity changed:
https://answers.launchpad.net/duplicity/+question/631423

    Status: Answered => Open

Howard Kaye is still having a problem:
Increasing the number of open files by 2 orders of magnitude allowed this
particular restore to proceed.

In debugging the issue though, I tried catching the exception, doing an
os.open() in the handler, and this succeeds, though a subsequent call to
open() does not.
The process does not have an excessive number of open files.
So, the root cause of this does not appear to be actual open files, but
perhaps a bug somewhere else.

Rather than mask it by increasing the ulimit for open files, I'd prefer to
find the root cause.

On Mon, May 8, 2017 at 3:23 PM, Kenneth Loafman <
question631423@xxxxxxxxxxxxxxxxxxxxx> wrote:

> Your question #631423 on Duplicity changed:
> https://answers.launchpad.net/duplicity/+question/631423
>
>     Status: Open => Answered
>
> Kenneth Loafman proposed the following answer:
> That bug applies only to runs made with --no-encryption and
> --no-compression.  He's got gpg processes, so he's not running that way.
>
> Also, that bug should only affect the memory use, some.
>
> Really should try my settings.
>
>
> On Mon, May 8, 2017 at 9:58 AM, Howard Kaye <
> question631423@xxxxxxxxxxxxxxxxxxxxx> wrote:
>
> > Question #631423 on Duplicity changed:
> > https://answers.launchpad.net/duplicity/+question/631423
> >
> >     Status: Answered => Open
> >
> > Howard Kaye is still having a problem:
> > That does not seem to be related to the open files.
> >
> > On Mon, May 8, 2017 at 6:43 AM, edso <question631423@answers.
> launchpad.net
> > >
> > wrote:
> >
> > > Your question #631423 on Duplicity changed:
> > > https://answers.launchpad.net/duplicity/+question/631423
> > >
> > > edso proposed the following answer:
> > > On 05.05.2017 20:13, Howard Kaye wrote:
> > > > New question #631423 on Duplicity:
> > > > https://answers.launchpad.net/duplicity/+question/631423
> > > >
> > > > In testing a full restore, I am seeing an exception get raised for
> "Too
> > > many open files" from within Path.open:
> > > >
> > > > I am running on macOS 10.12.4, duplicity 0.7.10
> > > >
> > > > I have raised the ulimit for the number of files up to 12000.
> > > > lsof shows there are only 89 open files
> > > > sample shows a number of threads blocked on subprocesses (gpg
> > processes)
> > > > sysctl -a shows:
> > > >   kern.maxfiles: 24576
> > > >   kern.maxfilesperproc: 10240
> > > >   kern.num_files: 5356
> > > >
> > > > There are currently only 17 incrementals following my latest full.
> We
> > > usually do monthly fulls.
> > > >
> > > > Is anyone else seeing this recently?
> > > >
> > >
> > > there's going to be a fix in 0.7.13, replacing the use of os.system('cp
> > > ...') with the native python routine, which should reduce the open
> files
> > > count dramatically
> > >   http://duplicity.nongnu.org/CHANGELOG
> > >
> > > it's explained in more detail here
> > >   https://bugs.launchpad.net/duplicity/+bug/1684312
> > >
> > > ..ede/duply.net
> > >
> > > --
> > > If this answers your question, please go to the following page to let
> us
> > > know that it is solved:
> > > https://answers.launchpad.net/duplicity/+question/631423/+
> > > confirm?answer_id=4
> > >
> > > If you still need help, you can reply to this email or go to the
> > > following page to enter your feedback:
> > > https://answers.launchpad.net/duplicity/+question/631423
> > >
> > > You received this question notification because you asked the question.
> > >
> >
> > --
> > You received this question notification because your team duplicity-team
> > is an answer contact for Duplicity.
> >
> > _______________________________________________
> > Mailing list: https://launchpad.net/~duplicity-team
> > Post to     : duplicity-team@xxxxxxxxxxxxxxxxxxx
> > Unsubscribe : https://launchpad.net/~duplicity-team
> > More help   : https://help.launchpad.net/ListHelp
> >
>
> --
> If this answers your question, please go to the following page to let us
> know that it is solved:
> https://answers.launchpad.net/duplicity/+question/631423/+
> confirm?answer_id=6
>
> If you still need help, you can reply to this email or go to the
> following page to enter your feedback:
> https://answers.launchpad.net/duplicity/+question/631423
>
> You received this question notification because you asked the question.
>

-- 
You received this question notification because your team duplicity-team
is an answer contact for Duplicity.