duplicity-team team mailing list archive

[Kenneth Loafman <kenneth@xxxxxxxxxxx>]

Scott,

Glad you found it.  Will fix it so it uses the same Python.

BTW, *debian/duplicity.sh* gets installed on macOS as
*/usr/local/bin/duplicity* without the .sh.

Thanks for being patient.  It really helps when dealing with packaging.  Be
prepared for more changes as we go along.

...Ken


On Sun, Feb 11, 2024 at 9:54 PM Scott Hannahs <sth@xxxxxxxxxx> wrote:

> Ok that was my thought, the Python had the module directory layout and
> there is no reason for duplicity to mess with it.  The python script that
> is installed runs python3.10 as it should.  The duplicity.sh in the /debian
>  directory is something else.
>
> My pytest install is hard linked to pytest-3.10 which explicitly call
> python-3.10.  Is there something to tell pytest to use python3.10 vs
> python3?
>
> So I assume if I have pytest installed for the current version of python,
> then I don’t need the other dependencies that I had for testing previously:
>         pylint-py310,
>         future-py310,
>         librsync-bin,
>         mock-py310,
>         pluggy-py310,
>         py-py310,
>         tox-py310
>
> My installed version of duplicity 2.1.5 seems to have no problem of
> finding fasteners.
> ——————————————————————
> Results from latest tweak.  Much more readable!  However pytest is using
> the command “python3" and not python-3.10 which throws the error.
>
> ...command: *'python3'*
> '/opt/sw/src/fink.build/duplicity-2.2.2-2/duplicity-rel.2.2.2/duplicity/__main__.py'
> 'full'
> '/var/folders/6b/85yg9bp94rv55dzbdp78thp40000kr/T/testfiles/various_file_types'
> 'file:///var/folders/6b/85yg9bp94rv55dzbdp78thp40000kr/T/testfiles/output'
> '--volsize=1' '-v0' '--no-print-statistics'
> '--archive-dir=/var/folders/6b/85yg9bp94rv55dzbdp78thp40000kr/T/testfiles/cache'
> < /dev/null
>
> Where is it getting that command?  Aha!  Found it.  It is in the python
> testing/functional/__init__.py!  Once that is patched:
>
> "================= 427 passed, 17 skipped in 641.77s (0:10:41)
> ==================“
>
> Much better, thanks for the help. I am completely unfamiliar with pytest…
>  I assume that the Makefile is removing/changing the test call?
>
> -Scott
>
> On Feb 11, 2024, at 12:54 PM, Kenneth Loafman <kenneth@xxxxxxxxxxx> wrote:
>
> Scott,
>
> Duplicity does not set the path for imports, that's Python's job.
>
> *duplicity.sh* is the entrypoint script I lifted from an install of
> duplicity.  It's generated by the install.  Fix the shebang and it's the
> script for duplicity instead of 'duplicity/__main__.py`.  __main__ came
> about because we needed duplicity to operate as 'python3 -m duplicity'.
> Kind of a pain for package management.
>
> I found a solution to the noise problem.  Run pytest as '*pytest
> --capture=fd*'.  It should only print if the test fails.  Will fix the
> config.
>
> ...Ken
>
>
> On Sun, Feb 11, 2024 at 11:41 AM Scott Hannahs <sth@xxxxxxxxxx> wrote:
>
>> Ken,
>>
>> I agree that fasteners doesn’t see to be imported.  Not sure why.  Does
>> duplicity have some place where it explicitly sets the path for python
>> modules?  This is non-standard on my system but python is built to find
>> them.
>>
>> I have 4 versions of python installed.  (Uggh)
>>
>> The system version which is python 3.9 (and linked to python3)
>>
>> And Package installs of
>> python2.7  (needed for nmap but I hope to be able to dump it soon)
>> python3.8  (Needed for the R package I believe)
>> python3.10 (definitely has fasteners, can import from command line)
>>
>> The first two installed pythons do not have fasteners.  Python 3.10
>> does.  That is why I edit all the shebangs to explicitly call python3.10.
>> It seems that the debian duplicity.sh is left as merely python3 but I don’t
>> think that the Debian shell script is used on MacOS for testing.  I will
>> get that last one fixed but I don’t think it is called by the test harness.
>>
>> I will remove tox as a testing dependency.
>>
>> -Scott
>>
>>
>> On Feb 11, 2024, at 12:08 PM, Kenneth Loafman <kenneth@xxxxxxxxxxx>
>> wrote:
>>
>> Scott,
>>
>> Looks like module *fasteners* is not installed.  It's required.  Maybe
>> you have multiple Python 3.10 versions installed and installed fasteners
>> into the wrong one?
>>
>> Look in *requirements.txt* for the total list of requirements and
>> versions.  Most importantly pay attention to the ones with '==' which means
>> that version exactly.
>>
>> No, tox is not required unless you want to test against multiple versions
>> of Python (GitLab CI).  I'll see what I can do to remove it.
>>
>> ...Ken
>>
>>
>>
>>
>> On Sat, Feb 10, 2024 at 5:29 PM Scott Hannahs <sth@xxxxxxxxxx> wrote:
>>
>>> Ken,
>>>
>>> Thanks, I don’t mind extra output, but it looked like duplicity was
>>> merely repeating an error for a bad command line option.  That was very
>>> different than when I setup duplicity 2.2.1.
>>>
>>> I am running the tests right now and will let you know how it goes.  For
>>> testing on macOS, I use the following parameters to install the
>>> dependencies and then to set the number of files to “large” and to
>>> explicitly run the correct pytest version.  Is tox not needed anymore?
>>>
>>>     TestDepends: <<
>>>         pylint-py310,
>>>         future-py310,
>>>         librsync-bin,
>>>         mock-py310,
>>>         pluggy-py310,
>>>         py-py310,
>>>         tox-py310
>>> <<
>>>     TestScript: <<
>>>         #!/bin/sh -ev
>>>         ulimit -n 8192
>>>         %p/bin/pytest3.10
>>> <<
>>>
>>> Using pytest I get 23 command line “usage” errors.
>>>
>>> I get a much larger number of errors:
>>> ""ModuleNotFoundError: No module named 'fasteners'"
>>>  Which is why I tested running python3.10 manually and import
>>> fasteners.  Do I need to set the “PYTHONPATH” variable to anything specific?
>>>
>>>
>>> And the very last line of testing shows way more errors that previously:
>>> =========== 124 failed, 303 passed, 17 skipped in 112.73s (0:01:52)
>>> ============
>>>
>>> -Scott
>>>
>>>
>>> On Feb 10, 2024, at 2:32 PM, Kenneth Loafman <kenneth@xxxxxxxxxxx>
>>> wrote:
>>>
>>> Scott,
>>>
>>> Bottom line is that you can ignore all of the front matter if the tests
>>> passed.  The very last line is the one you need to look for.  It should
>>> look like this:
>>>
>>> ======================= 437 passed, 7 skipped in 485.72s (0:08:05)
>>>> ==============================
>>>>
>>>> with ==== filling in the line.
>>>
>>> I can't suppress the output of the tests, for the most part, since we
>>> use that to verify the test worked.  Catch-22.
>>>
>>> ...Ken
>>>
>>>
>>> On Sat, Feb 10, 2024 at 1:06 PM Kenneth Loafman <kenneth@xxxxxxxxxxx>
>>> wrote:
>>>
>>>> On Sat, Feb 10, 2024 at 11:43 AM Scott Hannahs via Duplicity-talk <
>>>> duplicity-talk@xxxxxxxxxx> wrote:
>>>>
>>>>> I attempted to integrate this into my fink building package for
>>>>> MacOS.  However when running the test portion of the install as I had for
>>>>> previous versions, I got a massive number of failures.  It seems that the
>>>>> test is using a command line argument that is changed/deleted?  There is a
>>>>> massively long output log full of this message repeating the duplicity
>>>>> usage message.  See below the signature.
>>>>>
>>>>
>>>> 2.2.1 and 2.2.1 should behave the same.  Both test the CLI to make sure
>>>> that options not valid for a command would create an error.  Massive
>>>> output.  I'll try to suppress it.
>>>>
>>>>
>>>>> I think this all worked for Duplicity 2.2.1 but something seems to
>>>>> have gone terribly awry.   The test commands are:
>>>>>
>>>>> TestScript: <<
>>>>>         #!/bin/sh -ev
>>>>>         ulimit -n 8192
>>>>>         %p/bin/python3.10 setup.py test
>>>>> <<
>>>>>
>>>>>
>>>> Instead of 'setup.py test' use 'pytest' by itself.  pyproject.toml has
>>>> the pytest config defined, setup.py test does not.  I should remove it
>>>> completely.
>>>>
>>>> Lots of changes like these are due to the rapid changes being made in
>>>> PyPA (Python Packaging Authority).  Sucks, but we have to live with it.
>>>>
>>>> Let me know how it goes.  I run everything here through pytest and it
>>>> works correctly.  Noisy but correct.
>>>>
>>>> ...Ken
>>>>
>>>>
>>>
>>
>