duplicity-team team mailing list archive

[Kenneth Loafman <kenneth@xxxxxxxxxxx>]

We're making progress!  With the latest update:

   1. On an 18.04 Ubuntu VM:
      - py27 passes all tests
      - py3 passes all but 7 tests
   2. On a rebuilt docker instance:
      - fails a lot more tests on both
      - abandoned till later
   3. The "from future import ???" works as it should.

I'm fairly sure the testing code itself is responsible for some of the 7
failures, but we need to be sure.

...Thanks,
...Ken


On Mon, Jun 17, 2019 at 10:41 AM Kenneth Loafman <kenneth@xxxxxxxxxxx>
wrote:

> Just did a pipe cleaning:
>
> 2019-06-17  Kenneth Loafman  <kenneth@xxxxxxxxxxx>
>
>     * Fix some import conflicts with the "past" module
>       - Rename collections.py to dup_collections.py
>       - Remove all "from future.utils import old_div"
>       - Replace old_div() with "//" (in py27 for a while).
>       - All tests run for py3, unit tests run for py2.  The new
>         import fail is "from future import standard_library"
>     * Fixed bug #1831178 sequence item 0: expected str instance, int found
>       - Simply converted int to str when making list
>     * Merged in lp:~kaffeekiffer/duplicity/azure-python3-fix
>       - Use util.fsencode to encode file string
>
> The "from future" is doing the same thing.  Merely do the following:
>
>     $ python2 duplicity/asyncscheduler.py
>
> To see what's happening.  Now looking for the problem.
>
> ...Ken
>
>
> On Fri, Jun 14, 2019 at 10:51 AM Kenneth Loafman <kenneth@xxxxxxxxxxx>
> wrote:
>
>>
>> On Thu, Jun 13, 2019 at 11:06 AM Kenneth Loafman <kenneth@xxxxxxxxxxx>
>> wrote:
>>
>>>
>>> On Wed, Jun 12, 2019 at 4:41 PM Aaron Whitehouse <
>>> aaron@xxxxxxxxxxxxxxxxxx> wrote:
>>>
>>>> Ken,
>>>>
>>>> On Jun 2 2019, at 2:36 pm, Aaron Whitehouse <lists@xxxxxxxxxxxxxxxxxx>
>>>> wrote:
>>>>
>>>> > On py2 I cannot get *past* to import even though it's in tox and
>>>> local site-packages.  This causes all of the functional tests to fail.
>>>>
>>>> Do we need this on Py2? Isn't it to provide Python 2-equivalent
>>>> functionality in Python 3? Can we put the import within a test for Python
>>>> version (and then obviously in future move over to using Python 3
>>>> constructs).
>>>>
>>>> Yes, past.old_div() is used in multiple modules to keep away from py3's
>>>> broken division.
>>>>
>>>> I am still getting this when trying to do ./run-tests on trunk, but I
>>>> see you managed to get 0.8.00 out the door -- how did you fix it on your
>>>> machine?
>>>>
>>>>
>>>> I did a clean install on a Ubuntu Bionic VM and that worked.
>>>>
>>>>
>>>> I cannot make this work in a clean Bionic install, on my development
>>>> box or even the 16.04 docker image.
>>>>
>>>> I guess we can replace it with an equivalent old_div function in util,
>>>> but thought I would check there is not something I am missing that can get
>>>> this working. Any suggestions?
>>>>
>>>> Kind regards,
>>>>
>>>> Aaron
>>>>
>>>
>>> Aaron,
>>>
>>> Regrettably, I've managed to get myself back to the same situation, py3
>>> passes but py2 fails with 'past' imports.  Not good!
>>>
>>> I had not installed all of the backend imports, so when I uncommented
>>> those and installed them via
>>>     "pip install -U -r requirements.txt",
>>> py27 functional tests broke.  I think there is a name conflict somewhere.
>>>
>>> So, for now, I have one run of py2 that passed unittest and functional,
>>> but none sense.
>>>
>>> Anyone else running into this problem?  Did you find a fix?  Google
>>> seems to be full of unanswered or misleading questions about this!
>>>
>>> ...Thanks,
>>> ...Ken
>>>
>>
>> I found the problem and it's a mess.  By the time that 'from past.utils
>> import old_div' gets executed, duplicity/collections has been imported.  In
>> the imports that past does, it tries to import DefaultDict from
>> collections, which does not exist.  You can see it below.  Working now on
>> renaming our collections to dup_collections.  We're likely to hit this
>> again.
>>
>> ken@utest-18-04:~/workspace/duplicity-src8$ python duplicity/log.py
>> Traceback (most recent call last):
>>   File "duplicity/log.py", line 27, in <module>
>>     from builtins import str
>>   File "/usr/local/lib/python2.7/dist-packages/builtins/__init__.py",
>> line 8, in <module>
>>     from future.builtins import *
>>   File
>> "/usr/local/lib/python2.7/dist-packages/future/builtins/__init__.py", line
>> 10, in <module>
>>     from future.builtins.iterators import (filter, map, zip)
>>   File
>> "/usr/local/lib/python2.7/dist-packages/future/builtins/iterators.py", line
>> 38, in <module>
>>     from future import utils
>>   File "/usr/local/lib/python2.7/dist-packages/future/utils/__init__.py",
>> line 55, in <module>
>>     import inspect
>>   File "/usr/lib/python2.7/inspect.py", line 42, in <module>
>>     from collections import namedtuple
>>   File "/home/ken/workspace/duplicity-src8/duplicity/collections.py",
>> line 24, in <module>
>>     from past.builtins import cmp
>>   File "/usr/local/lib/python2.7/dist-packages/past/__init__.py", line
>> 88, in <module>
>>     from past.translation import install_hooks as autotranslate
>>   File
>> "/usr/local/lib/python2.7/dist-packages/past/translation/__init__.py", line
>> 42, in <module>
>>     from lib2to3.refactor import RefactoringTool
>>   File "/usr/lib/python2.7/lib2to3/refactor.py", line 30, in <module>
>>     from . import btm_matcher as bm
>>   File "/usr/lib/python2.7/lib2to3/btm_matcher.py", line 12, in <module>
>>     from collections import defaultdict
>> ImportError: cannot import name defaultdict
>>
>> ...Ken
>>
>>