← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 2038474] [NEW] many unit tests issues with objects compared to strings

 

Public bug reported:

Running unit tests of Horizon in Debian Unstable leads to many unit test
failures like this one below.

My instinct tells me that this is Python 3.12 related, but I'm not sure.
Here's a pip freeze output with the installed build-dependency of the
package:

alabaster==0.7.12
appdirs==1.4.4
asgiref==3.7.2
async-timeout==4.0.3
attrs==23.1.0
autopage==0.4.0
Babel==2.10.3
blinker==1.6.2
calmjs==3.4.2
calmjs.parse==1.2.5
calmjs.types==1.0.1
certifi==2023.7.22
chardet==5.2.0
charset-normalizer==3.2.0
cliff==4.2.0
cmd2==2.4.3+ds
coverage==7.2.7
cryptography==38.0.4
csscompressor==0.9.5
debtcollector==2.5.0
decorator==5.1.1
Deprecated==1.2.14
deprecation==2.0.7
Django==4.2.5
django-appconf==1.0.5
django-compressor==4.0
django-debreach==2.1.0
django-pyscss==2.0.2
dnspython==2.4.2
docutils==0.19
dogpile.cache==1.1.8
dulwich==0.21.6
enmerkar==0.7.1
eventlet==0.33.1
exceptiongroup==1.1.3
execnet==2.0.0
extras==1.0.0
fasteners==0.17.3
fixtures==4.0.1
flake8==5.0.4
freezegun==1.2.1
futurist==2.4.1
greenlet==2.0.2
h11==0.14.0
hacking==4.1.0
idna==3.3
imagesize==1.4.1
importlib-metadata==4.12.0
iniconfig==1.1.1
iso8601==1.0.2
jaraco.classes==3.2.1
jeepney==0.8.0
Jinja2==3.1.2
jmespath==1.0.1
jsonpatch==1.32
jsonpointer==2.3
jsonschema==4.10.3
keyring==24.2.0
keystoneauth1==5.3.0
lxml==4.9.3
Mako==1.2.4.dev0
MarkupSafe==2.1.3
mccabe==0.7.0
monotonic==1.6
more-itertools==10.1.0
msgpack==1.0.3
netaddr==0.8.0
netifaces==0.11.0
oauthlib==3.2.2
openstackdocstheme==1.20.0
openstacksdk==1.0.1
os-client-config==2.1.0
os-service-types==1.7.0
osc-lib==2.8.1
oslo.concurrency==5.1.1
oslo.config==9.1.1
oslo.context==5.1.1
oslo.i18n==6.0.0
oslo.log==5.2.0
oslo.policy==4.1.1
oslo.serialization==5.1.1
oslo.upgradecheck==2.1.1
oslo.utils==6.1.0
osprofiler==3.4.3
outcome==1.2.0
packaging==23.1
pbr==5.11.1
pep8==1.7.1
pluggy==1.3.0
ply==3.11
prettytable==3.6.0
pycodestyle==2.10.0
pyflakes==2.5.0
Pygments==2.15.1
pyinotify==0.9.6
PyJWT==2.7.0
pymongo==3.11.0
pyOpenSSL==23.0.0
pyparsing==3.1.1
pyperclip==1.8.2
pyrsistent==0.18.1
pyScss==1.4.0
pytest==7.4.2
pytest-django==4.5.2
pytest-xdist==3.3.1
python-cinderclient==9.3.0
python-dateutil==2.8.2
python-glanceclient==4.3.0
python-keystoneclient==5.1.0
python-memcached==1.58
python-neutronclient==9.0.0
python-novaclient==18.3.0
python-swiftclient==4.2.0
pytz==2023.3.post1
PyYAML==6.0.1
rcssmin==1.1.0
redis==4.3.4
requests==2.31.0
requestsexceptions==1.4.0
rfc3986==1.5.0
rjsmin==1.2.0
roman==3.3
SecretStorage==3.3.3
selenium==4.13.0
semantic-version==2.9.0
simplejson==3.19.1
six==1.16.0
sniffio==1.2.0
snowballstemmer==2.2.0
sortedcontainers==2.4.0
Sphinx==5.3.0
sqlparse==0.4.2
stevedore==5.1.0
testscenarios==0.5.0
testtools==2.5.0
trio==0.22.2
trio-websocket==0.10.3
urllib3==1.26.16
warlock==2.0.1
wcwidth==0.2.5
WebOb==1.8.6
wrapt==1.14.1
wsproto==1.2.0
XStatic==1.0.3
XStatic-Angular==1.8.2.2
XStatic-Angular-Bootstrap==2.5.0.0
XStatic-Angular-FileUpload==12.0.4.0
XStatic-Angular-Gettext==2.4.1.0
XStatic-Angular-lrdragndrop==1.0.2.2
XStatic-Angular-Schema-Form==0.8.13.0
XStatic-angular-ui-router==0.3.1.4
XStatic-Bootstrap-Datepicker==1.3.1.0
XStatic-Bootstrap-SCSS==3.4.1.0
XStatic-bootswatch==3.3.7.0
XStatic-D3==3.5.17.0
XStatic-Font-Awesome==4.7.0.0
XStatic-Hogan==2.0.0.2
XStatic-Jasmine==2.4.1.0
XStatic-jQuery==3.5.1.0
XStatic-JQuery-Migrate==3.3.2.1
XStatic-jquery-ui==1.12.0.1
XStatic-JQuery.quicksearch==2.0.4.1
XStatic-JQuery.TableSorter==2.14.5.1
XStatic-JSEncrypt==2.3.1.1
XStatic-Magic-Search==0.2.5.1
XStatic-mdi==1.6.50.2
XStatic-objectpath==1.2.1.0
XStatic-Rickshaw==1.5.0.2
XStatic-roboto-fontface==0.5.0.0
XStatic-smart-table==1.4.13.2
XStatic-Spin==1.2.8.2
XStatic-term.js==0.0.7.0
XStatic-tv4==1.2.7.0
xvfbwrapper==0.2.9
zipp==1.0.0

and here's a typical failure below. Note that there's maybe more than 3
dozen of issues like it:

__________________ WorkflowsTests.test_workflow_registration ___________________
[gw3] linux -- Python 3.11.5 /usr/bin/python3.11

self = <horizon.test.unit.workflows.test_workflows.WorkflowsTests
testMethod=test_workflow_registration>

    def test_workflow_registration(self):
        req = self.factory.get("/foo")
        flow = WorkflowForTesting(req)
>       self.assertQuerysetEqual(flow.steps,
                                 ['<StepOne: action_one>',
                                  '<StepTwo: action_two>'])

horizon/test/unit/workflows/test_workflows.py:328: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/usr/lib/python3/dist-packages/django/test/testcases.py:1330: in assertQuerysetEqual
    return self.assertQuerySetEqual(*args, **kw)
/usr/lib/python3/dist-packages/django/test/testcases.py:1346: in assertQuerySetEqual
    return self.assertEqual(list(items), values, msg=msg)
E   AssertionError: Lists differ: [<StepOne: action_one>, <StepTwo: action_two>] != ['<StepOne: action_one>', '<StepTwo: action_two>']
E   
E   First differing element 0:
E   <StepOne: action_one>
E   '<StepOne: action_one>'
E   
E   - [<StepOne: action_one>, <StepTwo: action_two>]
E   + ['<StepOne: action_one>', '<StepTwo: action_two>']
E   ?  +                     +  +                     +

Probably, it used to be that Python returned strings, but now it's not?
Anyways, calling repr() on each object fixes it, with something like
this:

       self.assertQuerysetEqual(map(repr, flow.steps),
                                 ['<StepOne: action_one>',
                                  '<StepTwo: action_two>'])

though, this breaks in earlier environment, as a map isn't a list, so
this really fixes all cases:

       self.assertQuerysetEqual(list(map(repr, flow.steps)),
                                 ['<StepOne: action_one>',
                                  '<StepTwo: action_two>'])

My patch here fixes them all:
https://review.opendev.org/c/openstack/horizon/+/897310

Cheers,

Thomas

** Affects: horizon
     Importance: Undecided
         Status: In Progress

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Dashboard (Horizon).
https://bugs.launchpad.net/bugs/2038474

Title:
  many unit tests issues with objects compared to strings

Status in OpenStack Dashboard (Horizon):
  In Progress

Bug description:
  Running unit tests of Horizon in Debian Unstable leads to many unit
  test failures like this one below.

  My instinct tells me that this is Python 3.12 related, but I'm not
  sure. Here's a pip freeze output with the installed build-dependency
  of the package:

  alabaster==0.7.12
  appdirs==1.4.4
  asgiref==3.7.2
  async-timeout==4.0.3
  attrs==23.1.0
  autopage==0.4.0
  Babel==2.10.3
  blinker==1.6.2
  calmjs==3.4.2
  calmjs.parse==1.2.5
  calmjs.types==1.0.1
  certifi==2023.7.22
  chardet==5.2.0
  charset-normalizer==3.2.0
  cliff==4.2.0
  cmd2==2.4.3+ds
  coverage==7.2.7
  cryptography==38.0.4
  csscompressor==0.9.5
  debtcollector==2.5.0
  decorator==5.1.1
  Deprecated==1.2.14
  deprecation==2.0.7
  Django==4.2.5
  django-appconf==1.0.5
  django-compressor==4.0
  django-debreach==2.1.0
  django-pyscss==2.0.2
  dnspython==2.4.2
  docutils==0.19
  dogpile.cache==1.1.8
  dulwich==0.21.6
  enmerkar==0.7.1
  eventlet==0.33.1
  exceptiongroup==1.1.3
  execnet==2.0.0
  extras==1.0.0
  fasteners==0.17.3
  fixtures==4.0.1
  flake8==5.0.4
  freezegun==1.2.1
  futurist==2.4.1
  greenlet==2.0.2
  h11==0.14.0
  hacking==4.1.0
  idna==3.3
  imagesize==1.4.1
  importlib-metadata==4.12.0
  iniconfig==1.1.1
  iso8601==1.0.2
  jaraco.classes==3.2.1
  jeepney==0.8.0
  Jinja2==3.1.2
  jmespath==1.0.1
  jsonpatch==1.32
  jsonpointer==2.3
  jsonschema==4.10.3
  keyring==24.2.0
  keystoneauth1==5.3.0
  lxml==4.9.3
  Mako==1.2.4.dev0
  MarkupSafe==2.1.3
  mccabe==0.7.0
  monotonic==1.6
  more-itertools==10.1.0
  msgpack==1.0.3
  netaddr==0.8.0
  netifaces==0.11.0
  oauthlib==3.2.2
  openstackdocstheme==1.20.0
  openstacksdk==1.0.1
  os-client-config==2.1.0
  os-service-types==1.7.0
  osc-lib==2.8.1
  oslo.concurrency==5.1.1
  oslo.config==9.1.1
  oslo.context==5.1.1
  oslo.i18n==6.0.0
  oslo.log==5.2.0
  oslo.policy==4.1.1
  oslo.serialization==5.1.1
  oslo.upgradecheck==2.1.1
  oslo.utils==6.1.0
  osprofiler==3.4.3
  outcome==1.2.0
  packaging==23.1
  pbr==5.11.1
  pep8==1.7.1
  pluggy==1.3.0
  ply==3.11
  prettytable==3.6.0
  pycodestyle==2.10.0
  pyflakes==2.5.0
  Pygments==2.15.1
  pyinotify==0.9.6
  PyJWT==2.7.0
  pymongo==3.11.0
  pyOpenSSL==23.0.0
  pyparsing==3.1.1
  pyperclip==1.8.2
  pyrsistent==0.18.1
  pyScss==1.4.0
  pytest==7.4.2
  pytest-django==4.5.2
  pytest-xdist==3.3.1
  python-cinderclient==9.3.0
  python-dateutil==2.8.2
  python-glanceclient==4.3.0
  python-keystoneclient==5.1.0
  python-memcached==1.58
  python-neutronclient==9.0.0
  python-novaclient==18.3.0
  python-swiftclient==4.2.0
  pytz==2023.3.post1
  PyYAML==6.0.1
  rcssmin==1.1.0
  redis==4.3.4
  requests==2.31.0
  requestsexceptions==1.4.0
  rfc3986==1.5.0
  rjsmin==1.2.0
  roman==3.3
  SecretStorage==3.3.3
  selenium==4.13.0
  semantic-version==2.9.0
  simplejson==3.19.1
  six==1.16.0
  sniffio==1.2.0
  snowballstemmer==2.2.0
  sortedcontainers==2.4.0
  Sphinx==5.3.0
  sqlparse==0.4.2
  stevedore==5.1.0
  testscenarios==0.5.0
  testtools==2.5.0
  trio==0.22.2
  trio-websocket==0.10.3
  urllib3==1.26.16
  warlock==2.0.1
  wcwidth==0.2.5
  WebOb==1.8.6
  wrapt==1.14.1
  wsproto==1.2.0
  XStatic==1.0.3
  XStatic-Angular==1.8.2.2
  XStatic-Angular-Bootstrap==2.5.0.0
  XStatic-Angular-FileUpload==12.0.4.0
  XStatic-Angular-Gettext==2.4.1.0
  XStatic-Angular-lrdragndrop==1.0.2.2
  XStatic-Angular-Schema-Form==0.8.13.0
  XStatic-angular-ui-router==0.3.1.4
  XStatic-Bootstrap-Datepicker==1.3.1.0
  XStatic-Bootstrap-SCSS==3.4.1.0
  XStatic-bootswatch==3.3.7.0
  XStatic-D3==3.5.17.0
  XStatic-Font-Awesome==4.7.0.0
  XStatic-Hogan==2.0.0.2
  XStatic-Jasmine==2.4.1.0
  XStatic-jQuery==3.5.1.0
  XStatic-JQuery-Migrate==3.3.2.1
  XStatic-jquery-ui==1.12.0.1
  XStatic-JQuery.quicksearch==2.0.4.1
  XStatic-JQuery.TableSorter==2.14.5.1
  XStatic-JSEncrypt==2.3.1.1
  XStatic-Magic-Search==0.2.5.1
  XStatic-mdi==1.6.50.2
  XStatic-objectpath==1.2.1.0
  XStatic-Rickshaw==1.5.0.2
  XStatic-roboto-fontface==0.5.0.0
  XStatic-smart-table==1.4.13.2
  XStatic-Spin==1.2.8.2
  XStatic-term.js==0.0.7.0
  XStatic-tv4==1.2.7.0
  xvfbwrapper==0.2.9
  zipp==1.0.0

  and here's a typical failure below. Note that there's maybe more than
  3 dozen of issues like it:

  __________________ WorkflowsTests.test_workflow_registration ___________________
  [gw3] linux -- Python 3.11.5 /usr/bin/python3.11

  self = <horizon.test.unit.workflows.test_workflows.WorkflowsTests
  testMethod=test_workflow_registration>

      def test_workflow_registration(self):
          req = self.factory.get("/foo")
          flow = WorkflowForTesting(req)
  >       self.assertQuerysetEqual(flow.steps,
                                   ['<StepOne: action_one>',
                                    '<StepTwo: action_two>'])

  horizon/test/unit/workflows/test_workflows.py:328: 
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  /usr/lib/python3/dist-packages/django/test/testcases.py:1330: in assertQuerysetEqual
      return self.assertQuerySetEqual(*args, **kw)
  /usr/lib/python3/dist-packages/django/test/testcases.py:1346: in assertQuerySetEqual
      return self.assertEqual(list(items), values, msg=msg)
  E   AssertionError: Lists differ: [<StepOne: action_one>, <StepTwo: action_two>] != ['<StepOne: action_one>', '<StepTwo: action_two>']
  E   
  E   First differing element 0:
  E   <StepOne: action_one>
  E   '<StepOne: action_one>'
  E   
  E   - [<StepOne: action_one>, <StepTwo: action_two>]
  E   + ['<StepOne: action_one>', '<StepTwo: action_two>']
  E   ?  +                     +  +                     +

  Probably, it used to be that Python returned strings, but now it's
  not? Anyways, calling repr() on each object fixes it, with something
  like this:

         self.assertQuerysetEqual(map(repr, flow.steps),
                                   ['<StepOne: action_one>',
                                    '<StepTwo: action_two>'])

  though, this breaks in earlier environment, as a map isn't a list, so
  this really fixes all cases:

         self.assertQuerysetEqual(list(map(repr, flow.steps)),
                                   ['<StepOne: action_one>',
                                    '<StepTwo: action_two>'])

  My patch here fixes them all:
  https://review.opendev.org/c/openstack/horizon/+/897310

  Cheers,

  Thomas

To manage notifications about this bug go to:
https://bugs.launchpad.net/horizon/+bug/2038474/+subscriptions



Follow ups