yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1551312] [NEW] Python unit tests don't run in a clean 15.10 environment

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Sean Dague <sean@xxxxxxxxx>
Date: Mon, 29 Feb 2016 16:12:11 -0000
Reply-to: Bug 1551312 <1551312@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

Attempting to test fixes for keystone but tox -e py27 doesn't work for
me:

ribos:~/code/openstack/keystone(master)> rm -rf .tox
ribos:~/code/openstack/keystone(master)> tox -e py27
py27 create: /home/sdague/code/openstack/keystone/.tox/py27
py27 installdeps: -r/home/sdague/code/openstack/keystone/test-requirements.txt, .[ldap,memcache,mongodb]
py27 develop-inst: /home/sdague/code/openstack/keystone
py27 installed: -f /home/sdague/.pip/wheelhouse,alembic==0.8.4,amqp==1.4.9,anyjson==0.3.3,appdirs==1.4.0,Babel==2.2.0,bashate==0.4.0,beautifulsoup4==4.4.1,cachetools==1.1.5,cffi==1.5.2,contextlib2==0.5.1,coverage==4.0.3,cryptography==1.2.2,debtcollector==1.3.0,decorator==4.0.9,docutils==0.12,dogpile.cache==0.5.7,dogpile.core==0.4.1,ecdsa==0.13,enum34==1.1.2,eventlet==0.18.4,extras==0.0.3,fasteners==0.14.1,fixtures==1.4.0,flake8==2.2.4,flake8-docstrings==0.2.1.post1,funcsigs==0.4,functools32==3.2.3.post2,futures==3.0.5,futurist==0.13.0,greenlet==0.4.9,hacking==0.10.2,httplib2==0.9.2,idna==2.0,ipaddress==1.0.16,iso8601==0.1.11,Jinja2==2.8,jsonschema==2.5.1,-e git+https://github.com/openstack/keystone.git@428cbeec71aa063fe627ea89b23d77a1d4556763#egg=keystone,keystoneauth1==2.3.0,keystonemiddleware==4.3.0,kombu==3.0.33,ldappool==1.0,linecache2==1.0.0,lxml==3.5.0,Mako==1.0.3,MarkupSafe==0.23,mccabe==0.2.1,mock==1.3.0,monotonic==0.6,mox3==0.14.0,msgpack-python==0.4.7,netaddr==0.7.18,netifaces==0.10.4,oauthlib==1.0.3,os-client-config==1.16.0,os-testr==0.6.0,oslo.cache==1.4.0,oslo.concurrency==3.6.0,oslo.config==3.9.0,oslo.context==2.2.0,oslo.db==4.6.0,oslo.i18n==3.4.0,oslo.log==3.2.0,oslo.messaging==4.5.0,oslo.middleware==3.7.0,oslo.policy==1.5.0,oslo.serialization==2.4.0,oslo.service==1.7.0,oslo.utils==3.7.0,oslosphinx==4.3.0,oslotest==2.3.0,paramiko==1.16.0,passlib==1.6.5,Paste==2.0.2,PasteDeploy==1.5.2,pbr==1.8.1,pep257==0.7.0,pep8==1.5.7,pika==0.10.0,pika-pool==0.1.3,positional==1.0.1,prettytable==0.7.2,pyasn1==0.1.9,pycadf==2.1.0,pycparser==2.14,pycrypto==2.6.1,pyflakes==0.8.1,Pygments==2.1.2,pyinotify==0.9.6,pymongo==3.2.1,pyOpenSSL==0.15.1,pyrsistent==0.11.12,pysaml2==4.0.2,python-dateutil==2.5.0,python-editor==0.5,python-keystoneclient==2.3.1,python-ldap==2.4.25,python-memcached==1.57,python-mimeparse==1.5.1,python-subunit==1.2.0,pytz==2015.7,PyYAML==3.11,reno==1.5.0,repoze.lru==0.6,repoze.who==2.2,requests==2.9.1,requestsexceptions==1.1.3,retrying==1.3.3,Routes==2.2,six==1.10.0,Sphinx==1.2.3,SQLAlchemy==1.0.12,sqlalchemy-migrate==0.10.0,sqlparse==0.1.18,stevedore==1.12.0,tempest-lib==0.14.0,Tempita==0.5.2,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.0.0,traceback2==1.4.0,unittest2==1.1.0,waitress==0.8.10,WebOb==1.5.1,WebTest==2.0.20,wheel==0.26.0,wrapt==1.10.6,zope.interface==4.1.3
py27 runtests: PYTHONHASHSEED='446975225'
py27 runtests: commands[0] | bash tools/pretty_tox.sh 
running testr
runningNon-zero exit code (2) from test listing.
=error: testr failed (3)

OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
OS_LOG_CAPTURE=${OS_LOG_CAPTURE:-1} \
${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./keystone/tests/unit} --list 
--- import errors ---
Failed to import test module: keystone.tests.unit.test_backend_ldap_pool
Traceback (most recent call last):
  File "/home/sdague/code/openstack/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", line 456, in _find_test_path
    module = self._get_module_from_name(name)
  File "/home/sdague/code/openstack/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", line 395, in _get_module_from_name
    __import__(name)
  File "keystone/tests/unit/test_backend_ldap_pool.py", line 23, in <module>
    from keystone.identity.backends import ldap
  File "/home/sdague/code/openstack/keystone/keystone/identity/backends/ldap/__init__.py", line 17, in <module>
ImportError: No module named core

Failed to import test module: keystone.tests.unit.test_ldap_livetest
Traceback (most recent call last):
  File "/home/sdague/code/openstack/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", line 456, in _find_test_path
    module = self._get_module_from_name(name)
  File "/home/sdague/code/openstack/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", line 395, in _get_module_from_name
    __import__(name)
  File "keystone/tests/unit/test_ldap_livetest.py", line 23, in <module>
    from keystone.identity.backends import ldap as identity_ldap
  File "/home/sdague/code/openstack/keystone/keystone/identity/backends/ldap/__init__.py", line 17, in <module>
ImportError: No module named core

Failed to import test module: keystone.tests.unit.test_ldap_pool_livetest
Traceback (most recent call last):
  File "/home/sdague/code/openstack/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", line 456, in _find_test_path
    module = self._get_module_from_name(name)
  File "/home/sdague/code/openstack/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", line 395, in _get_module_from_name
    __import__(name)
  File "keystone/tests/unit/test_ldap_pool_livetest.py", line 21, in <module>
    from keystone.identity.backends import ldap
  File "/home/sdague/code/openstack/keystone/keystone/identity/backends/ldap/__init__.py", line 17, in <module>
ImportError: No module named core

Failed to import test module: keystone.tests.unit.test_ldap_tls_livetest
Traceback (most recent call last):
  File "/home/sdague/code/openstack/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", line 456, in _find_test_path
    module = self._get_module_from_name(name)
  File "/home/sdague/code/openstack/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", line 395, in _get_module_from_name
    __import__(name)
  File "keystone/tests/unit/test_ldap_tls_livetest.py", line 22, in <module>
    from keystone.tests.unit import test_ldap_livetest
  File "keystone/tests/unit/test_ldap_livetest.py", line 23, in <module>
    from keystone.identity.backends import ldap as identity_ldap
  File "/home/sdague/code/openstack/keystone/keystone/identity/backends/ldap/__init__.py", line 17, in <module>
ImportError: No module named core
The test run didn't actually run any tests

Slowest Tests:

ERROR: InvocationError: '/bin/bash tools/pretty_tox.sh '
________________________________________________________________________________ summary ________________________________________________________________________________
ERROR:   py27: commands failed


> lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 15.10
Release:	15.10
Codename:	wily

> tox --version
2.0.0 imported from /usr/local/lib/python2.7/dist-packages/tox/__init__.pyc

> virtualenv --version
14.0.5

> pip --version
pip 8.0.2 from /usr/local/lib/python2.7/dist-packages (python 2.7)

> dpkg -l | grep ldap
ii  erlang-eldap                                         1:18.0-dfsg-1ubuntu1                                amd64        Erlang/OTP LDAP library
ii  libaprutil1-ldap:amd64                               1.5.4-1                                             amd64        Apache Portable Runtime Utility Library - LDAP Driver
ii  libkldap4                                            4:4.14.10-1ubuntu2                                  amd64        library for accessing LDAP
ii  libldap-2.4-2:amd64                                  2.4.41+dfsg-1ubuntu2                                amd64        OpenLDAP libraries
rc  libldap-2.4-2:i386                                   2.4.31-1+nmu2ubuntu11                               i386         OpenLDAP libraries
ii  libldap2-dev:amd64                                   2.4.41+dfsg-1ubuntu2                                amd64        OpenLDAP development libraries

** Affects: keystone
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Identity (keystone).
https://bugs.launchpad.net/bugs/1551312

Title:
  Python unit tests don't run in a clean 15.10 environment

Status in OpenStack Identity (keystone):
  New

Bug description:
  Attempting to test fixes for keystone but tox -e py27 doesn't work for
  me:

  ribos:~/code/openstack/keystone(master)> rm -rf .tox
  ribos:~/code/openstack/keystone(master)> tox -e py27
  py27 create: /home/sdague/code/openstack/keystone/.tox/py27
  py27 installdeps: -r/home/sdague/code/openstack/keystone/test-requirements.txt, .[ldap,memcache,mongodb]
  py27 develop-inst: /home/sdague/code/openstack/keystone
  py27 installed: -f /home/sdague/.pip/wheelhouse,alembic==0.8.4,amqp==1.4.9,anyjson==0.3.3,appdirs==1.4.0,Babel==2.2.0,bashate==0.4.0,beautifulsoup4==4.4.1,cachetools==1.1.5,cffi==1.5.2,contextlib2==0.5.1,coverage==4.0.3,cryptography==1.2.2,debtcollector==1.3.0,decorator==4.0.9,docutils==0.12,dogpile.cache==0.5.7,dogpile.core==0.4.1,ecdsa==0.13,enum34==1.1.2,eventlet==0.18.4,extras==0.0.3,fasteners==0.14.1,fixtures==1.4.0,flake8==2.2.4,flake8-docstrings==0.2.1.post1,funcsigs==0.4,functools32==3.2.3.post2,futures==3.0.5,futurist==0.13.0,greenlet==0.4.9,hacking==0.10.2,httplib2==0.9.2,idna==2.0,ipaddress==1.0.16,iso8601==0.1.11,Jinja2==2.8,jsonschema==2.5.1,-e git+https://github.com/openstack/keystone.git@428cbeec71aa063fe627ea89b23d77a1d4556763#egg=keystone,keystoneauth1==2.3.0,keystonemiddleware==4.3.0,kombu==3.0.33,ldappool==1.0,linecache2==1.0.0,lxml==3.5.0,Mako==1.0.3,MarkupSafe==0.23,mccabe==0.2.1,mock==1.3.0,monotonic==0.6,mox3==0.14.0,msgpack-python==0.4.7,netaddr==0.7.18,netifaces==0.10.4,oauthlib==1.0.3,os-client-config==1.16.0,os-testr==0.6.0,oslo.cache==1.4.0,oslo.concurrency==3.6.0,oslo.config==3.9.0,oslo.context==2.2.0,oslo.db==4.6.0,oslo.i18n==3.4.0,oslo.log==3.2.0,oslo.messaging==4.5.0,oslo.middleware==3.7.0,oslo.policy==1.5.0,oslo.serialization==2.4.0,oslo.service==1.7.0,oslo.utils==3.7.0,oslosphinx==4.3.0,oslotest==2.3.0,paramiko==1.16.0,passlib==1.6.5,Paste==2.0.2,PasteDeploy==1.5.2,pbr==1.8.1,pep257==0.7.0,pep8==1.5.7,pika==0.10.0,pika-pool==0.1.3,positional==1.0.1,prettytable==0.7.2,pyasn1==0.1.9,pycadf==2.1.0,pycparser==2.14,pycrypto==2.6.1,pyflakes==0.8.1,Pygments==2.1.2,pyinotify==0.9.6,pymongo==3.2.1,pyOpenSSL==0.15.1,pyrsistent==0.11.12,pysaml2==4.0.2,python-dateutil==2.5.0,python-editor==0.5,python-keystoneclient==2.3.1,python-ldap==2.4.25,python-memcached==1.57,python-mimeparse==1.5.1,python-subunit==1.2.0,pytz==2015.7,PyYAML==3.11,reno==1.5.0,repoze.lru==0.6,repoze.who==2.2,requests==2.9.1,requestsexceptions==1.1.3,retrying==1.3.3,Routes==2.2,six==1.10.0,Sphinx==1.2.3,SQLAlchemy==1.0.12,sqlalchemy-migrate==0.10.0,sqlparse==0.1.18,stevedore==1.12.0,tempest-lib==0.14.0,Tempita==0.5.2,testrepository==0.0.20,testscenarios==0.5.0,testtools==2.0.0,traceback2==1.4.0,unittest2==1.1.0,waitress==0.8.10,WebOb==1.5.1,WebTest==2.0.20,wheel==0.26.0,wrapt==1.10.6,zope.interface==4.1.3
  py27 runtests: PYTHONHASHSEED='446975225'
  py27 runtests: commands[0] | bash tools/pretty_tox.sh 
  running testr
  runningNon-zero exit code (2) from test listing.
  =error: testr failed (3)

  OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \
  OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \
  OS_LOG_CAPTURE=${OS_LOG_CAPTURE:-1} \
  ${PYTHON:-python} -m subunit.run discover -t ./ ${OS_TEST_PATH:-./keystone/tests/unit} --list 
  --- import errors ---
  Failed to import test module: keystone.tests.unit.test_backend_ldap_pool
  Traceback (most recent call last):
    File "/home/sdague/code/openstack/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", line 456, in _find_test_path
      module = self._get_module_from_name(name)
    File "/home/sdague/code/openstack/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", line 395, in _get_module_from_name
      __import__(name)
    File "keystone/tests/unit/test_backend_ldap_pool.py", line 23, in <module>
      from keystone.identity.backends import ldap
    File "/home/sdague/code/openstack/keystone/keystone/identity/backends/ldap/__init__.py", line 17, in <module>
  ImportError: No module named core

  Failed to import test module: keystone.tests.unit.test_ldap_livetest
  Traceback (most recent call last):
    File "/home/sdague/code/openstack/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", line 456, in _find_test_path
      module = self._get_module_from_name(name)
    File "/home/sdague/code/openstack/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", line 395, in _get_module_from_name
      __import__(name)
    File "keystone/tests/unit/test_ldap_livetest.py", line 23, in <module>
      from keystone.identity.backends import ldap as identity_ldap
    File "/home/sdague/code/openstack/keystone/keystone/identity/backends/ldap/__init__.py", line 17, in <module>
  ImportError: No module named core

  Failed to import test module: keystone.tests.unit.test_ldap_pool_livetest
  Traceback (most recent call last):
    File "/home/sdague/code/openstack/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", line 456, in _find_test_path
      module = self._get_module_from_name(name)
    File "/home/sdague/code/openstack/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", line 395, in _get_module_from_name
      __import__(name)
    File "keystone/tests/unit/test_ldap_pool_livetest.py", line 21, in <module>
      from keystone.identity.backends import ldap
    File "/home/sdague/code/openstack/keystone/keystone/identity/backends/ldap/__init__.py", line 17, in <module>
  ImportError: No module named core

  Failed to import test module: keystone.tests.unit.test_ldap_tls_livetest
  Traceback (most recent call last):
    File "/home/sdague/code/openstack/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", line 456, in _find_test_path
      module = self._get_module_from_name(name)
    File "/home/sdague/code/openstack/keystone/.tox/py27/local/lib/python2.7/site-packages/unittest2/loader.py", line 395, in _get_module_from_name
      __import__(name)
    File "keystone/tests/unit/test_ldap_tls_livetest.py", line 22, in <module>
      from keystone.tests.unit import test_ldap_livetest
    File "keystone/tests/unit/test_ldap_livetest.py", line 23, in <module>
      from keystone.identity.backends import ldap as identity_ldap
    File "/home/sdague/code/openstack/keystone/keystone/identity/backends/ldap/__init__.py", line 17, in <module>
  ImportError: No module named core
  The test run didn't actually run any tests

  Slowest Tests:

  ERROR: InvocationError: '/bin/bash tools/pretty_tox.sh '
  ________________________________________________________________________________ summary ________________________________________________________________________________
  ERROR:   py27: commands failed

  
  > lsb_release -a
  No LSB modules are available.
  Distributor ID:	Ubuntu
  Description:	Ubuntu 15.10
  Release:	15.10
  Codename:	wily

  > tox --version
  2.0.0 imported from /usr/local/lib/python2.7/dist-packages/tox/__init__.pyc

  > virtualenv --version
  14.0.5

  > pip --version
  pip 8.0.2 from /usr/local/lib/python2.7/dist-packages (python 2.7)

  > dpkg -l | grep ldap
  ii  erlang-eldap                                         1:18.0-dfsg-1ubuntu1                                amd64        Erlang/OTP LDAP library
  ii  libaprutil1-ldap:amd64                               1.5.4-1                                             amd64        Apache Portable Runtime Utility Library - LDAP Driver
  ii  libkldap4                                            4:4.14.10-1ubuntu2                                  amd64        library for accessing LDAP
  ii  libldap-2.4-2:amd64                                  2.4.41+dfsg-1ubuntu2                                amd64        OpenLDAP libraries
  rc  libldap-2.4-2:i386                                   2.4.31-1+nmu2ubuntu11                               i386         OpenLDAP libraries
  ii  libldap2-dev:amd64                                   2.4.41+dfsg-1ubuntu2                                amd64        OpenLDAP development libraries

To manage notifications about this bug go to:
https://bugs.launchpad.net/keystone/+bug/1551312/+subscriptions
Follow ups

[Bug 1551312] Re: Python unit tests don't run in a clean 15.10 environment
From: OpenStack Infra, 2016-03-02