yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1377043] Re: problem with selenium unittest setup

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Thierry Carrez <thierry.carrez+lp@xxxxxxxxx>
Date: Thu, 05 Feb 2015 16:00:46 -0000
Reply-to: Bug 1377043 <1377043@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
** Changed in: horizon
       Status: Fix Committed => Fix Released

** Changed in: horizon
    Milestone: None => kilo-2

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Dashboard (Horizon).
https://bugs.launchpad.net/bugs/1377043

Title:
  problem with selenium unittest setup

Status in OpenStack Dashboard (Horizon):
  Fix Released

Bug description:
  There is a problem with selenium unittests environment set-up

  If you have a US english environment, "./run_tests.sh --only-selenium"
  works like a charm (but not --with-selenium, see other traceback
  below).

  But if your locales are not US english you need to run the selenium
  tests with "LC_ALL=C ./run_tests.sh --only-selenium" because if you
  don't you get::

      ./run_tests.sh --only-selenium
      Checking environment.
      Environment is up to date.
      Running Horizon application tests
      nosetests horizon --nocapture --nologcapture --exclude-dir=horizon/conf/ --exclude-dir=horizon/test/customization --cover-package=horizon --cover-inclusive --all-modules --exclude-dir=openstack_dashboard/test/integration_tests --verbosity=1
      Creating test database for alias 'default'...
      SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS..F.....
      ======================================================================
      FAIL: test_qunit (horizon.test.tests.selenium_tests.BrowserTests)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/home/yves/openstack/horizon/horizon/test/tests/selenium_tests.py", line 31, in test_qunit
          self.assertEqual(int(failed.text), 0)
      AssertionError: 4 != 0
          '4 != 0' = '%s != %s' % (safe_repr(4), safe_repr(0))
          '4 != 0' = self._formatMessage('4 != 0', '4 != 0')
      >>  raise self.failureException('4 != 0')
          
      
      ----------------------------------------------------------------------
      Ran 134 tests in 56.114s
      
      FAILED (SKIP=126, failures=1)
      Destroying test database for alias 'default'...
      Running openstack_dashboard tests
      nosetests openstack_dashboard --nocapture --nologcapture --cover-package=openstack_dashboard --cover-inclusive --all-modules --exclude-dir=openstack_dashboard/test/integration_tests --verbosity=1
      Creating test database for alias 'default'...
      SSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSSS...FF.....F
      ======================================================================
      FAIL: test_modal_create_user_with_passwords_not_matching (openstack_dashboard.dashboards.identity.users.tests.SeleniumTests)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/home/yves/openstack/horizon/openstack_dashboard/test/helpers.py", line 80, in instance_stub_out
          return fn(self, *args, **kwargs)
        File "/home/yves/openstack/horizon/openstack_dashboard/dashboards/identity/users/tests.py", line 595, in test_modal_create_user_with_passwords_not_matching
          "Error message not found in body")
      AssertionError: Error message not found in body
          'Error message not found in body' = self._formatMessage('Error message not found in body', "%s is not true" % safe_repr(False))
      >>  raise self.failureException('Error message not found in body')
          
      
      ======================================================================
      FAIL: test_update_user_with_passwords_not_matching (openstack_dashboard.dashboards.identity.users.tests.SeleniumTests)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/home/yves/openstack/horizon/openstack_dashboard/test/helpers.py", line 80, in instance_stub_out
          return fn(self, *args, **kwargs)
        File "/home/yves/openstack/horizon/openstack_dashboard/dashboards/identity/users/tests.py", line 621, in test_update_user_with_passwords_not_matching
          "Error message not found in body")
      AssertionError: Error message not found in body
          'Error message not found in body' = self._formatMessage('Error message not found in body', "%s is not true" % safe_repr(False))
      >>  raise self.failureException('Error message not found in body')
          
      
      ======================================================================
      FAIL: test_splash (openstack_dashboard.test.tests.selenium_tests.BrowserTests)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/home/yves/openstack/horizon/openstack_dashboard/test/tests/selenium_tests.py", line 22, in test_splash
          self.assertEqual("Sign In", button.text)
      AssertionError: 'Sign In' != u'Se Connecter'
          "'Sign In' != u'Se Connecter'" = '%s != %s' % (safe_repr('Sign In'), safe_repr(u'Se Connecter'))
          "'Sign In' != u'Se Connecter'" = self._formatMessage("'Sign In' != u'Se Connecter'", "'Sign In' != u'Se Connecter'")
      >>  raise self.failureException("'Sign In' != u'Se Connecter'")
          
      
      ----------------------------------------------------------------------
      Ran 1127 tests in 33.178s
      
      FAILED (SKIP=1116, failures=3)
      Destroying test database for alias 'default'...
      Tests failed.
      

  
  If you run *all* unittests with selenium, some selenium errors are thrown because of a 500 error probably due to something missing in a tearDown or setUp (or both)::

      LC_ALL=C ./run_tests.sh --with-selenium
      Checking environment.
      Environment is up to date.
      Running Horizon application tests
      nosetests horizon --nocapture --nologcapture --exclude-dir=horizon/conf/ --exclude-dir=horizon/test/customization --cover-package=horizon --cover-inclusive --all-modules --verbosity=1
      Creating test database for alias 'default'...
      ......................................................................................................................................
      ----------------------------------------------------------------------
      Ran 134 tests in 66.986s
      
      OK
      Destroying test database for alias 'default'...
      Running openstack_dashboard tests
      nosetests openstack_dashboard --nocapture --nologcapture --cover-package=openstack_dashboard --cover-inclusive --all-modules --verbosity=1
      Creating test database for alias 'default'...
      ........................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................S...................................................................................................................................................................................................................................................EEEEEEEEE....SSSSS..
      ======================================================================
      ERROR: test_inline_editing_cancel (openstack_dashboard.dashboards.identity.projects.tests.SeleniumTests)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/home/yves/openstack/horizon/openstack_dashboard/test/helpers.py", line 80, in instance_stub_out
          return fn(self, *args, **kwargs)
        File "/home/yves/openstack/horizon/openstack_dashboard/dashboards/identity/projects/tests.py", line 1821, in test_inline_editing_cancel
          "//td[@data-update-url='/identity/?action=cell_update"
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 230, in find_element_by_xpath
          return self.find_element(by=By.XPATH, value=xpath)
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 662, in find_element
          {'using': by, 'value': value})['value']
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 173, in execute
          self.error_handler.check_response(response)
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 166, in check_response
          raise exception_class(message, screen, stacktrace)
      NoSuchElementException: Message: u'Unable to locate element: {"method":"xpath","selector":"//td[@data-update-url=\'/identity/?action=cell_update&table=tenants&cell_name=name&obj_id=1\']"}' ; Stacktrace: 
          at FirefoxDriver.prototype.findElementInternal_ (file:///tmp/tmplw460l/extensions/fxdriver@xxxxxxxxxxxxxx/components/driver-component.js:9618:7)
          at fxdriver.Timer.prototype.setTimeout/<.notify (file:///tmp/tmplw460l/extensions/fxdriver@xxxxxxxxxxxxxx/components/driver-component.js:548:5) 
      
      ======================================================================
      ERROR: test_inline_editing_cancel (openstack_dashboard.dashboards.identity.projects.tests.SeleniumTests)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/home/yves/openstack/horizon/openstack_dashboard/test/helpers.py", line 404, in tearDown
          self.mox.VerifyAll()
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/mox.py", line 286, in VerifyAll
          mock_obj._Verify()
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/mox.py", line 506, in _Verify
          raise ExpectedMethodCallsError(self._expected_calls_queue)
      ExpectedMethodCallsError: Verify: Expected methods never called:
        0.  tenant_get.__call__(<IgnoreArg>, u'1', admin=True) -> <Tenant {'description': 'a test tenant.', 'enabled': True, 'domain_name': 'test_domain', 'id': '1', 'domain_id': '1', 'name': 'test_tenant'}>
      
      ======================================================================
      ERROR: test_inline_editing_update (openstack_dashboard.dashboards.identity.projects.tests.SeleniumTests)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/home/yves/openstack/horizon/openstack_dashboard/test/helpers.py", line 80, in instance_stub_out
          return fn(self, *args, **kwargs)
        File "/home/yves/openstack/horizon/openstack_dashboard/dashboards/identity/projects/tests.py", line 1757, in test_inline_editing_update
          "//td[@data-update-url='/identity/?action=cell_update"
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 230, in find_element_by_xpath
          return self.find_element(by=By.XPATH, value=xpath)
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 662, in find_element
          {'using': by, 'value': value})['value']
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 173, in execute
          self.error_handler.check_response(response)
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 166, in check_response
          raise exception_class(message, screen, stacktrace)
      NoSuchElementException: Message: u'Unable to locate element: {"method":"xpath","selector":"//td[@data-update-url=\'/identity/?action=cell_update&table=tenants&cell_name=name&obj_id=1\']"}' ; Stacktrace: 
          at FirefoxDriver.prototype.findElementInternal_ (file:///tmp/tmplw460l/extensions/fxdriver@xxxxxxxxxxxxxx/components/driver-component.js:9618:7)
          at fxdriver.Timer.prototype.setTimeout/<.notify (file:///tmp/tmplw460l/extensions/fxdriver@xxxxxxxxxxxxxx/components/driver-component.js:548:5) 
      
      ======================================================================
      ERROR: test_inline_editing_update (openstack_dashboard.dashboards.identity.projects.tests.SeleniumTests)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/home/yves/openstack/horizon/openstack_dashboard/test/helpers.py", line 404, in tearDown
          self.mox.VerifyAll()
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/mox.py", line 286, in VerifyAll
          mock_obj._Verify()
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/mox.py", line 506, in _Verify
          raise ExpectedMethodCallsError(self._expected_calls_queue)
      ExpectedMethodCallsError: Verify: Expected methods never called:
        0.  tenant_get.__call__(<IgnoreArg>, u'1', admin=True) -> <Tenant {'description': 'a test tenant.', 'enabled': True, 'domain_name': 'test_domain', 'id': '1', 'domain_id': '1', 'name': 'test_tenant'}>
        1.  tenant_get.__call__(<IgnoreArg>, u'1', admin=True) -> <Tenant {'description': 'a test tenant.', 'enabled': True, 'domain_name': 'test_domain', 'id': '1', 'domain_id': '1', 'name': 'test_tenant'}>
        2.  tenant_get.__call__(<IgnoreArg>, u'1', admin=True) -> <Tenant {'description': 'a test tenant.', 'enabled': True, 'domain_name': 'test_domain', 'id': '1', 'domain_id': '1', 'name': 'test_tenant'}>
      
      ======================================================================
      ERROR: test_membership_list_loads_correctly (openstack_dashboard.dashboards.identity.projects.tests.SeleniumTests)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/home/yves/openstack/horizon/openstack_dashboard/test/helpers.py", line 80, in instance_stub_out
          return fn(self, *args, **kwargs)
        File "/home/yves/openstack/horizon/openstack_dashboard/dashboards/identity/projects/tests.py", line 1888, in test_membership_list_loads_correctly
          members = self.selenium.find_element_by_css_selector(member_css_class)
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 374, in find_element_by_css_selector
          return self.find_element(by=By.CSS_SELECTOR, value=css_selector)
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 662, in find_element
          {'using': by, 'value': value})['value']
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 173, in execute
          self.error_handler.check_response(response)
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 166, in check_response
          raise exception_class(message, screen, stacktrace)
      NoSuchElementException: Message: u'Unable to locate element: {"method":"css selector","selector":".available_members"}' ; Stacktrace: 
          at FirefoxDriver.prototype.findElementInternal_ (file:///tmp/tmplw460l/extensions/fxdriver@xxxxxxxxxxxxxx/components/driver-component.js:9618:7)
          at fxdriver.Timer.prototype.setTimeout/<.notify (file:///tmp/tmplw460l/extensions/fxdriver@xxxxxxxxxxxxxx/components/driver-component.js:548:5) 
      
      ======================================================================
      ERROR: test_membership_list_loads_correctly (openstack_dashboard.dashboards.identity.projects.tests.SeleniumTests)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/home/yves/openstack/horizon/openstack_dashboard/test/helpers.py", line 404, in tearDown
          self.mox.VerifyAll()
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/mox.py", line 286, in VerifyAll
          mock_obj._Verify()
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/mox.py", line 506, in _Verify
          raise ExpectedMethodCallsError(self._expected_calls_queue)
      ExpectedMethodCallsError: Verify: Expected methods never called:
        0.  <MultipleTimesGroup "default">
      
      ======================================================================
      ERROR: test_modal_create_user_with_passwords_not_matching (openstack_dashboard.dashboards.identity.users.tests.SeleniumTests)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/home/yves/openstack/horizon/openstack_dashboard/test/helpers.py", line 80, in instance_stub_out
          return fn(self, *args, **kwargs)
        File "/home/yves/openstack/horizon/openstack_dashboard/dashboards/identity/users/tests.py", line 580, in test_modal_create_user_with_passwords_not_matching
          self.selenium.find_element_by_id("users__action_create") \
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 206, in find_element_by_id
          return self.find_element(by=By.ID, value=id_)
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 662, in find_element
          {'using': by, 'value': value})['value']
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 173, in execute
          self.error_handler.check_response(response)
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 166, in check_response
          raise exception_class(message, screen, stacktrace)
      NoSuchElementException: Message: u'Unable to locate element: {"method":"id","selector":"users__action_create"}' ; Stacktrace: 
          at FirefoxDriver.prototype.findElementInternal_ (file:///tmp/tmpwjRuK0/extensions/fxdriver@xxxxxxxxxxxxxx/components/driver-component.js:9618:7)
          at fxdriver.Timer.prototype.setTimeout/<.notify (file:///tmp/tmpwjRuK0/extensions/fxdriver@xxxxxxxxxxxxxx/components/driver-component.js:548:5) 
      
      ======================================================================
      ERROR: test_modal_create_user_with_passwords_not_matching (openstack_dashboard.dashboards.identity.users.tests.SeleniumTests)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/home/yves/openstack/horizon/openstack_dashboard/test/helpers.py", line 404, in tearDown
          self.mox.VerifyAll()
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/mox.py", line 286, in VerifyAll
          mock_obj._Verify()
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/mox.py", line 506, in _Verify
          raise ExpectedMethodCallsError(self._expected_calls_queue)
      ExpectedMethodCallsError: Verify: Expected methods never called:
        0.  get_default_domain.__call__(<IgnoreArg>) -> <APIDictWrapper: {'id': None, 'name': None}>
      
      ======================================================================
      ERROR: test_update_user_with_passwords_not_matching (openstack_dashboard.dashboards.identity.users.tests.SeleniumTests)
      ----------------------------------------------------------------------
      Traceback (most recent call last):
        File "/home/yves/openstack/horizon/openstack_dashboard/test/helpers.py", line 80, in instance_stub_out
          return fn(self, *args, **kwargs)
        File "/home/yves/openstack/horizon/openstack_dashboard/dashboards/identity/users/tests.py", line 616, in test_update_user_with_passwords_not_matching
          self.selenium.find_element_by_id("id_password").send_keys("test")
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 206, in find_element_by_id
          return self.find_element(by=By.ID, value=id_)
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 662, in find_element
          {'using': by, 'value': value})['value']
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/webdriver.py", line 173, in execute
          self.error_handler.check_response(response)
        File "/home/yves/openstack/horizon/.venv/local/lib/python2.7/site-packages/selenium/webdriver/remote/errorhandler.py", line 166, in check_response
          raise exception_class(message, screen, stacktrace)
      NoSuchElementException: Message: u'Unable to locate element: {"method":"id","selector":"id_password"}' ; Stacktrace: 
          at FirefoxDriver.prototype.findElementInternal_ (file:///tmp/tmpwjRuK0/extensions/fxdriver@xxxxxxxxxxxxxx/components/driver-component.js:9618:7)
          at fxdriver.Timer.prototype.setTimeout/<.notify (file:///tmp/tmpwjRuK0/extensions/fxdriver@xxxxxxxxxxxxxx/components/driver-component.js:548:5) 
      
      ----------------------------------------------------------------------
      Ran 1132 tests in 227.234s
      
      FAILED (SKIP=6, errors=9)
      Destroying test database for alias 'default'...
      Running flake8 ...
      Tests failed.

To manage notifications about this bug go to:
https://bugs.launchpad.net/horizon/+bug/1377043/+subscriptions
References

[Bug 1377043] [NEW] problem with selenium unittest setup
From: Yves-Gwenael Bourhis, 2014-10-03