yahoo-eng-team team mailing list archive

[Matt Riedemann <mriedem.os@xxxxxxxxx>]

Public bug reported:

Seen here:

http://logs.openstack.org/45/649345/7/check/openstack-tox-py36/ba15c17
/job-output.txt.gz#_2019-04-18_18_12_52_121182

2019-04-18 18:12:52.121182 | ubuntu-bionic | {3} nova.tests.unit.test_rpc.TestRPC.test_get_transport_url_null [] ... inprogress
2019-04-18 18:12:52.138024 | ubuntu-bionic | '>' not supported between instances of 'NoneType' and 'datetime.datetime'
2019-04-18 18:12:52.353474 | ubuntu-bionic | ERROR: InvocationError for command /home/zuul/src/git.openstack.org/openstack/nova/.tox/py36/bin/stestr run (exited with code 1)

Which seems to kill the test and stestr.

There is also a stack overflow here:

http://logs.openstack.org/45/649345/7/check/openstack-tox-py36/ba15c17
/job-output.txt.gz#_2019-04-18_18_10_53_423952

2019-04-18 18:10:53.423952 | ubuntu-bionic | Fatal Python error: Cannot recover from stack overflow.
2019-04-18 18:10:53.423999 | ubuntu-bionic |
2019-04-18 18:10:53.424162 | ubuntu-bionic | Current thread 0x00007f34d2bcc740 (most recent call first):
2019-04-18 18:10:53.424472 | ubuntu-bionic |   File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/oslo_config/cfg.py", line 2614 in _get
2019-04-18 18:10:53.424794 | ubuntu-bionic |   File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/oslo_config/cfg.py", line 2183 in __getattr__
...
2019-04-18 18:10:53.476982 | ubuntu-bionic |   File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/oslo_config/cfg.py", line 2183 in __getattr__
2019-04-18 18:10:53.477236 | ubuntu-bionic |   ...
2019-04-18 18:10:53.477293 | ubuntu-bionic | Aborted

The stack overflow seems to be just nova since April 15:

http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22%7C%20Fatal%20Python%20error%3A%20Cannot%20recover%20from%20stack%20overflow.%5C%22%20AND%20tags%3A%5C%22console%5C%22&from=7d

I don't see anything that looks related for nova changes around April
15, but maybe something that was released to upper-constraints on April
15:

https://github.com/openstack/requirements/commit/a96ee0e258aafb2880149b3e25dd5959f7b37c09

Nothing in there looks obvious though.

** Affects: nova
     Importance: Undecided
         Status: Confirmed


** Tags: gate-failure testing

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1825435

Title:
  TestRPC unit tests intermittently fail with "'>' not supported between
  instances of 'NoneType' and 'datetime.datetime'"

Status in OpenStack Compute (nova):
  Confirmed

Bug description:
  Seen here:

  http://logs.openstack.org/45/649345/7/check/openstack-tox-py36/ba15c17
  /job-output.txt.gz#_2019-04-18_18_12_52_121182

  2019-04-18 18:12:52.121182 | ubuntu-bionic | {3} nova.tests.unit.test_rpc.TestRPC.test_get_transport_url_null [] ... inprogress
  2019-04-18 18:12:52.138024 | ubuntu-bionic | '>' not supported between instances of 'NoneType' and 'datetime.datetime'
  2019-04-18 18:12:52.353474 | ubuntu-bionic | ERROR: InvocationError for command /home/zuul/src/git.openstack.org/openstack/nova/.tox/py36/bin/stestr run (exited with code 1)

  Which seems to kill the test and stestr.

  There is also a stack overflow here:

  http://logs.openstack.org/45/649345/7/check/openstack-tox-py36/ba15c17
  /job-output.txt.gz#_2019-04-18_18_10_53_423952

  2019-04-18 18:10:53.423952 | ubuntu-bionic | Fatal Python error: Cannot recover from stack overflow.
  2019-04-18 18:10:53.423999 | ubuntu-bionic |
  2019-04-18 18:10:53.424162 | ubuntu-bionic | Current thread 0x00007f34d2bcc740 (most recent call first):
  2019-04-18 18:10:53.424472 | ubuntu-bionic |   File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/oslo_config/cfg.py", line 2614 in _get
  2019-04-18 18:10:53.424794 | ubuntu-bionic |   File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/oslo_config/cfg.py", line 2183 in __getattr__
  ...
  2019-04-18 18:10:53.476982 | ubuntu-bionic |   File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/py36/lib/python3.6/site-packages/oslo_config/cfg.py", line 2183 in __getattr__
  2019-04-18 18:10:53.477236 | ubuntu-bionic |   ...
  2019-04-18 18:10:53.477293 | ubuntu-bionic | Aborted

  The stack overflow seems to be just nova since April 15:

  http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22%7C%20Fatal%20Python%20error%3A%20Cannot%20recover%20from%20stack%20overflow.%5C%22%20AND%20tags%3A%5C%22console%5C%22&from=7d

  I don't see anything that looks related for nova changes around April
  15, but maybe something that was released to upper-constraints on
  April 15:

  https://github.com/openstack/requirements/commit/a96ee0e258aafb2880149b3e25dd5959f7b37c09

  Nothing in there looks obvious though.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1825435/+subscriptions