yahoo-eng-team team mailing list archive

[Matt Riedemann <mriedem.os@xxxxxxxxx>]

Looking at logstash:

http://logstash.openstack.org/#dashboard/file/logstash.json?query=message%3A%5C%22_send_notification%5C%22%20AND%20message%3A%5C%22ValueError%3A%20Circular%20reference%20detected%5C%22&from=7d

This seems to only show up on that aggregate negative test, and it's
probably this error notification wrapper:

https://github.com/openstack/nova/blob/2380659e358770a3f36253b93a112b9779a23958/nova/compute/api.py#L4602

The AggregateHostExists duplicate error comes out of the call to
"aggregate.add_host(host_name)".

I'm not sure what's breaking in the actual notification driver. It also
looks like this is happening on non-py35 jobs too.

** Summary changed:

- Duplicate entry for key 'uniq_aggregate_hosts0host0aggregate_id
+ "ValueError: Circular reference detected" in send_notification

** Also affects: oslo.messaging
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1673375

Title:
  "ValueError: Circular reference detected" in send_notification

Status in OpenStack Compute (nova):
  New
Status in oslo.messaging:
  New

Bug description:
  I guess something is wrong with nova, causing a tempest failure.

  http://logs.openstack.org/08/445308/3/check/gate-tempest-dsvm-py35
  -ubuntu-
  xenial/7bf0d72/logs/screen-n-api.txt.gz#_2017-03-16_05_31_09_399

  
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging Traceback (most recent call last):
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     context)
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     cursor.execute(statement, parameters)
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/pymysql/cursors.py", line 166, in execute
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     result = self._query(query)
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/pymysql/cursors.py", line 322, in _query
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     conn.query(q)
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 852, in query
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1053, in _read_query_result
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     result.read()
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1336, in read
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     first_packet = self.connection._read_packet()
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1010, in _read_packet
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     packet.check_error()
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 393, in check_error
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     err.raise_mysql_exception(self._data)
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/pymysql/err.py", line 107, in raise_mysql_exception
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     raise errorclass(errno, errval)
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging pymysql.err.IntegrityError: (1062, "Duplicate entry 'ubuntu-xenial-rax-iad-7906449-6' for key 'uniq_aggregate_hosts0host0aggregate_id'")

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1673375/+subscriptions