← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1673375] Re: "ValueError: Circular reference detected" in send_notification

 

Reviewed:  https://review.openstack.org/446948
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=3bf177a59cfd0b4e74dba256c3466ba2ea9bfbf7
Submitter: Jenkins
Branch:    master

commit 3bf177a59cfd0b4e74dba256c3466ba2ea9bfbf7
Author: Balazs Gibizer <balazs.gibizer@xxxxxxxxxxxx>
Date:   Fri Mar 17 11:24:49 2017 +0100

    do not include context to exception notification
    
    The wrap_exception decorator optionally emited a notification.
    Based on the code comments the original intention was not to include the
    context to that notification due to security reasons. However the
    implementation did included the context to the payload of the legacy
    notification.
    
    Recently we saw circural reference errors during the payload serialization
    of this notification. Based on the logs the only complex data structure
    that could cause circural reference is the context. So this patch
    removes the context from the legacy exception notification.
    
    The versioned exception notification is not affected as it does not
    contain the args of the decorated function.
    
    Closes-Bug: #1673375
    Change-Id: I1d217620e52d45595a3e0e49ed57b4ab33cd1688


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1673375

Title:
  "ValueError: Circular reference detected" in send_notification

Status in OpenStack Compute (nova):
  Fix Released
Status in oslo.messaging:
  New

Bug description:
  I guess something is wrong with nova, causing a tempest failure.

  http://logs.openstack.org/08/445308/3/check/gate-tempest-dsvm-py35
  -ubuntu-
  xenial/7bf0d72/logs/screen-n-api.txt.gz#_2017-03-16_05_31_09_399

  
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging Traceback (most recent call last):
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1139, in _execute_context
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     context)
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/default.py", line 450, in do_execute
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     cursor.execute(statement, parameters)
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/pymysql/cursors.py", line 166, in execute
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     result = self._query(query)
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/pymysql/cursors.py", line 322, in _query
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     conn.query(q)
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 852, in query
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     self._affected_rows = self._read_query_result(unbuffered=unbuffered)
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1053, in _read_query_result
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     result.read()
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1336, in read
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     first_packet = self.connection._read_packet()
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 1010, in _read_packet
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     packet.check_error()
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/pymysql/connections.py", line 393, in check_error
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     err.raise_mysql_exception(self._data)
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging   File "/usr/local/lib/python3.5/dist-packages/pymysql/err.py", line 107, in raise_mysql_exception
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging     raise errorclass(errno, errval)
  2017-03-16 05:31:09.399 23355 ERROR oslo_messaging.notify.messaging pymysql.err.IntegrityError: (1062, "Duplicate entry 'ubuntu-xenial-rax-iad-7906449-6' for key 'uniq_aggregate_hosts0host0aggregate_id'")

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1673375/+subscriptions


References