yahoo-eng-team team mailing list archive

[sammyliu <esunsnote@xxxxxxx>]

Public bug reported:

Env:

Node 1: controller, running controller services, Mysql, RabbitMQ
Node 2: compute1, running nova_compute
Node 3: network, running neutron_server
Node 4: block1, running cinder-volume

version: Juno stable
Nova version: 2.19

Procedure for reproduce:

Volume creation works. when launching instance in dashboard or with Nova
Cli, exception was thrown in instance.save() function in file
"/usr/lib/python2.7/dist-packages/nova/compute/manager.py" on compute
node, line 2273. Following is the trace in nova-compute log files with
debug=true:

**RemoteError: Remote error: OperationalError (OperationalError) (1048,
"Column 'instance_uuid' cannot be null") 'UPDATE instance_extra SET
updated_at=%s, instance_uuid=%s WHERE instance_extra.id = %s'
(datetime.datetime(2014, 12, 25, 11, 11, 36, 186194), None, 13L)**

19:11:36.105 20544 TRACE oslo.messaging.rpc.dispatcher [u'Traceback (most recent call last):\n', u'  
File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 400, in _object_dispatch\n    return getattr(target, method)(context, *args, **kwargs)\n', u'  
File "/usr/lib/python2.7/dist-packages/nova/objects/base.py", line 204, in wrapper\n    return fn(self, ctxt, *args, **kwargs)\n', u'  
File "/usr/lib/python2.7/dist-packages/nova/objects/instance.py", line 500, in save\n    columns_to_join=_expected_cols(expected_attrs))\n', u'  
File "/usr/lib/python2.7/dist-packages/nova/db/api.py", line 746, in instance_update_and_get_original\n    columns_to_join=columns_to_join)\n', u'  
File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 143, in wrapper\n    return f(*args, **kwargs)\n', u'  
File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 2289, in instance_update_and_get_original\n    columns_to_join=columns_to_join)\n', u'  
File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 2380, in _instance_update\n    session.add(instance_ref)\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 470, in __exit__\n    self.rollback()\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__\n    compat.reraise(exc_type, exc_value, exc_tb)\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 467, in __exit__\n    self.commit()\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 377, in commit\n    self._prepare_impl()\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 357, in _prepare_impl\n    self.session.flush()\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1919, in flush\n    self._flush(objects)\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2037, in _flush\n    transaction.rollback(_capture_exception=True)\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__\n    compat.reraise(exc_type, exc_value, exc_tb)\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2001, in _flush\n    flush_context.execute()\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 372, in execute\n    rec.execute(self)\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 526, in execute\n    uow\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 60, in save_obj\n    mapper, table, update)\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 518, in _emit_update_statements\n    execute(statement, params)\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 729, in execute\n    return meth(self, multiparams, params)\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 321, in _execute_on_connection\n    return connection._execute_clauseelement(self, multiparams, params)\n', u'  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement\n    compiled_sql, distilled_params\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 958, in _execute_context\n    context)\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1156, in _handle_dbapi_exception\n    util.raise_from_cause(newraise, exc_info)\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause\n    reraise(type(exception), exception, tb=exc_tb)\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 951, in _execute_context\n    context)\n', u'  
File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 436, in do_execute\n    cursor.execute(statement, parameters)\n', u'  
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute\n    self.errorhandler(self, exc, value)\n', u'  
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler\n    raise errorclass, errorvalue\n', u'OperationalError: (OperationalError) (1048, "Column \'instance_uuid\' cannot be null") \'UPDATE instance_extra SET updated_at=%s, instance_uuid=%s WHERE instance_extra.id = %s\' (datetime.datetime(2014, 12, 25, 11, 11, 36, 186194), None, 13L)\n'].
After commenting this code line, the operation can move forward, but the similar problem occurred in other instance.save() invoking positions.

thank you.

** Affects: nova
     Importance: Undecided
         Status: New

** Attachment added: "nova-compute log file"
   https://bugs.launchpad.net/bugs/1405741/+attachment/4287682/+files/nova-compute.log

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1405741

Title:
  OpenStack Juno release Nova error: RemoteError: Remote error:
  OperationalError (OperationalError) (1048, “Column 'instance_uuid'
  cannot be null”)

Status in OpenStack Compute (Nova):
  New

Bug description:
  Env:

  Node 1: controller, running controller services, Mysql, RabbitMQ
  Node 2: compute1, running nova_compute
  Node 3: network, running neutron_server
  Node 4: block1, running cinder-volume

  version: Juno stable
  Nova version: 2.19

  Procedure for reproduce:

  Volume creation works. when launching instance in dashboard or with
  Nova Cli, exception was thrown in instance.save() function in file
  "/usr/lib/python2.7/dist-packages/nova/compute/manager.py" on compute
  node, line 2273. Following is the trace in nova-compute log files with
  debug=true:

  **RemoteError: Remote error: OperationalError (OperationalError)
  (1048, "Column 'instance_uuid' cannot be null") 'UPDATE instance_extra
  SET updated_at=%s, instance_uuid=%s WHERE instance_extra.id = %s'
  (datetime.datetime(2014, 12, 25, 11, 11, 36, 186194), None, 13L)**

  19:11:36.105 20544 TRACE oslo.messaging.rpc.dispatcher [u'Traceback (most recent call last):\n', u'  
  File "/usr/lib/python2.7/dist-packages/nova/conductor/manager.py", line 400, in _object_dispatch\n    return getattr(target, method)(context, *args, **kwargs)\n', u'  
  File "/usr/lib/python2.7/dist-packages/nova/objects/base.py", line 204, in wrapper\n    return fn(self, ctxt, *args, **kwargs)\n', u'  
  File "/usr/lib/python2.7/dist-packages/nova/objects/instance.py", line 500, in save\n    columns_to_join=_expected_cols(expected_attrs))\n', u'  
  File "/usr/lib/python2.7/dist-packages/nova/db/api.py", line 746, in instance_update_and_get_original\n    columns_to_join=columns_to_join)\n', u'  
  File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 143, in wrapper\n    return f(*args, **kwargs)\n', u'  
  File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 2289, in instance_update_and_get_original\n    columns_to_join=columns_to_join)\n', u'  
  File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 2380, in _instance_update\n    session.add(instance_ref)\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 470, in __exit__\n    self.rollback()\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__\n    compat.reraise(exc_type, exc_value, exc_tb)\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 467, in __exit__\n    self.commit()\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 377, in commit\n    self._prepare_impl()\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 357, in _prepare_impl\n    self.session.flush()\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1919, in flush\n    self._flush(objects)\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2037, in _flush\n    transaction.rollback(_capture_exception=True)\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 60, in __exit__\n    compat.reraise(exc_type, exc_value, exc_tb)\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 2001, in _flush\n    flush_context.execute()\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 372, in execute\n    rec.execute(self)\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 526, in execute\n    uow\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 60, in save_obj\n    mapper, table, update)\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 518, in _emit_update_statements\n    execute(statement, params)\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 729, in execute\n    return meth(self, multiparams, params)\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 321, in _execute_on_connection\n    return connection._execute_clauseelement(self, multiparams, params)\n', u'  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement\n    compiled_sql, distilled_params\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 958, in _execute_context\n    context)\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1156, in _handle_dbapi_exception\n    util.raise_from_cause(newraise, exc_info)\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause\n    reraise(type(exception), exception, tb=exc_tb)\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 951, in _execute_context\n    context)\n', u'  
  File "/usr/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 436, in do_execute\n    cursor.execute(statement, parameters)\n', u'  
  File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute\n    self.errorhandler(self, exc, value)\n', u'  
  File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler\n    raise errorclass, errorvalue\n', u'OperationalError: (OperationalError) (1048, "Column \'instance_uuid\' cannot be null") \'UPDATE instance_extra SET updated_at=%s, instance_uuid=%s WHERE instance_extra.id = %s\' (datetime.datetime(2014, 12, 25, 11, 11, 36, 186194), None, 13L)\n'].
  After commenting this code line, the operation can move forward, but the similar problem occurred in other instance.save() invoking positions.

  thank you.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1405741/+subscriptions