yahoo-eng-team team mailing list archive

[Thierry Carrez <thierry.carrez+lp@xxxxxxxxx>]

** Changed in: nova
       Status: Fix Committed => Fix Released

** Changed in: nova
    Milestone: None => havana-2

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1193720

Title:
  XenAPI nova-agent incorrect handling in timeouts

Status in OpenStack Compute (Nova):
  Fix Released

Bug description:
  This is mostly for Windows VMs that have an extra reboot due to
  sysprep, but:

  The XenAPI agent/xenstore plugins try to look for a reply in xenstore
  from the agent.  If the timeout period is reached, we attempt to
  remove the original request... BUT... if the DomID has changed or if
  the agent picked up the request (and didn't respond in time), we get
  an exception on line 226 of the agent.py plugin code:

  223     # No response within the timeout period; bail out
  224     # First, delete the request record
  225     arg_dict["path"] = "data/host/%s" % request_id
  226     xenstore.delete_record(self, arg_dict)
  227     raise TimeoutError(_("TIMEOUT: No response from agent within"
  228                          " %s seconds.") % timeout)

  Which raises back to nova:

  2013-06-22 15:07:17.462 27806 DEBUG nova.virt.xenapi.driver [req-382ff4ee-506f-416f-bdf2-67243170b708 171019 658803] Got exception: ['XENAPI_PLUGIN_FAILURE', 'version', 'XenstoreError', "cmd: ['xenstore-rm', '/local/domain/53/data/host/b5cdcd3f-2d7a-425e-9c86-9804a0a48fd9']; returncode: 1; stderr: xenstore-rm: could not remove path /local/domain/53/data/host/b5cdcd3f-2d7a-425e-9c86-9804a0a48fd9\n; stdout: "] _unwrap_plugin_exceptions /opt/nova/virt/xenapi/driver.py:756
  2013-06-22 15:07:17.463 27806 ERROR nova.virt.xenapi.agent [req-382ff4ee-506f-416f-bdf2-67243170b708 171019 658803] [instance: ab5bd265-9faf-48e8-9653-2ec0479f91e4] The call to version returned an error: ['XENAPI_PLUGIN_FAILURE', 'version', 'XenstoreError', "cmd: ['xenstore-rm', '/local/domain/53/data/host/b5cdcd3f-2d7a-425e-9c86-9804a0a48fd9']; returncode: 1; stderr: xenstore-rm: could not remove path /local/domain/53/data/host/b5cdcd3f-2d7a-425e-9c86-9804a0a48fd9\n; stdout: "]. args={'dom_id': '53', 'id': 'b5cdcd3f-2d7a-425e-9c86-9804a0a48fd9', 'timeout': '30', 'host_uuid': '650ee5c2-92b1-415f-b65f-96adc97e8886'}

  This exception is not caught in nova and the build fails... instead of
  retrying on TIMEOUT.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1193720/+subscriptions