yahoo-eng-team team mailing list archive

[Boden R <bodenvmw@xxxxxxxxx>]

Public bug reported:

During the instance boot (spawn/run) process, neutron ports are
allocated for the instance if necessary. If the instance fails to spawn
(say as a result of a compute host failure), the default behavior is to
reschedule the instance and leave it's networking resources in-tact for
potential reuse on the rescheduled host (as per
deallocate_networks_on_reschedule() [1] which returns False for most
compute drivers).

All is good if the instance is successfully rescheduled, but if the
reschedule fails (say no more applicable hosts) the allocated ports are
left as-is and effectively orphaned.

There are some related defects ([2] and [3]), but they don't quite touch
on the particular behavior described herein.

Obviously there are a number of ways to address this issue, but the most
obvious is perhaps nova should be aware of the reschedule failure and
deallocate any resources which may have been left in-tact for the
reschedule.

I'm running devstack all-in-one setup from openstack master branches.

nova --version
2.32.0
neutron --version
3.1.0

The easiest way to repo is to use an all-in-one devstack (only 1 compute
host) simulate a host spawn failure by editing the spwan() method of
your compute driver to raise an exception at the end of the method and
simply try to boot a server. In this setup there's only 1 host so the
reschedule will fail and you can verify the port allocated for the
instance still exists after trying to boot the instance.


[1] https://github.com/openstack/nova/blob/master/nova/virt/driver.py#L1273
[2] https://bugs.launchpad.net/nova/+bug/1410739
[3] https://bugs.launchpad.net/nova/+bug/1327124

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1510979

Title:
  Instance reschedule failure leaves orphaned neutron ports

Status in OpenStack Compute (nova):
  New

Bug description:
  During the instance boot (spawn/run) process, neutron ports are
  allocated for the instance if necessary. If the instance fails to
  spawn (say as a result of a compute host failure), the default
  behavior is to reschedule the instance and leave it's networking
  resources in-tact for potential reuse on the rescheduled host (as per
  deallocate_networks_on_reschedule() [1] which returns False for most
  compute drivers).

  All is good if the instance is successfully rescheduled, but if the
  reschedule fails (say no more applicable hosts) the allocated ports
  are left as-is and effectively orphaned.

  There are some related defects ([2] and [3]), but they don't quite
  touch on the particular behavior described herein.

  Obviously there are a number of ways to address this issue, but the
  most obvious is perhaps nova should be aware of the reschedule failure
  and deallocate any resources which may have been left in-tact for the
  reschedule.

  I'm running devstack all-in-one setup from openstack master branches.

  nova --version
  2.32.0
  neutron --version
  3.1.0

  The easiest way to repo is to use an all-in-one devstack (only 1
  compute host) simulate a host spawn failure by editing the spwan()
  method of your compute driver to raise an exception at the end of the
  method and simply try to boot a server. In this setup there's only 1
  host so the reschedule will fail and you can verify the port allocated
  for the instance still exists after trying to boot the instance.

  
  [1] https://github.com/openstack/nova/blob/master/nova/virt/driver.py#L1273
  [2] https://bugs.launchpad.net/nova/+bug/1410739
  [3] https://bugs.launchpad.net/nova/+bug/1327124

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1510979/+subscriptions