← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1609526] [NEW] nova doesn't clean up network ports when an image fails to download from glance

 

Public bug reported:

Consider this environment:

* Running stable/mitaka (latest available)
* Four hypervisors
* Two glance nodes (A and B)
* The glance nodes are storing images locally but the image files aren't in sync between both hosts

When I request a new instance, the following happens:

* Instance is scheduled to hypervisor A
* Hypervisor A checks to see if the image is available for use -- SUCCESS
* Hypervisor A calls neutron for a network port -- SUCCESS
* Hypervisor A tries to download image from glance server A -- FAILURE (glance server A doesn't have the image cached on its filesystem)
* Instance is rescheduled to hypervisor B
* Hypervisor B checks to see if the image is available for use -- SUCCESS
* Hypervisor B calls neutron for a network port -- SUCCESS
* Hypervisor B downloads an image from glance server B -- SUCCESS (glance server B has the image on its filesystem)

The instance will come up on hypervisor B with two ports attached to the
instance.  The second one (requested by hypervisor B) will be up and
fully functional.  The first port (requested by hypervisor A) will be
marked as 'down' and won't be usable.

It seems like nova-compute should call neutron to say "I don't need that
network port any longer since I can't get what I need to build the rest
of the instance" and clean up that port.  Without the cleanup, an
instance can end up with a lot of ports attached and potentially waste a
lot of IPv4 address space.

I wrote more details on this issue here: https://major.io/2016/08/03
/openstack-instances-come-online-with-multiple-network-ports-attached/

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1609526

Title:
  nova doesn't clean up network ports when an image fails to download
  from glance

Status in OpenStack Compute (nova):
  New

Bug description:
  Consider this environment:

  * Running stable/mitaka (latest available)
  * Four hypervisors
  * Two glance nodes (A and B)
  * The glance nodes are storing images locally but the image files aren't in sync between both hosts

  When I request a new instance, the following happens:

  * Instance is scheduled to hypervisor A
  * Hypervisor A checks to see if the image is available for use -- SUCCESS
  * Hypervisor A calls neutron for a network port -- SUCCESS
  * Hypervisor A tries to download image from glance server A -- FAILURE (glance server A doesn't have the image cached on its filesystem)
  * Instance is rescheduled to hypervisor B
  * Hypervisor B checks to see if the image is available for use -- SUCCESS
  * Hypervisor B calls neutron for a network port -- SUCCESS
  * Hypervisor B downloads an image from glance server B -- SUCCESS (glance server B has the image on its filesystem)

  The instance will come up on hypervisor B with two ports attached to
  the instance.  The second one (requested by hypervisor B) will be up
  and fully functional.  The first port (requested by hypervisor A) will
  be marked as 'down' and won't be usable.

  It seems like nova-compute should call neutron to say "I don't need
  that network port any longer since I can't get what I need to build
  the rest of the instance" and clean up that port.  Without the
  cleanup, an instance can end up with a lot of ports attached and
  potentially waste a lot of IPv4 address space.

  I wrote more details on this issue here: https://major.io/2016/08/03
  /openstack-instances-come-online-with-multiple-network-ports-attached/

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1609526/+subscriptions


Follow ups