yahoo-eng-team team mailing list archive

[Matt Riedemann <mriedem.os@xxxxxxxxx>]

** Also affects: nova/newton
   Importance: Undecided
       Status: New

** Also affects: nova/pike
   Importance: Undecided
       Status: New

** Also affects: nova/ocata
   Importance: Undecided
       Status: New

** Changed in: nova/ocata
       Status: New => Confirmed

** Changed in: nova/pike
       Status: New => Confirmed

** Changed in: nova/pike
   Importance: Undecided => High

** Changed in: nova/ocata
   Importance: Undecided => High

** Changed in: nova
     Assignee: melanie witt (melwitt) => int32bit (int32bit)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1744325

Title:
  If a rebuild is refused by the scheduler, the instance's imageref is
  not rolled back

Status in OpenStack Compute (nova):
  In Progress
Status in OpenStack Compute (nova) newton series:
  New
Status in OpenStack Compute (nova) ocata series:
  Confirmed
Status in OpenStack Compute (nova) pike series:
  Confirmed

Bug description:
  Description
  ===========

  Since CVE-2017-16239, we now go through the scheduler for rebuilds. If
  the scheduler refuses a rebuild with a new image because of filter
  constraints (for example IsolatedHostsFilter), the instance's imageref
  is set to the new image and never rolled back.

  Steps to reproduce
  ==================

  1. Configure IsolatedHostsFilter:
   
     [filter_scheduler]
     enabled_filters = [...],IsolatedHostsFilter
     isolated_images = 41d3e5ca-14cf-436c-9413-4826b5c8bdb1
     isolated_hosts = ubuntu
     restrict_isolated_hosts_to_isolated_images = true

  2. Have two images, one isolated and one not:

     $ openstack image list

       8d0581a5-ed9d-4b98-a766-a41efbc99929 | centos                   | active
       41d3e5ca-14cf-436c-9413-4826b5c8bdb1 | cirros-0.3.5-x86_64-disk | active

       cirros is the isolated one

  3. Have only one hypervisor (the isolated one):

     $ openstack hypervisor list

       ubuntu              | QEMU            | 192.168.100.194 | up

  4. To confirm, boot a centos (non-isolated) image, expecting it to be
  refused by the scheduler:

     $ openstack server create \
       --image 8d0581a5-ed9d-4b98-a766-a41efbc99929 \
       --flavor \
       m1.nano centos-test-expect-fail

     $ openstack server list

       centos-test-expect-fail | ERROR  |          | centos | m1.nano

  5. Boot a cirros (isolated) image:

     $ openstack server create \
       --image 41d3e5ca-14cf-436c-9413-4826b5c8bdb1 \
       --flavor m1.nano \
       cirros-test-expect-success

     $ openstack server list

       cirros-test-expect-success | ACTIVE | [...] |
  cirros-0.3.5-x86_64-disk | m1.nano

  6. Rebuild the cirros instance with centos:

     $ nova --debug rebuild cirros-test-expect-success centos
     
       DEBUG (session:722) POST call to compute for 
       http://192.168.100.194/compute/v2.1/servers/d9d98bf7-623e-4587-b82c-06f36abf59cb/action
       used request id req-c234346a-6e05-47cf-a0cd-45f89d11e15d

  7. Observer the rebuild being refused in the conductor:

     WARNING nova.conductor.manager
     [None req-c234346a-6e05-47cf-a0cd-45f89d11e15d demo admin]
     [instance: d9d98bf7-623e-4587-b82c-06f36abf59cb]
     No valid host found for rebuild: NoValidHost_Remote:
     No valid host was found. There are not enough hosts available.

  8. Observe the API is showing the new centos image for the instance:

     $ nova show cirros-test-expect-success

       [...]
       image | centos (8d0581a5-ed9d-4b98-a766-a41efbc99929)
       [...]

  Expected result
  ===============

  Some indication that the rebuild was refused, or at least rolling back
  the instance's imageref.

  Actual result
  =============

  No indication that the rebuild was refused, and worse, we now have a
  wrong imageref for the instance.

  Environment
  ===========

  1. Exact version of OpenStack you are running. See the following
     
     This was picked up by QE for stable/pike, and is still present in master,
     and presumably in all versions affected by the CVE fix, including newton,
     which is now EOL.

  2. Which hypervisor did you use?
     libvirt+kvm

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1744325/+subscriptions