yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1794126] Re: typo in IVS related privsep method, VM creation failure

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Matt Riedemann <mriedem.os@xxxxxxxxx>
Date: Wed, 03 Oct 2018 19:28:56 -0000
Reply-to: Bug 1794126 <1794126@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
** Also affects: nova/queens
   Importance: Undecided
       Status: New

** Changed in: nova
       Status: New => Won't Fix

** Changed in: nova/queens
       Status: New => In Progress

** Changed in: nova/queens
     Assignee: (unassigned) => Aditya Vaja (wolverine-av)

** Changed in: nova/queens
   Importance: Undecided => High

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1794126

Title:
  typo in IVS related privsep method, VM creation failure

Status in OpenStack Compute (nova):
  Won't Fix
Status in OpenStack Compute (nova) queens series:
  In Progress

Bug description:
  Description
  ===========
  As part of the change here [1], a few calls to edit files as root were moved to a privsep class that encompasses all those edits.
  A typo creeped in, causing a failure when creating a VM with VIF type IVS.

  Method in question:
  @nova.privsep.dacnet_admin_pctxt.entrypoint
  def disable_ipv6(interface):
      """Disable ipv6 for a bridge."""
      with open('/proc/sys/net/ipv6/conf/%s/disable_ipv' % interface, 'w') as f:
          f.write('1')

  The name of the file should be 'disable_ipv6' instead of 'disable_ipv'.
  Missing '6' at the end.

  Logs & Configs
  ==============
  log excerpt for failure:
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [req-e2560c77-8950-4dc6-8f4f-44254b184bbb 17bbd148d3f34a42bede6bdb8fcc84aa 62daca4c6dff406685f3fcf754cb82c4 - default default] [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d] Instance failed to spawn: IOError: [Errno 2] No such file or directory
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d] Traceback (most recent call last):
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2236, in _build_resources
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]     yield resources
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 2016, in _build_and_run_instance
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]     block_device_info=block_device_info)
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3100, in spawn
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]     destroy_disks_on_failure=True)
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5590, in _create_domain_and_network
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]     destroy_disks_on_failure)
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]     self.force_reraise()
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]     six.reraise(self.type_, self.value, self.tb)
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 5549, in _create_domain_and_network
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]     self.plug_vifs(instance, network_info)
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 829, in plug_vifs
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]     self.vif_driver.plug(instance, vif)
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 896, in plug
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]     func(instance, vif)
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 722, in plug_ivs
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]     self.plug_ivs_hybrid(instance, vif)
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/vif.py", line 703, in plug_ivs_hybrid
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]     nova.privsep.libvirt.disable_ipv6(br_name)
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]   File "/usr/lib/python2.7/site-packages/oslo_privsep/priv_context.py", line 207, in _wrap
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]     return self.channel.remote_call(name, args, kwargs)
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]   File "/usr/lib/python2.7/site-packages/oslo_privsep/daemon.py", line 202, in remote_call
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]     raise exc_type(*result[2])
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d] IOError: [Errno 2] No such file or directory
  2018-09-24 14:21:47.167 1 ERROR nova.compute.manager [instance: a631eff3-9d68-4e06-a7ce-4338e29e8e2d]

  I'll open a fix PR as well. And add a check that file exists before
  editing - apparently that's not true for this particular case and
  wasn't carried over from existing code.

  Note: only affects stable/queens branch. Later releases has proper
  calling to os-vif-bigswitch and this code has been cleaned up from
  Nova.

  [1] https://review.openstack.org/#/c/489438/

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1794126/+subscriptions
References

[Bug 1794126] [NEW] typo in IVS related privsep method, VM creation failure
From: Aditya Vaja, 2018-09-24