yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1267191] Re: openstack-nova-compute service fails with - libvirtError: internal error: CPU feature `avx' specified more than once

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Kashyap Chamarthy <kchamart@xxxxxxxxxx>
Date: Mon, 03 Feb 2014 16:44:27 -0000
Reply-to: Bug 1267191 <1267191@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Hmm, just wondering. . .

Ideally, these kind of bugs  under the hood technologies that affect
OpenStack could be closed as reason  -- "CLOSED, UPSTREAM"

But, closing it as "Fix Committed" is a misleading -- as there's no
actual fix committed in openstack-nova.

So, closing it as "Invalid, Not a bug" (technically, this is right -- as
it's not a Nova bug; but a bug in libvirt that _affected_ Nova).


** Changed in: nova
       Status: Fix Committed => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1267191

Title:
   openstack-nova-compute service fails with - libvirtError: internal
  error: CPU feature `avx' specified more than once

Status in OpenStack Compute (Nova):
  Invalid

Bug description:
  Description of problem
  ---------------------------

  Restarting openstack-nova-compute serivce fails with:

    libvirtError: internal error: CPU feature `avx' specified more than
  once

  Version
  ---------

      $ rpm -q openstack-nova libvirt qemu-system-x86
      openstack-nova-2014.1-0.4.b1.fc21.noarch
      libvirt-1.1.3.2-1.fc20.x86_64
      qemu-system-x86-1.6.1-3.fc20.x86_64

  
  Test env
  ----------

  A two node OpenStack RDO set-up configured manually on two Fedora 20
  VMs:

    - Controller node: Nova, Keystone, Cinder, Glance, Neutron (using Open
      vSwitch plugin and GRE tunneling).

    - Compute node: Nova (nova-compute), Neutron (openvswitch-agent)

  
  How reproducible: Consistently.

  
  Steps to Reproduce
  ------------------

      $ systemctl restart openstack-nova-compute

  Observe /var/log/nova/compute.log

  
  Actual results 
  --------------

  
  $ tail -f /var/log/nova/compute.log
  2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit
  2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup     result = proxy_call(self._autowrap, f, *args, **kwargs)
  2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call
  2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup     rv = execute(f,*args,**kwargs)
  2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker
  2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup     rv = meth(*args,**kwargs)
  2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup   File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU
  2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup     if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
  2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup libvirtError: internal error: CPU feature `avx' specified more than once
  2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup 

  
  Expected results
  ----------------

  Compute service should start successfully.

  Additional info
  ---------------

  Status of openstack-nova-compute service

  $ systemctl status openstack-nova-compute
  openstack-nova-compute.service - OpenStack Nova Compute Server
     Loaded: loaded (/usr/lib/systemd/system/openstack-nova-compute.service; enabled)
     Active: inactive (dead) since Tue 2014-01-08 07:00:07 EST; 13min ago
    Process: 1529 ExecStart=/usr/bin/nova-compute --logfile /var/log/nova/compute.log (code=exited, status=0/SUCCESS)
   Main PID: 1529 (code=exited, status=0/SUCCESS)

  Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup   File "/usr/lib/pyth...in doit
  Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup     result = proxy_ca...kwargs)
  Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup   File "/usr/lib/pyth...xy_call
  Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup     rv = execute(f,*a...kwargs)
  Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup   File "/usr/lib/pyth...tworker
  Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup     rv = meth(*args,**kwargs)
  Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup   File "/usr/lib64/py...lineCPU
  Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup     if ret is None: r...n=self)
  Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup libvirtError: interna...an once
  Jan 07 07:00:07 node2-compute nova-compute[1529]: 2014-01-08 07:00:07.200 1529 TRACE nova.openstack.common.threadgroup

  [reply] [−]
  Private
  Comment 1 Kashyap Chamarthy 2014-01-08 09:21:17 EST

  More contextual trace from compute.log:

  [. . .]
  2014-12-10 22:12:38.789 1429 TRACE nova.virt.libvirt.driver 
  2014-12-10 22:12:39.319 1429 ERROR nova.openstack.common.threadgroup [-] internal error: CPU feature `avx' specified more than once
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup Traceback (most recent call last):
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 117, in wait
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     x.wait()
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/nova/openstack/common/threadgroup.py", line 49, in wait
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     return self.thread.wait()
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 168, in wait
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     return self._exit_event.wait()
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/eventlet/event.py", line 116, in wait
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     return hubs.get_hub().switch()
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 187, in switch
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     return self.greenlet.switch()
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/eventlet/greenthread.py", line 194, in main
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     result = function(*args, **kwargs)
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/nova/openstack/common/service.py", line 448, in run_service
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     service.start()
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/nova/service.py", line 164, in start
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     self.manager.pre_start_hook()
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 822, in pre_start_hook
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     self.update_available_resource(nova.context.get_admin_context())
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/nova/compute/manager.py", line 4971, in update_available_resource
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     nodenames = set(self.driver.get_available_nodes())
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/nova/virt/driver.py", line 980, in get_available_nodes
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     stats = self.get_host_stats(refresh=refresh)
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4569, in get_host_stats
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     return self.host_state.get_host_stats(refresh=refresh)
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 429, in host_state
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     self._host_state = HostState(self)
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4960, in __init__
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     self.update_status()
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 4999, in update_status
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     self.driver.get_instance_capabilities()
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 3702, in get_instance_capabilities
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     caps = self.get_host_capabilities()
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py", line 2742, in get_host_capabilities
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     libvirt.VIR_CONNECT_BASELINE_CPU_EXPAND_FEATURES)
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 187, in doit
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     result = proxy_call(self._autowrap, f, *args, **kwargs)
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 147, in proxy_call
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     rv = execute(f,*args,**kwargs)
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib/python2.7/site-packages/eventlet/tpool.py", line 76, in tworker
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     rv = meth(*args,**kwargs)
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup   File "/usr/lib64/python2.7/site-packages/libvirt.py", line 3622, in baselineCPU
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup     if ret is None: raise libvirtError ('virConnectBaselineCPU() failed', conn=self)
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup libvirtError: internal error: CPU feature `avx' specified more than once
  2014-12-10 22:12:39.319 1429 TRACE nova.openstack.common.threadgroup 
  2014-12-10 22:12:58.904 1467 WARNING nova.virt.libvirt.driver [req-88723bba-e677-4956-9b70-1f7fe66dbff7 None None] Cannot update service status on host: node2-compute,due to an unexpected exception.
  [. . .]

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1267191/+subscriptions
References

[Bug 1267191] [NEW] openstack-nova-compute service fails with - libvirtError: internal error: CPU feature `avx' specified more than once
From: Kashyap Chamarthy, 2014-01-08