yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1338822] [NEW] CPU limit not used in instance resize

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Daniel Snider <daniel.snider@xxxxxxxxxx>
Date: Mon, 07 Jul 2014 23:02:12 -0000
Reply-to: Bug 1338822 <1338822@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

If I resize an instance to a flavor with more cpus than should be
possible, even more cpus than cpu_allocation_ratio would allow, then
nova proceeds with the resize and the instance state goes to error and
it does exist anymore in the hypervisor.

My environment:
Nova 2014.1 (Icehouse)
Libvirt / KVM hypervisor
Ceph RBD for volume storage
1 nova-manage node, 2 nova-compute nodes (all virtualized on my laptop)
Ubuntu 14.04 for all OSes except for the instance
Cirros 0.3.2 for the instance 
/etc/nova/nova.conf on all nodes contain:
allow_resize_to_same_host=true (although this didn't make a difference!)
ram_allocation_ratio=0.95
cpu_allocation_ratio=5

CPU and RAM allocation_ratio limits are set but only the memory limit took effect. This is from the log below:
memory limit: 1900.95 MB, free: 1388.95 MB
CPUs limit not specified, defaulting to unlimited

This is the log from nova-compute.log when I resize an instance to a
flavor with 500 vcpus:

AUDIT nova.compute.resource_tracker [-] Auditing locally available compute resources
AUDIT nova.compute.resource_tracker [-] Free ram (MB): 1489
AUDIT nova.compute.resource_tracker [-] Free disk (GB): 34
AUDIT nova.compute.resource_tracker [-] Free VCPUS: 2
INFO nova.compute.resource_tracker [-] Compute_service record updated for r-ABCDEF1234:r-ABCDEF1234.vagrant.local
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Attempting claim: memory 500 MB, disk 1 GB, VCPUs 500
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Total memory: 2001 MB, used: 512.00 MB
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] memory limit: 1900.95 MB, free: 1388.95 MB
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Total disk: 34 GB, used: 0.00 GB
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] disk limit not specified, defaulting to unlimited
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Total CPUs: 2 VCPUs, used: 0.00 VCPUs
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] CPUs limit not specified, defaulting to unlimited
AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Claim successful
AUDIT nova.compute.resource_tracker [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] Updating from migration 75204b82-70e9-4e13-b56f-d7b78613cac7
AUDIT nova.compute.manager [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Migrating
WARNING nova.virt.libvirt.utils [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] systool is not installed
WARNING nova.virt.libvirt.utils [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] systool is not installed
INFO urllib3.connectionpool [-] Starting new HTTP connection (1): 192.168.55.253
INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 192.168.55.11:5672
INFO nova.virt.libvirt.driver [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Creating image
ERROR glanceclient.common.http [-] Request returned failure status.
WARNING nova.compute.utils [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Can't access image : Image  could not be found.
INFO nova.virt.libvirt.firewall [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Called setup_basic_filtering in nwfilter
INFO nova.virt.libvirt.firewall [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Ensuring static filters
ERROR nova.virt.libvirt.driver [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] An error occurred while trying to launch a defined domain with xml: <domain type='qemu'>
  <name>instance-00000006</name>
  <uuid>75204b82-70e9-4e13-b56f-d7b78613cac7</uuid>
  <memory unit='KiB'>512000</memory>
  <currentMemory unit='KiB'>512000</currentMemory>
  <vcpu placement='static'>500</vcpu>
  <sysinfo type='smbios'>
    <system>
      <entry name='manufacturer'>OpenStack Foundation</entry>
      <entry name='product'>OpenStack Nova</entry>
      <entry name='version'>2014.1</entry>
      <entry name='serial'>cffca416-65a2-48b5-ba0c-40ad122dc9d2</entry>
      <entry name='uuid'>75204b82-70e9-4e13-b56f-d7b78613cac7</entry>
    </system>
  </sysinfo>
  <os>
    <type arch='x86_64' machine='pc-i440fx-trusty'>hvm</type>
    <boot dev='hd'/>
    <smbios mode='sysinfo'/>
  </os>
  <features>
    <acpi/>
    <apic/>
  </features>
  <cpu mode='host-model'>
    <model fallback='allow'/>
  </cpu>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type='network' device='disk'>
      <driver name='qemu' type='raw' cache='none'/>
      <auth username='volumes'>
        <secret type='ceph' uuid='d9002041-37e4-b53c-1d9e-a5e5ee7e3466'/>
      </auth>
      <source protocol='rbd' name='volumes/volume-22a7ae26-e55b-4aad-a3a1-bbee6ecd6abb'>
        <host name='192.168.55.11' port='6789'/>
      </source>
      <target dev='vda' bus='virtio'/>
      <serial>22a7ae26-e55b-4aad-a3a1-bbee6ecd6abb</serial>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </disk>
    <controller type='usb' index='0'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'/>
    <interface type='bridge'>
      <mac address='fa:16:3e:e3:8f:5d'/>
      <source bridge='br101'/>
      <model type='virtio'/>
      <driver name='qemu'/>
      <filterref filter='nova-instance-instance-00000006-fa163ee38f5d'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </interface>
    <serial type='file'>
      <source path='/var/lib/nova/instances/75204b82-70e9-4e13-b56f-d7b78613cac7/console.log'/>
      <target port='0'/>
    </serial>
    <serial type='pty'>
      <target port='1'/>
    </serial>
    <console type='file'>
      <source path='/var/lib/nova/instances/75204b82-70e9-4e13-b56f-d7b78613cac7/console.log'/>
      <target type='serial' port='0'/>
    </console>
    <input type='tablet' bus='usb'/>
    <input type='mouse' bus='ps2'/>
    <input type='keyboard' bus='ps2'/>
    <graphics type='vnc' port='-1' autoport='yes' listen='192.168.55.101' keymap='en-us'>
      <listen type='address' address='192.168.55.101'/>
    </graphics>
    <video>
      <model type='cirrus' vram='9216' heads='1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </video>
    <memballoon model='virtio'>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </memballoon>
  </devices>
</domain>
ERROR nova.compute.manager [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Setting instance vm_state to ERROR
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Traceback (most recent call last):
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3559, in finish_resize
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     disk_info, image)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3530, in _finish_resize
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     block_device_info, power_on)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5004, in finish_migration
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     block_device_info, power_on)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3644, in _create_domain_and_network
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     power_on=power_on)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3547, in _create_domain
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     domain.XMLDesc(0))
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     six.reraise(self.type_, self.value, self.tb)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3542, in _create_domain
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     domain.createWithFlags(launch_flags)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     result = proxy_call(self._autowrap, f, *args, **kwargs)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     rv = execute(f,*args,**kwargs)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     rv = meth(*args,**kwargs)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 896, in createWithFlags
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] libvirtError: unsupported configuration: Maximum CPUs greater than specified machine type limit
TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] 
ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: unsupported configuration: Maximum CPUs greater than specified machine type limit
TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply
TRACE oslo.messaging.rpc.dispatcher     incoming.message))
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch
TRACE oslo.messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch
TRACE oslo.messaging.rpc.dispatcher     result = getattr(endpoint, method)(ctxt, **new_args)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped
TRACE oslo.messaging.rpc.dispatcher     payload)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped
TRACE oslo.messaging.rpc.dispatcher     return f(self, context, *args, **kw)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 280, in decorated_function
TRACE oslo.messaging.rpc.dispatcher     pass
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 266, in decorated_function
TRACE oslo.messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 333, in decorated_function
TRACE oslo.messaging.rpc.dispatcher     function(self, context, *args, **kwargs)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 254, in decorated_function
TRACE oslo.messaging.rpc.dispatcher     break
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 235, in decorated_function
TRACE oslo.messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 309, in decorated_function
TRACE oslo.messaging.rpc.dispatcher     e, sys.exc_info())
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 296, in decorated_function
TRACE oslo.messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3571, in finish_resize
TRACE oslo.messaging.rpc.dispatcher     self._set_instance_error_state(context, instance['uuid'])
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3559, in finish_resize
TRACE oslo.messaging.rpc.dispatcher     disk_info, image)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3530, in _finish_resize
TRACE oslo.messaging.rpc.dispatcher     block_device_info, power_on)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5004, in finish_migration
TRACE oslo.messaging.rpc.dispatcher     block_device_info, power_on)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3644, in _create_domain_and_network
TRACE oslo.messaging.rpc.dispatcher     power_on=power_on)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3547, in _create_domain
TRACE oslo.messaging.rpc.dispatcher     domain.XMLDesc(0))
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3542, in _create_domain
TRACE oslo.messaging.rpc.dispatcher     domain.createWithFlags(launch_flags)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit
TRACE oslo.messaging.rpc.dispatcher     result = proxy_call(self._autowrap, f, *args, **kwargs)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call
TRACE oslo.messaging.rpc.dispatcher     rv = execute(f,*args,**kwargs)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker
TRACE oslo.messaging.rpc.dispatcher     rv = meth(*args,**kwargs)
TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 896, in createWithFlags
TRACE oslo.messaging.rpc.dispatcher     if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
TRACE oslo.messaging.rpc.dispatcher libvirtError: unsupported configuration: Maximum CPUs greater than specified machine type limit
AUDIT nova.compute.resource_tracker [-] Auditing locally available compute resources
WARNING nova.compute.resource_tracker [-] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Instance not resizing, skipping migration.
AUDIT nova.compute.resource_tracker [-] Free ram (MB): 989
AUDIT nova.compute.resource_tracker [-] Free disk (GB): 33
AUDIT nova.compute.resource_tracker [-] Free VCPUS: -498
WARNING nova.compute.manager [-] Found 1 in the database and 0 on the hypervisor.

** Affects: nova
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1338822

Title:
  CPU limit not used in instance resize

Status in OpenStack Compute (Nova):
  New

Bug description:
  If I resize an instance to a flavor with more cpus than should be
  possible, even more cpus than cpu_allocation_ratio would allow, then
  nova proceeds with the resize and the instance state goes to error and
  it does exist anymore in the hypervisor.

  My environment:
  Nova 2014.1 (Icehouse)
  Libvirt / KVM hypervisor
  Ceph RBD for volume storage
  1 nova-manage node, 2 nova-compute nodes (all virtualized on my laptop)
  Ubuntu 14.04 for all OSes except for the instance
  Cirros 0.3.2 for the instance 
  /etc/nova/nova.conf on all nodes contain:
  allow_resize_to_same_host=true (although this didn't make a difference!)
  ram_allocation_ratio=0.95
  cpu_allocation_ratio=5

  CPU and RAM allocation_ratio limits are set but only the memory limit took effect. This is from the log below:
  memory limit: 1900.95 MB, free: 1388.95 MB
  CPUs limit not specified, defaulting to unlimited

  This is the log from nova-compute.log when I resize an instance to a
  flavor with 500 vcpus:

  AUDIT nova.compute.resource_tracker [-] Auditing locally available compute resources
  AUDIT nova.compute.resource_tracker [-] Free ram (MB): 1489
  AUDIT nova.compute.resource_tracker [-] Free disk (GB): 34
  AUDIT nova.compute.resource_tracker [-] Free VCPUS: 2
  INFO nova.compute.resource_tracker [-] Compute_service record updated for r-ABCDEF1234:r-ABCDEF1234.vagrant.local
  AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Attempting claim: memory 500 MB, disk 1 GB, VCPUs 500
  AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Total memory: 2001 MB, used: 512.00 MB
  AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] memory limit: 1900.95 MB, free: 1388.95 MB
  AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Total disk: 34 GB, used: 0.00 GB
  AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] disk limit not specified, defaulting to unlimited
  AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Total CPUs: 2 VCPUs, used: 0.00 VCPUs
  AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] CPUs limit not specified, defaulting to unlimited
  AUDIT nova.compute.claims [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Claim successful
  AUDIT nova.compute.resource_tracker [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] Updating from migration 75204b82-70e9-4e13-b56f-d7b78613cac7
  AUDIT nova.compute.manager [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Migrating
  WARNING nova.virt.libvirt.utils [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] systool is not installed
  WARNING nova.virt.libvirt.utils [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] systool is not installed
  INFO urllib3.connectionpool [-] Starting new HTTP connection (1): 192.168.55.253
  INFO oslo.messaging._drivers.impl_rabbit [-] Connected to AMQP server on 192.168.55.11:5672
  INFO nova.virt.libvirt.driver [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Creating image
  ERROR glanceclient.common.http [-] Request returned failure status.
  WARNING nova.compute.utils [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Can't access image : Image  could not be found.
  INFO nova.virt.libvirt.firewall [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Called setup_basic_filtering in nwfilter
  INFO nova.virt.libvirt.firewall [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Ensuring static filters
  ERROR nova.virt.libvirt.driver [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] An error occurred while trying to launch a defined domain with xml: <domain type='qemu'>
    <name>instance-00000006</name>
    <uuid>75204b82-70e9-4e13-b56f-d7b78613cac7</uuid>
    <memory unit='KiB'>512000</memory>
    <currentMemory unit='KiB'>512000</currentMemory>
    <vcpu placement='static'>500</vcpu>
    <sysinfo type='smbios'>
      <system>
        <entry name='manufacturer'>OpenStack Foundation</entry>
        <entry name='product'>OpenStack Nova</entry>
        <entry name='version'>2014.1</entry>
        <entry name='serial'>cffca416-65a2-48b5-ba0c-40ad122dc9d2</entry>
        <entry name='uuid'>75204b82-70e9-4e13-b56f-d7b78613cac7</entry>
      </system>
    </sysinfo>
    <os>
      <type arch='x86_64' machine='pc-i440fx-trusty'>hvm</type>
      <boot dev='hd'/>
      <smbios mode='sysinfo'/>
    </os>
    <features>
      <acpi/>
      <apic/>
    </features>
    <cpu mode='host-model'>
      <model fallback='allow'/>
    </cpu>
    <clock offset='utc'/>
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>destroy</on_crash>
    <devices>
      <emulator>/usr/bin/qemu-system-x86_64</emulator>
      <disk type='network' device='disk'>
        <driver name='qemu' type='raw' cache='none'/>
        <auth username='volumes'>
          <secret type='ceph' uuid='d9002041-37e4-b53c-1d9e-a5e5ee7e3466'/>
        </auth>
        <source protocol='rbd' name='volumes/volume-22a7ae26-e55b-4aad-a3a1-bbee6ecd6abb'>
          <host name='192.168.55.11' port='6789'/>
        </source>
        <target dev='vda' bus='virtio'/>
        <serial>22a7ae26-e55b-4aad-a3a1-bbee6ecd6abb</serial>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
      </disk>
      <controller type='usb' index='0'>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x01' function='0x2'/>
      </controller>
      <controller type='pci' index='0' model='pci-root'/>
      <interface type='bridge'>
        <mac address='fa:16:3e:e3:8f:5d'/>
        <source bridge='br101'/>
        <model type='virtio'/>
        <driver name='qemu'/>
        <filterref filter='nova-instance-instance-00000006-fa163ee38f5d'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
      </interface>
      <serial type='file'>
        <source path='/var/lib/nova/instances/75204b82-70e9-4e13-b56f-d7b78613cac7/console.log'/>
        <target port='0'/>
      </serial>
      <serial type='pty'>
        <target port='1'/>
      </serial>
      <console type='file'>
        <source path='/var/lib/nova/instances/75204b82-70e9-4e13-b56f-d7b78613cac7/console.log'/>
        <target type='serial' port='0'/>
      </console>
      <input type='tablet' bus='usb'/>
      <input type='mouse' bus='ps2'/>
      <input type='keyboard' bus='ps2'/>
      <graphics type='vnc' port='-1' autoport='yes' listen='192.168.55.101' keymap='en-us'>
        <listen type='address' address='192.168.55.101'/>
      </graphics>
      <video>
        <model type='cirrus' vram='9216' heads='1'/>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
      </video>
      <memballoon model='virtio'>
        <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
      </memballoon>
    </devices>
  </domain>
  ERROR nova.compute.manager [req-960d005b-0ca5-434b-96dc-e1e7575aef0f 49e16f440dcc4dffa485132bf4c1fceb a5a08d831062498995090b1fb6e7fdf2] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Setting instance vm_state to ERROR
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Traceback (most recent call last):
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3559, in finish_resize
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     disk_info, image)
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3530, in _finish_resize
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     block_device_info, power_on)
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5004, in finish_migration
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     block_device_info, power_on)
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3644, in _create_domain_and_network
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     power_on=power_on)
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3547, in _create_domain
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     domain.XMLDesc(0))
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     six.reraise(self.type_, self.value, self.tb)
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3542, in _create_domain
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     domain.createWithFlags(launch_flags)
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     result = proxy_call(self._autowrap, f, *args, **kwargs)
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     rv = execute(f,*args,**kwargs)
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     rv = meth(*args,**kwargs)
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 896, in createWithFlags
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7]     if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] libvirtError: unsupported configuration: Maximum CPUs greater than specified machine type limit
  TRACE nova.compute.manager [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] 
  ERROR oslo.messaging.rpc.dispatcher [-] Exception during message handling: unsupported configuration: Maximum CPUs greater than specified machine type limit
  TRACE oslo.messaging.rpc.dispatcher Traceback (most recent call last):
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 133, in _dispatch_and_reply
  TRACE oslo.messaging.rpc.dispatcher     incoming.message))
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 176, in _dispatch
  TRACE oslo.messaging.rpc.dispatcher     return self._do_dispatch(endpoint, method, ctxt, args)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/oslo/messaging/rpc/dispatcher.py", line 122, in _do_dispatch
  TRACE oslo.messaging.rpc.dispatcher     result = getattr(endpoint, method)(ctxt, **new_args)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 88, in wrapped
  TRACE oslo.messaging.rpc.dispatcher     payload)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
  TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/exception.py", line 71, in wrapped
  TRACE oslo.messaging.rpc.dispatcher     return f(self, context, *args, **kw)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 280, in decorated_function
  TRACE oslo.messaging.rpc.dispatcher     pass
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
  TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 266, in decorated_function
  TRACE oslo.messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 333, in decorated_function
  TRACE oslo.messaging.rpc.dispatcher     function(self, context, *args, **kwargs)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 254, in decorated_function
  TRACE oslo.messaging.rpc.dispatcher     break
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
  TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 235, in decorated_function
  TRACE oslo.messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 309, in decorated_function
  TRACE oslo.messaging.rpc.dispatcher     e, sys.exc_info())
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
  TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 296, in decorated_function
  TRACE oslo.messaging.rpc.dispatcher     return function(self, context, *args, **kwargs)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3571, in finish_resize
  TRACE oslo.messaging.rpc.dispatcher     self._set_instance_error_state(context, instance['uuid'])
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
  TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3559, in finish_resize
  TRACE oslo.messaging.rpc.dispatcher     disk_info, image)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 3530, in _finish_resize
  TRACE oslo.messaging.rpc.dispatcher     block_device_info, power_on)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 5004, in finish_migration
  TRACE oslo.messaging.rpc.dispatcher     block_device_info, power_on)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3644, in _create_domain_and_network
  TRACE oslo.messaging.rpc.dispatcher     power_on=power_on)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3547, in _create_domain
  TRACE oslo.messaging.rpc.dispatcher     domain.XMLDesc(0))
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/openstack/common/excutils.py", line 68, in __exit__
  TRACE oslo.messaging.rpc.dispatcher     six.reraise(self.type_, self.value, self.tb)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/driver.py", line 3542, in _create_domain
  TRACE oslo.messaging.rpc.dispatcher     domain.createWithFlags(launch_flags)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 179, in doit
  TRACE oslo.messaging.rpc.dispatcher     result = proxy_call(self._autowrap, f, *args, **kwargs)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 139, in proxy_call
  TRACE oslo.messaging.rpc.dispatcher     rv = execute(f,*args,**kwargs)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/eventlet/tpool.py", line 77, in tworker
  TRACE oslo.messaging.rpc.dispatcher     rv = meth(*args,**kwargs)
  TRACE oslo.messaging.rpc.dispatcher   File "/usr/lib/python2.7/dist-packages/libvirt.py", line 896, in createWithFlags
  TRACE oslo.messaging.rpc.dispatcher     if ret == -1: raise libvirtError ('virDomainCreateWithFlags() failed', dom=self)
  TRACE oslo.messaging.rpc.dispatcher libvirtError: unsupported configuration: Maximum CPUs greater than specified machine type limit
  AUDIT nova.compute.resource_tracker [-] Auditing locally available compute resources
  WARNING nova.compute.resource_tracker [-] [instance: 75204b82-70e9-4e13-b56f-d7b78613cac7] Instance not resizing, skipping migration.
  AUDIT nova.compute.resource_tracker [-] Free ram (MB): 989
  AUDIT nova.compute.resource_tracker [-] Free disk (GB): 33
  AUDIT nova.compute.resource_tracker [-] Free VCPUS: -498
  WARNING nova.compute.manager [-] Found 1 in the database and 0 on the hypervisor.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1338822/+subscriptions
Follow ups

[Bug 1338822] Re: CPU limit not used in instance resize
From: melanie witt, 2014-07-25
[Bug 1338822] [NEW] CPU limit not used in instance resize
From: Daniel Snider, 2014-07-07
References

[Bug 1338822] [NEW] CPU limit not used in instance resize
From: Daniel Snider, 2014-07-07