yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #47527
[Bug 1515490] Re: Fail to create instance when single numa_node vcpus that set in flavor is odd number (cpu_policy=dedicated）
So I have a system with 40 cores (2 sockets, 10 cores, hypethreading enabled).
The NUMA topology is as follows:
$ numactl --hardware
available: 2 nodes (0-1)
node 0 cpus: 0 1 2 3 4 5 6 7 8 9 20 21 22 23 24 25 26 27 28 29
node 0 size: 32083 MB
node 0 free: 16652 MB
node 1 cpus: 10 11 12 13 14 15 16 17 18 19 30 31 32 33 34 35 36 37 38 39
node 1 size: 32237 MB
node 1 free: 25386 MB
node distances:
node 0 1
0: 10 21
1: 21 10
I'm using OpenStack provisioned by DevStack on a Fedora 23 host:
$ cat /etc/*-release*
Fedora release 23 (Twenty Three)
...
$ uname -r
4.3.5-300.fc23.x86_64
$ cd /opt/stack/nova
$ git show --oneline
8bafc99 Merge "remove the unnecessary parem of set_vm_state_and_notify"
I defined a flavor similar to yours, but without the unnecessary swap and
disk space and with a smaller RAM allocation (KISS?).
$ openstack flavor create bug.1515490 --id 100 --ram 512 --disk 0 \
--vcpus 30
$ openstack flavor set bug.1515490 \
--property "hw:cpu_policy=dedicated"
--property "hw:numa_nodes=2" \
--property "hw:numa_cpus.0=0-14" \
--property "hw:numa_mem.0=256" \
--property "hw:numa_cpus.1=15-29" \
--property "hw:numa_mem.1=256"
$ openstack flavor show bug.1464286
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| disk | 0 |
| id | 100 |
| name | bug.1515490 |
| os-flavor-access:is_public | True |
| properties | hw:cpu_policy='dedicated', hw:numa_cpus.0='0-14', hw:numa_cpus.1='15-29', hw:numa_mem.0='256', hw:numa_mem.1='256', hw:numa_nodes='2' |
| ram | 512 |
| rxtx_factor | 1.0 |
| swap | |
| vcpus | 30 |
+----------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
I also modified the default quotas to allow allocation of more than 20
cores:
$ openstack quota set --cores 40 demo
I boot one instance...
$ openstack server create --flavor=bug.1515490 \
--image=cirros-0.3.4-x86_64-uec --wait test1
$ sudo virsh list
Id Name State
----------------------------------------------------
22 instance-00000013 running
$ sudo virsh dumpxml 22
<domain type='kvm' id='22'>
<name>instance-00000013</name>
...
<memory unit='KiB'>524288</memory>
<currentMemory unit='KiB'>524288</currentMemory>
<vcpu placement='static'>30</vcpu>
<cputune>
<shares>30720</shares>
<vcpupin vcpu='0' cpuset='1'/>
<vcpupin vcpu='1' cpuset='21'/>
<vcpupin vcpu='2' cpuset='0'/>
<vcpupin vcpu='3' cpuset='20'/>
<vcpupin vcpu='4' cpuset='25'/>
<vcpupin vcpu='5' cpuset='5'/>
<vcpupin vcpu='6' cpuset='8'/>
<vcpupin vcpu='7' cpuset='28'/>
<vcpupin vcpu='8' cpuset='9'/>
<vcpupin vcpu='9' cpuset='29'/>
<vcpupin vcpu='10' cpuset='24'/>
<vcpupin vcpu='11' cpuset='4'/>
<vcpupin vcpu='12' cpuset='27'/>
<vcpupin vcpu='13' cpuset='7'/>
<vcpupin vcpu='14' cpuset='2'/>
<vcpupin vcpu='15' cpuset='35'/>
<vcpupin vcpu='16' cpuset='15'/>
<vcpupin vcpu='17' cpuset='10'/>
<vcpupin vcpu='18' cpuset='30'/>
<vcpupin vcpu='19' cpuset='16'/>
<vcpupin vcpu='20' cpuset='36'/>
<vcpupin vcpu='21' cpuset='11'/>
<vcpupin vcpu='22' cpuset='31'/>
<vcpupin vcpu='23' cpuset='32'/>
<vcpupin vcpu='24' cpuset='12'/>
<vcpupin vcpu='25' cpuset='17'/>
<vcpupin vcpu='26' cpuset='37'/>
<vcpupin vcpu='27' cpuset='18'/>
<vcpupin vcpu='28' cpuset='38'/>
<vcpupin vcpu='29' cpuset='19'/>
<emulatorpin cpuset='0-2,4-5,7-12,15-21,24-25,27-32,35-38'/>
</cputune>
<numatune>
<memory mode='strict' nodeset='0-1'/>
<memnode cellid='0' mode='strict' nodeset='0'/>
<memnode cellid='1' mode='strict' nodeset='1'/>
</numatune>
...
<cpu>
<topology sockets='30' cores='1' threads='1'/>
<numa>
<cell id='0' cpus='0-14' memory='262144' unit='KiB'/>
<cell id='1' cpus='15-29' memory='262144' unit='KiB'/>
</numa>
</cpu>
...
</domain>
So, based on the above, it seems this bug has been resolved in Mitaka and no
longer applies. I think this is an identical issue to bug 1467927 and the
fixes for that also fix this. In fact, I think these patches were backported
to Kilo so they should be fixed there also (though I'm not testing this - feel
free to do so, if required). As such, I'm going to mark this as "fix released".
** Changed in: nova
Assignee: jinquanni（ZTE） (ni-jinquan) => Stephen Finucane (sfinucan)
** Changed in: nova
Status: Confirmed => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1515490
Title:
Fail to create instance when single numa_node vcpus that set in
flavor is odd number (cpu_policy=dedicated）
Status in OpenStack Compute (nova):
Fix Released
Bug description:
1. version
kilo 2015.1.0
2. Relevant log files:
nova-scheduler.log
2015-11-12 12:56:50.472 27408 INFO nova.filters [req-70882f90-d48e-4ca0-8b09-b15a82df792f 9c67877ee37b47e989148a776862c7b8 40fc54dc632c4a02b44bf31d7ff15c82 - - -] Filter NUMATopologyFilter returned 0 hosts for instance 890d0215-5033-44ac-bb8c-3b90ce0db6a4
3. Reproduce steps:
3.1 environment described
I have one compute node with 32 cpus，
It’s numa_topology is as follows：
2 sockets * 8 cores * 2 thread = 32
this mean this host have 2 numa_node，and have 16 cpu on per numa_node
3.2 create a flavor with 30 cpus：
[root@nail-SBCJ-5-3-13 nova(keystone_admin)]# nova flavor-show 30cpu
+----------------------------+-------+
| Property | Value |
+----------------------------+-------+
| OS-FLV-DISABLED:disabled | False |
| OS-FLV-EXT-DATA:ephemeral | 0 |
| disk | 1 |
| extra_specs | {} |
| id | 7 |
| name | 30cpu |
| os-flavor-access:is_public | True |
| ram | 512 |
| rxtx_factor | 1.0 |
| swap | |
| vcpus | 30 |
+----------------------------+-------+
3.3 Set the numa properties as follows（numa_node1:15 vcpu numa_node2:15 vcpu），then，create vm use the flavor
nova flavor-key 30cpu set hw:numa_nodes=2 hw:numa_cpus.0="0-14" hw:numa_mem.0=256 hw:numa_cpus.1="15-29" hw:numa_mem.1=256 hw:cpu_policy=dedicated
Expected result: create success
Actual result ： create failed
Error log: ----> 2. Relevant log files
3.4 Set the numa properties as follows again（numa_node1: 14 vcpu numa_node2: 16 vcpu），then，create vm use the flavor
nova flavor-key 30cpu set hw:numa_nodes=2 hw:numa_cpus.0="0-13" hw:numa_mem.0=256 hw:numa_cpus.1="14-29" hw:numa_mem.1=256 hw:cpu_policy=dedicated
Expected result: create success
Actual result ： create success
3.5 create another flavor with 14 cpus（numa_node1:7 vcpu numa_node2:7 vcpu），Set the numa properties as follows，then，create vm use the flavor
nova flavor-key 14cpu set hw:numa_nodes=2 hw:numa_cpus.0="0-6" hw:numa_mem.0=256 hw:numa_cpus.1="7-13" hw:numa_mem.1=256 hw:cpu_policy=dedicated
Expected result: create success
Actual result ： create success
OR （numa_node1：6 vcpu numa_node2:8 vcpu）
nova flavor-key 14cpu set hw:numa_nodes=2 hw:numa_cpus.0="0-5" hw:numa_mem.0=256 hw:numa_cpus.1="6-13" hw:numa_mem.1=256 hw:cpu_policy=dedicated
Expected result: create success
Actual result ： create success
4
The above results show that：
It must be less than cores per socket of host，when specify the VM ’s single numa_node vcpus is odd number （when cpu pining policy = dedicated）。
but if the number is even number does not exist this kind of situation。
I don't think it is reasonable
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1515490/+subscriptions
References
