yahoo-eng-team team mailing list archive

[QiangGuan <hzguanqiang@xxxxxxxxxxxxxxxx>]

Sorry guys, Daniel is right. I mixed cputune setting of kvm up with that
of lxc.

On 2013-10-08 19:14 , hzguanqiang@xxxxxxxxxxxxxxxx wrote:
Well, Daniel. If the thing is as you said, I'm wrong.
But The fact is with the same cputune xml configure under different version
of libvirt, It will lead to different results in cgroup.

Details are as following:
hzguanqiang@debian1:~$ sudo virsh version
Compiled against library: libvir 0.9.12
Using library: libvir 0.9.12
Using API: QEMU 0.9.12
Running hypervisor: QEMU 1.1.2
hzguanqiang@debian1:/data/nova/instances/instance-00000053$ cat libvirt.xml 
<domain type="kvm">
  <uuid>2bc80c41-ed34-4a7e-8645-e9af789b29cb</uuid>
  <name>instance-00000053</name>
  <memory>8388608</memory>
  <vcpu cpuset="4-63">8</vcpu>
  <os>
    <type>hvm</type>
    <boot dev="hd"/>
  </os>
  <features>
    <acpi/>
  </features>
  <cputune>
    <shares>32768</shares>
    <quota>100000</quota>
    <period>100000</period>
  </cputune>
  <clock offset="utc">
    <timer name="pit" tickpolicy="delay"/>
    <timer name="rtc" tickpolicy="catchup"/>
  </clock>
  <cpu mode="host-model" match="exact"/>
  <devices>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2" cache="none"/>
      <source file="/data/nova/instances/instance-00000053/disk"/>
      <target bus="virtio" dev="vda"/>
    </disk>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2" cache="none"/>
      <source file="/data/nova/instances/instance-00000053/disk.local"/>
      <target bus="virtio" dev="vdb"/>
    </disk>
    <interface type="bridge">
      <mac address="fa:16:3e:79:04:5c"/>
      <model type="virtio"/>
      <source bridge="br100"/>
      <filterref filter="nova-instance-instance-00000053-fa163e79045c">
        <parameter name="IP" value="10.160.161.21"/>
        <parameter name="DHCPSERVER" value="10.160.160.11"/>
      </filterref>
    </interface>
    <serial type="file">
      <source path="/data/nova/instances/instance-00000053/console.log"/>
    </serial>
    <serial type="pty"/>
    <input type="tablet" bus="usb"/>
    <graphics type="vnc" autoport="yes" keymap="en-us" listen="127.0.0.1"/>
  </devices>
</domain>
hzguanqiang@debian1:/data/nova/instances/instance-00000053$ cat /sys/fs/cgroup/cpu/libvirt/qemu/instance-00000053/cpu.cfs_quota_us 
800000


hzguanqiang@debian2:~$ vir version
Compiled against library: libvirt 1.1.2
Using library: libvirt 1.1.2
Using API: LXC 1.1.2
Running hypervisor: LXC 3.9.6

hzguanqiang@debian2:~$ vir dumpxml instance-0000006c
<domain type='lxc' id='10303'>
  <name>instance-0000006c</name>
  <uuid>ab43db36-ee30-4223-92e8-7652255cffb7</uuid>
  <memory unit='KiB'>1048576</memory>
  <currentMemory unit='KiB'>1048576</currentMemory>
  <vcpu placement='static' cpuset='4-23'>3</vcpu>
  <cputune>
    <shares>6144</shares>
    <period>100000</period>
    <quota>57499</quota>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64'>exe</type>
    <init>/sbin/init</init>
    <cmdline>console=tty0 console=ttyS0</cmdline>
  </os>
  <clock offset='utc'/>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <devices>
    <emulator>/usr/libexec/libvirt_lxc</emulator>
    <filesystem type='mount' accessmode='passthrough'>
      <source dir='/opt/stack/data/nova/instances/ab43db36-ee30-4223-92e8-7652255cffb7/rootfs'/>
      <target dir='/'/>
    </filesystem>
    <interface type='bridge'>
      <mac address='fa:16:3e:16:41:47'/>
      <source bridge='br100'/>
      <target dev='veth0'/>
      <filterref filter='nova-instance-instance-0000006c-fa163e164147'/>
    </interface>
    <console type='pty' tty='/dev/pts/0'>
      <source path='/dev/pts/0'/>
      <target type='lxc' port='0'/>
      <alias name='console0'/>
    </console>
  </devices>
  <seclabel type='none'/>
</domain>
hzguanqiang@debian2:~$ cat /sys/fs/cgroup/cpu/machine/instance-0000006c.libvirt-lxc/cpu.cfs_quota_us 
57499

>From the above, you can see with libvirt version of 0.9.12, the 'cpu.cfs_quota_us' parameter of 
instance cgroup is the value of cputune.period multiplied by vcpu numbers.
And with libvirt version of 1.1.2, they are equal.

On 2013-10-08 17:58 , Daniel Berrange wrote:
I don't believe this is correct.

In libvirt < 0.10.0 we created a single cgroup

$ROOT/libvirt/qemu/$GUESTNAME

and set the 'quota * nvcpus' value at that level

In libvirt >= 0.10.0 we create one cgroup per vCPU

$ROOT/libvirt/qemu/$GUESTNAME/{vcpu0,vcpu1,....}

and set the 'quota' value at the vcpuN level.

The end result is the same in both cases - you get 'quota * nvcpus'
worth of time allowed.

-- 
You received this bug notification because you are subscribed to the bug
report.
https://bugs.launchpad.net/bugs/1230050

Title:
  setting of cputune.quota need to recoginse libvirt version

Status in OpenStack Compute (Nova):
  In Progress

Bug description:
  Setting of cputune.quota has changed in libvirt from version 0.10.0. So the
  cputune.quota parameter of cpu qos which is writed into the libvirt xml should
  be set differently under different version of libvirt.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1230050/+subscriptions


------------------     
Best regards!
GuanQiang
19:04:26


------------------     
Best regards!
GuanQiang
15:12:33

** Changed in: nova
       Status: In Progress => Invalid

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1230050

Title:
  setting of cputune.quota need to recoginse libvirt version

Status in OpenStack Compute (Nova):
  Invalid

Bug description:
  Setting of cputune.quota has changed in libvirt from version 0.10.0. So the
  cputune.quota parameter of cpu qos which is writed into the libvirt xml should
  be set differently under different version of libvirt.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1230050/+subscriptions