yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #18127
[Bug 1151845] Re: Instance counter is incorrect after live-migration is done
** Changed in: nova
Assignee: Jason Dillaman (jdillaman) => (unassigned)
** Changed in: nova
Status: In Progress => Invalid
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1151845
Title:
Instance counter is incorrect after live-migration is done
Status in OpenStack Compute (Nova):
Invalid
Bug description:
Environment: grizzly
After live-migration is done, check DB table compute_nodes, the
counter of running_vms of source hypervisor will be decreased but the
counter of running_vms of destination hypervisor won't be increased.
(Actually the counter "num_instances" from compute_node_stats will not
be increased as well)
1. Three compute nodes in the system, and for each of them have one active instance:
[root@rhel8233 ~]# nova list
+--------------------------------------+-------+--------+----------+
| ID | Name | Status | Networks |
+--------------------------------------+-------+--------+----------+
| 4b25db8e-b3a1-4177-aed1-32506b94eb88 | myvm1 | ACTIVE | | ---> on host rhel8234
| 00565dce-4828-4eb2-ab29-9ef0403e5ec0 | myvm2 | ACTIVE | | ---> on host rhel8235
| e4871da2-3a6f-4f34-93cf-bb2511b46cf8 | myvm3 | ACTIVE | | ---> on host rhel8236
+--------------------------------------+-------+--------+----------+
Check mysql, the counter is correct.
mysql> select * from compute_nodes;
+---------------------+---------------------+------------+----+------------+-------+-----------+----------+------------+----------------+---------------+-----------------+--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------+-------------+--------------+------------------+-------------+---------------------+---------+
| created_at | updated_at | deleted_at | id | service_id | vcpus | memory_mb | local_gb | vcpus_used | memory_mb_used | local_gb_used | hypervisor_type | hypervisor_version | cpu_info | disk_available_least | free_ram_mb | free_disk_gb | current_workload | running_vms | hypervisor_hostname | deleted |
+---------------------+---------------------+------------+----+------------+-------+-----------+----------+------------+----------------+---------------+-----------------+--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------+-------------+--------------+------------------+-------------+---------------------+---------+
| 2013-03-05 05:05:29 | 2013-03-07 08:37:44 | NULL | 1 | 5 | 4 | 1877 | 546 | 1 | 1024 | 0 | QEMU | 12001 | {"vendor": "Intel", "model": "Penryn", "arch": "x86_64", "features": ["hypervisor", "osxsave", "xsave", "ht", "ss", "acpi", "ds", "vme"], "topology": {"cores": 2, "threads": 1, "sockets": 2}} | 88 | 853 | 546 | 1 | 1 | rhel8234 | 0 |
| 2013-03-05 05:11:26 | 2013-03-07 07:48:41 | NULL | 2 | 7 | 4 | 1877 | 546 | 1 | 1024 | 0 | QEMU | 12001 | {"vendor": "Intel", "model": "Penryn", "arch": "x86_64", "features": ["hypervisor", "osxsave", "xsave", "ht", "ss", "acpi", "ds", "vme"], "topology": {"cores": 2, "threads": 1, "sockets": 2}} | 87 | 853 | 546 | 0 | 1 | rhel8235 | 0 |
| 2013-03-05 05:12:36 | 2013-03-07 07:48:07 | NULL | 3 | 9 | 4 | 1877 | 546 | 1 | 1024 | 0 | QEMU | 12001 | {"vendor": "Intel", "model": "Penryn", "arch": "x86_64", "features": ["hypervisor", "osxsave", "xsave", "ht", "ss", "acpi", "ds", "vme"], "topology": {"cores": 2, "threads": 1, "sockets": 2}} | 87 | 853 | 546 | 0 | 1 | rhel8236 | 0 |
+---------------------+---------------------+------------+----+------------+-------+-----------+----------+------------+----------------+---------------+-----------------+--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------+-------------+--------------+------------------+-------------+---------------------+---------+
2. trigger live-migration for myvm1 to rhel8235
+--------------------------------------+-------+-----------+----------+
| ID | Name | Status | Networks |
+--------------------------------------+-------+-----------+----------+
| 4b25db8e-b3a1-4177-aed1-32506b94eb88 | myvm1 | MIGRATING | |
| 00565dce-4828-4eb2-ab29-9ef0403e5ec0 | myvm2 | ACTIVE | |
| e4871da2-3a6f-4f34-93cf-bb2511b46cf8 | myvm3 | ACTIVE | |
+--------------------------------------+-------+-----------+----------+
3. after live-migration is done, wait for a while, check DB:
+--------------------------------------+-------+--------+----------+
| ID | Name | Status | Networks |
+--------------------------------------+-------+--------+----------+
| 4b25db8e-b3a1-4177-aed1-32506b94eb88 | myvm1 | ACTIVE | |
| 00565dce-4828-4eb2-ab29-9ef0403e5ec0 | myvm2 | ACTIVE | |
| e4871da2-3a6f-4f34-93cf-bb2511b46cf8 | myvm3 | ACTIVE | |
+--------------------------------------+-------+--------+----------+
[root@rhel8233 ~]# nova show myvm1
+-------------------------------------+------------------------------------------------------------+
| Property | Value |
+-------------------------------------+------------------------------------------------------------+
| status | ACTIVE |
| updated | 2013-03-07T08:37:45Z |
| OS-EXT-STS:task_state | None |
| OS-EXT-SRV-ATTR:host | rhel8235 |
| key_name | None |
| image | cirros-0.3.0-x86_64 (15b8a04c-9207-4ab1-b69e-43c5c5a46525) |
| hostId | a9e1ff745b20b8c20c7f1b9f55f1957609cc40d57e2a68959bcf1b80 |
| OS-EXT-STS:vm_state | active |
| OS-EXT-SRV-ATTR:instance_name | instance-00000014 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | rhel8235 |
| flavor | m1.tiny (1) |
| id | 4b25db8e-b3a1-4177-aed1-32506b94eb88 |
| security_groups | [{u'name': u'default'}] |
| user_id | 75a5e821c5db4c3aba3280c28bb6db2c |
| name | myvm1 |
| created | 2013-03-07T07:47:48Z |
| tenant_id | c6e0a21705b04c0fb06185c72d5173d3 |
| OS-DCF:diskConfig | MANUAL |
| metadata | {} |
| accessIPv4 | |
| accessIPv6 | |
| progress | 0 |
| OS-EXT-STS:power_state | 1 |
| OS-EXT-AZ:availability_zone | nova |
| config_drive | |
+-------------------------------------+------------------------------------------------------------+
The running_vms from the source hypervisor has been decreased, that's
correct, but the counter in destination hypervisor was not increased:
mysql> select * from compute_nodes;
+---------------------+---------------------+------------+----+------------+-------+-----------+----------+------------+----------------+---------------+-----------------+--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------+-------------+--------------+------------------+-------------+---------------------+---------+
| created_at | updated_at | deleted_at | id | service_id | vcpus | memory_mb | local_gb | vcpus_used | memory_mb_used | local_gb_used | hypervisor_type | hypervisor_version | cpu_info | disk_available_least | free_ram_mb | free_disk_gb | current_workload | running_vms | hypervisor_hostname | deleted |
+---------------------+---------------------+------------+----+------------+-------+-----------+----------+------------+----------------+---------------+-----------------+--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------+-------------+--------------+------------------+-------------+---------------------+---------+
| 2013-03-05 05:05:29 | 2013-03-07 08:38:45 | NULL | 1 | 5 | 4 | 1877 | 546 | 0 | 512 | 0 | QEMU | 12001 | {"vendor": "Intel", "model": "Penryn", "arch": "x86_64", "features": ["hypervisor", "osxsave", "xsave", "ht", "ss", "acpi", "ds", "vme"], "topology": {"cores": 2, "threads": 1, "sockets": 2}} | 88 | 1365 | 546 | 0 | 0 | rhel8234 | 0 |
| 2013-03-05 05:11:26 | 2013-03-07 07:48:41 | NULL | 2 | 7 | 4 | 1877 | 546 | 1 | 1024 | 0 | QEMU | 12001 | {"vendor": "Intel", "model": "Penryn", "arch": "x86_64", "features": ["hypervisor", "osxsave", "xsave", "ht", "ss", "acpi", "ds", "vme"], "topology": {"cores": 2, "threads": 1, "sockets": 2}} | 87 | 853 | 546 | 0 | 1 | rhel8235 | 0 |
| 2013-03-05 05:12:36 | 2013-03-07 07:48:07 | NULL | 3 | 9 | 4 | 1877 | 546 | 1 | 1024 | 0 | QEMU | 12001 | {"vendor": "Intel", "model": "Penryn", "arch": "x86_64", "features": ["hypervisor", "osxsave", "xsave", "ht", "ss", "acpi", "ds", "vme"], "topology": {"cores": 2, "threads": 1, "sockets": 2}} | 87 | 853 | 546 | 0 | 1 | rhel8236 | 0 |
+---------------------+---------------------+------------+----+------------+-------+-----------+----------+------------+----------------+---------------+-----------------+--------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------------------+-------------+--------------+------------------+-------------+---------------------+---------+
mysql> select * from compute_node_stats where compute_node_id=2;
| 2013-03-07 07:47:53 | 2013-03-07 08:38:49 | NULL | 218 | 2 | num_instances | 1 | 0 |
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1151845/+subscriptions
