yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #80620
[Bug 1851587] [NEW] HypervisorUnavailable error leaks compute host fqdn to non-admin users
Public bug reported:
Description
===========
When an instance encounters a HypervisorUnavailable error, the non-admin user gets the info of the compute host fqdn in the error message.
Steps to reproduce
==================
1. Spin up an instance with non-admin user credentials
2. To reproduce the error, stop the libvirtd service on the compute host containing instance
3. Delete the instance
4. Deletion fails providing HypervisorUnavailable error
Expected result
===============
Error does not show compute host fqdn to a non-admin user
Actual result
=============
#spin up an instance
+--------------------------------------+------------+--------+------------+-------------+-------------------------------------+------------------------------+--------------------------------------+-------------+-----------+-------------------+------+------------+
| ID | Name | Status | Task State | Power State | Networks | Image Name | Image ID | Flavor Name | Flavor ID | Availability Zone | Host | Properties |
+--------------------------------------+------------+--------+------------+-------------+-------------------------------------+------------------------------+--------------------------------------+-------------+-----------+-------------------+------+------------+
| 4f42886d-e1f8-4607-a09d-0dc12a681880 | test-11869 | ACTIVE | None | Running | private=192.168.100.158, 10.0.0.243 | cirros-0.4.0-x86_64-disk.img | 5d0bd6a5-7331-4ebe-9328-d126189897e2 | | | nova | | |
+--------------------------------------+------------+--------+------------+-------------+-------------------------------------+------------------------------+--------------------------------------+-------------+-----------+-------------------+------+------------+
#instance is running on compute-0 node (only admin knows this)
[heat-admin@compute-0 ~]$ sudo virsh list --all
Id Name State
----------------------------------------------------
108 instance-00000092 running
#stop libvirtd service
[root@compute-0 heat-admin]# systemctl stop tripleo_nova_libvirt.service
[root@compute-0 heat-admin]# systemctl status tripleo_nova_libvirt.service
● tripleo_nova_libvirt.service - nova_libvirt container
Loaded: loaded (/etc/systemd/system/tripleo_nova_libvirt.service; enabled; vendor preset: disabled)
Active: inactive (dead) since Wed 2019-11-06 22:48:25 UTC; 5s ago
Process: 8514 ExecStop=/usr/bin/podman stop -t 10 nova_libvirt (code=exited, status=0/SUCCESS)
Main PID: 3783
Nov 06 22:29:48 compute-0 podman[3396]: 2019-11-06 22:29:48.443603571 +0000 UTC m=+1.325620613 container init a3e32121d12929e663b899b57cb7bc87581ddf5bdfb19cf8fee4bace41cb19bb (image=undercloud-0.ctlpla>
Nov 06 22:29:48 compute-0 podman[3396]: 2019-11-06 22:29:48.475946808 +0000 UTC m=+1.357963869 container start a3e32121d12929e663b899b57cb7bc87581ddf5bdfb19cf8fee4bace41cb19bb (image=undercloud-0.ctlpl>
Nov 06 22:29:48 compute-0 paunch-start-podman-container[3385]: nova_libvirt
Nov 06 22:29:48 compute-0 paunch-start-podman-container[3385]: Creating additional drop-in dependency for "nova_libvirt" (a3e32121d12929e663b899b57cb7bc87581ddf5bdfb19cf8fee4bace41cb19bb)
Nov 06 22:29:49 compute-0 systemd[1]: Started nova_libvirt container.
Nov 06 22:48:24 compute-0 systemd[1]: Stopping nova_libvirt container...
Nov 06 22:48:25 compute-0 podman[8514]: 2019-11-06 22:48:25.595405651 +0000 UTC m=+1.063832024 container died a3e32121d12929e663b899b57cb7bc87581ddf5bdfb19cf8fee4bace41cb19bb (image=undercloud-0.ctlpla>
Nov 06 22:48:25 compute-0 podman[8514]: 2019-11-06 22:48:25.597210594 +0000 UTC m=+1.065636903 container stop a3e32121d12929e663b899b57cb7bc87581ddf5bdfb19cf8fee4bace41cb19bb (image=undercloud-0.ctlpla>
Nov 06 22:48:25 compute-0 podman[8514]: a3e32121d12929e663b899b57cb7bc87581ddf5bdfb19cf8fee4bace41cb19bb
Nov 06 22:48:25 compute-0 systemd[1]: Stopped nova_libvirt container.
#delete the instance, it leaks compute host fqdn to the non-admin user
(overcloud) [stack@undercloud-0 ~]$ nova delete test-11869
Request to delete server test-11869 has been accepted.
(overcloud) [stack@undercloud-0 ~]$ openstack server list --long
+--------------------------------------+------------+--------+------------+-------------+----------+------------------------------+--------------------------------------+-------------+-----------+-------------------+------+------------+
| ID | Name | Status | Task State | Power State | Networks | Image Name | Image ID | Flavor Name | Flavor ID | Availability Zone | Host | Properties |
+--------------------------------------+------------+--------+------------+-------------+----------+------------------------------+--------------------------------------+-------------+-----------+-------------------+------+------------+
| 4f42886d-e1f8-4607-a09d-0dc12a681880 | test-11869 | ERROR | None | Running | | cirros-0.4.0-x86_64-disk.img | 5d0bd6a5-7331-4ebe-9328-d126189897e2 | | | nova | | |
+--------------------------------------+------------+--------+------------+-------------+----------+------------------------------+--------------------------------------+-------------+-----------+-------------------+------+------------+
(overcloud) [stack@undercloud-0 ~]$ openstack server show test-11869 <---debug output attached in logs
+-----------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-STS:power_state | Running |
| OS-EXT-STS:task_state | None |
| OS-EXT-STS:vm_state | error |
| OS-SRV-USG:launched_at | 2019-11-06T22:13:08.000000 |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| config_drive | |
| created | 2019-11-06T22:12:57Z |
| description | None |
| fault | {'code': 500, 'created': '2019-11-06T23:01:45Z', 'message': 'Connection to the hypervisor is broken on host: compute-0.redhat.local'} |
| flavor | disk='1', ephemeral='0', , original_name='m1.tiny', ram='512', swap='0', vcpus='1' |
| hostId | c7e6bf58b57f435659bb0aa9637c7f830f776ec202a0d6e430ee3168 |
| id | 4f42886d-e1f8-4607-a09d-0dc12a681880 |
| image | cirros-0.4.0-x86_64-disk.img (5d0bd6a5-7331-4ebe-9328-d126189897e2) |
| key_name | None |
| locked | False |
| locked_reason | None |
| name | test-11869 |
| project_id | 6e39619e17a9478580c93120e1cb16bc |
| properties | |
| server_groups | [] |
| status | ERROR |
| tags | [] |
| trusted_image_certificates | None |
| updated | 2019-11-06T23:01:45Z |
| user_id | 3cd6a8cb88eb49d3a84f9e67d89df598 |
| volumes_attached | |
+-----------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
** Affects: nova
Importance: Undecided
Status: New
** Attachment added: "full steps to reproduce"
https://bugs.launchpad.net/bugs/1851587/+attachment/5303376/+files/logtrace.log
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1851587
Title:
HypervisorUnavailable error leaks compute host fqdn to non-admin users
Status in OpenStack Compute (nova):
New
Bug description:
Description
===========
When an instance encounters a HypervisorUnavailable error, the non-admin user gets the info of the compute host fqdn in the error message.
Steps to reproduce
==================
1. Spin up an instance with non-admin user credentials
2. To reproduce the error, stop the libvirtd service on the compute host containing instance
3. Delete the instance
4. Deletion fails providing HypervisorUnavailable error
Expected result
===============
Error does not show compute host fqdn to a non-admin user
Actual result
=============
#spin up an instance
+--------------------------------------+------------+--------+------------+-------------+-------------------------------------+------------------------------+--------------------------------------+-------------+-----------+-------------------+------+------------+
| ID | Name | Status | Task State | Power State | Networks | Image Name | Image ID | Flavor Name | Flavor ID | Availability Zone | Host | Properties |
+--------------------------------------+------------+--------+------------+-------------+-------------------------------------+------------------------------+--------------------------------------+-------------+-----------+-------------------+------+------------+
| 4f42886d-e1f8-4607-a09d-0dc12a681880 | test-11869 | ACTIVE | None | Running | private=192.168.100.158, 10.0.0.243 | cirros-0.4.0-x86_64-disk.img | 5d0bd6a5-7331-4ebe-9328-d126189897e2 | | | nova | | |
+--------------------------------------+------------+--------+------------+-------------+-------------------------------------+------------------------------+--------------------------------------+-------------+-----------+-------------------+------+------------+
#instance is running on compute-0 node (only admin knows this)
[heat-admin@compute-0 ~]$ sudo virsh list --all
Id Name State
----------------------------------------------------
108 instance-00000092 running
#stop libvirtd service
[root@compute-0 heat-admin]# systemctl stop tripleo_nova_libvirt.service
[root@compute-0 heat-admin]# systemctl status tripleo_nova_libvirt.service
● tripleo_nova_libvirt.service - nova_libvirt container
Loaded: loaded (/etc/systemd/system/tripleo_nova_libvirt.service; enabled; vendor preset: disabled)
Active: inactive (dead) since Wed 2019-11-06 22:48:25 UTC; 5s ago
Process: 8514 ExecStop=/usr/bin/podman stop -t 10 nova_libvirt (code=exited, status=0/SUCCESS)
Main PID: 3783
Nov 06 22:29:48 compute-0 podman[3396]: 2019-11-06 22:29:48.443603571 +0000 UTC m=+1.325620613 container init a3e32121d12929e663b899b57cb7bc87581ddf5bdfb19cf8fee4bace41cb19bb (image=undercloud-0.ctlpla>
Nov 06 22:29:48 compute-0 podman[3396]: 2019-11-06 22:29:48.475946808 +0000 UTC m=+1.357963869 container start a3e32121d12929e663b899b57cb7bc87581ddf5bdfb19cf8fee4bace41cb19bb (image=undercloud-0.ctlpl>
Nov 06 22:29:48 compute-0 paunch-start-podman-container[3385]: nova_libvirt
Nov 06 22:29:48 compute-0 paunch-start-podman-container[3385]: Creating additional drop-in dependency for "nova_libvirt" (a3e32121d12929e663b899b57cb7bc87581ddf5bdfb19cf8fee4bace41cb19bb)
Nov 06 22:29:49 compute-0 systemd[1]: Started nova_libvirt container.
Nov 06 22:48:24 compute-0 systemd[1]: Stopping nova_libvirt container...
Nov 06 22:48:25 compute-0 podman[8514]: 2019-11-06 22:48:25.595405651 +0000 UTC m=+1.063832024 container died a3e32121d12929e663b899b57cb7bc87581ddf5bdfb19cf8fee4bace41cb19bb (image=undercloud-0.ctlpla>
Nov 06 22:48:25 compute-0 podman[8514]: 2019-11-06 22:48:25.597210594 +0000 UTC m=+1.065636903 container stop a3e32121d12929e663b899b57cb7bc87581ddf5bdfb19cf8fee4bace41cb19bb (image=undercloud-0.ctlpla>
Nov 06 22:48:25 compute-0 podman[8514]: a3e32121d12929e663b899b57cb7bc87581ddf5bdfb19cf8fee4bace41cb19bb
Nov 06 22:48:25 compute-0 systemd[1]: Stopped nova_libvirt container.
#delete the instance, it leaks compute host fqdn to the non-admin user
(overcloud) [stack@undercloud-0 ~]$ nova delete test-11869
Request to delete server test-11869 has been accepted.
(overcloud) [stack@undercloud-0 ~]$ openstack server list --long
+--------------------------------------+------------+--------+------------+-------------+----------+------------------------------+--------------------------------------+-------------+-----------+-------------------+------+------------+
| ID | Name | Status | Task State | Power State | Networks | Image Name | Image ID | Flavor Name | Flavor ID | Availability Zone | Host | Properties |
+--------------------------------------+------------+--------+------------+-------------+----------+------------------------------+--------------------------------------+-------------+-----------+-------------------+------+------------+
| 4f42886d-e1f8-4607-a09d-0dc12a681880 | test-11869 | ERROR | None | Running | | cirros-0.4.0-x86_64-disk.img | 5d0bd6a5-7331-4ebe-9328-d126189897e2 | | | nova | | |
+--------------------------------------+------------+--------+------------+-------------+----------+------------------------------+--------------------------------------+-------------+-----------+-------------------+------+------------+
(overcloud) [stack@undercloud-0 ~]$ openstack server show test-11869 <---debug output attached in logs
+-----------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
| OS-DCF:diskConfig | MANUAL |
| OS-EXT-AZ:availability_zone | nova |
| OS-EXT-STS:power_state | Running |
| OS-EXT-STS:task_state | None |
| OS-EXT-STS:vm_state | error |
| OS-SRV-USG:launched_at | 2019-11-06T22:13:08.000000 |
| OS-SRV-USG:terminated_at | None |
| accessIPv4 | |
| accessIPv6 | |
| addresses | |
| config_drive | |
| created | 2019-11-06T22:12:57Z |
| description | None |
| fault | {'code': 500, 'created': '2019-11-06T23:01:45Z', 'message': 'Connection to the hypervisor is broken on host: compute-0.redhat.local'} |
| flavor | disk='1', ephemeral='0', , original_name='m1.tiny', ram='512', swap='0', vcpus='1' |
| hostId | c7e6bf58b57f435659bb0aa9637c7f830f776ec202a0d6e430ee3168 |
| id | 4f42886d-e1f8-4607-a09d-0dc12a681880 |
| image | cirros-0.4.0-x86_64-disk.img (5d0bd6a5-7331-4ebe-9328-d126189897e2) |
| key_name | None |
| locked | False |
| locked_reason | None |
| name | test-11869 |
| project_id | 6e39619e17a9478580c93120e1cb16bc |
| properties | |
| server_groups | [] |
| status | ERROR |
| tags | [] |
| trusted_image_certificates | None |
| updated | 2019-11-06T23:01:45Z |
| user_id | 3cd6a8cb88eb49d3a84f9e67d89df598 |
| volumes_attached | |
+-----------------------------+---------------------------------------------------------------------------------------------------------------------------------------+
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1851587/+subscriptions
Follow ups
-
[Bug 1851587] Fix included in openstack/nova rocky-eol
From: OpenStack Infra, 2022-11-11
-
[Bug 1851587] Fix included in openstack/nova queens-eol
From: OpenStack Infra, 2022-11-11
-
[Bug 1851587] Fix included in openstack/nova pike-eol
From: OpenStack Infra, 2022-08-01
-
[Bug 1851587] Re: HypervisorUnavailable error leaks compute host fqdn to non-admin users
From: Elod Illes, 2020-11-24
-
[Bug 1851587] Re: HypervisorUnavailable error leaks compute host fqdn to non-admin users
From: Elod Illes, 2020-11-24
-
[Bug 1851587] Re: HypervisorUnavailable error leaks compute host fqdn to non-admin users
From: Elod Illes, 2020-11-24
-
[Bug 1851587] Re: HypervisorUnavailable error leaks compute host fqdn to non-admin users
From: melanie witt, 2020-10-01
-
[Bug 1851587] Re: HypervisorUnavailable error leaks compute host fqdn to non-admin users
From: melanie witt, 2020-09-03
-
[Bug 1851587] Re: HypervisorUnavailable error leaks compute host fqdn to non-admin users
From: OpenStack Infra, 2020-08-05
-
[Bug 1851587] Re: HypervisorUnavailable error leaks compute host fqdn to non-admin users
From: Jeremy Stanley, 2019-11-08
