yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #53882
[Bug 1603124] [NEW] [stable/liberty] LBaaS v2 haproxy: pool member status shown wrongly
Public bug reported:
Setup:
1. Loadbalancer with one listener and pool.
2. 4 webservers as members of pool.
3. create health-monitor for the pool
Test:
1. Send 4 curl requests, requests are distributed equally among the members as ROUND_ROBIN lb_alorithm is selected.
2. using "nova stop <>" SHUTDOWN one VM and send 4 requests again.
3. health monitor seems to detect member down and it stopped forwarding request to that particular webserver.
Problem:
1. CLI "neutron lbaas-member-list <>" says all members are admin_up
stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-member-list test-lb-pool-http
+--------------------------------------+-----------+---------------+--------+--------------------------------------+----------------+
| id | address | protocol_port | weight | subnet_id | admin_state_up |
+--------------------------------------+-----------+---------------+--------+--------------------------------------+----------------+
| 002e68a0-03db-4f46-9c82-e35d395ada6b | 20.0.0.9 | 80 | 1 | 63cbeccd-6887-4dda-b4d2-b7503bce870a | True |
| 0a97fecf-30c8-483d-8158-17523a726594 | 20.0.0.28 | 80 | 1 | 63cbeccd-6887-4dda-b4d2-b7503bce870a | True |
| 348e0f42-9ab3-417e-a6e3-9857a988b6f4 | 20.0.0.29 | 80 | 1 | 63cbeccd-6887-4dda-b4d2-b7503bce870a | True |
| 823d0f22-a442-4450-b7c8-0038e3f142b6 | 20.0.0.8 | 80 | 1 | 63cbeccd-6887-4dda-b4d2-b7503bce870a | True |
+--------------------------------------+-----------+---------------+--------+--------------------------------------+----------------+
stack@runner:~/prash/nsbu_cqe_openstack/tools$
2. CLI "neutron lbaas-loadbalancer-status <>" states all members are
"ACTIVE" and "ONLINE".
stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-loadbalancer-status test-lb
{
"loadbalancer": {
"listeners": [
{
"pools": [
{
"name": "test-lb-pool-http",
"provisioning_status": "ACTIVE",
"healthmonitor": {
"type": "HTTP",
"id": "96ec79bc-d212-431b-87da-2a053064675a",
"provisioning_status": "ACTIVE"
},
"members": [
{
"provisioning_status": "ACTIVE",
"protocol_port": 80,
"id": "002e68a0-03db-4f46-9c82-e35d395ada6b",
"operating_status": "ONLINE",
"address": "20.0.0.9"
},
{
"provisioning_status": "ACTIVE",
"protocol_port": 80,
"id": "823d0f22-a442-4450-b7c8-0038e3f142b6",
"operating_status": "ONLINE",
"address": "20.0.0.8"
},
{
"provisioning_status": "ACTIVE",
"protocol_port": 80,
"id": "348e0f42-9ab3-417e-a6e3-9857a988b6f4",
"operating_status": "ONLINE",
"address": "20.0.0.29"
},
{
"provisioning_status": "ACTIVE",
"protocol_port": 80,
"id": "0a97fecf-30c8-483d-8158-17523a726594",
"operating_status": "ONLINE",
"address": "20.0.0.28"
}
],
"id": "4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f",
"operating_status": "ONLINE"
}
],
"provisioning_status": "ACTIVE",
"name": "test-lb-http",
"operating_status": "ONLINE",
"id": "dedabd6e-eb51-435f-9fda-e9b90eef4108"
}
],
"provisioning_status": "ACTIVE",
"name": "test-lb",
"operating_status": "ONLINE",
"id": "57814c66-731b-4dc7-abad-7d582c546873"
}
}
stack@runner:~/prash/nsbu_cqe_openstack/tools$
Nova output:
stack@runner:~/prash/nsbu_cqe_openstack/tools$ nova list
+--------------------------------------+------------+---------+------------+-------------+---------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------------+---------+------------+-------------+---------------------------+
| 10ddd66b-3bdf-4811-97cd-28d6c94b579e | test_esx-1 | SHUTOFF | - | Shutdown | vn=20.0.0.28, 192.168.0.5 |
| 93db6815-e209-4c48-bce7-308f769cf745 | test_esx-2 | ACTIVE | - | Running | vn=20.0.0.29, 192.168.0.6 |
| 1a9657e6-3037-49a6-b1f3-c79b280772d3 | test_esx-3 | ACTIVE | - | Running | vn=20.0.0.32 |
| 63808996-af5c-41b6-aabc-f1a7c8a6abd4 | test_esx-4 | ACTIVE | - | Running | vn=20.0.0.30 |
| f5101231-0e34-49b2-81cb-a4e204b3c2d4 | test_esx-5 | ACTIVE | - | Running | vn=20.0.0.31 |
| 8e743e4c-b1da-4cfd-ac47-cd2c26504cb6 | test_kvm-1 | ACTIVE | - | Running | vn=20.0.0.8, 192.168.0.2 |
| 8ea23ce9-f317-4288-b345-3cb6d4483bce | test_kvm-2 | ACTIVE | - | Running | vn=20.0.0.10 |
| 576f6b43-c4c4-4997-b18a-bf9375ad9e47 | test_kvm-3 | ACTIVE | - | Running | vn=20.0.0.9, 192.168.0.4 |
| 755de1d3-b05d-42ed-b2ef-506d92e170a9 | test_kvm-4 | ACTIVE | - | Running | vn=20.0.0.11 |
| a489472f-923e-44c4-9e25-9015bddeb3e2 | test_kvm-5 | ACTIVE | - | Running | vn=20.0.0.12 |
+--------------------------------------+------------+---------+------------+-------------+---------------------------+
stack@runner:~/prash/nsbu_cqe_openstack/tools$
stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-healthmonitor-show 96ec79bc-d212-431b-87da-2a053064675a
+----------------+------------------------------------------------+
| Field | Value |
+----------------+------------------------------------------------+
| admin_state_up | True |
| delay | 5 |
| expected_codes | 200 |
| http_method | GET |
| id | 96ec79bc-d212-431b-87da-2a053064675a |
| max_retries | 2 |
| pools | {"id": "4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f"} |
| tenant_id | ce1d087209c64df4b7e8007dc35def22 |
| timeout | 10 |
| type | HTTP |
| url_path | / |
+----------------+------------------------------------------------+
stack@runner:~/prash/nsbu_cqe_openstack/tools$
stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-listener-list
+--------------------------------------+--------------------------------------+--------------+----------+---------------+----------------+
| id | default_pool_id | name | protocol | protocol_port | admin_state_up |
+--------------------------------------+--------------------------------------+--------------+----------+---------------+----------------+
| dedabd6e-eb51-435f-9fda-e9b90eef4108 | 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f | test-lb-http | HTTP | 80 | True |
+--------------------------------------+--------------------------------------+--------------+----------+---------------+----------------+
stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-listener-show dedabd6e-eb51-435f-9fda-e9b90eef4108
+---------------------------+------------------------------------------------+
| Field | Value |
+---------------------------+------------------------------------------------+
| admin_state_up | True |
| connection_limit | -1 |
| default_pool_id | 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f |
| default_tls_container_ref | |
| description | |
| id | dedabd6e-eb51-435f-9fda-e9b90eef4108 |
| loadbalancers | {"id": "57814c66-731b-4dc7-abad-7d582c546873"} |
| name | test-lb-http |
| protocol | HTTP |
| protocol_port | 80 |
| sni_container_refs | |
| tenant_id | ce1d087209c64df4b7e8007dc35def22 |
+---------------------------+------------------------------------------------+
stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron pool-list
Unknown command [u'pool-list']
stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-pool-list
+--------------------------------------+-------------------+----------+----------------+
| id | name | protocol | admin_state_up |
+--------------------------------------+-------------------+----------+----------------+
| 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f | test-lb-pool-http | HTTP | True |
+--------------------------------------+-------------------+----------+----------------+
stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-pool-show 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f
+---------------------+------------------------------------------------+
| Field | Value |
+---------------------+------------------------------------------------+
| admin_state_up | True |
| description | |
| healthmonitor_id | 96ec79bc-d212-431b-87da-2a053064675a |
| id | 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f |
| lb_algorithm | ROUND_ROBIN |
| listeners | {"id": "dedabd6e-eb51-435f-9fda-e9b90eef4108"} |
| members | 002e68a0-03db-4f46-9c82-e35d395ada6b |
| | 0a97fecf-30c8-483d-8158-17523a726594 |
| | 348e0f42-9ab3-417e-a6e3-9857a988b6f4 |
| | 823d0f22-a442-4450-b7c8-0038e3f142b6 |
| name | test-lb-pool-http |
| protocol | HTTP |
| session_persistence | |
| tenant_id | ce1d087209c64df4b7e8007dc35def22 |
+---------------------+------------------------------------------------+
stack@runner:~/prash/nsbu_cqe_openstack/tools$
** Affects: neutron
Importance: Undecided
Status: New
** Tags: lbaas
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1603124
Title:
[stable/liberty] LBaaS v2 haproxy: pool member status shown wrongly
Status in neutron:
New
Bug description:
Setup:
1. Loadbalancer with one listener and pool.
2. 4 webservers as members of pool.
3. create health-monitor for the pool
Test:
1. Send 4 curl requests, requests are distributed equally among the members as ROUND_ROBIN lb_alorithm is selected.
2. using "nova stop <>" SHUTDOWN one VM and send 4 requests again.
3. health monitor seems to detect member down and it stopped forwarding request to that particular webserver.
Problem:
1. CLI "neutron lbaas-member-list <>" says all members are admin_up
stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-member-list test-lb-pool-http
+--------------------------------------+-----------+---------------+--------+--------------------------------------+----------------+
| id | address | protocol_port | weight | subnet_id | admin_state_up |
+--------------------------------------+-----------+---------------+--------+--------------------------------------+----------------+
| 002e68a0-03db-4f46-9c82-e35d395ada6b | 20.0.0.9 | 80 | 1 | 63cbeccd-6887-4dda-b4d2-b7503bce870a | True |
| 0a97fecf-30c8-483d-8158-17523a726594 | 20.0.0.28 | 80 | 1 | 63cbeccd-6887-4dda-b4d2-b7503bce870a | True |
| 348e0f42-9ab3-417e-a6e3-9857a988b6f4 | 20.0.0.29 | 80 | 1 | 63cbeccd-6887-4dda-b4d2-b7503bce870a | True |
| 823d0f22-a442-4450-b7c8-0038e3f142b6 | 20.0.0.8 | 80 | 1 | 63cbeccd-6887-4dda-b4d2-b7503bce870a | True |
+--------------------------------------+-----------+---------------+--------+--------------------------------------+----------------+
stack@runner:~/prash/nsbu_cqe_openstack/tools$
2. CLI "neutron lbaas-loadbalancer-status <>" states all members are
"ACTIVE" and "ONLINE".
stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-loadbalancer-status test-lb
{
"loadbalancer": {
"listeners": [
{
"pools": [
{
"name": "test-lb-pool-http",
"provisioning_status": "ACTIVE",
"healthmonitor": {
"type": "HTTP",
"id": "96ec79bc-d212-431b-87da-2a053064675a",
"provisioning_status": "ACTIVE"
},
"members": [
{
"provisioning_status": "ACTIVE",
"protocol_port": 80,
"id": "002e68a0-03db-4f46-9c82-e35d395ada6b",
"operating_status": "ONLINE",
"address": "20.0.0.9"
},
{
"provisioning_status": "ACTIVE",
"protocol_port": 80,
"id": "823d0f22-a442-4450-b7c8-0038e3f142b6",
"operating_status": "ONLINE",
"address": "20.0.0.8"
},
{
"provisioning_status": "ACTIVE",
"protocol_port": 80,
"id": "348e0f42-9ab3-417e-a6e3-9857a988b6f4",
"operating_status": "ONLINE",
"address": "20.0.0.29"
},
{
"provisioning_status": "ACTIVE",
"protocol_port": 80,
"id": "0a97fecf-30c8-483d-8158-17523a726594",
"operating_status": "ONLINE",
"address": "20.0.0.28"
}
],
"id": "4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f",
"operating_status": "ONLINE"
}
],
"provisioning_status": "ACTIVE",
"name": "test-lb-http",
"operating_status": "ONLINE",
"id": "dedabd6e-eb51-435f-9fda-e9b90eef4108"
}
],
"provisioning_status": "ACTIVE",
"name": "test-lb",
"operating_status": "ONLINE",
"id": "57814c66-731b-4dc7-abad-7d582c546873"
}
}
stack@runner:~/prash/nsbu_cqe_openstack/tools$
Nova output:
stack@runner:~/prash/nsbu_cqe_openstack/tools$ nova list
+--------------------------------------+------------+---------+------------+-------------+---------------------------+
| ID | Name | Status | Task State | Power State | Networks |
+--------------------------------------+------------+---------+------------+-------------+---------------------------+
| 10ddd66b-3bdf-4811-97cd-28d6c94b579e | test_esx-1 | SHUTOFF | - | Shutdown | vn=20.0.0.28, 192.168.0.5 |
| 93db6815-e209-4c48-bce7-308f769cf745 | test_esx-2 | ACTIVE | - | Running | vn=20.0.0.29, 192.168.0.6 |
| 1a9657e6-3037-49a6-b1f3-c79b280772d3 | test_esx-3 | ACTIVE | - | Running | vn=20.0.0.32 |
| 63808996-af5c-41b6-aabc-f1a7c8a6abd4 | test_esx-4 | ACTIVE | - | Running | vn=20.0.0.30 |
| f5101231-0e34-49b2-81cb-a4e204b3c2d4 | test_esx-5 | ACTIVE | - | Running | vn=20.0.0.31 |
| 8e743e4c-b1da-4cfd-ac47-cd2c26504cb6 | test_kvm-1 | ACTIVE | - | Running | vn=20.0.0.8, 192.168.0.2 |
| 8ea23ce9-f317-4288-b345-3cb6d4483bce | test_kvm-2 | ACTIVE | - | Running | vn=20.0.0.10 |
| 576f6b43-c4c4-4997-b18a-bf9375ad9e47 | test_kvm-3 | ACTIVE | - | Running | vn=20.0.0.9, 192.168.0.4 |
| 755de1d3-b05d-42ed-b2ef-506d92e170a9 | test_kvm-4 | ACTIVE | - | Running | vn=20.0.0.11 |
| a489472f-923e-44c4-9e25-9015bddeb3e2 | test_kvm-5 | ACTIVE | - | Running | vn=20.0.0.12 |
+--------------------------------------+------------+---------+------------+-------------+---------------------------+
stack@runner:~/prash/nsbu_cqe_openstack/tools$
stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-healthmonitor-show 96ec79bc-d212-431b-87da-2a053064675a
+----------------+------------------------------------------------+
| Field | Value |
+----------------+------------------------------------------------+
| admin_state_up | True |
| delay | 5 |
| expected_codes | 200 |
| http_method | GET |
| id | 96ec79bc-d212-431b-87da-2a053064675a |
| max_retries | 2 |
| pools | {"id": "4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f"} |
| tenant_id | ce1d087209c64df4b7e8007dc35def22 |
| timeout | 10 |
| type | HTTP |
| url_path | / |
+----------------+------------------------------------------------+
stack@runner:~/prash/nsbu_cqe_openstack/tools$
stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-listener-list
+--------------------------------------+--------------------------------------+--------------+----------+---------------+----------------+
| id | default_pool_id | name | protocol | protocol_port | admin_state_up |
+--------------------------------------+--------------------------------------+--------------+----------+---------------+----------------+
| dedabd6e-eb51-435f-9fda-e9b90eef4108 | 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f | test-lb-http | HTTP | 80 | True |
+--------------------------------------+--------------------------------------+--------------+----------+---------------+----------------+
stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-listener-show dedabd6e-eb51-435f-9fda-e9b90eef4108
+---------------------------+------------------------------------------------+
| Field | Value |
+---------------------------+------------------------------------------------+
| admin_state_up | True |
| connection_limit | -1 |
| default_pool_id | 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f |
| default_tls_container_ref | |
| description | |
| id | dedabd6e-eb51-435f-9fda-e9b90eef4108 |
| loadbalancers | {"id": "57814c66-731b-4dc7-abad-7d582c546873"} |
| name | test-lb-http |
| protocol | HTTP |
| protocol_port | 80 |
| sni_container_refs | |
| tenant_id | ce1d087209c64df4b7e8007dc35def22 |
+---------------------------+------------------------------------------------+
stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron pool-list
Unknown command [u'pool-list']
stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-pool-list
+--------------------------------------+-------------------+----------+----------------+
| id | name | protocol | admin_state_up |
+--------------------------------------+-------------------+----------+----------------+
| 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f | test-lb-pool-http | HTTP | True |
+--------------------------------------+-------------------+----------+----------------+
stack@runner:~/prash/nsbu_cqe_openstack/tools$ neutron lbaas-pool-show 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f
+---------------------+------------------------------------------------+
| Field | Value |
+---------------------+------------------------------------------------+
| admin_state_up | True |
| description | |
| healthmonitor_id | 96ec79bc-d212-431b-87da-2a053064675a |
| id | 4a4b3d7e-f061-4ef6-aef4-1322aa4e6c6f |
| lb_algorithm | ROUND_ROBIN |
| listeners | {"id": "dedabd6e-eb51-435f-9fda-e9b90eef4108"} |
| members | 002e68a0-03db-4f46-9c82-e35d395ada6b |
| | 0a97fecf-30c8-483d-8158-17523a726594 |
| | 348e0f42-9ab3-417e-a6e3-9857a988b6f4 |
| | 823d0f22-a442-4450-b7c8-0038e3f142b6 |
| name | test-lb-pool-http |
| protocol | HTTP |
| session_persistence | |
| tenant_id | ce1d087209c64df4b7e8007dc35def22 |
+---------------------+------------------------------------------------+
stack@runner:~/prash/nsbu_cqe_openstack/tools$
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1603124/+subscriptions
