← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1956745] [NEW] [ovn-octavia-provider] Load Balancer remained with ACTIVE state even with PENDING_UPDATE listener

 

Public bug reported:

Description of problem:

The Load Balancer remained with ACTIVE and ONLINE state even when the
listener had provisioning_status PENDING_UPDATE and the pool was with
ERROR, what made the load-balancer to be considered functional by Kuryr
and a load-balancer member removal was attempt without success as pool
was immutable:

2021-11-03 17:04:53.893 1 ERROR kuryr_kubernetes.handlers.logging openstack.exceptions.ConflictException: ConflictException: 409: Client Error for url: https://10.x.x.x:13876/v2.0/lbaas/po
ols/302709f7-d122-49d7-9d05-8e57ccf2ad11/members/214e892a-4e25-46ce-8bba-9ae005304931, Pool 302709f7-d122-49d7-9d05-8e57ccf2ad11 is immutable and cannot be updated.                          
2021-11-03 17:04:53.893 1 ERROR kuryr_kubernetes.handlers.logging 

(shiftstack) [stack@undercloud-0 ~]$ openstack loadbalancer show 8d03213c-ab26-4391-9d26-4878fa6b5d02     
+---------------------+-------------------------------------------+
| Field               | Value                                     |
+---------------------+-------------------------------------------+
| admin_state_up      | True                                      |
| created_at          | 2021-11-02T05:21:15                       |
| description         |                                           |
| flavor_id           | None                                      |
| id                  | 8d03213c-ab26-4391-9d26-4878fa6b5d02      |
| listeners           | 39f36082-3a07-44c0-9b00-41ba12db49fa      |
| name                | openshift-marketplace/community-operators |
| operating_status    | ONLINE                                    |
| pools               | 302709f7-d122-49d7-9d05-8e57ccf2ad11      |
| project_id          | fd0d6a21436d4ff987682c3a0419569d          |
| provider            | ovn                                       |
| provisioning_status | ACTIVE                                    |
| updated_at          | 2021-11-02T13:45:52                       |
| vip_address         | 172.30.237.127                            |
| vip_network_id      | f738aa91-61e1-4571-8eac-f3979164393f      |
| vip_port_id         | 72502dfe-e6f8-4e9b-9f00-b846bcf693f8      |
| vip_qos_policy_id   | None                                      |
| vip_subnet_id       | 14d2bae5-7de6-4f77-882b-1d5c919251fc      |
+---------------------+-------------------------------------------+

(shiftstack) [stack@undercloud-0 ~]$ openstack loadbalancer listener show 39f36082-3a07-44c0-9b00-41ba12db49fa
+-----------------------------+-----------------------------------------------------+
| Field                       | Value                                               |
+-----------------------------+-----------------------------------------------------+
| admin_state_up              | True                                                |
| connection_limit            | -1                                                  |
| created_at                  | 2021-11-02T05:21:44                                 |
| default_pool_id             | 302709f7-d122-49d7-9d05-8e57ccf2ad11                |
| default_tls_container_ref   | None                                                |
| description                 |                                                     |
| id                          | 39f36082-3a07-44c0-9b00-41ba12db49fa                |
| insert_headers              | None                                                |
| l7policies                  |                                                     |
| loadbalancers               | 8d03213c-ab26-4391-9d26-4878fa6b5d02                |
| name                        | openshift-marketplace/community-operators:TCP:50051 |
| operating_status            | ONLINE                                              |
| project_id                  | fd0d6a21436d4ff987682c3a0419569d                    |
| protocol                    | TCP                                                 |
| protocol_port               | 50051                                               |
| provisioning_status         | PENDING_UPDATE                                      |
| sni_container_refs          | []                                                  |
| timeout_client_data         | 50000                                               |
| timeout_member_connect      | 5000                                                |
| timeout_member_data         | 50000                                               |
| timeout_tcp_inspect         | 0                                                   |
| updated_at                  | 2021-11-02T13:44:27                                 |
| client_ca_tls_container_ref | None                                                |
| client_authentication       | NONE                                                |
| client_crl_container_ref    | None                                                |
| allowed_cidrs               | None                                                |
+-----------------------------+-----------------------------------------------------+

(shiftstack) [stack@undercloud-0 ~]$ openstack loadbalancer pool show 302709f7-d122-49d7-9d05-8e57ccf2ad11
+----------------------+-----------------------------------------------------+
| Field                | Value                                               |
+----------------------+-----------------------------------------------------+
| admin_state_up       | True                                                |
| created_at           | 2021-11-02T05:21:49                                 |
| description          |                                                     |
| healthmonitor_id     |                                                     |
| id                   | 302709f7-d122-49d7-9d05-8e57ccf2ad11                |
| lb_algorithm         | SOURCE_IP_PORT                                      |
| listeners            | 39f36082-3a07-44c0-9b00-41ba12db49fa                |
| loadbalancers        | 8d03213c-ab26-4391-9d26-4878fa6b5d02                |
| members              | 214e892a-4e25-46ce-8bba-9ae005304931                |
| name                 | openshift-marketplace/community-operators:TCP:50051 |
| operating_status     | OFFLINE                                             |
| project_id           | fd0d6a21436d4ff987682c3a0419569d                    |
| protocol             | TCP                                                 |
| provisioning_status  | ERROR                                               |
| session_persistence  | None                                                |
| updated_at           | 2021-11-02T13:45:52                                 |
| tls_container_ref    | None                                                |
| ca_tls_container_ref | None                                                |
| crl_container_ref    | None                                                |
| tls_enabled          | False                                               |
+----------------------+-----------------------------------------------------+

(shiftstack) [stack@undercloud-0 ~]$ openstack loadbalancer member list 302709f7-d122-49d7-9d05-8e57ccf2ad11
+--------------------------------------+-------------------------------------------------------+----------------------------------+---------------------+---------------+---------------+------------------+--------+
| id                                   | name                                                  | project_id                       | provisioning_status | address       | protocol_port | operating_status | weight |
+--------------------------------------+-------------------------------------------------------+----------------------------------+---------------------+---------------+---------------+------------------+--------+
| 214e892a-4e25-46ce-8bba-9ae005304931 | openshift-marketplace/community-operators-zbmxw:50051 | fd0d6a21436d4ff987682c3a0419569d | ACTIVE              | 10.128.46.128 |         50051 | NO_MONITOR       |      1 |
+--------------------------------------+-------------------------------------------------------+----------------------------------+---------------------+---------------+---------------+------------------+--------+

** Affects: neutron
     Importance: Undecided
     Assignee: Luis Tomas Bolivar (ltomasbo)
         Status: In Progress

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1956745

Title:
  [ovn-octavia-provider]  Load Balancer remained with ACTIVE state even
  with PENDING_UPDATE listener

Status in neutron:
  In Progress

Bug description:
  Description of problem:

  The Load Balancer remained with ACTIVE and ONLINE state even when the
  listener had provisioning_status PENDING_UPDATE and the pool was with
  ERROR, what made the load-balancer to be considered functional by
  Kuryr and a load-balancer member removal was attempt without success
  as pool was immutable:

  2021-11-03 17:04:53.893 1 ERROR kuryr_kubernetes.handlers.logging openstack.exceptions.ConflictException: ConflictException: 409: Client Error for url: https://10.x.x.x:13876/v2.0/lbaas/po
  ols/302709f7-d122-49d7-9d05-8e57ccf2ad11/members/214e892a-4e25-46ce-8bba-9ae005304931, Pool 302709f7-d122-49d7-9d05-8e57ccf2ad11 is immutable and cannot be updated.                          
  2021-11-03 17:04:53.893 1 ERROR kuryr_kubernetes.handlers.logging 

  (shiftstack) [stack@undercloud-0 ~]$ openstack loadbalancer show 8d03213c-ab26-4391-9d26-4878fa6b5d02     
  +---------------------+-------------------------------------------+
  | Field               | Value                                     |
  +---------------------+-------------------------------------------+
  | admin_state_up      | True                                      |
  | created_at          | 2021-11-02T05:21:15                       |
  | description         |                                           |
  | flavor_id           | None                                      |
  | id                  | 8d03213c-ab26-4391-9d26-4878fa6b5d02      |
  | listeners           | 39f36082-3a07-44c0-9b00-41ba12db49fa      |
  | name                | openshift-marketplace/community-operators |
  | operating_status    | ONLINE                                    |
  | pools               | 302709f7-d122-49d7-9d05-8e57ccf2ad11      |
  | project_id          | fd0d6a21436d4ff987682c3a0419569d          |
  | provider            | ovn                                       |
  | provisioning_status | ACTIVE                                    |
  | updated_at          | 2021-11-02T13:45:52                       |
  | vip_address         | 172.30.237.127                            |
  | vip_network_id      | f738aa91-61e1-4571-8eac-f3979164393f      |
  | vip_port_id         | 72502dfe-e6f8-4e9b-9f00-b846bcf693f8      |
  | vip_qos_policy_id   | None                                      |
  | vip_subnet_id       | 14d2bae5-7de6-4f77-882b-1d5c919251fc      |
  +---------------------+-------------------------------------------+

  (shiftstack) [stack@undercloud-0 ~]$ openstack loadbalancer listener show 39f36082-3a07-44c0-9b00-41ba12db49fa
  +-----------------------------+-----------------------------------------------------+
  | Field                       | Value                                               |
  +-----------------------------+-----------------------------------------------------+
  | admin_state_up              | True                                                |
  | connection_limit            | -1                                                  |
  | created_at                  | 2021-11-02T05:21:44                                 |
  | default_pool_id             | 302709f7-d122-49d7-9d05-8e57ccf2ad11                |
  | default_tls_container_ref   | None                                                |
  | description                 |                                                     |
  | id                          | 39f36082-3a07-44c0-9b00-41ba12db49fa                |
  | insert_headers              | None                                                |
  | l7policies                  |                                                     |
  | loadbalancers               | 8d03213c-ab26-4391-9d26-4878fa6b5d02                |
  | name                        | openshift-marketplace/community-operators:TCP:50051 |
  | operating_status            | ONLINE                                              |
  | project_id                  | fd0d6a21436d4ff987682c3a0419569d                    |
  | protocol                    | TCP                                                 |
  | protocol_port               | 50051                                               |
  | provisioning_status         | PENDING_UPDATE                                      |
  | sni_container_refs          | []                                                  |
  | timeout_client_data         | 50000                                               |
  | timeout_member_connect      | 5000                                                |
  | timeout_member_data         | 50000                                               |
  | timeout_tcp_inspect         | 0                                                   |
  | updated_at                  | 2021-11-02T13:44:27                                 |
  | client_ca_tls_container_ref | None                                                |
  | client_authentication       | NONE                                                |
  | client_crl_container_ref    | None                                                |
  | allowed_cidrs               | None                                                |
  +-----------------------------+-----------------------------------------------------+

  (shiftstack) [stack@undercloud-0 ~]$ openstack loadbalancer pool show 302709f7-d122-49d7-9d05-8e57ccf2ad11
  +----------------------+-----------------------------------------------------+
  | Field                | Value                                               |
  +----------------------+-----------------------------------------------------+
  | admin_state_up       | True                                                |
  | created_at           | 2021-11-02T05:21:49                                 |
  | description          |                                                     |
  | healthmonitor_id     |                                                     |
  | id                   | 302709f7-d122-49d7-9d05-8e57ccf2ad11                |
  | lb_algorithm         | SOURCE_IP_PORT                                      |
  | listeners            | 39f36082-3a07-44c0-9b00-41ba12db49fa                |
  | loadbalancers        | 8d03213c-ab26-4391-9d26-4878fa6b5d02                |
  | members              | 214e892a-4e25-46ce-8bba-9ae005304931                |
  | name                 | openshift-marketplace/community-operators:TCP:50051 |
  | operating_status     | OFFLINE                                             |
  | project_id           | fd0d6a21436d4ff987682c3a0419569d                    |
  | protocol             | TCP                                                 |
  | provisioning_status  | ERROR                                               |
  | session_persistence  | None                                                |
  | updated_at           | 2021-11-02T13:45:52                                 |
  | tls_container_ref    | None                                                |
  | ca_tls_container_ref | None                                                |
  | crl_container_ref    | None                                                |
  | tls_enabled          | False                                               |
  +----------------------+-----------------------------------------------------+

  (shiftstack) [stack@undercloud-0 ~]$ openstack loadbalancer member list 302709f7-d122-49d7-9d05-8e57ccf2ad11
  +--------------------------------------+-------------------------------------------------------+----------------------------------+---------------------+---------------+---------------+------------------+--------+
  | id                                   | name                                                  | project_id                       | provisioning_status | address       | protocol_port | operating_status | weight |
  +--------------------------------------+-------------------------------------------------------+----------------------------------+---------------------+---------------+---------------+------------------+--------+
  | 214e892a-4e25-46ce-8bba-9ae005304931 | openshift-marketplace/community-operators-zbmxw:50051 | fd0d6a21436d4ff987682c3a0419569d | ACTIVE              | 10.128.46.128 |         50051 | NO_MONITOR       |      1 |
  +--------------------------------------+-------------------------------------------------------+----------------------------------+---------------------+---------------+---------------+------------------+--------+

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1956745/+subscriptions



Follow ups