yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1958561] Re: [ovn-octavia-provider] Listner/Pool Stuck in PENDING_CREATE Using Fully Populated LB API

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Akihiro Motoki <1958561@xxxxxxxxxxxxxxxxxx>
Date: Tue, 15 Mar 2022 12:26:45 -0000
Reply-to: Bug 1958561 <1958561@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
*** This bug is a duplicate of bug 1958964 ***
    https://bugs.launchpad.net/bugs/1958964

** This bug has been marked a duplicate of bug 1958964
   Fully Populated Load Balancer creation with OVN provider leaves pools as PENDING_CREATE

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1958561

Title:
  [ovn-octavia-provider] Listner/Pool Stuck in PENDING_CREATE Using
  Fully Populated LB API

Status in neutron:
  Fix Released

Bug description:
  Description
  ===========
  When using creating a fully populated LB with ovn as provider, the listener/pools will stuck in PENDING_CREATE STATE.

  I Haven't tested what the result for other Octavia provider as I only
  have OVN in my environment.

  Found this issue while using cloud-provider-openstack with K8S to
  create LB, as it creates LB with fully populated request, it will have
  listeners and pools in PENDING_CREATE status.

  Steps to reproduce
  ==================

  Create a fully populated LB with API (only includes listener in this
  example):

  curl -i -X POST -H "Content-Type: application/json" -H "X-Auth-Token:
  $OS_TOKEN" -d
  '{"loadbalancer":{"admin_state_up":true,"listeners":[{"name":"http_listener","protocol":"TCP","protocol_port":80}],"vip_subnet_id":"SUBNET_ID","provider":"ovn","name":"best_load_balancer","tags":["test_tag"]}}'
  https://XXXXXXXXXXXXX:9876/v2.0/lbaas/loadbalancers

  Expected Result
  ===============

  LB and listener are both created and in ACTIVE state

  Actual Result
  =============

  openstack loadbalancer listener show http_listener
  +-----------------------------+--------------------------------------+
  | Field                       | Value                                |
  +-----------------------------+--------------------------------------+
  | admin_state_up              | True                                 |
  | connection_limit            | -1                                   |
  | created_at                  | 2022-01-20T16:37:58                  |
  | default_pool_id             | None                                 |
  | default_tls_container_ref   | None                                 |
  | description                 |                                      |
  | id                          | de2bbd23-ca08-445f-b58b-e36c8ea709ad |
  | insert_headers              | None                                 |
  | l7policies                  |                                      |
  | loadbalancers               | 467846cf-7b1a-49f8-ae2f-398da103e6e4 |
  | name                        | http_listener                        |
  | operating_status            | OFFLINE                              |
  | project_id                  | 76235f59db4f4b289513b08c3739d2d6     |
  | protocol                    | TCP                                  |
  | protocol_port               | 80                                   |
  | provisioning_status         | PENDING_CREATE                       |
  | sni_container_refs          | []                                   |
  | timeout_client_data         | 50000                                |
  | timeout_member_connect      | 5000                                 |
  | timeout_member_data         | 50000                                |
  | timeout_tcp_inspect         | 0                                    |
  | updated_at                  | None                                 |
  | client_ca_tls_container_ref | None                                 |
  | client_authentication       | NONE                                 |
  | client_crl_container_ref    | None                                 |
  | allowed_cidrs               | None                                 |
  | tls_ciphers                 | None                                 |
  | tls_versions                | None                                 |
  | alpn_protocols              | None                                 |
  | tags                        |                                      |
  +-----------------------------+--------------------------------------+

  The listener will remain in PENDING_CREATE state forever.

  Environment
  ===========

  Deployment method: Kolla-Ansible

  Octavia version:
  octavia-api --version
  9.0.1.dev8

  OpenStack Version: stabe/xena

  octavia.conf is generated with Kolla-Ansible with no modification.

  Logs
  ====

  2022-01-20 15:52:57.991 24 DEBUG ovn_octavia_provider.helper [-] Handling request lb_create with info {'id': '5b743d3f-3c66-4735-89d3-f02f26cd63cc', 'vip_address': '10.112.7.174', 'vip_network_id': 'a5bb4f81-fbe7-4a3c-b6d6-b25d22e48ec0', 'admin_state_up': True} request_handler /var/lib/kolla/venv/lib/python3.8/site-packages/ovn_octavia_provider/helper.py:318
  2022-01-20 15:52:58.037 24 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(table=Load_Balancer, columns={'name': '5b743d3f-3c66-4735-89d3-f02f26cd63cc', 'protocol': [], 'external_ids': {'neutron:vip': '10.112.7.174', 'neutron:vip_port_id': '3935726a-93bb-4930-8a61-6eb07cbe22af', 'enabled': 'True'}, 'selection_fields': ['ip_src', 'ip_dst', 'tp_src', 'tp_dst']}, row=False) do_commit /var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
  2022-01-20 15:52:58.167 24 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): LsLbAddCommand(switch=42ec735c-75c0-4930-a4c7-504e48877605, lb=ac4b72e3-ef04-4c02-ae47-c398db4b409c, may_exist=True) do_commit /var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
  2022-01-20 15:52:58.168 24 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(table=Load_Balancer, record=ac4b72e3-ef04-4c02-ae47-c398db4b409c, col_values=(('external_ids', {'ls_refs': '{"neutron-a5bb4f81-fbe7-4a3c-b6d6-b25d22e48ec0": 1}'}),)) do_commit /var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
  2022-01-20 15:52:58.168 24 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): LrLbAddCommand(router=8a21ef14-97cb-428b-bfc1-a806c3a48243, lb=ac4b72e3-ef04-4c02-ae47-c398db4b409c, may_exist=True) do_commit /var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
  2022-01-20 15:52:58.168 24 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): LsLbAddCommand(switch=neutron-a5bb4f81-fbe7-4a3c-b6d6-b25d22e48ec0, lb=ac4b72e3-ef04-4c02-ae47-c398db4b409c, may_exist=True) do_commit /var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
  2022-01-20 15:52:58.169 24 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): LsLbAddCommand(switch=neutron-a5bb4f81-fbe7-4a3c-b6d6-b25d22e48ec0, lb=ac4b72e3-ef04-4c02-ae47-c398db4b409c, may_exist=True) do_commit /var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
  2022-01-20 15:52:58.169 24 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DbSetCommand(table=Load_Balancer, record=ac4b72e3-ef04-4c02-ae47-c398db4b409c, col_values=(('external_ids', {'lr_ref': 'neutron-9f704c9f-b469-40cc-b31e-b6575bcc0a51'}),)) do_commit /var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
  2022-01-20 15:52:58.210 24 DEBUG ovn_octavia_provider.helper [-] Updating status to octavia: {'loadbalancers': [{'id': '5b743d3f-3c66-4735-89d3-f02f26cd63cc', 'provisioning_status': 'ACTIVE', 'operating_status': 'ONLINE'}]} _update_status_to_octavia /var/lib/kolla/venv/lib/python3.8/site-packages/ovn_octavia_provider/helper.py:345

  Logs when creating LB and listener seperately
  =============================================
  2022-01-20 15:54:55.583 20 INFO octavia.api.v2.controllers.load_balancer [req-8d50c4c2-37c7-4e66-ab5b-61dbcbaa78d9 - 76235f59db4f4b289513b08c3739d2d6 - default default] Sending create Load Balancer c120740a-c6b3-431f-8419-29b824973fde to provider ovn
  2022-01-20 15:54:55.584 20 DEBUG ovn_octavia_provider.helper [-] Handling request lb_create with info {'id': 'c120740a-c6b3-431f-8419-29b824973fde', 'vip_address': '10.112.7.102', 'vip_network_id': 'a5bb4f81-fbe7-4a3c-b6d6-b25d22e48ec0', 'admin_state_up': True} request_handler /var/lib/kolla/venv/lib/python3.8/site-packages/ovn_octavia_provider/helper.py:318
  2022-01-20 15:54:55.625 20 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbCreateCommand(table=Load_Balancer, columns={'name': 'c120740a-c6b3-431f-8419-29b824973fde', 'protocol': [], 'external_ids': {'neutron:vip': '10.112.7.102', 'neutron:vip_port_id': '5d26f051-a891-4934-ba99-e1d76a83b8cd', 'enabled': 'True'}, 'selection_fields': ['ip_src', 'ip_dst', 'tp_src', 'tp_dst']}, row=False) do_commit /var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
  2022-01-20 15:54:55.789 20 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): LsLbAddCommand(switch=42ec735c-75c0-4930-a4c7-504e48877605, lb=e5cc4da9-05be-4428-9600-e202c29adcb5, may_exist=True) do_commit /var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
  2022-01-20 15:54:55.790 20 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(table=Load_Balancer, record=e5cc4da9-05be-4428-9600-e202c29adcb5, col_values=(('external_ids', {'ls_refs': '{"neutron-a5bb4f81-fbe7-4a3c-b6d6-b25d22e48ec0": 1}'}),)) do_commit /var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
  2022-01-20 15:54:55.791 20 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): LrLbAddCommand(router=8a21ef14-97cb-428b-bfc1-a806c3a48243, lb=e5cc4da9-05be-4428-9600-e202c29adcb5, may_exist=True) do_commit /var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
  2022-01-20 15:54:55.791 20 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): LsLbAddCommand(switch=neutron-a5bb4f81-fbe7-4a3c-b6d6-b25d22e48ec0, lb=e5cc4da9-05be-4428-9600-e202c29adcb5, may_exist=True) do_commit /var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
  2022-01-20 15:54:55.792 20 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=4): LsLbAddCommand(switch=neutron-a5bb4f81-fbe7-4a3c-b6d6-b25d22e48ec0, lb=e5cc4da9-05be-4428-9600-e202c29adcb5, may_exist=True) do_commit /var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
  2022-01-20 15:54:55.792 20 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=5): DbSetCommand(table=Load_Balancer, record=e5cc4da9-05be-4428-9600-e202c29adcb5, col_values=(('external_ids', {'lr_ref': 'neutron-9f704c9f-b469-40cc-b31e-b6575bcc0a51'}),)) do_commit /var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
  2022-01-20 15:54:55.859 20 DEBUG ovn_octavia_provider.helper [-] Updating status to octavia: {'loadbalancers': [{'id': 'c120740a-c6b3-431f-8419-29b824973fde', 'provisioning_status': 'ACTIVE', 'operating_status': 'ONLINE'}]} _update_status_to_octavia /var/lib/kolla/venv/lib/python3.8/site-packages/ovn_octavia_provider/helper.py:345
  2022-01-20 15:54:56.302 20 INFO octavia.api.v2.controllers.listener [req-bec2b26f-6c47-43d5-8952-c9640b5c129f - 76235f59db4f4b289513b08c3739d2d6 - default default] Sending create Listener 54f59773-7e9c-4030-9523-c241b025cdf4 to provider ovn
  2022-01-20 15:54:56.302 20 DEBUG ovn_octavia_provider.helper [-] Handling request listener_create with info {'id': '54f59773-7e9c-4030-9523-c241b025cdf4', 'protocol': 'TCP', 'loadbalancer_id': 'c120740a-c6b3-431f-8419-29b824973fde', 'protocol_port': 443, 'default_pool_id': None, 'admin_state_up': True} request_handler /var/lib/kolla/venv/lib/python3.8/site-packages/ovn_octavia_provider/helper.py:318
  2022-01-20 15:54:56.303 20 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(table=Load_Balancer, record=e5cc4da9-05be-4428-9600-e202c29adcb5, col_values=(('protocol', 'tcp'),)) do_commit /var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
  2022-01-20 15:54:56.342 20 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=0): DbSetCommand(table=Load_Balancer, record=e5cc4da9-05be-4428-9600-e202c29adcb5, col_values=(('external_ids', {'listener_54f59773-7e9c-4030-9523-c241b025cdf4': '443:'}),)) do_commit /var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
  2022-01-20 15:54:56.343 20 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=1): DbSetCommand(table=Load_Balancer, record=e5cc4da9-05be-4428-9600-e202c29adcb5, col_values=(('protocol', 'tcp'),)) do_commit /var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
  2022-01-20 15:54:56.343 20 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=2): DbClearCommand(table=Load_Balancer, record=e5cc4da9-05be-4428-9600-e202c29adcb5, column=vips) do_commit /var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
  2022-01-20 15:54:56.343 20 DEBUG ovsdbapp.backend.ovs_idl.transaction [-] Running txn n=1 command(idx=3): DbSetCommand(table=Load_Balancer, record=e5cc4da9-05be-4428-9600-e202c29adcb5, col_values=(('vips', {}),)) do_commit /var/lib/kolla/venv/lib/python3.8/site-packages/ovsdbapp/backend/ovs_idl/transaction.py:90
  2022-01-20 15:54:56.373 20 DEBUG ovn_octavia_provider.helper [-] Updating status to octavia: {'listeners': [{'id': '54f59773-7e9c-4030-9523-c241b025cdf4', 'provisioning_status': 'ACTIVE', 'operating_status': 'ONLINE'}], 'loadbalancers': [{'id': 'c120740a-c6b3-431f-8419-29b824973fde', 'provisioning_status': 'ACTIVE'}]} _update_status_to_octavia /var/lib/kolla/venv/lib/python3.8/site-packages/ovn_octavia_provider/helper.py:345

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1958561/+subscriptions
References

[Bug 1958561] [NEW] [ovn-octavia-provider] Listner/Pool Stuck in PENDING_CREATE Using Fully Populated LB API
From: Ching Kuo, 2022-01-20