← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 2025946] [NEW] Neutron 504 Gateway Timeout Openstack Kolla-Ansible : Ussuri

 

Public bug reported:

i have 3 controller openstack, but the network agent often 504 Gateway
timeout. when i see neutron_server.log, this logs showed up in one of my
controllers

2023-05-24 10:00:23.314 687 ERROR neutron.api.v2.resource [req-a1f3e58a-00f7-4ed9-b8e5-6c538dc5d5a3 3fe50ccef00f49e3b1b0bbd58705a930 c7d2001e7a2c4c32b9f2a3657f29b6b0 - default default] index failed: No details.: ovsdbapp.exceptions.TimeoutException: Commands [<neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.commands.CheckLivenessCommand object at 0x7f5ce81a98d0>] exceeded timeout 180 seconds
2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource Traceback (most recent call last):                                                                                                                                                                
2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource   File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 153, in queue_txn                                                                           
2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource     self.txns.put(txn, timeout=self.timeout)                                                                                                                                                      
2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource   File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 51, in put                                                                                  
2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource     super(TransactionQueue, self).put(*args, **kwargs)                                                                                                                                            
2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource   File "/var/lib/kolla/venv/lib/python3.6/site-packages/eventlet/queue.py", line 264, in put                                                                                                      
2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource     result = waiter.wait()                                                                                                                                                                        
2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource   File "/var/lib/kolla/venv/lib/python3.6/site-packages/eventlet/queue.py", line 141, in wait                                                                                                     
2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource     return get_hub().switch()                                                                                                                                                                     
2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource   File "/var/lib/kolla/venv/lib/python3.6/site-packages/eventlet/hubs/hub.py", line 298, in switch                                                                                                
2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource     return self.greenlet.switch()                                                                                                                                                                 
2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource queue.Full                                                                                                                                                                                        
2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource                                                                                                                                                                                                   
2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource During handling of the above exception, another exception occurred:                                                                                                                               
2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource  

How do i solve this?

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2025946

Title:
  Neutron 504 Gateway Timeout Openstack Kolla-Ansible : Ussuri

Status in neutron:
  New

Bug description:
  i have 3 controller openstack, but the network agent often 504 Gateway
  timeout. when i see neutron_server.log, this logs showed up in one of
  my controllers

  2023-05-24 10:00:23.314 687 ERROR neutron.api.v2.resource [req-a1f3e58a-00f7-4ed9-b8e5-6c538dc5d5a3 3fe50ccef00f49e3b1b0bbd58705a930 c7d2001e7a2c4c32b9f2a3657f29b6b0 - default default] index failed: No details.: ovsdbapp.exceptions.TimeoutException: Commands [<neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.commands.CheckLivenessCommand object at 0x7f5ce81a98d0>] exceeded timeout 180 seconds
  2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource Traceback (most recent call last):                                                                                                                                                                
  2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource   File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 153, in queue_txn                                                                           
  2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource     self.txns.put(txn, timeout=self.timeout)                                                                                                                                                      
  2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource   File "/var/lib/kolla/venv/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 51, in put                                                                                  
  2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource     super(TransactionQueue, self).put(*args, **kwargs)                                                                                                                                            
  2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource   File "/var/lib/kolla/venv/lib/python3.6/site-packages/eventlet/queue.py", line 264, in put                                                                                                      
  2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource     result = waiter.wait()                                                                                                                                                                        
  2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource   File "/var/lib/kolla/venv/lib/python3.6/site-packages/eventlet/queue.py", line 141, in wait                                                                                                     
  2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource     return get_hub().switch()                                                                                                                                                                     
  2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource   File "/var/lib/kolla/venv/lib/python3.6/site-packages/eventlet/hubs/hub.py", line 298, in switch                                                                                                
  2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource     return self.greenlet.switch()                                                                                                                                                                 
  2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource queue.Full                                                                                                                                                                                        
  2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource                                                                                                                                                                                                   
  2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource During handling of the above exception, another exception occurred:                                                                                                                               
  2023-07-05 09:49:03.453 670 ERROR neutron.api.v2.resource  

  How do i solve this?

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2025946/+subscriptions



Follow ups