yahoo-eng-team team mailing list archive

[Ihar Hrachyshka <1997262@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Examples:

https://zuul.opendev.org/t/openstack/build/a1d2c87b23e641539cff111abc8c5e4a/log/job-output.txt
https://5b948c35ca2c61dd95cb-b83277dfb85347aca874505fa2930911.ssl.cf2.rackcdn.com/862358/2/check/networking-ovn-dsvm-functional-py27/073519e/job-output.txt

The job takes more than 2h and is eventually killed.

Log example:

2022-11-20 01:04:17.245895 | primary | 2022-11-20 01:04:17.245 | {7} networking_ovn.tests.functional.test_ovn_db_sync.TestOvnNbSync.test_ovn_nb_sync_off [67.421953s] ... ok
2022-11-20 01:04:17.247309 | primary | 2022-11-20 01:04:17.246 | {0} networking_ovn.tests.functional.test_router.TestRouter.test_gateway_chassis_with_bridge_mappings [12.473030s] ... ok
2022-11-20 01:04:17.248841 | primary | 2022-11-20 01:04:17.248 | {2} networking_ovn.tests.functional.test_router.TestRouter.test_router_port_ipv6_ra_configs_ipv4 [6.018013s] ... ok
2022-11-20 01:04:17.250444 | primary | 2022-11-20 01:04:17.250 | {0} networking_ovn.tests.functional.test_router.TestRouter.test_router_port_ipv6_ra_configs_addr_mode_slaac [5.654530s] ... ok
2022-11-20 01:04:17.252024 | primary | 2022-11-20 01:04:17.251 | {2} networking_ovn.tests.functional.test_trunk_driver.TestOVNTrunkDriver.test_subport_add [4.991551s] ... ok
2022-11-20 01:04:17.255232 | primary | 2022-11-20 01:04:17.254 | {0} networking_ovn.tests.functional.test_trunk_driver.TestOVNTrunkDriver.test_trunk_create_with_subports [4.687337s] ... ok
2022-11-20 01:04:17.256586 | primary | 2022-11-20 01:04:17.256 | {6} networking_ovn.tests.functional.test_ovn_db_sync.TestOvnNbSync.test_ovn_nb_sync_repair [73.895267s] ... ok
2022-11-20 01:04:17.257905 | primary | 2022-11-20 01:04:17.257 | {4} networking_ovn.tests.functional.test_ovn_db_sync.TestOvnNbSyncOverSsl.test_ovn_nb_sync_off [64.915188s] ... ok
2022-11-20 01:04:17.263929 | primary | 2022-11-20 01:04:17.259 | {6} networking_ovn.tests.functional.test_ovsdb_monitor.TestNBDbMonitor.test_distributed_lock [3.913342s] ... ok
2022-11-20 01:04:17.265641 | primary | 2022-11-20 01:04:17.265 | {6} networking_ovn.tests.functional.test_ovsdb_monitor.TestNBDbMonitor.test_floatingip_mac_bindings [11.765796s] ... ok
2022-11-20 01:04:17.267119 | primary | 2022-11-20 01:04:17.266 | {6} networking_ovn.tests.functional.test_ovsdb_monitor.TestNBDbMonitorOverSsl.test_port_up_down_events [6.316748s] ... ok
2022-11-20 01:04:17.271742 | primary | 2022-11-20 01:04:17.271 | {6} networking_ovn.tests.functional.test_qos_driver.TestOVNQosDriver.test_port_qos_options_with_rule [3.441892s] ... ok
2022-11-20 02:51:16.344359 | primary | /home/zuul/workspace/devstack-gate/functions.sh: line 1009: 26167 Killed                  timeout -s 9 ${REMAINING_TIME}m bash -c "source $WORKSPACE/devstack-gate/functions.sh && $cmd"

Observation:

both failed runs were executing the following test cases just before the
test runner froze:

2022-11-20 01:04:17.256586 | primary | 2022-11-20 01:04:17.256 | {6} networking_ovn.tests.functional.test_ovn_db_sync.TestOvnNbSync.test_ovn_nb_sync_repair [73.895267s] ... ok
2022-11-20 01:04:17.257905 | primary | 2022-11-20 01:04:17.257 | {4} networking_ovn.tests.functional.test_ovn_db_sync.TestOvnNbSyncOverSsl.test_ovn_nb_sync_off [64.915188s] ... ok

(I don't know if this is related though)

** Affects: neutron
     Importance: Undecided
         Status: New


** Tags: functional-tests gate-failure ovn

** Tags added: ovn

** Tags added: gate-failure

** Tags added: functional-tests

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1997262

Title:
  networking-ovn-dsvm-functional-py27 job killed on timeout in
  stable/train branch

Status in neutron:
  New

Bug description:
  Examples:

  https://zuul.opendev.org/t/openstack/build/a1d2c87b23e641539cff111abc8c5e4a/log/job-output.txt
  https://5b948c35ca2c61dd95cb-b83277dfb85347aca874505fa2930911.ssl.cf2.rackcdn.com/862358/2/check/networking-ovn-dsvm-functional-py27/073519e/job-output.txt

  The job takes more than 2h and is eventually killed.

  Log example:

  2022-11-20 01:04:17.245895 | primary | 2022-11-20 01:04:17.245 | {7} networking_ovn.tests.functional.test_ovn_db_sync.TestOvnNbSync.test_ovn_nb_sync_off [67.421953s] ... ok
  2022-11-20 01:04:17.247309 | primary | 2022-11-20 01:04:17.246 | {0} networking_ovn.tests.functional.test_router.TestRouter.test_gateway_chassis_with_bridge_mappings [12.473030s] ... ok
  2022-11-20 01:04:17.248841 | primary | 2022-11-20 01:04:17.248 | {2} networking_ovn.tests.functional.test_router.TestRouter.test_router_port_ipv6_ra_configs_ipv4 [6.018013s] ... ok
  2022-11-20 01:04:17.250444 | primary | 2022-11-20 01:04:17.250 | {0} networking_ovn.tests.functional.test_router.TestRouter.test_router_port_ipv6_ra_configs_addr_mode_slaac [5.654530s] ... ok
  2022-11-20 01:04:17.252024 | primary | 2022-11-20 01:04:17.251 | {2} networking_ovn.tests.functional.test_trunk_driver.TestOVNTrunkDriver.test_subport_add [4.991551s] ... ok
  2022-11-20 01:04:17.255232 | primary | 2022-11-20 01:04:17.254 | {0} networking_ovn.tests.functional.test_trunk_driver.TestOVNTrunkDriver.test_trunk_create_with_subports [4.687337s] ... ok
  2022-11-20 01:04:17.256586 | primary | 2022-11-20 01:04:17.256 | {6} networking_ovn.tests.functional.test_ovn_db_sync.TestOvnNbSync.test_ovn_nb_sync_repair [73.895267s] ... ok
  2022-11-20 01:04:17.257905 | primary | 2022-11-20 01:04:17.257 | {4} networking_ovn.tests.functional.test_ovn_db_sync.TestOvnNbSyncOverSsl.test_ovn_nb_sync_off [64.915188s] ... ok
  2022-11-20 01:04:17.263929 | primary | 2022-11-20 01:04:17.259 | {6} networking_ovn.tests.functional.test_ovsdb_monitor.TestNBDbMonitor.test_distributed_lock [3.913342s] ... ok
  2022-11-20 01:04:17.265641 | primary | 2022-11-20 01:04:17.265 | {6} networking_ovn.tests.functional.test_ovsdb_monitor.TestNBDbMonitor.test_floatingip_mac_bindings [11.765796s] ... ok
  2022-11-20 01:04:17.267119 | primary | 2022-11-20 01:04:17.266 | {6} networking_ovn.tests.functional.test_ovsdb_monitor.TestNBDbMonitorOverSsl.test_port_up_down_events [6.316748s] ... ok
  2022-11-20 01:04:17.271742 | primary | 2022-11-20 01:04:17.271 | {6} networking_ovn.tests.functional.test_qos_driver.TestOVNQosDriver.test_port_qos_options_with_rule [3.441892s] ... ok
  2022-11-20 02:51:16.344359 | primary | /home/zuul/workspace/devstack-gate/functions.sh: line 1009: 26167 Killed                  timeout -s 9 ${REMAINING_TIME}m bash -c "source $WORKSPACE/devstack-gate/functions.sh && $cmd"

  Observation:

  both failed runs were executing the following test cases just before
  the test runner froze:

  2022-11-20 01:04:17.256586 | primary | 2022-11-20 01:04:17.256 | {6} networking_ovn.tests.functional.test_ovn_db_sync.TestOvnNbSync.test_ovn_nb_sync_repair [73.895267s] ... ok
  2022-11-20 01:04:17.257905 | primary | 2022-11-20 01:04:17.257 | {4} networking_ovn.tests.functional.test_ovn_db_sync.TestOvnNbSyncOverSsl.test_ovn_nb_sync_off [64.915188s] ... ok

  (I don't know if this is related though)

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1997262/+subscriptions