yahoo-eng-team team mailing list archive

[Julia Kreger <2127038@xxxxxxxxxxxxxxxxxx>]

So thus far, I've not been able to produce this issue locally on my
local environment. However I've only been going as far as enabling
cleaning which has seemed to work just fine because the network has been
created long in advance.

What appears to fail though is our basic ops test where we end
interacting in fairly rapid succession which will really require
reproduction.

For a first run yesterday, I was able to observe the PXE job partially
working but then failing on additional PXE operations. I'm going to tag
this over to Neutron as well since the behavior is just now inconsistent
as compared to before these patches.

** Also affects: neutron
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2127038

Title:
  OVN CI jobs on ironic are broken

Status in Ironic:
  New
Status in neutron:
  New

Bug description:
  The Ironic OVN CI jobs, represented by:

  ironic-tempest-ovn-uefi-ipmi-pxe
  ironic-tempest-ovn-uefi-ipxe-ipv6

  Both jobs utilize OVN which supports DHCP, one for IPv4 and one for
  IPv6 which appears to have broken starting around midnight, on October
  5th, 2025.

  The state we find the CI jobs in is where the baremetal nodes appear
  not to get DHCP packet handling and ultimately never boot.

  tl;dr is that we end up with errors like the line below in the console
  log.

  
  >>Start PXE over IPv4.

  or a connection timed out if some data gets across.

  It appears IPv6 jobs are not as susceptible to this issue, but still
  do sporadically fail.

  https://zuul.opendev.org/t/openstack/builds?job_name=ironic-tempest-ovn-uefi-ipmi-pxe&project=openstack/ironic
  https://zuul.opendev.org/t/openstack/builds?job_name=ironic-tempest-ovn-uefi-ipxe-ipv6&project=openstack/ironic

  
  Console logs sourced via: 
  - https://storage.gra.cloud.ovh.net/v1/AUTH_dcaab5e32b234d56b626f72581e3644c/zuul_opendev_logs_244/openstack/244c6b9611c94e879c73f4c9a630c45d/controller/logs/ironic-bm-logs/index.html
  - https://e31ba1ae87a54b333074-6d75abbe2d1bba564eb60ae65c4ffb8a.ssl.cf1.rackcdn.com/openstack/d9ecf0ccbe094371bdea42c7f8e14c7b/controller/logs/ironic-bm-logs/index.html
  - https://fab7ac47e8647a645ec4-46682989e655431f9d8a4a56f6d0c4c2.ssl.cf2.rackcdn.com/openstack/5a74c98655d745e5afc9ad7a93bd0df0/controller/logs/ironic-bm-logs/index.html

  Since Ironic had not merged any changes in the time window where this
  started to error, this quickly led to reviewing changes against the
  Neutron repository.

  During the time window, the following changes merged (in order):

  - https://review.opendev.org/c/openstack/neutron/+/962191/
  - https://review.opendev.org/c/openstack/neutron/+/962402/
  - https://review.opendev.org/c/openstack/neutron/+/962404/
  - https://review.opendev.org/c/openstack/neutron/+/962406/ (Unit test only change)
  - https://review.opendev.org/c/openstack/neutron/+/962568/

To manage notifications about this bug go to:
https://bugs.launchpad.net/ironic/+bug/2127038/+subscriptions