yahoo-eng-team team mailing list archive

Thread
Date
[Bug 2034684] [NEW] UEFI (edk2/ovmf) network boot with OVN fail because no DHCP release reply

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Harald Jensås <2034684@xxxxxxxxxxxxxxxxxx>
Date: Thu, 07 Sep 2023 09:08:31 -0000
Reply-to: Bug 2034684 <2034684@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Public bug reported:

When attempting to verify neutron change[1], we discovered that despite
options in DHCPv6 ADV and REQ/REPLY are correct network booting still
fails.

When comparing traffic capture between openvswitch+neutron-dhcp-agent setup to the ovn setup a significant difference is that:
* neutron-dhcp-ageent(dnsmasq) does REPLY to RELEASE with a packet including a dhcpv6 option type Status code (13) success to confirm the release. edk2/ovmf does TFTP transfer of the NBP immediately after recieving this reply.
* OVN does not respond with a REPLY to the clients RELEASE. In traffic capture we can see the client repeates the RELEASE several times, but finally give up and raise an error:

>>Start PXE over IPv6..
  Station IP address is FC01:0:0:0:0:0:0:206
  Server IP address is FC00:0:0:0:0:0:0:1
  NBP filename is snponly.efi
  NBP filesize is 0 Bytes
  PXE-E53: No boot filename received.

--------------------------------------------------
FAILING - sequence on OVN
--------------------------------------------------
No.	Time	Source	Destination	Protocol	Length	Info
1	0.000000	fe80::f816:3eff:fe6f:a0ab	::	ICMPv6	118	Router Advertisement from fa:16:3e:6f:a0:ab
2	51.931422	fe80::5054:ff:feb1:a5b0	ff02::1:2	DHCPv6	177	Solicit XID: 0x4f04ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 
3	51.931840	fe80::f816:3eff:feeb:b176	fe80::5054:ff:feb1:a5b0	DHCPv6	198	Advertise XID: 0x4f04ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 IAA: fc01::2ad 
4	56.900421	fe80::5054:ff:feb1:a5b0	ff02::1:2	DHCPv6	219	Request XID: 0x5004ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 IAA: fc01::2ad 
5	56.900726	fe80::f816:3eff:feeb:b176	fe80::5054:ff:feb1:a5b0	DHCPv6	198	Reply XID: 0x5004ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 IAA: fc01::2ad 
6	68.861979	fe80::5054:ff:feb1:a5b0	ff02::1:2	DHCPv6	152	Release XID: 0x5104ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 IAA: fc01::2ad 
7	69.900715	fe80::5054:ff:feb1:a5b0	ff02::1:2	DHCPv6	152	Release XID: 0x5104ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 IAA: fc01::2ad 
8	72.900784	fe80::5054:ff:feb1:a5b0	ff02::1:2	DHCPv6	152	Release XID: 0x5104ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 IAA: fc01::2ad 
9	77.900774	fe80::5054:ff:feb1:a5b0	ff02::1:2	DHCPv6	152	Release XID: 0x5104ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 IAA: fc01::2ad 
10	86.900759	fe80::5054:ff:feb1:a5b0	ff02::1:2	DHCPv6	152	Release XID: 0x5104ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 IAA: fc01::2ad 
11	103.900786	fe80::5054:ff:feb1:a5b0	ff02::1:2	DHCPv6	152	Release XID: 0x5104ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 IAA: fc01::2ad 


--------------------------------------------------
WORKING - sequence on neutron-dhcp-agent (dnsmasq)
--------------------------------------------------
No.	Time	Source	Destination	Protocol	Length	Info
1	0.000000	fe80::f816:3eff:fe38:eef0	ff02::1	ICMPv6	142	Router Advertisement from fa:16:3e:38:ee:f0
2	0.001102	fe80::5054:ff:fed9:3d5c	ff02::1:2	DHCPv6	116	Solicit XID: 0x71d892 CID: 0004c9b0caa37bce994e85633d7572708047 
3	0.001245	fe80::f816:3eff:fef5:ef7a	fe80::5054:ff:fed9:3d5c	DHCPv6	208	Advertise XID: 0x71d892 CID: 0004c9b0caa37bce994e85633d7572708047 IAA: fc01::87 
4	0.002436	fe80::5054:ff:fed9:3d5c	ff02::1:2	DHCPv6	162	Request XID: 0x72d892 CID: 0004c9b0caa37bce994e85633d7572708047 IAA: fc01::87 
5	0.002508	fe80::f816:3eff:fef5:ef7a	fe80::5054:ff:fed9:3d5c	DHCPv6	219	Reply XID: 0x72d892 CID: 0004c9b0caa37bce994e85633d7572708047 IAA: fc01::87 
6	3.130605	fe80::5054:ff:fed9:3d5c	ff02::1:2	DHCPv6	223	Request XID: 0x73d892 CID: 0004c9b0caa37bce994e85633d7572708047 IAA: fc01::87 
7	3.130791	fe80::f816:3eff:fef5:ef7a	fe80::5054:ff:fed9:3d5c	DHCPv6	256	Reply XID: 0x73d892 CID: 0004c9b0caa37bce994e85633d7572708047 IAA: fc01::2a0 
8	3.132060	fe80::5054:ff:fed9:3d5c	ff02::1:2	DHCPv6	156	Release XID: 0x74d892 CID: 0004c9b0caa37bce994e85633d7572708047 IAA: fc01::87 
9	3.132126	fe80::f816:3eff:fef5:ef7a	fe80::5054:ff:fed9:3d5c	DHCPv6	128	Reply XID: 0x74d892 CID: 0004c9b0caa37bce994e85633d7572708047 
10	5.477847	fc01::2a0	fc00::1	TFTP	116	Read Request, File: snponly.efi, Transfer type: octet, tsize=0, blksize=1228, windowsize=4
--------------------------------------------------


Conclusion is that OVN DHCPv6 implementation need to be fixed, it should reply when the client send a dhcpv6 release.

Attached file contain traffic capture files from both the working
(dnsmasq dhcp) set-up and the failing (OVN dhcp) set-up.


[1] https://review.opendev.org/c/openstack/neutron/+/890683

** Affects: ironic
     Importance: Undecided
         Status: New

** Affects: neutron
     Importance: Undecided
         Status: New

** Attachment added: "ovn-snippet.pcap and ovs-snippet.pcap, contains RA + DHCPv6 network boot sequence from working OVS(dnsmasq) and failing OVN environment"
   https://bugs.launchpad.net/bugs/2034684/+attachment/5698203/+files/pcap-snippets.tar.gz

** Also affects: neutron
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2034684

Title:
  UEFI (edk2/ovmf) network boot with OVN fail because no DHCP release
  reply

Status in Ironic:
  New
Status in neutron:
  New

Bug description:
  When attempting to verify neutron change[1], we discovered that
  despite options in DHCPv6 ADV and REQ/REPLY are correct network
  booting still fails.

  When comparing traffic capture between openvswitch+neutron-dhcp-agent setup to the ovn setup a significant difference is that:
  * neutron-dhcp-ageent(dnsmasq) does REPLY to RELEASE with a packet including a dhcpv6 option type Status code (13) success to confirm the release. edk2/ovmf does TFTP transfer of the NBP immediately after recieving this reply.
  * OVN does not respond with a REPLY to the clients RELEASE. In traffic capture we can see the client repeates the RELEASE several times, but finally give up and raise an error:

  >>Start PXE over IPv6..
    Station IP address is FC01:0:0:0:0:0:0:206
    Server IP address is FC00:0:0:0:0:0:0:1
    NBP filename is snponly.efi
    NBP filesize is 0 Bytes
    PXE-E53: No boot filename received.

  --------------------------------------------------
  FAILING - sequence on OVN
  --------------------------------------------------
  No.	Time	Source	Destination	Protocol	Length	Info
  1	0.000000	fe80::f816:3eff:fe6f:a0ab	::	ICMPv6	118	Router Advertisement from fa:16:3e:6f:a0:ab
  2	51.931422	fe80::5054:ff:feb1:a5b0	ff02::1:2	DHCPv6	177	Solicit XID: 0x4f04ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 
  3	51.931840	fe80::f816:3eff:feeb:b176	fe80::5054:ff:feb1:a5b0	DHCPv6	198	Advertise XID: 0x4f04ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 IAA: fc01::2ad 
  4	56.900421	fe80::5054:ff:feb1:a5b0	ff02::1:2	DHCPv6	219	Request XID: 0x5004ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 IAA: fc01::2ad 
  5	56.900726	fe80::f816:3eff:feeb:b176	fe80::5054:ff:feb1:a5b0	DHCPv6	198	Reply XID: 0x5004ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 IAA: fc01::2ad 
  6	68.861979	fe80::5054:ff:feb1:a5b0	ff02::1:2	DHCPv6	152	Release XID: 0x5104ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 IAA: fc01::2ad 
  7	69.900715	fe80::5054:ff:feb1:a5b0	ff02::1:2	DHCPv6	152	Release XID: 0x5104ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 IAA: fc01::2ad 
  8	72.900784	fe80::5054:ff:feb1:a5b0	ff02::1:2	DHCPv6	152	Release XID: 0x5104ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 IAA: fc01::2ad 
  9	77.900774	fe80::5054:ff:feb1:a5b0	ff02::1:2	DHCPv6	152	Release XID: 0x5104ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 IAA: fc01::2ad 
  10	86.900759	fe80::5054:ff:feb1:a5b0	ff02::1:2	DHCPv6	152	Release XID: 0x5104ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 IAA: fc01::2ad 
  11	103.900786	fe80::5054:ff:feb1:a5b0	ff02::1:2	DHCPv6	152	Release XID: 0x5104ed CID: 000430a25dc55972534aa516ff9c9f7c7ac4 IAA: fc01::2ad 

  
  --------------------------------------------------
  WORKING - sequence on neutron-dhcp-agent (dnsmasq)
  --------------------------------------------------
  No.	Time	Source	Destination	Protocol	Length	Info
  1	0.000000	fe80::f816:3eff:fe38:eef0	ff02::1	ICMPv6	142	Router Advertisement from fa:16:3e:38:ee:f0
  2	0.001102	fe80::5054:ff:fed9:3d5c	ff02::1:2	DHCPv6	116	Solicit XID: 0x71d892 CID: 0004c9b0caa37bce994e85633d7572708047 
  3	0.001245	fe80::f816:3eff:fef5:ef7a	fe80::5054:ff:fed9:3d5c	DHCPv6	208	Advertise XID: 0x71d892 CID: 0004c9b0caa37bce994e85633d7572708047 IAA: fc01::87 
  4	0.002436	fe80::5054:ff:fed9:3d5c	ff02::1:2	DHCPv6	162	Request XID: 0x72d892 CID: 0004c9b0caa37bce994e85633d7572708047 IAA: fc01::87 
  5	0.002508	fe80::f816:3eff:fef5:ef7a	fe80::5054:ff:fed9:3d5c	DHCPv6	219	Reply XID: 0x72d892 CID: 0004c9b0caa37bce994e85633d7572708047 IAA: fc01::87 
  6	3.130605	fe80::5054:ff:fed9:3d5c	ff02::1:2	DHCPv6	223	Request XID: 0x73d892 CID: 0004c9b0caa37bce994e85633d7572708047 IAA: fc01::87 
  7	3.130791	fe80::f816:3eff:fef5:ef7a	fe80::5054:ff:fed9:3d5c	DHCPv6	256	Reply XID: 0x73d892 CID: 0004c9b0caa37bce994e85633d7572708047 IAA: fc01::2a0 
  8	3.132060	fe80::5054:ff:fed9:3d5c	ff02::1:2	DHCPv6	156	Release XID: 0x74d892 CID: 0004c9b0caa37bce994e85633d7572708047 IAA: fc01::87 
  9	3.132126	fe80::f816:3eff:fef5:ef7a	fe80::5054:ff:fed9:3d5c	DHCPv6	128	Reply XID: 0x74d892 CID: 0004c9b0caa37bce994e85633d7572708047 
  10	5.477847	fc01::2a0	fc00::1	TFTP	116	Read Request, File: snponly.efi, Transfer type: octet, tsize=0, blksize=1228, windowsize=4
  --------------------------------------------------

  
  Conclusion is that OVN DHCPv6 implementation need to be fixed, it should reply when the client send a dhcpv6 release.

  Attached file contain traffic capture files from both the working
  (dnsmasq dhcp) set-up and the failing (OVN dhcp) set-up.

  
  [1] https://review.opendev.org/c/openstack/neutron/+/890683

To manage notifications about this bug go to:
https://bugs.launchpad.net/ironic/+bug/2034684/+subscriptions
Follow ups

[Bug 2034684] Re: UEFI (edk2/ovmf) network boot with OVN fail because no DHCP release reply
From: Dmitry Tantsur, 2024-02-28
[Bug 2034684] Re: UEFI (edk2/ovmf) network boot with OVN fail because no DHCP release reply
From: Rodolfo Alonso, 2023-09-12