canonical-ubuntu-qa team mailing list archive

Thread
Date
[Bug 2028755] [NEW] udpgro_frglist.sh in net from ubuntu_kernel_selftests crashes Power10 LPAR

To: canonical-ubuntu-qa@xxxxxxxxxxxxxxxxxxx
From: Po-Hsu Lin <2028755@xxxxxxxxxxxxxxxxxx>
Date: Wed, 26 Jul 2023 09:07:24 -0000
Reply-to: Bug 2028755 <2028755@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Public bug reported:

The upd test in udpgro_frglist.sh from ubuntu_kernel_selftests/net will
crash the Power10 LPAR with Lunar 6.2.0-27.28. This should not be
considered as this is the first cycle that we have P10 support.

It 100% reproducible when you run the udpgro_frglist.sh script int linux/tools/testing/selftests/net/:
$ sudo ./udpgro_frglist.sh
ipv6
tcp - over veth touching data
-l 4 -6 -D 2001:db8::1 -t rx -4 -t
-4 -t
tcp tx:    531 MB/s     9015 calls/s   9015 msg/s
tcp rx:    531 MB/s     5699 calls/s
tcp tx:    539 MB/s     9149 calls/s   9149 msg/s
tcp rx:    539 MB/s     5353 calls/s
tcp tx:    529 MB/s     8975 calls/s   8975 msg/s
tcp rx:    546 MB/s     5742 calls/s
tcp tx:    534 MB/s     9063 calls/s   9063 msg/s
udp gso - over veth touching data
-l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
-4 -v
packet_write_wait: Connection to 10.245.68.210 port 22: Broken pipe

With "set -x" added before starting the udp test we can see that it's crashing with "./in_netns.sh ./udpgro_frglist.sh __subprocess -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v":
+ run_udp '-l 4 -6  -D 2001:db8::1'
+ local -r 'args=-l 4 -6  -D 2001:db8::1'
+ echo 'udp gso - over veth touching data'
udp gso - over veth touching data
+ run_in_netns -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
+ local -r 'args=-l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v'
+ echo -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
-l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
+ ./in_netns.sh ./udpgro_frglist.sh __subprocess -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
-4 -v
packet_write_wait: Connection to 10.245.68.210 port 22: Broken pipe

No output captured from demsg:
[ 3197.958338] IPv6: ADDRCONF(NETDEV_CHANGE): veth1: link becomes ready
[ 3197.958372] IPv6: ADDRCONF(NETDEV_CHANGE): veth0: link becomes ready
[ 3197.970123] NET: Registered PF_ALG protocol family
[ 3202.293708] IPv6: ADDRCONF(NETDEV_CHANGE): veth1: link becomes ready
[ 3202.293739] IPv6: ADDRCONF(NETDEV_CHANGE): veth0: link becomes ready
packet_write_wait: Connection to 10.245.68.210 port 22: Broken pipe

We might need to use console to see what's going on here.

** Affects: ubuntu-kernel-tests
     Importance: Undecided
         Status: New


** Tags: 6.2 lunar ppc64el sru-20230710 ubuntu-kernel-selftests

** Summary changed:

-  udpgro_frglist.sh in net from ubuntu_kernel_selftests crashes Power10 VM on openstack
+ udpgro_frglist.sh in net from ubuntu_kernel_selftests crashes Power10

** Summary changed:

- udpgro_frglist.sh in net from ubuntu_kernel_selftests crashes Power10
+ udpgro_frglist.sh in net from ubuntu_kernel_selftests crashes Power10 LPAR

** Description changed:

  The upd test in udpgro_frglist.sh from ubuntu_kernel_selftests/net will
- crash the Power10 VM on openstack with Lunar 6.2.0-27.28. This should
- not be considered as this is the first cycle that we have P10 support.
+ crash the Power10 LPAR with Lunar 6.2.0-27.28. This should not be
+ considered as this is the first cycle that we have P10 support.
  
- $ sudo ./udpgro_frglist.sh 
+ $ sudo ./udpgro_frglist.sh
  ipv6
  tcp - over veth touching data
  -l 4 -6 -D 2001:db8::1 -t rx -4 -t
  -4 -t
  tcp tx:    531 MB/s     9015 calls/s   9015 msg/s
  tcp rx:    531 MB/s     5699 calls/s
  tcp tx:    539 MB/s     9149 calls/s   9149 msg/s
  tcp rx:    539 MB/s     5353 calls/s
  tcp tx:    529 MB/s     8975 calls/s   8975 msg/s
  tcp rx:    546 MB/s     5742 calls/s
  tcp tx:    534 MB/s     9063 calls/s   9063 msg/s
  udp gso - over veth touching data
  -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
  -4 -v
  packet_write_wait: Connection to 10.245.68.210 port 22: Broken pipe
  
  With "set -x" added before starting the udp test we can see that it's crashing with "./in_netns.sh ./udpgro_frglist.sh __subprocess -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v":
  + run_udp '-l 4 -6  -D 2001:db8::1'
  + local -r 'args=-l 4 -6  -D 2001:db8::1'
  + echo 'udp gso - over veth touching data'
  udp gso - over veth touching data
  + run_in_netns -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
  + local -r 'args=-l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v'
  + echo -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
  -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
  + ./in_netns.sh ./udpgro_frglist.sh __subprocess -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
  -4 -v
  packet_write_wait: Connection to 10.245.68.210 port 22: Broken pipe
  
- 
  No output captured from demsg:
  [ 3197.958338] IPv6: ADDRCONF(NETDEV_CHANGE): veth1: link becomes ready
  [ 3197.958372] IPv6: ADDRCONF(NETDEV_CHANGE): veth0: link becomes ready
  [ 3197.970123] NET: Registered PF_ALG protocol family
  [ 3202.293708] IPv6: ADDRCONF(NETDEV_CHANGE): veth1: link becomes ready
  [ 3202.293739] IPv6: ADDRCONF(NETDEV_CHANGE): veth0: link becomes ready
  packet_write_wait: Connection to 10.245.68.210 port 22: Broken pipe
  
  We might need to use console to see what's going on here.

** Description changed:

  The upd test in udpgro_frglist.sh from ubuntu_kernel_selftests/net will
  crash the Power10 LPAR with Lunar 6.2.0-27.28. This should not be
  considered as this is the first cycle that we have P10 support.
  
+ It 100% reproducible when you run the udpgro_frglist.sh script int linux/tools/testing/selftests/net/:
  $ sudo ./udpgro_frglist.sh
  ipv6
  tcp - over veth touching data
  -l 4 -6 -D 2001:db8::1 -t rx -4 -t
  -4 -t
  tcp tx:    531 MB/s     9015 calls/s   9015 msg/s
  tcp rx:    531 MB/s     5699 calls/s
  tcp tx:    539 MB/s     9149 calls/s   9149 msg/s
  tcp rx:    539 MB/s     5353 calls/s
  tcp tx:    529 MB/s     8975 calls/s   8975 msg/s
  tcp rx:    546 MB/s     5742 calls/s
  tcp tx:    534 MB/s     9063 calls/s   9063 msg/s
  udp gso - over veth touching data
  -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
  -4 -v
  packet_write_wait: Connection to 10.245.68.210 port 22: Broken pipe
  
  With "set -x" added before starting the udp test we can see that it's crashing with "./in_netns.sh ./udpgro_frglist.sh __subprocess -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v":
  + run_udp '-l 4 -6  -D 2001:db8::1'
  + local -r 'args=-l 4 -6  -D 2001:db8::1'
  + echo 'udp gso - over veth touching data'
  udp gso - over veth touching data
  + run_in_netns -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
  + local -r 'args=-l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v'
  + echo -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
  -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
  + ./in_netns.sh ./udpgro_frglist.sh __subprocess -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
  -4 -v
  packet_write_wait: Connection to 10.245.68.210 port 22: Broken pipe
  
  No output captured from demsg:
  [ 3197.958338] IPv6: ADDRCONF(NETDEV_CHANGE): veth1: link becomes ready
  [ 3197.958372] IPv6: ADDRCONF(NETDEV_CHANGE): veth0: link becomes ready
  [ 3197.970123] NET: Registered PF_ALG protocol family
  [ 3202.293708] IPv6: ADDRCONF(NETDEV_CHANGE): veth1: link becomes ready
  [ 3202.293739] IPv6: ADDRCONF(NETDEV_CHANGE): veth0: link becomes ready
  packet_write_wait: Connection to 10.245.68.210 port 22: Broken pipe
  
  We might need to use console to see what's going on here.

-- 
You received this bug notification because you are a member of Canonical
Platform QA Team, which is subscribed to ubuntu-kernel-tests.
https://bugs.launchpad.net/bugs/2028755

Title:
  udpgro_frglist.sh in net from ubuntu_kernel_selftests crashes Power10
  LPAR

Status in ubuntu-kernel-tests:
  New

Bug description:
  The upd test in udpgro_frglist.sh from ubuntu_kernel_selftests/net
  will crash the Power10 LPAR with Lunar 6.2.0-27.28. This should not be
  considered as this is the first cycle that we have P10 support.

  It 100% reproducible when you run the udpgro_frglist.sh script int linux/tools/testing/selftests/net/:
  $ sudo ./udpgro_frglist.sh
  ipv6
  tcp - over veth touching data
  -l 4 -6 -D 2001:db8::1 -t rx -4 -t
  -4 -t
  tcp tx:    531 MB/s     9015 calls/s   9015 msg/s
  tcp rx:    531 MB/s     5699 calls/s
  tcp tx:    539 MB/s     9149 calls/s   9149 msg/s
  tcp rx:    539 MB/s     5353 calls/s
  tcp tx:    529 MB/s     8975 calls/s   8975 msg/s
  tcp rx:    546 MB/s     5742 calls/s
  tcp tx:    534 MB/s     9063 calls/s   9063 msg/s
  udp gso - over veth touching data
  -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
  -4 -v
  packet_write_wait: Connection to 10.245.68.210 port 22: Broken pipe

  With "set -x" added before starting the udp test we can see that it's crashing with "./in_netns.sh ./udpgro_frglist.sh __subprocess -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v":
  + run_udp '-l 4 -6  -D 2001:db8::1'
  + local -r 'args=-l 4 -6  -D 2001:db8::1'
  + echo 'udp gso - over veth touching data'
  udp gso - over veth touching data
  + run_in_netns -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
  + local -r 'args=-l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v'
  + echo -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
  -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
  + ./in_netns.sh ./udpgro_frglist.sh __subprocess -l 4 -6 -D 2001:db8::1 -u -S 0 rx -4 -v
  -4 -v
  packet_write_wait: Connection to 10.245.68.210 port 22: Broken pipe

  No output captured from demsg:
  [ 3197.958338] IPv6: ADDRCONF(NETDEV_CHANGE): veth1: link becomes ready
  [ 3197.958372] IPv6: ADDRCONF(NETDEV_CHANGE): veth0: link becomes ready
  [ 3197.970123] NET: Registered PF_ALG protocol family
  [ 3202.293708] IPv6: ADDRCONF(NETDEV_CHANGE): veth1: link becomes ready
  [ 3202.293739] IPv6: ADDRCONF(NETDEV_CHANGE): veth0: link becomes ready
  packet_write_wait: Connection to 10.245.68.210 port 22: Broken pipe

  We might need to use console to see what's going on here.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-kernel-tests/+bug/2028755/+subscriptions
Follow ups

[Bug 2028755] Re: udpgro_frglist.sh in net from ubuntu_kernel_selftests crashes Power10 LPAR
From: Po-Hsu Lin, 2024-06-07