canonical-ubuntu-qa team mailing list archive

Thread
Date
[Bug 2090819] [NEW] cpuset_hotplug failed on N-aws since 6.8.0-1017-aws on AWS x1e.xlarge

To: canonical-ubuntu-qa@xxxxxxxxxxxxxxxxxxx
From: Po-Hsu Lin <2090819@xxxxxxxxxxxxxxxxxx>
Date: Mon, 02 Dec 2024 06:59:01 -0000
Reply-to: Bug 2090819 <2090819@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Public bug reported:

Issue found on 6.8.0-1020.20, looking back into our test history we can
see it's failing with 6.8.0-1018.20 on the same instance in
sru-20240930.

And it has passed with 6.8.0-1015.16 in sru-20240805 and 6.8.0-1016.17
in sru-s20240805

A manual test shows this failure begins with 6.8.0-1017-aws

Test result with 6.8.0-1016.17:
<<<test_start>>>
tag=cpuset_hotplug stime=1733121824
cmdline="cpuset_hotplug_test.sh"
contacts=""
analysis=exit
<<<test_output>>>
incrementing stop
cpuset_hotplug 1 TINFO: CPUs are numbered continuously starting at 0 (0-3)
cpuset_hotplug 1 TINFO: Nodes are numbered continuously starting at 0 (0)
cpuset_hotplug 1 TCONF: Could not mount cgroup filesystem with cpuset on /dev/cpuset
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=32 corefile=no
cutime=1 cstime=1
<<<test_end>>>

Test result with 6.8.0-1017:
<<<test_start>>>
tag=cpuset_hotplug stime=1733122053
cmdline="cpuset_hotplug_test.sh"
contacts=""
analysis=exit
<<<test_output>>>
incrementing stop
cpuset_hotplug 1 TINFO: CPUs are numbered continuously starting at 0 (0-3)
cpuset_hotplug 1 TINFO: Nodes are numbered continuously starting at 0 (0)
cpuset_hotplug 1 TPASS: Cpuset vs CPU hotplug test succeeded.
/bin/echo: write error: Input/output error
cpuset_hotplug 3 TINFO: /bin/echo: write error: Input/output error
cpuset_hotplug 3 TFAIL:  CPU#1 failed.
/bin/echo: write error: Input/output error
cpuset_hotplug 5 TINFO: /bin/echo: write error: Invalid argument
cpuset_hotplug 5 TFAIL: set general group parameter failed.
/bin/echo: write error: Input/output error
cpuset_hotplug 7 TINFO: /bin/echo: write error: Invalid argument
cpuset_hotplug 7 TFAIL: set general group parameter failed.
/bin/echo: write error: Input/output error
cpuset_hotplug 9 TPASS: Cpuset vs CPU hotplug test succeeded.
/bin/echo: write error: Input/output error
cpuset_hotplug 11 TINFO: /bin/echo: write error: Input/output error
cpuset_hotplug 11 TFAIL:  CPU#1 failed.
/bin/echo: write error: Input/output error
<<<execution_status>>>
initiation_status="ok"
duration=82 termination_type=exited termination_id=1 corefile=no
cutime=136 cstime=1168
<<<test_end>>>

This failure will cause the following cpuset_regression_test to fail with:
<<<test_start>>>
tag=cpuset_regression_test stime=1733122896
cmdline="cpuset_regression_test.sh"
contacts=""
analysis=exit
<<<test_output>>>
incrementing stop
cpuset_regression_test 1 TINFO: Running: cpuset_regression_test.sh 
cpuset_regression_test 1 TINFO: Tested kernel: Linux ip-172-31-3-132 6.8.0-1017-aws #18-Ubuntu SMP Wed Oct  2 20:17:03 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
cpuset_regression_test 1 TINFO: Using /tmp/ltp-SdsOVtsygU/LTP_cpuset_regression_test.vBePRqZkZM as tmpdir (ext2/ext3 filesystem)
cpuset_regression_test 1 TINFO: timeout per run is 0h 5m 0s
tst_cgroup.c:776: TINFO: Mounted V1 cpuset CGroup on /tmp/ltp-SdsOVtsygU/cgroup_cpuset
cpuset_regression_test 1 TINFO: test starts with cgroup version 1
/opt/ltp/testcases/bin/cpuset_regression_test.sh: 78: echo: echo: I/O error
cpuset_regression_test 1 TBROK: Unable to prepare existing cpuset cgroups
/opt/ltp/testcases/bin/cpuset_regression_test.sh: 111: echo: echo: I/O error
cpuset_regression_test 1 TINFO: AppArmor enabled, this may affect test results
cpuset_regression_test 1 TINFO: it can be disabled with TST_DISABLE_APPARMOR=1 (requires super/root)
cpuset_regression_test 1 TINFO: loaded AppArmor profiles: none

Summary:
passed   0
failed   0
broken   1
skipped  0
warnings 0
<<<execution_status>>>

If we don't run the cpuset_hotplug test, this test will be skipped:
<<<test_start>>>
tag=cpuset_regression_test stime=1733122991
cmdline="cpuset_regression_test.sh"
contacts=""
analysis=exit
<<<test_output>>>
incrementing stop
cpuset_regression_test 1 TINFO: Running: cpuset_regression_test.sh 
cpuset_regression_test 1 TINFO: Tested kernel: Linux ip-172-31-3-132 6.8.0-1017-aws #18-Ubuntu SMP Wed Oct  2 20:17:03 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
cpuset_regression_test 1 TINFO: Using /tmp/ltp-NQZT7RC4j4/LTP_cpuset_regression_test.YG7BLBZW9X as tmpdir (ext2/ext3 filesystem)
cpuset_regression_test 1 TINFO: timeout per run is 0h 5m 0s
cpuset_regression_test 1 TINFO: test starts with cgroup version 2
cpuset_regression_test 1 TCONF: cgroup v2 found, skipping test

Summary:
passed   0
failed   0
broken   0
skipped  1
warnings 0
<<<execution_status>>>
initiation_status="ok"
duration=0 termination_type=exited termination_id=32 corefile=no
cutime=2 cstime=2
<<<test_end>>>

** Affects: ubuntu-kernel-tests
     Importance: Undecided
         Status: New

** Affects: linux-aws (Ubuntu)
     Importance: Undecided
         Status: New

** Affects: linux-aws (Ubuntu Noble)
     Importance: Undecided
     Assignee: Philip Cox (philcox)
         Status: New


** Tags: 6.8 aws noble sru-20240930 sru-20241028 ubuntu-ltp-controllers

** Description changed:

  Issue found on 6.8.0-1020.20, looking back into our test history we can
  see it's failing with 6.8.0-1018.20 on the same instance in
  sru-20240930.
  
  And it has passed with 6.8.0-1015.16 in sru-20240805 and 6.8.0-1016.17
  in sru-s20240805
  
  A manual test shows this failure begins with 6.8.0-1017-aws
  
  Test result with 6.8.0-1016.17:
  <<<test_start>>>
  tag=cpuset_hotplug stime=1733121824
  cmdline="cpuset_hotplug_test.sh"
  contacts=""
  analysis=exit
  <<<test_output>>>
  incrementing stop
  cpuset_hotplug 1 TINFO: CPUs are numbered continuously starting at 0 (0-3)
  cpuset_hotplug 1 TINFO: Nodes are numbered continuously starting at 0 (0)
  cpuset_hotplug 1 TCONF: Could not mount cgroup filesystem with cpuset on /dev/cpuset
  <<<execution_status>>>
  initiation_status="ok"
  duration=0 termination_type=exited termination_id=32 corefile=no
  cutime=1 cstime=1
  <<<test_end>>>
  
  Test result with 6.8.0-1017:
  <<<test_start>>>
  tag=cpuset_hotplug stime=1733122053
  cmdline="cpuset_hotplug_test.sh"
  contacts=""
  analysis=exit
  <<<test_output>>>
  incrementing stop
  cpuset_hotplug 1 TINFO: CPUs are numbered continuously starting at 0 (0-3)
  cpuset_hotplug 1 TINFO: Nodes are numbered continuously starting at 0 (0)
  cpuset_hotplug 1 TPASS: Cpuset vs CPU hotplug test succeeded.
  /bin/echo: write error: Input/output error
  cpuset_hotplug 3 TINFO: /bin/echo: write error: Input/output error
  cpuset_hotplug 3 TFAIL:  CPU#1 failed.
  /bin/echo: write error: Input/output error
  cpuset_hotplug 5 TINFO: /bin/echo: write error: Invalid argument
  cpuset_hotplug 5 TFAIL: set general group parameter failed.
  /bin/echo: write error: Input/output error
  cpuset_hotplug 7 TINFO: /bin/echo: write error: Invalid argument
  cpuset_hotplug 7 TFAIL: set general group parameter failed.
  /bin/echo: write error: Input/output error
  cpuset_hotplug 9 TPASS: Cpuset vs CPU hotplug test succeeded.
  /bin/echo: write error: Input/output error
  cpuset_hotplug 11 TINFO: /bin/echo: write error: Input/output error
  cpuset_hotplug 11 TFAIL:  CPU#1 failed.
  /bin/echo: write error: Input/output error
  <<<execution_status>>>
  initiation_status="ok"
  duration=82 termination_type=exited termination_id=1 corefile=no
  cutime=136 cstime=1168
  <<<test_end>>>
+ 
+ This failure will cause the following cpuset_regression_test to fail with:
+ <<<test_start>>>
+ tag=cpuset_regression_test stime=1733122896
+ cmdline="cpuset_regression_test.sh"
+ contacts=""
+ analysis=exit
+ <<<test_output>>>
+ incrementing stop
+ cpuset_regression_test 1 TINFO: Running: cpuset_regression_test.sh 
+ cpuset_regression_test 1 TINFO: Tested kernel: Linux ip-172-31-3-132 6.8.0-1017-aws #18-Ubuntu SMP Wed Oct  2 20:17:03 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
+ cpuset_regression_test 1 TINFO: Using /tmp/ltp-SdsOVtsygU/LTP_cpuset_regression_test.vBePRqZkZM as tmpdir (ext2/ext3 filesystem)
+ cpuset_regression_test 1 TINFO: timeout per run is 0h 5m 0s
+ tst_cgroup.c:776: TINFO: Mounted V1 cpuset CGroup on /tmp/ltp-SdsOVtsygU/cgroup_cpuset
+ cpuset_regression_test 1 TINFO: test starts with cgroup version 1
+ /opt/ltp/testcases/bin/cpuset_regression_test.sh: 78: echo: echo: I/O error
+ cpuset_regression_test 1 TBROK: Unable to prepare existing cpuset cgroups
+ /opt/ltp/testcases/bin/cpuset_regression_test.sh: 111: echo: echo: I/O error
+ cpuset_regression_test 1 TINFO: AppArmor enabled, this may affect test results
+ cpuset_regression_test 1 TINFO: it can be disabled with TST_DISABLE_APPARMOR=1 (requires super/root)
+ cpuset_regression_test 1 TINFO: loaded AppArmor profiles: none
+ 
+ Summary:
+ passed   0
+ failed   0
+ broken   1
+ skipped  0
+ warnings 0
+ <<<execution_status>>>
+ 
+ If we don't run the cpuset_hotplug test, this test will be skipped:
+ <<<test_start>>>
+ tag=cpuset_regression_test stime=1733122991
+ cmdline="cpuset_regression_test.sh"
+ contacts=""
+ analysis=exit
+ <<<test_output>>>
+ incrementing stop
+ cpuset_regression_test 1 TINFO: Running: cpuset_regression_test.sh 
+ cpuset_regression_test 1 TINFO: Tested kernel: Linux ip-172-31-3-132 6.8.0-1017-aws #18-Ubuntu SMP Wed Oct  2 20:17:03 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
+ cpuset_regression_test 1 TINFO: Using /tmp/ltp-NQZT7RC4j4/LTP_cpuset_regression_test.YG7BLBZW9X as tmpdir (ext2/ext3 filesystem)
+ cpuset_regression_test 1 TINFO: timeout per run is 0h 5m 0s
+ cpuset_regression_test 1 TINFO: test starts with cgroup version 2
+ cpuset_regression_test 1 TCONF: cgroup v2 found, skipping test
+ 
+ Summary:
+ passed   0
+ failed   0
+ broken   0
+ skipped  1
+ warnings 0
+ <<<execution_status>>>
+ initiation_status="ok"
+ duration=0 termination_type=exited termination_id=32 corefile=no
+ cutime=2 cstime=2
+ <<<test_end>>>

** Also affects: linux-aws (Ubuntu)
   Importance: Undecided
       Status: New

** Also affects: linux-aws (Ubuntu Noble)
   Importance: Undecided
       Status: New

** Changed in: linux-aws (Ubuntu Noble)
     Assignee: (unassigned) => Philip Cox (philcox)

-- 
You received this bug notification because you are a member of Canonical
Platform QA Team, which is subscribed to ubuntu-kernel-tests.
https://bugs.launchpad.net/bugs/2090819

Title:
  cpuset_hotplug failed on N-aws  since 6.8.0-1017-aws on AWS x1e.xlarge

Status in ubuntu-kernel-tests:
  New
Status in linux-aws package in Ubuntu:
  New
Status in linux-aws source package in Noble:
  New

Bug description:
  Issue found on 6.8.0-1020.20, looking back into our test history we
  can see it's failing with 6.8.0-1018.20 on the same instance in
  sru-20240930.

  And it has passed with 6.8.0-1015.16 in sru-20240805 and 6.8.0-1016.17
  in sru-s20240805

  A manual test shows this failure begins with 6.8.0-1017-aws

  Test result with 6.8.0-1016.17:
  <<<test_start>>>
  tag=cpuset_hotplug stime=1733121824
  cmdline="cpuset_hotplug_test.sh"
  contacts=""
  analysis=exit
  <<<test_output>>>
  incrementing stop
  cpuset_hotplug 1 TINFO: CPUs are numbered continuously starting at 0 (0-3)
  cpuset_hotplug 1 TINFO: Nodes are numbered continuously starting at 0 (0)
  cpuset_hotplug 1 TCONF: Could not mount cgroup filesystem with cpuset on /dev/cpuset
  <<<execution_status>>>
  initiation_status="ok"
  duration=0 termination_type=exited termination_id=32 corefile=no
  cutime=1 cstime=1
  <<<test_end>>>

  Test result with 6.8.0-1017:
  <<<test_start>>>
  tag=cpuset_hotplug stime=1733122053
  cmdline="cpuset_hotplug_test.sh"
  contacts=""
  analysis=exit
  <<<test_output>>>
  incrementing stop
  cpuset_hotplug 1 TINFO: CPUs are numbered continuously starting at 0 (0-3)
  cpuset_hotplug 1 TINFO: Nodes are numbered continuously starting at 0 (0)
  cpuset_hotplug 1 TPASS: Cpuset vs CPU hotplug test succeeded.
  /bin/echo: write error: Input/output error
  cpuset_hotplug 3 TINFO: /bin/echo: write error: Input/output error
  cpuset_hotplug 3 TFAIL:  CPU#1 failed.
  /bin/echo: write error: Input/output error
  cpuset_hotplug 5 TINFO: /bin/echo: write error: Invalid argument
  cpuset_hotplug 5 TFAIL: set general group parameter failed.
  /bin/echo: write error: Input/output error
  cpuset_hotplug 7 TINFO: /bin/echo: write error: Invalid argument
  cpuset_hotplug 7 TFAIL: set general group parameter failed.
  /bin/echo: write error: Input/output error
  cpuset_hotplug 9 TPASS: Cpuset vs CPU hotplug test succeeded.
  /bin/echo: write error: Input/output error
  cpuset_hotplug 11 TINFO: /bin/echo: write error: Input/output error
  cpuset_hotplug 11 TFAIL:  CPU#1 failed.
  /bin/echo: write error: Input/output error
  <<<execution_status>>>
  initiation_status="ok"
  duration=82 termination_type=exited termination_id=1 corefile=no
  cutime=136 cstime=1168
  <<<test_end>>>

  This failure will cause the following cpuset_regression_test to fail with:
  <<<test_start>>>
  tag=cpuset_regression_test stime=1733122896
  cmdline="cpuset_regression_test.sh"
  contacts=""
  analysis=exit
  <<<test_output>>>
  incrementing stop
  cpuset_regression_test 1 TINFO: Running: cpuset_regression_test.sh 
  cpuset_regression_test 1 TINFO: Tested kernel: Linux ip-172-31-3-132 6.8.0-1017-aws #18-Ubuntu SMP Wed Oct  2 20:17:03 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
  cpuset_regression_test 1 TINFO: Using /tmp/ltp-SdsOVtsygU/LTP_cpuset_regression_test.vBePRqZkZM as tmpdir (ext2/ext3 filesystem)
  cpuset_regression_test 1 TINFO: timeout per run is 0h 5m 0s
  tst_cgroup.c:776: TINFO: Mounted V1 cpuset CGroup on /tmp/ltp-SdsOVtsygU/cgroup_cpuset
  cpuset_regression_test 1 TINFO: test starts with cgroup version 1
  /opt/ltp/testcases/bin/cpuset_regression_test.sh: 78: echo: echo: I/O error
  cpuset_regression_test 1 TBROK: Unable to prepare existing cpuset cgroups
  /opt/ltp/testcases/bin/cpuset_regression_test.sh: 111: echo: echo: I/O error
  cpuset_regression_test 1 TINFO: AppArmor enabled, this may affect test results
  cpuset_regression_test 1 TINFO: it can be disabled with TST_DISABLE_APPARMOR=1 (requires super/root)
  cpuset_regression_test 1 TINFO: loaded AppArmor profiles: none

  Summary:
  passed   0
  failed   0
  broken   1
  skipped  0
  warnings 0
  <<<execution_status>>>

  If we don't run the cpuset_hotplug test, this test will be skipped:
  <<<test_start>>>
  tag=cpuset_regression_test stime=1733122991
  cmdline="cpuset_regression_test.sh"
  contacts=""
  analysis=exit
  <<<test_output>>>
  incrementing stop
  cpuset_regression_test 1 TINFO: Running: cpuset_regression_test.sh 
  cpuset_regression_test 1 TINFO: Tested kernel: Linux ip-172-31-3-132 6.8.0-1017-aws #18-Ubuntu SMP Wed Oct  2 20:17:03 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
  cpuset_regression_test 1 TINFO: Using /tmp/ltp-NQZT7RC4j4/LTP_cpuset_regression_test.YG7BLBZW9X as tmpdir (ext2/ext3 filesystem)
  cpuset_regression_test 1 TINFO: timeout per run is 0h 5m 0s
  cpuset_regression_test 1 TINFO: test starts with cgroup version 2
  cpuset_regression_test 1 TCONF: cgroup v2 found, skipping test

  Summary:
  passed   0
  failed   0
  broken   0
  skipped  1
  warnings 0
  <<<execution_status>>>
  initiation_status="ok"
  duration=0 termination_type=exited termination_id=32 corefile=no
  cutime=2 cstime=2
  <<<test_end>>>

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-kernel-tests/+bug/2090819/+subscriptions