canonical-ubuntu-qa team mailing list archive

Thread
Date
[Bug 1836694] Re: memcg_test_3 from controllers in LTP timeout with Bionic/Focal/Hirsute (arm64)

To: canonical-ubuntu-qa@xxxxxxxxxxxxxxxxxxx
From: Mehmet Basaran <1836694@xxxxxxxxxxxxxxxxxx>
Date: Fri, 09 Aug 2024 13:46:17 -0000
Reply-to: Bug 1836694 <1836694@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
I have run into this (event with 75m) using jammy:linux-lowlatency-
hwe-6.8 6.8.0-40.40.1~22.04.1 metal:appleton-kernel

Kernel being lowlatency might cause this error if this is one of those
throughtput tests (we might need to increase timeout even more).

14:09:18 INFO |     START    ubuntu_ltp_controllers.memcg_test_3    ubuntu_ltp_controllers.memcg_test_3    timestamp=1723126158    timeout=4500    localtime=Aug 08 14:09:18    
14:09:18 DEBUG| Persistent state client._record_indent now set to 2
14:09:18 DEBUG| Persistent state client.unexpected_reboot now set to ('ubuntu_ltp_controllers.memcg_test_3', 'ubuntu_ltp_controllers.memcg_test_3')
14:09:18 DEBUG| Waiting for pid 116800 for 4500 seconds
14:09:18 WARNI| System python is too old, crash handling disabled
14:09:18 INFO | Set timeout multiplier LTP_TIMEOUT_MUL=150 for memcg_test_3 (lp:1836694)
15:24:18 ERROR| Exception escaping from test:
Traceback (most recent call last):
File "/home/ubuntu/autotest/client/shared/test.py", line 411, in _exec
_call_test_function(self.execute, *p_args, **p_dargs)
File "/home/ubuntu/autotest/client/shared/test.py", line 823, in _call_test_function
return func(*args, **dargs)
File "/home/ubuntu/autotest/client/shared/test.py", line 291, in execute
postprocess_profiled_run, args, dargs)
File "/home/ubuntu/autotest/client/shared/test.py", line 212, in _call_run_once
self.run_once(*args, **dargs)
File "/home/ubuntu/autotest/client/tests/ubuntu_ltp_controllers/ubuntu_ltp_controllers.py", line 137, in run_once
print(utils.system_output(cmd, verbose=False))
File "/home/ubuntu/autotest/client/shared/utils.py", line 1271, in system_output
verbose=verbose, args=args).stdout
File "/home/ubuntu/autotest/client/shared/utils.py", line 918, in run
"Command returned non-zero exit status")
CmdError: Command  failed, rc=1, Command returned non-zero exit status
* Command: 
/opt/ltp/runltp -f /tmp/target -q -C /dev/null -l /dev/null -T /dev/null
Exit status: 1
Duration: 4500.08709598

stdout:
Checking for required user/group ids

'root' user id and group found.
'nobody' user id and group found.
'bin' user id and group found.
'daemon' user id and group found.
Users group found.
Sys group found.
Required users/groups exist.
no big block device was specified on commandline.
Tests which require a big block device are disabled.
You can specify it with option -z
INFO: Test start time: Thu Aug  8 14:09:18 UTC 2024
COMMAND:    /opt/ltp/bin/ltp-pan -q  -e -S   -a 116804     -n 116804  -f /tmp/ltp-XFn4xt7RJQ/alltests -l /dev/null  -C /dev/null -T /dev/null
LOG File: /dev/null
FAILED COMMAND File: /dev/null
TCONF COMMAND File: /dev/null
Running tests.......
tst_test.c:1733: TINFO: LTP version: 20240524
tst_test.c:1617: TINFO: Timeout per run is 1h 15m 00s
Test timeouted, sending SIGKILL!
tst_test.c:1665: TINFO: Killed the leftover descendant processes
tst_test.c:1671: TINFO: If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1
tst_test.c:1673: TBROK: Test killed! (timeout?)

Summary:
passed   0
failed   0
broken   1
skipped  0
warnings 0
INFO: ltp-pan reported some tests FAIL
LTP Version: 20240524
INFO: Test end time: Thu Aug  8 15:24:18 UTC 2024
15:24:18 INFO | Restore timeout multiplier LTP_TIMEOUT_MUL back to default
15:24:18 ERROR| child process failed
15:24:18 DEBUG| Traceback (most recent call last):
15:24:18 DEBUG|   File "/home/ubuntu/autotest/client/parallel.py", line 25, in fork_start
15:24:18 DEBUG|     l()
15:24:18 DEBUG|   File "/home/ubuntu/autotest/client/job.py", line 505, in 
15:24:18 DEBUG|     l = lambda: test.runtest(self, url, tag, args, dargs)
15:24:18 DEBUG|   File "/home/ubuntu/autotest/client/test.py", line 125, in runtest
15:24:18 DEBUG|     job.sysinfo.log_after_each_iteration)
15:24:18 DEBUG|   File "/home/ubuntu/autotest/client/shared/test.py", line 913, in runtest
15:24:18 DEBUG|     mytest._exec(args, dargs)
15:24:18 DEBUG|   File "/home/ubuntu/autotest/client/shared/test.py", line 411, in _exec
15:24:18 DEBUG|     _call_test_function(self.execute, *p_args, **p_dargs)
15:24:18 DEBUG|   File "/home/ubuntu/autotest/client/shared/test.py", line 823, in _call_test_function
15:24:18 DEBUG|     return func(*args, **dargs)
15:24:18 DEBUG|   File "/home/ubuntu/autotest/client/shared/test.py", line 291, in execute
15:24:18 DEBUG|     postprocess_profiled_run, args, dargs)
15:24:18 DEBUG|   File "/home/ubuntu/autotest/client/shared/test.py", line 212, in _call_run_once
15:24:18 DEBUG|     self.run_once(*args, **dargs)
15:24:18 DEBUG|   File "/home/ubuntu/autotest/client/tests/ubuntu_ltp_controllers/ubuntu_ltp_controllers.py", line 137, in run_once
15:24:18 DEBUG|     print(utils.system_output(cmd, verbose=False))
15:24:18 DEBUG|   File "/home/ubuntu/autotest/client/shared/utils.py", line 1271, in system_output
15:24:18 DEBUG|     verbose=verbose, args=args).stdout
15:24:18 DEBUG|   File "/home/ubuntu/autotest/client/shared/utils.py", line 918, in run
15:24:18 DEBUG|     "Command returned non-zero exit status")
15:24:18 DEBUG| CmdError: Command  failed, rc=1, Command returned non-zero exit status
15:24:18 DEBUG| * Command: 
15:24:18 DEBUG|     /opt/ltp/runltp -f /tmp/target -q -C /dev/null -l /dev/null -T /dev/null
15:24:18 DEBUG| Exit status: 1
15:24:18 DEBUG| Duration: 4500.08709598
15:24:18 DEBUG| 
15:24:18 DEBUG| stdout:
15:24:18 DEBUG| Checking for required user/group ids
15:24:18 DEBUG| 
15:24:18 DEBUG| 'root' user id and group found.
15:24:18 DEBUG| 'nobody' user id and group found.
15:24:18 DEBUG| 'bin' user id and group found.
15:24:18 DEBUG| 'daemon' user id and group found.
15:24:18 DEBUG| Users group found.
15:24:18 DEBUG| Sys group found.
15:24:18 DEBUG| Required users/groups exist.
15:24:18 DEBUG| no big block device was specified on commandline.
15:24:18 DEBUG| Tests which require a big block device are disabled.
15:24:18 DEBUG| You can specify it with option -z
15:24:18 DEBUG| INFO: Test start time: Thu Aug  8 14:09:18 UTC 2024
15:24:18 DEBUG| COMMAND:    /opt/ltp/bin/ltp-pan -q  -e -S   -a 116804     -n 116804  -f /tmp/ltp-XFn4xt7RJQ/alltests -l /dev/null  -C /dev/null -T /dev/null
15:24:18 DEBUG| LOG File: /dev/null
15:24:18 DEBUG| FAILED COMMAND File: /dev/null
15:24:18 DEBUG| TCONF COMMAND File: /dev/null
15:24:18 DEBUG| Running tests.......
15:24:18 DEBUG| tst_test.c:1733: TINFO: LTP version: 20240524
15:24:18 DEBUG| tst_test.c:1617: TINFO: Timeout per run is 1h 15m 00s
15:24:18 DEBUG| Test timeouted, sending SIGKILL!
15:24:18 DEBUG| tst_test.c:1665: TINFO: Killed the leftover descendant processes
15:24:18 DEBUG| tst_test.c:1671: TINFO: If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1
15:24:18 DEBUG| tst_test.c:1673: TBROK: Test killed! (timeout?)
15:24:18 DEBUG| 
15:24:18 DEBUG| Summary:
15:24:18 DEBUG| passed   0
15:24:18 DEBUG| failed   0
15:24:18 DEBUG| broken   1
15:24:18 DEBUG| skipped  0
15:24:18 DEBUG| warnings 0
15:24:18 DEBUG| INFO: ltp-pan reported some tests FAIL
15:24:18 DEBUG| LTP Version: 20240524
15:24:18 DEBUG| INFO: Test end time: Thu Aug  8 15:24:18 UTC 2024
15:24:19 INFO |         ERROR    ubuntu_ltp_controllers.memcg_test_3    ubuntu_ltp_controllers.memcg_test_3    timestamp=1723130659    localtime=Aug 08 15:24:19    Command  failed, rc=1, Command returned non-zero exit status
* Command: 
/opt/ltp/runltp -f /tmp/target -q -C /dev/null -l /dev/null -T /dev/null
Exit status: 1
Duration: 4500.08709598

stdout:
Checking for required user/group ids

'root' user id and group found.
'nobody' user id and group found.
'bin' user id and group found.
'daemon' user id and group found.
Users group found.
Sys group found.
Required users/groups exist.
no big block device was specified on commandline.
Tests which require a big block device are disabled.
You can specify it with option -z
INFO: Test start time: Thu Aug  8 14:09:18 UTC 2024
COMMAND:    /opt/ltp/bin/ltp-pan -q  -e -S   -a 116804     -n 116804  -f /tmp/ltp-XFn4xt7RJQ/alltests -l /dev/null  -C /dev/null -T /dev/null
LOG File: /dev/null
FAILED COMMAND File: /dev/null
TCONF COMMAND File: /dev/null
Running tests.......
tst_test.c:1733: TINFO: LTP version: 20240524
tst_test.c:1617: TINFO: Timeout per run is 1h 15m 00s
Test timeouted, sending SIGKILL!
tst_test.c:1665: TINFO: Killed the leftover descendant processes
tst_test.c:1671: TINFO: If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1
tst_test.c:1673: TBROK: Test killed! (timeout?)

Summary:
passed   0
failed   0
broken   1
skipped  0
warnings 0
INFO: ltp-pan reported some tests FAIL
LTP Version: 20240524
INFO: Test end time: Thu Aug  8 15:24:18 UTC 2024
15:24:19 INFO |     END ERROR    ubuntu_ltp_controllers.memcg_test_3    ubuntu_ltp_controllers.memcg_test_3    timestamp=1723130659    localtime=Aug 08 15:24:19

-- 
You received this bug notification because you are a member of Canonical
Platform QA Team, which is subscribed to ubuntu-kernel-tests.
https://bugs.launchpad.net/bugs/1836694

Title:
  memcg_test_3 from controllers in LTP timeout with Bionic/Focal/Hirsute
  (arm64)

Status in ubuntu-kernel-tests:
  In Progress
Status in linux package in Ubuntu:
  Invalid
Status in linux source package in Bionic:
  Invalid
Status in linux source package in Focal:
  Invalid
Status in linux source package in Hirsute:
  Won't Fix
Status in linux source package in Impish:
  Won't Fix

Bug description:
  This failure could be found in the LTP test suite on a Moonshot ARM64
  node with B-4.15, but sometimes it will pass if you try to run it
  manually. (Sometimes not.)

  <<<test_start>>>
  tag=memcg_test_3 stime=1563249678
  cmdline="memcg_test_3"
  contacts=""
  analysis=exit
  <<<test_output>>>
  incrementing stop
  tst_test.c:1100: INFO: Timeout per run is 0h 05m 00s
  Test timeouted, sending SIGKILL!
  tst_test.c:1140: INFO: If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1
  tst_test.c:1141: BROK: Test killed! (timeout?)

  Summary:
  passed   0
  failed   0
  skipped  0
  warnings 0
  tst_tmpdir.c:330: WARN: tst_rmdir: rmobj(/tmp/ltp-nJ05WiJDR1/06EzUc) failed: unlink(/tmp/ltp-nJ05WiJDR1/06EzUc/memcg/cgroup.clone_children) failed; errno=1: EPERM
  <<<execution_status>>>

  
  When it fails, the attempt to remove files will fail, and most of the cgroup_fj_* test will fail:
    * cgroup_fj_function_memory
    * cgroup_fj_stress_memory_10_3_each
    * cgroup_fj_stress_memory_10_3_none
    * cgroup_fj_stress_memory_10_3_one
    * cgroup_fj_stress_memory_1_200_each
    * cgroup_fj_stress_memory_1_200_none
    * cgroup_fj_stress_memory_1_200_one
    * cgroup_fj_stress_memory_200_1_each
    * cgroup_fj_stress_memory_200_1_none
    * cgroup_fj_stress_memory_200_1_one
    * cgroup_fj_stress_memory_2_2_each
    * cgroup_fj_stress_memory_2_2_none
    * cgroup_fj_stress_memory_2_2_one
    * cgroup_fj_stress_memory_2_9_each
    * cgroup_fj_stress_memory_2_9_none
    * cgroup_fj_stress_memory_2_9_one
    * cgroup_fj_stress_memory_3_3_each
    * cgroup_fj_stress_memory_3_3_none
    * cgroup_fj_stress_memory_3_3_one
    * cgroup_fj_stress_memory_4_4_each
    * cgroup_fj_stress_memory_4_4_none
    * cgroup_fj_stress_memory_4_4_one

  Steps to run this:
    git clone --depth=1 https://github.com/linux-test-project/ltp.git
    cd ltp; make autotools; ./configure; make; sudo make install
    echo "memcg_test_3    memcg_test_3" > /tmp/jobs
    sudo /opt/ltp/runltp -f /tmp/jobs

  ProblemType: Bug
  DistroRelease: Ubuntu 18.04
  Package: linux-image-4.15.0-54-generic 4.15.0-54.58
  ProcVersionSignature: User Name 4.15.0-54.58-generic 4.15.18
  Uname: Linux 4.15.0-54-generic aarch64
  AlsaDevices:
   total 0
   crw-rw---- 1 root audio 116,  1 Jul 16 03:46 seq
   crw-rw---- 1 root audio 116, 33 Jul 16 03:46 timer
  AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay'
  ApportVersion: 2.20.9-0ubuntu7.6
  Architecture: arm64
  ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord'
  AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
  Date: Tue Jul 16 04:16:14 2019
  IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig'
  Lsusb: Error: command ['lsusb'] failed with exit code 1:
  PciMultimedia:
   
  ProcFB:
   
  ProcKernelCmdLine: console=ttyS0,9600n8r ro
  RelatedPackageVersions:
   linux-restricted-modules-4.15.0-54-generic N/A
   linux-backports-modules-4.15.0-54-generic  N/A
   linux-firmware                             1.173.8
  RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill'
  SourcePackage: linux
  UpgradeStatus: No upgrade log present (probably fresh install)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu-kernel-tests/+bug/1836694/+subscriptions