← Back to team overview

lttng team mailing list archive

[Bug 2017006] Re: fix build issue for v5.19 kernels (jammy-hwe, kinetic, lunar-riscv)

 

** Also affects: linux (Ubuntu)
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Ubuntu
LTTng, which is subscribed to lttng-modules in Ubuntu.
Matching subscriptions: lttng-modules-bugs
https://bugs.launchpad.net/bugs/2017006

Title:
  fix build issue for v5.19 kernels (jammy-hwe, kinetic, lunar-riscv)

Status in linux package in Ubuntu:
  New
Status in lttng-modules package in Ubuntu:
  In Progress
Status in linux source package in Jammy:
  New
Status in lttng-modules source package in Jammy:
  Fix Committed
Status in linux source package in Kinetic:
  New
Status in lttng-modules source package in Kinetic:
  Fix Committed
Status in linux source package in Lunar:
  New
Status in lttng-modules source package in Lunar:
  Fix Committed

Bug description:
  SRU Justification

  [ Impact ]

  Upstream stable added a change in the format of jbd2 in 5.15.87 that
  was cherry-picked to kinetic:linux.

  This is incompatible with the current changes in the lttng-module
  kinetic.

  We previously encountered this issue for focal https://bugs.launchpad.net/ubuntu-kernel-tests/+bug/2004644
  and we proactively released a new version for kinetic.
  The problem is the fix does not really work as expected.

  As shown in the logs below,
  The build error is triggered because some function declaration does not match the kernel's.
  In `include/instrumentation/events/jbd2.h` in lttng-modules, there is a conditional declaration based on the kernel version. 5.19 is missing there, therefore kinetic will fall back to the old declaration. The reason 5.19 is not there is probably because this version has reached end of life last year.

  Build logs:
  35:35 DEBUG| [stdout] In file included from /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/../../include/lttng/define_trace.h:87,
        860 	  	02:35:35 DEBUG| [stdout]                  from /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/../../include/instrumentation/events/jbd2.h:293,
        861 	  	02:35:35 DEBUG| [stdout]                  from /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-jbd2.c:29:
        862 	  	02:35:35 DEBUG| [stdout] /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/../../include/lttng/tracepoint-event-impl.h:133:6: error: conflicting types for ‘trace_jbd2_run_stats’; have ‘void(dev_t,  long unsigned int,  struct transaction_run_stats_s *)’ {aka ‘void(unsigned int,  long unsigned int,  struct transaction_run_stats_s *)’}
        863 	  	02:35:35 DEBUG| [stdout]   133 | void trace_##_name(_proto);
        864 	  	02:35:35 DEBUG| [stdout]       |      ^~~~~~
        865 	  	02:35:35 DEBUG| [stdout] /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/../../include/lttng/tracepoint-event-impl.h:45:9: note: in expansion of macro ‘LTTNG_TRACEPOINT_EVENT_INSTANCE_MAP’
        866 	  	02:35:35 DEBUG| [stdout]    45 |         LTTNG_TRACEPOINT_EVENT_INSTANCE_MAP(map, name, map, PARAMS(proto), PARAMS(args))
        867 	  	02:35:35 DEBUG| [stdout]       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        868 	  	02:35:35 DEBUG| [stdout] /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/../../include/lttng/tracepoint-event-impl.h:87:9: note: in expansion of macro ‘LTTNG_TRACEPOINT_EVENT_MAP’
        869 	  	02:35:35 DEBUG| [stdout]    87 |         LTTNG_TRACEPOINT_EVENT_MAP(name, name,                          \
        870 	  	02:35:35 DEBUG| [stdout]       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~
        871 	  	02:35:35 DEBUG| [stdout] /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/../../include/instrumentation/events/jbd2.h:180:1: note: in expansion of macro ‘LTTNG_TRACEPOINT_EVENT’
        872 	  	02:35:35 DEBUG| [stdout]   180 | LTTNG_TRACEPOINT_EVENT(jbd2_run_stats,
        873 	  	02:35:35 DEBUG| [stdout]       | ^~~~~~~~~~~~~~~~~~~~~~
        874 	  	02:35:35 DEBUG| [stdout] In file included from ./include/trace/events/jbd2.h:9,
        875 	  	02:35:35 DEBUG| [stdout]                  from /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-jbd2.c:18:
        876 	  	02:35:35 DEBUG| [stdout] ./include/linux/tracepoint.h:245:28: note: previous definition of ‘trace_jbd2_run_stats’ with type ‘void(dev_t,  tid_t,  struct transaction_run_stats_s *)’ {aka ‘void(unsigned int,  unsigned int,  struct transaction_run_stats_s *)’}
        877 	  	02:35:35 DEBUG| [stdout]   245 |         static inline void trace_##name(proto)                          \
        878 	  	02:35:35 DEBUG| [stdout]       |                            ^~~~~~
        879 	  	02:35:35 DEBUG| [stdout] ./include/linux/tracepoint.h:419:9: note: in expansion of macro ‘__DECLARE_TRACE’
        880 	  	02:35:35 DEBUG| [stdout]   419 |         __DECLARE_TRACE(name, PARAMS(proto), PARAMS(args),              \
        881 	  	02:35:35 DEBUG| [stdout]       |         ^~~~~~~~~~~~~~~
        882 	  	02:35:35 DEBUG| [stdout] ./include/linux/tracepoint.h:553:9: note: in expansion of macro ‘DECLARE_TRACE’
        883 	  	02:35:35 DEBUG| [stdout]   553 |         DECLARE_TRACE(name, PARAMS(proto), PARAMS(args))
        884 	  	02:35:35 DEBUG| [stdout]       |         ^~~~~~~~~~~~~
        885 	  	02:35:35 DEBUG| [stdout] ./include/trace/events/jbd2.h:248:1: note: in expansion of macro ‘TRACE_EVENT’
        886 	  	02:35:35 DEBUG| [stdout]   248 | TRACE_EVENT(jbd2_run_stats,
        887 	  	02:35:35 DEBUG| [stdout]       | ^~~~~~~~~~~
        888 	  	02:35:35 DEBUG| [stdout] /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/../../include/lttng/tracepoint-event-impl.h:133:6: error: conflicting types for ‘trace_jbd2_checkpoint_stats’; have ‘void(dev_t,  long unsigned int,  struct transaction_chp_stats_s *)’ {aka ‘void(unsigned int,  long unsigned int,  struct transaction_chp_stats_s *)’}
        889 	  	02:35:35 DEBUG| [stdout]   133 | void trace_##_name(_proto);
        890 	  	02:35:35 DEBUG| [stdout]       |      ^~~~~~
        891 	  	02:35:35 DEBUG| [stdout] /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/../../include/lttng/tracepoint-event-impl.h:45:9: note: in expansion of macro ‘LTTNG_TRACEPOINT_EVENT_INSTANCE_MAP’
        892 	  	02:35:35 DEBUG| [stdout]    45 |         LTTNG_TRACEPOINT_EVENT_INSTANCE_MAP(map, name, map, PARAMS(proto), PARAMS(args))
        893 	  	02:35:35 DEBUG| [stdout]       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        894 	  	02:35:35 DEBUG| [stdout] /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/../../include/lttng/tracepoint-event-impl.h:87:9: note: in expansion of macro ‘LTTNG_TRACEPOINT_EVENT_MAP’
        895 	  	02:35:35 DEBUG| [stdout]    87 |         LTTNG_TRACEPOINT_EVENT_MAP(name, name,                          \
        896 	  	02:35:35 DEBUG| [stdout]       |         ^~~~~~~~~~~~~~~~~~~~~~~~~~
        897 	  	02:35:35 DEBUG| [stdout] /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/../../include/instrumentation/events/jbd2.h:200:1: note: in expansion of macro ‘LTTNG_TRACEPOINT_EVENT’
        898 	  	02:35:35 DEBUG| [stdout]   200 | LTTNG_TRACEPOINT_EVENT(jbd2_checkpoint_stats,
        899 	  	02:35:35 DEBUG| [stdout]       | ^~~~~~~~~~~~~~~~~~~~~~
        900 	  	02:35:35 DEBUG| [stdout] ./include/linux/tracepoint.h:245:28: note: previous definition of ‘trace_jbd2_checkpoint_stats’ with type ‘void(dev_t,  tid_t,  struct transaction_chp_stats_s *)’ {aka ‘void(unsigned int,  unsigned int,  struct transaction_chp_stats_s *)’}
        901 	  	02:35:35 DEBUG| [stdout]   245 |         static inline void trace_##name(proto)                          \
        902 	  	02:35:35 DEBUG| [stdout]       |                            ^~~~~~
        903 	  	02:35:35 DEBUG| [stdout] ./include/linux/tracepoint.h:419:9: note: in expansion of macro ‘__DECLARE_TRACE’
        904 	  	02:35:35 DEBUG| [stdout]   419 |         __DECLARE_TRACE(name, PARAMS(proto), PARAMS(args),              \
        905 	  	02:35:35 DEBUG| [stdout]       |         ^~~~~~~~~~~~~~~
        906 	  	02:35:35 DEBUG| [stdout] ./include/linux/tracepoint.h:553:9: note: in expansion of macro ‘DECLARE_TRACE’
        907 	  	02:35:35 DEBUG| [stdout]   553 |         DECLARE_TRACE(name, PARAMS(proto), PARAMS(args))
        908 	  	02:35:35 DEBUG| [stdout]       |         ^~~~~~~~~~~~~
        909 	  	02:35:35 DEBUG| [stdout] ./include/trace/events/jbd2.h:296:1: note: in expansion of macro ‘TRACE_EVENT’
        910 	  	02:35:35 DEBUG| [stdout]   296 | TRACE_EVENT(jbd2_checkpoint_stats,
        911 	  	02:35:35 DEBUG| [stdout]       | ^~~~~~~~~~~
        912 	  	02:35:35 DEBUG| [stdout] make[3]: *** [scripts/Makefile.build:257: /var/lib/dkms/lttng-modules/2.13.8/build/src/probes/lttng-probe-jbd2.o] Error 1
        913 	  	02:35:35 DEBUG| [stdout] make[2]: *** [scripts/Makefile.build:474: /var/lib/dkms/lttng-modules/2.13.8/build/src/probes] Error 2
        914 	  	02:35:35 DEBUG| [stdout] make[1]: *** [scripts/Makefile.build:474: /var/lib/dkms/lttng-modules/2.13.8/build/src] Error 2
        915 	  	02:35:35 DEBUG| [stdout] make: *** [Makefile:1850: /var/lib/dkms/lttng-modules/2.13.8/build] Error 2
        916 	  	02:35:35 DEBUG| [stdout] make: Leaving directory '/usr/src/linux-headers-5.19.0-42-generic'
        917 	  	02:35:35 ERROR| Exception escaping from test:
        918 	  	Traceback (most recent call last):
        919 	  	  File "/home/ubuntu/autotest/client/shared/test.py", line 378, in _exec
        920 	  	    _cherry_pick_call(self.initialize, *args, **dargs)
        921 	  	  File "/home/ubuntu/autotest/client/shared/test.py", line 738, in _cherry_pick_call
        922 	  	    return func(*p_args, **p_dargs)
        923 	  	  File "/home/ubuntu/autotest/client/tests/ubuntu_lttng_smoke_test/ubuntu_lttng_smoke_test.py", line 58, in initialize
        924 	  	    raise error.TestError('DKMS failed to install')

  [ Fix ]

  Add [5.19.0 - 5.19.19] interval
  This requires a new release for kinetic and jammy because jammy 5.19 backports are affected as well. Also lunar, because some riscv kernels are still on 5.19 as well.

  [ Where problems would occur ]

  We may notice new failures in ubuntu_lttng_smoke_test once this will
  compile and run. But it was tested locally and the results were good,
  so the probability is very very low.

  [ Test Plan ]

  1. Install kinetic latest kernel 5.19.0-42 or any other 5.19 kernel
  2. sudo apt install ltnng-modules-dkms
  3. Without the fix it will fail the build with the error from above.
  with the fix, the module should be installed.
  4. Additionally lttng-modules regression test should be triggered to confirm everything works file.

  Steps to run rt:
  # Install packages for autotest
  sudo apt-get install git python2 gdb python-yaml -y || sudo apt-get install git python gdb python-yaml -y
  # Enable deb-src
  sudo sed -i 's/^# deb-src/deb-src/' /etc/apt/sources.list
  sudo apt-get update
  git clone --depth=1 https://git.launchpad.net/~canonical-kernel-team/+git/autotest
  git clone --depth=1 https://git.launchpad.net/~canonical-kernel-team/+git/autotest-client-tests
  rm -fr autotest/client/tests
  ln -sf ~/autotest-client-tests autotest/client/tests
  AUTOTEST_PATH=/home/ubuntu/autotest sudo -E autotest/client/autotest-local --verbose autotest/client/tests/ubuntu_lttng_smoke_test/control

  
  [ Local test results ]
  I tested on a VM with the latest kinetic version : 5.19.0-42-generic:

  ubuntu@kinetic:~/lttng-modules$ dpkg -l | grep lttng-modules
  ii lttng-modules-dkms 2.13.8-1~ubuntu22.10.1 all Linux Trace Toolkit (LTTng) kernel modules (DKMS)

  RT results:
  08:54:11 INFO | Summary: 7 passed, 0 failed
  08:54:11 DEBUG| Running 'apt-get remove --yes lttng-modules-dkms'
  08:54:20 INFO |                 GOOD    ubuntu_lttng_smoke_test.lttng-smoke-test        ubuntu_lttng_smoke_test.lttng-smoke-test        timestamp=1683536060    localtime=May 08 08:54:20       completed successfully
  08:54:20 INFO |         END GOOD        ubuntu_lttng_smoke_test.lttng-smoke-test        ubuntu_lttng_smoke_test.lttng-smoke-test        timestamp=1683536060    localtime=May 08 08:54:20
  08:54:20 DEBUG| Persistent state client._record_indent now set to 1
  08:54:20 DEBUG| Persistent state client.unexpected_reboot deleted
  08:54:20 INFO | END GOOD        ----    ----    timestamp=1683536060    localtime=May 08 08:54:20

  
  For jammy, I applied the commit that broke it, built the kernel on cbd and booted in a vm:

  ubuntu@jammy-generic:~/lttng-modules$ dpkg -l | grep lttng-modules
  ii lttng-modules-dkms 2.13.8-1~ubuntu22.04.1 all Linux Trace Toolkit (LTTng) kernel modules (DKMS)

  RT results:
  09:00:24 INFO | Summary: 7 passed, 0 failed
  09:00:24 DEBUG| Running 'apt-get remove --yes lttng-modules-dkms'
  09:00:31 INFO |                 GOOD    ubuntu_lttng_smoke_test.lttng-smoke-test        ubuntu_lttng_smoke_test.lttng-smoke-test      timestamp=1683536431    localtime=May 08 09:00:31       completed successfully
  09:00:31 INFO |         END GOOD        ubuntu_lttng_smoke_test.lttng-smoke-test        ubuntu_lttng_smoke_test.lttng-smoke-test      timestamp=1683536431    localtime=May 08 09:00:31
  09:00:31 DEBUG| Persistent state client._record_indent now set to 1
  09:00:31 DEBUG| Persistent state client.unexpected_reboot deleted
  09:00:31 INFO | END GOOD        ----    ----    timestamp=1683536431    localtime=May 08 09:00:31

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2017006/+subscriptions



References