touch-packages team mailing list archive
-
touch-packages team
-
Mailing list archive
-
Message #23175
[Bug 1350598] Re: AppArmor policy compile improvements
** Description changed:
apparmor_parser can take a long time to compile policy especially when there is a lot of policy, so we want to utilize compiled cache profile as much as possible. Cache files will have to be regenerated in the following cases:
* the kernel .features file is updated (eg, new features are added to
apparmor in the new kernel)
* apparmor itself is updated
* on devices with click packages, apparmor-easyprof-ubuntu is updated
As of 2014-10-02, what can be expected is:
- Systems with system-image updates (eg, Ubuntu Touch):
- First boot will use the precompiled cache files in the rootfs or custom
tarball and be fast
- Reboots will use the cache files on the device and be fast
- First boots after upgrades will use the cache files on the device if the
above conditions are not met and be fast
- Production devices will not meet any of those conditions except under
exceptional and rare circumstances (eg, major OS upgrades like 14.10 to
15.04) and be fast
- First boots after upgrades that meet one of the conditions will need to
regenerate the cache. This can happen on development releases where the
kernel features file, apparmor or apparmor-easyprof-ubuntu is still
under development and getting updates
- Systems with apt updates (eg, current Ubuntu Desktop and Server):
- First boot will compile cache files
- Reboots will use the cache files on the machine and be fast
- First boots after upgrades will use the cache files on the machine if the
above conditions are not met and be fast
- Stable releases of Ubuntu will not meet any of those conditions except
under exceptional and rare circumstances (eg, major OS upgrades like
14.10 to 15.04) and be fast
- First boots after upgrades that meet one of the above conditions will
need to regenerate the cache. This can happen on development releases
where the kernel features file, apparmor or apparmor-easyprof-ubuntu
is still under development and getting updates
In addition to the above, updates to only apparmor-easyprof-ubuntu will
regenerate the cache files for only the policy that is affected (eg, if
there is a change to the location policy group in policy version 1.2,
only apps using this policy version and this policy group will need to
be recompiled).
Planned improvements (in order of most likely to be done first):
1. Finetuning the checks to invalidate the cache (eg, .md5sums could only
be for /etc/apparmor.d/abstractions, ...)
- 2. Improve cache handling for app store apps (eg, having the app store
+ 2. Investigate ways to utilize the custom tarball and rootfs precompiled
+ cache files on upgrades when apparmor and apparmor-easyprof-ubuntu are
+ updated
+ 3. Improve cache handling for app store apps (eg, having the app store
server precompile them so that the device can download them when it needs
to rather than having to regenerate them itself)
- 3. For systems with apt upgrades, compile the policy either during install or
+ 4. For systems with apt upgrades, compile the policy either during install or
on kernel upgrade rather than on boot
- 4. Support cache files per kernel .features file (or kernel version). This
+ 5. Support cache files per kernel .features file (or kernel version). This
will allow people to boot into a previous kernel without having to
recompile policy
- 5. Improve parser compile time
- 6. Investigate how to utilize profile composition and profile stacking to
+ 6. Improve parser compile time
+ 7. Investigate how to utilize profile composition and profile stacking to
decrease compile and load times (eg, one idea is that the policy template
is compiled once and each policy group once such that the parser need
only stitch the already compiled bits together)
- Work is planned for the medium term for '1' and '2' with '3' and '4'
- coming as needed. '5' and '6' will be handled in the long term.
+ Work is planned for the medium term for '1-3' with '4' and '5' coming as
+ needed. '6' and '7' will be handled in the long term.
Original description:
Just updated my Nexus 7 2013 from #160 to #161. It's been sat at the Google logo for 15 minutes now. It looks and feels like it's hung. As a user I'd be rebooting it thinking it had crashed by now. I shell in and find apparmor_parser using a lot of cpu for a long time.
top - 00:14:01 up 15 min, 2 users, load average: 5.12, 4.85, 3.21
Tasks: 202 total, 2 running, 200 sleeping, 0 stopped, 0 zombie
%Cpu(s): 50.5 us, 0.8 sy, 0.0 ni, 48.5 id, 0.2 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem: 1848024 total, 787400 used, 1060624 free, 54216 buffers
KiB Swap: 32764 total, 0 used, 32764 free. 579228 cached Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1970 root 20 0 4976 3652 852 R 99.8 0.2 14:31.04 apparmor_parser
2596 phablet 20 0 5996 1264 824 R 1.3 0.1 0:08.79 top
914 root 0 -20 7572 552 396 S 0.7 0.0 0:05.02 mpdecision
21 root 20 0 0 0 0 S 0.3 0.0 0:00.92 kworker/0:1
229 root 20 0 0 0 0 S 0.3 0.0 0:00.10 jbd2/mmcblk0p30
982 root 20 0 38856 1164 868 S 0.3 0.1 0:01.77 adbd
2570 phablet 20 0 10540 1456 692 S 0.3 0.1 0:02.30 sshd
1 root 20 0 3884 2648 1068 S 0.0 0.1 0:05.98 init
2 root -2 0 0 0 0 S 0.0 0.0 0:00.01 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 0:00.04 ksoftirqd/0
... it eventually finished after 18 minutes.
--
You received this bug notification because you are a member of Ubuntu
Touch seeded packages, which is subscribed to apparmor in Ubuntu.
https://bugs.launchpad.net/bugs/1350598
Title:
AppArmor policy compile improvements
Status in “apparmor” package in Ubuntu:
Triaged
Bug description:
apparmor_parser can take a long time to compile policy especially when there is a lot of policy, so we want to utilize compiled cache profile as much as possible. Cache files will have to be regenerated in the following cases:
* the kernel .features file is updated (eg, new features are added to
apparmor in the new kernel)
* apparmor itself is updated
* on devices with click packages, apparmor-easyprof-ubuntu is updated
As of 2014-10-02, what can be expected is:
- Systems with system-image updates (eg, Ubuntu Touch):
- First boot will use the precompiled cache files in the rootfs or custom
tarball and be fast
- Reboots will use the cache files on the device and be fast
- First boots after upgrades will use the cache files on the device if the
above conditions are not met and be fast
- Production devices will not meet any of those conditions except under
exceptional and rare circumstances (eg, major OS upgrades like 14.10 to
15.04) and be fast
- First boots after upgrades that meet one of the conditions will need to
regenerate the cache. This can happen on development releases where the
kernel features file, apparmor or apparmor-easyprof-ubuntu is still
under development and getting updates
- Systems with apt updates (eg, current Ubuntu Desktop and Server):
- First boot will compile cache files
- Reboots will use the cache files on the machine and be fast
- First boots after upgrades will use the cache files on the machine if the
above conditions are not met and be fast
- Stable releases of Ubuntu will not meet any of those conditions except
under exceptional and rare circumstances (eg, major OS upgrades like
14.10 to 15.04) and be fast
- First boots after upgrades that meet one of the above conditions will
need to regenerate the cache. This can happen on development releases
where the kernel features file, apparmor or apparmor-easyprof-ubuntu
is still under development and getting updates
In addition to the above, updates to only apparmor-easyprof-ubuntu
will regenerate the cache files for only the policy that is affected
(eg, if there is a change to the location policy group in policy
version 1.2, only apps using this policy version and this policy group
will need to be recompiled).
Planned improvements (in order of most likely to be done first):
1. Finetuning the checks to invalidate the cache (eg, .md5sums could only
be for /etc/apparmor.d/abstractions, ...)
2. Investigate ways to utilize the custom tarball and rootfs precompiled
cache files on upgrades when apparmor and apparmor-easyprof-ubuntu are
updated
3. Improve cache handling for app store apps (eg, having the app store
server precompile them so that the device can download them when it needs
to rather than having to regenerate them itself)
4. For systems with apt upgrades, compile the policy either during install or
on kernel upgrade rather than on boot
5. Support cache files per kernel .features file (or kernel version). This
will allow people to boot into a previous kernel without having to
recompile policy
6. Improve parser compile time
7. Investigate how to utilize profile composition and profile stacking to
decrease compile and load times (eg, one idea is that the policy template
is compiled once and each policy group once such that the parser need
only stitch the already compiled bits together)
Work is planned for the medium term for '1-3' with '4' and '5' coming
as needed. '6' and '7' will be handled in the long term.
Original description:
Just updated my Nexus 7 2013 from #160 to #161. It's been sat at the Google logo for 15 minutes now. It looks and feels like it's hung. As a user I'd be rebooting it thinking it had crashed by now. I shell in and find apparmor_parser using a lot of cpu for a long time.
top - 00:14:01 up 15 min, 2 users, load average: 5.12, 4.85, 3.21
Tasks: 202 total, 2 running, 200 sleeping, 0 stopped, 0 zombie
%Cpu(s): 50.5 us, 0.8 sy, 0.0 ni, 48.5 id, 0.2 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem: 1848024 total, 787400 used, 1060624 free, 54216 buffers
KiB Swap: 32764 total, 0 used, 32764 free. 579228 cached Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1970 root 20 0 4976 3652 852 R 99.8 0.2 14:31.04 apparmor_parser
2596 phablet 20 0 5996 1264 824 R 1.3 0.1 0:08.79 top
914 root 0 -20 7572 552 396 S 0.7 0.0 0:05.02 mpdecision
21 root 20 0 0 0 0 S 0.3 0.0 0:00.92 kworker/0:1
229 root 20 0 0 0 0 S 0.3 0.0 0:00.10 jbd2/mmcblk0p30
982 root 20 0 38856 1164 868 S 0.3 0.1 0:01.77 adbd
2570 phablet 20 0 10540 1456 692 S 0.3 0.1 0:02.30 sshd
1 root 20 0 3884 2648 1068 S 0.0 0.1 0:05.98 init
2 root -2 0 0 0 0 S 0.0 0.0 0:00.01 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 0:00.04 ksoftirqd/0
... it eventually finished after 18 minutes.
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/apparmor/+bug/1350598/+subscriptions
References
