← Back to team overview

kernel-packages team mailing list archive

[Bug 1115753] Re: Umount of Multiple LVM Snapshots Causes 'soft lockup CPU#0 stuck for'

 

Markus:
Looking back at an earlier version of my script that I was using when I first reported this bug, I used to do exactly what you describe:  umount'ing all the partitions before doing any lvremove on any partition.  Now I umount and lvremove each partition before proceeding to the next partition.  It does seem to point the finger at umount rather than lvremove.

I do not run the scipt in the context of bacula-fd.  That is to say, it
is not a bacula RunBefore or RunAfter script.  I'm assuming that's what
you meant.  Instead, I have a cron job bourne shell script which does
all of the commands that are external to bacula (lvcreate, mount,
umount, lvremove, etc), and uses bconsole only to do the bacula backup
job.  So it looks like this:

...postgresql stuff...
...lvcreate...
...mount...

 sudo -E -u bacula /usr/bin/bconsole -c /etc/bacula/bconsole.conf <<EOF
@output /dev/null
messages
@tee "${TMP_LOG}"
run job=brain-all-job client="${CLIENT}" level="${BACKUP_LEVEL}" pool="${BACKUP_POOL}" yes
wait jobname=brain-all-job
messages
@output
quit
EOF

...umount...
...lvremove...

So the bug is occuring for me in the context I've shown above.

Previously I had inserted some code in the scipt to try to fix and/or
debug the problem, and then have since commented it out.  I was
unsuccesful before, but will try again.  This is what I had in the
script.  If anyone has any additional debugging suggestions, please
advise:

#   time -p udevadm settle
#   logger --priority syslog.info --stderr --tag "`basename ${0}`" "umount'ing ${i}"-snap
#   blockdev --report /dev/volgroup0/"${i}"-snap /dev/volgroup0/"${i}" /dev/mapper/volgroup0-"${i}"*
#   time -p sync
#   blockdev --flushbufs /dev/volgroup0/"${i}"-snap /dev/volgroup0/"${i}" /dev/mapper/volgroup0-"${i}"*
    # Could also try disabling journal using tune2fs??
#   cat /proc/locks > /etc/bacula/debugLocks.txt
#   echo >> /etc/bacula/debugLocks.txt
#   ps axl >> /etc/bacula/debugLocks.txt
#   echo >> /etc/bacula/debugLocks.txt
#   fuser -mv /mnt/"${i}"-snap >>/etc/bacula/debugLocks.txt
sync
umount /mnt/"${i}"-snap

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1115753

Title:
  Umount of Multiple LVM Snapshots Causes  'soft lockup CPU#0 stuck for'

Status in “linux” package in Ubuntu:
  Incomplete

Bug description:
  Simultaneous umount of multiple lvm snapshots causes system to hang /
  freeze / deadlock.  Window manager freezes.  If umount is executed in
  a non window-manager tty,  messages containing "BUG", "blocked for
  more than xx seconds", and a list of tasks associated with each of the
  8 cpu cores will scroll up at regular intervals.  The tasks assigned
  to each cpu core are unchanging over time, appearing to be deadlocked.
  No relevant information is left in system logs.  Hard reset / poweroff
  is required.  Problem does not occur 100% of the time.  It is less
  likely to occur if umount is done shortly after mount.  Problem still
  occurs -- but less frequently -- if lazy umount -l is used.  Snapshots
  are read-only, and are mounted read-only.

  ProblemType: Bug
  DistroRelease: Ubuntu 12.04
  Package: linux-image-3.2.0-37-generic 3.2.0-37.58
  ProcVersionSignature: Ubuntu 3.2.0-37.58-generic 3.2.35
  Uname: Linux 3.2.0-37-generic x86_64
  AlsaVersion: Advanced Linux Sound Architecture Driver Version 1.0.24.
  ApportVersion: 2.0.1-0ubuntu17.1
  Architecture: amd64
  AudioDevicesInUse:
   USER        PID ACCESS COMMAND
   /dev/snd/controlC1:  craig      3257 F.... pulseaudio
   /dev/snd/controlC0:  craig      3257 F.... pulseaudio
  CRDA: Error: command ['iw', 'reg', 'get'] failed with exit code 1: nl80211 not found.
  Card0.Amixer.info:
   Card hw:0 'SB'/'HDA ATI SB at 0xfeb00000 irq 16'
     Mixer name	: 'Realtek ALC889'
     Components	: 'HDA:10ec0889,1043846b,00100004'
     Controls      : 49
     Simple ctrls  : 24
  Card1.Amixer.info:
   Card hw:1 'HDMI'/'HDA ATI HDMI at 0xfea30000 irq 98'
     Mixer name	: 'ATI R6xx HDMI'
     Components	: 'HDA:1002aa01,00aa0100,00100100'
     Controls      : 6
     Simple ctrls  : 1
  Card1.Amixer.values:
   Simple mixer control 'IEC958',0
     Capabilities: pswitch pswitch-joined penum
     Playback channels: Mono
     Mono: Playback [on]
  Date: Mon Feb  4 18:58:37 2013
  HibernationDevice: RESUME=UUID=b788f786-ba76-4eb0-991f-d3bbc339d3f7
  InstallationMedia: Ubuntu 11.10 "Oneiric Ocelot" - Release amd64 (20111011)
  IwConfig:
   lo        no wireless extensions.
   
   eth0      no wireless extensions.
  MachineType: To be filled by O.E.M. To be filled by O.E.M.
  MarkForUpload: True
  ProcEnviron:
   TERM=xterm
   PATH=(custom, no user)
   LANG=en_US.UTF-8
   SHELL=/bin/bash
  ProcFB: 0 VESA VGA
  ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-3.2.0-37-generic root=/dev/mapper/volgroup0-root ro
  RelatedPackageVersions:
   linux-restricted-modules-3.2.0-37-generic N/A
   linux-backports-modules-3.2.0-37-generic  N/A
   linux-firmware                            1.79.1
  RfKill:
   
  SourcePackage: linux
  UpgradeStatus: Upgraded to precise on 2012-10-23 (104 days ago)
  dmi.bios.date: 09/27/2011
  dmi.bios.vendor: American Megatrends Inc.
  dmi.bios.version: 0813
  dmi.board.asset.tag: To be filled by O.E.M.
  dmi.board.name: Crosshair V Formula
  dmi.board.vendor: ASUSTeK Computer INC.
  dmi.board.version: Rev 1.xx
  dmi.chassis.asset.tag: To Be Filled By O.E.M.
  dmi.chassis.type: 3
  dmi.chassis.vendor: To Be Filled By O.E.M.
  dmi.chassis.version: To Be Filled By O.E.M.
  dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr0813:bd09/27/2011:svnTobefilledbyO.E.M.:pnTobefilledbyO.E.M.:pvrTobefilledbyO.E.M.:rvnASUSTeKComputerINC.:rnCrosshairVFormula:rvrRev1.xx:cvnToBeFilledByO.E.M.:ct3:cvrToBeFilledByO.E.M.:
  dmi.product.name: To be filled by O.E.M.
  dmi.product.version: To be filled by O.E.M.
  dmi.sys.vendor: To be filled by O.E.M.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1115753/+subscriptions


Follow ups