canonical-partner-dev team mailing list archive

[Aleksandr Kichigin <1820001@xxxxxxxxxxxxxxxxxx>]

Public bug reported:

Hello

I am using lvm snapshots over sofware raid with 4 devices for rsync
backups.

Recently I found out that it is stuck and any operaton with lvm is stuck as well.
This also causes 1000+ processes of systemd-udevd to be in interpretable sleep and cannot be killed.


My backup script looks like this:

/sbin/vgcfgbackup -qq main
/sbin/lvcreate -qq --size 20G --snapshot --name backup /dev/mapper/main-root
mount /dev/mapper/main-backup /mnt/root_snapshot
*rsync stuff*
umount /mnt/root_snapshot
/sbin/lvremove -yqq /dev/mapper/main-backup


Launching as not root `pvs --unbuffered --readonly -t -vvvv` is stuck at:
#filters/filter-persistent.c:51     Wiping cache of LVM-capable devices

Launching as not root `lvs -vvvv --unbuffered --readonly -vvvv` us stuck at:
#toollib.c:3736         Get list of VGs on system


I have a lot of processes stucked (total number ~1000):

root       756  0.0  0.0  56400  8428 ?        D    фев26   0:00 /sbin/vgcfgbackup -qq main
root     21723  0.0  0.0  36940  1860 ?        D    мар13   0:00 /lib/systemd/systemd-udevd
root     21724  0.0  0.0  36940  1808 ?        D    мар13   0:00 /lib/systemd/systemd-udevd
root     21728  0.0  0.0  36940  1808 ?        D    мар13   0:00 /lib/systemd/systemd-udevd
root     14541  0.0  0.0   7780  2524 ?        D    06:25   0:00 /sbin/mdadm --monitor --scan --oneshot
root      8792  0.0  0.0  56548  8308 ?        D    05:59   0:00 lvs
root     11343  0.0  0.0 119424  7632 ?        S    06:10   0:00 sudo lvs -vd
admin    12112  0.0  0.0  56596  8368 ?        D    06:14   0:00 lvs -vvvv -dddddd --readonly -t --unbuffered


In dmesg I have following logs:

[16983388.190818] general protection fault: 0000 [#41] SMP PTI
[16987988.464353] general protection fault: 0000 [#42] SMP PTI


In system journal:

Mar 14 06:47:35 - systemd[1]: systemd-udevd.service: Found left-over process 17886 (systemd-udevd) in control group while starting unit. Ignoring.
Mar 14 06:47:35 - systemd[1]: This usually indicates unclean termination of a previous run, or service implementation deficiencies.
Mar 14 06:47:35- systemd[1]: systemd-udevd.service: Found left-over process 17887 (systemd-udevd) in control group while starting unit. Ignoring.
Mar 14 06:50:28 - systemd-udevd[20282]: seq 1116768 '/devices/pci0000:00/0000:00:1f.2/ata1/host0/target0:0:0/0:0:0:0/block/sda/sda1' is taking a long time
Mar 14 06:50:28 - systemd-udevd[20282]: seq 1116780 '/devices/pci0000:00/0000:00:1f.2/ata2/host1/target1:0:1/1:0:1:0/block/sdc/sdc1' is taking a long time


lvmdump hangs at `Gathering /dev listening`, but I was able to terminate it.


mdstat:
md0 : active raid10 sdb1[1] sdc1[2] sda1[0] sdd1[3]
      1953259520 blocks super 1.2 512K chunks 2 near-copies [4/4] [UUUU]
      bitmap: 1/15 pages [4KB], 65536KB chunk


OS: Ubuntu 18.04.1 LTS
Kernel: 4.15.0-29-generic
lvm2: 2.02.176-4.1ubuntu3

** Affects: lvm2 (Ubuntu)
     Importance: Undecided
         Status: New

** Package changed: skype (Ubuntu) => lvm2 (Ubuntu)

-- 
You received this bug notification because you are a member of Canonical
Partner Developers, which is subscribed to skype in Ubuntu.
https://bugs.launchpad.net/bugs/1820001

Title:
  vgcfgbackup stuck in kernel, causing lvm and udevd block forever

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/lvm2/+bug/1820001/+subscriptions