group.of.nepali.translators team mailing list archive

[ChristianEhrhardt <1738864@xxxxxxxxxxxxxxxxxx>]

Hi Laz,
thank you for your report.

Thanks for the log, I mostly looked at the condensed version like:
$ awk '/iscsiadm/ {gsub("[0-9]*: debug : virCommandRunAsync:2429 :",""); gsub("+0000",""); gsub("^2017-12-18 ",""); print $0}' 20171218-libvirt.txt | pastebinit
=> http://paste.ubuntu.com/26214102/

The version in 17.10 is libvirt 3.6.
So something between 1.3.1 (Xenial) and 3.6 (Artful) must have fixed it upstream according to your report.

I think what you are seeing is the effect of [3] that is active and essentially iterates to set "node.startup=>manual" on all targets.
The fix for that came in 1.3.5 via [4] which makes use of iscsiadm option "nonpersistent" to get the same done in a better way.

That would mean for a fix in Xenial backporting [4], but I'm not sure if backporting those changes in newer libvirt would not impose too much general regression risk for an SRU [1].
I agree it is uncomfortably slow in your case, but the change is a rather big behavioral change (wanted in your case I totally agree), but I'm always afraid of [5] as I ran into issues almost like it.
And since it is "only slow" (I hate saying that being a performance engineer in the past) the severity isn't as high as if it is broken.

OTOH the code seems to apply and the option on iscsiadm is available
even in trusty (important for backports).

I wonder if instead of taking the risk of changing that in an SRU for
you the best option might be via [2] Ubuntu Cloud Archive which would
allow you to stick to 16.04 but at the same time get a supported newer
virtualization stack?

I'm tagging up the bug tasks accordingly and look forward to a
discussion about the best but also safe way for you and other Ubuntu
users overall.

[1]: https://wiki.ubuntu.com/StableReleaseUpdates
[2]: https://wiki.ubuntu.com/OpenStack/CloudArchive
[3]: https://libvirt.org/git/?p=libvirt.git;a=commit;h=3c12b654
[4]: https://libvirt.org/git/?p=libvirt.git;a=commit;h=56057900
[5]: https://xkcd.com/1172/

** Also affects: libvirt (Ubuntu Trusty)
   Importance: Undecided
       Status: New

** Also affects: libvirt (Ubuntu Xenial)
   Importance: Undecided
       Status: New

** Changed in: libvirt (Ubuntu Trusty)
       Status: New => Won't Fix

** Changed in: libvirt (Ubuntu Xenial)
       Status: New => Confirmed

** Changed in: libvirt (Ubuntu)
       Status: New => Fix Released

-- 
You received this bug notification because you are a member of नेपाली
भाषा समायोजकहरुको समूह, which is subscribed to Xenial.
Matching subscriptions: Ubuntu 16.04 Bugs
https://bugs.launchpad.net/bugs/1738864

Title:
  libvirt updates all iSCSI targets from host each time a pool is
  started

Status in libvirt package in Ubuntu:
  Fix Released
Status in libvirt source package in Trusty:
  Won't Fix
Status in libvirt source package in Xenial:
  Confirmed

Bug description:
  Hello everyone, I'm a little confused about the behavior of libvirt in
  Ubuntu 16.04.3.

  We have up to 140 iSCSI targets on a single storage host, and all of
  these are made available to our VM hosts. If I stop one of the iSCSI
  pools through virsh ("virsh pool-destroy iscsipool1") and start it
  back up again while running libvirt in debug, I see that it runs a
  discovery and proceeds to go through and update every single target
  available on that host -- even targets that we do not use, instead of
  simply connecting to that target.

  This turns a <1 second process into a minimum of 30 seconds, and I
  just ran it with the stopwatch and clocked it at 64 seconds. So if we
  are doing maintenance on these hosts and go for a reboot, it takes
  90-120+ minutes to finish auto starting all of the iSCSI pools. And of
  course, during this period of time, the server is completely worthless
  as a VM host. Libvirt is just stuck until it finishes connecting
  everything. Manually connecting to the targets using iscsiadm without
  doing all the same hubbub that libvirt is doing connects these targets
  immediately, as I would expect from libvirt.

  And for each of the 140 iSCSI targets, it goes through and runs an
  iscsiadm sendtargets and then updates every single target before
  finally connecting the respective pool.

  We also noticed that libvirt in Ubuntu 17.10 does not have this
  behavior. Well maybe it does, but it connects the iSCSI targets
  immediately. It is a much different process than Ubuntu 16.04.3.

  Any help would be greatly appreciated. Thank you so much.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/libvirt/+bug/1738864/+subscriptions