← Back to team overview

group.of.nepali.translators team mailing list archive

[Bug 1597974] Re: ISST-LTE:pVM:monklp5:Ubuntu16.04.1:system crashed at lpfc_sli4_scmd_to_wqidx_distr

 

This bug was fixed in the package linux - 4.4.0-36.55

---------------
linux (4.4.0-36.55) xenial; urgency=low

  [ Stefan Bader ]

  * Release Tracking Bug
    - LP: #1612305

  * I2C touchpad does not work on AMD platform (LP: #1612006)
    - SAUCE: pinctrl/amd: Remove the default de-bounce time

  * CVE-2016-5696
    - tcp: make challenge acks less predictable

linux (4.4.0-35.54) xenial; urgency=low

  [ Stefan Bader ]

  * Release Tracking Bug
    - LP: #1611215

  * [i915_bpo] Sync with v4.7 (LP: #1609742)
    - SAUCE: i915_bpo: Sync with v4.7

  * s390/cio: fix reset of channel measurement block (LP: #1609415)
    - s390/cio: allow to reset channel measurement block

  * in Ubuntu16.10: Hit on Call traces  and system goes down when transactional
    memory  tests are running in 32TB Brazos system (LP: #1606786)
    - powerpc/tm: Avoid SLB faults in treclaim/trecheckpoint when RI=0
    - powerpc/tm: Fix stack pointer corruption in __tm_recheckpoint()

  *  Power Menu does not display after press the Power Button (LP: #1609204)
    - intel-vbtn: new driver for Intel Virtual Button
    - [config] enable CONFIG_INTEL_VBTN=m

  * OptiPlex 7450 AIO hangs when rebooting (LP: #1608762)
    - x86/reboot: Add Dell Optiplex 7450 AIO reboot quirk

  * virtualbox+usb 3.0 breaks boot, -28 kernel works (LP: #1604058)
    - SAUCE: xhci: Fix soft lockup in xhci_pci_probe path when XHCI_STATE_HALTED

  * linux-kernel: Freeing IRQ from IRQ context (LP: #1597908)
    - block: defer timeouts to a workqueue

  * Tunnel offload indications not stripped from encapsulated packets, causing
    performance overhead (LP: #1602755)
    - tunnels: Remove encapsulation offloads on decap.

  * lm-sensors is throwing "ERROR: Can't get value of subfeature temp1_input:
    I/O error" for be2net driver (LP: #1607387)
    - be2net: perform temperature query in adapter regardless of its interface
      state

  * Dell dock MAC Address pass through doesn't work in Ubuntu (LP: #1579984)
    - r8152: Add support for setting pass through MAC address on RTL8153-AD

  * vmxnet3 LRO IPv6 performance issues (stalling TCP) (LP: #1605494)
    - Driver: Vmxnet3: set CHECKSUM_UNNECESSARY for IPv6 packets

  * ISST-LTE:pVM:monklp5:Ubuntu16.04.1:system crashed at
    lpfc_sli4_scmd_to_wqidx_distr (LP: #1597974)
    - SAUCE: lpfc: fix oops in lpfc_sli4_scmd_to_wqidx_distr() from
      lpfc_send_taskmgmt()

  * Backport cxlflash shutdown patch to Xenial SRU (LP: #1605405)
    - SAUCE: cxlflash: Verify problem state area is mapped before notifying
      shutdown

  * Xenial update to v4.4.16 stable release (LP: #1607404)
    - mac80211: fix fast_tx header alignment
    - mac80211: mesh: flush mesh paths unconditionally
    - mac80211_hwsim: Add missing check for HWSIM_ATTR_SIGNAL
    - mac80211: Fix mesh estab_plinks counting in STA removal case
    - EDAC, sb_edac: Fix rank lookup on Broadwell
    - IB/cm: Fix a recently introduced locking bug
    - IB/mlx4: Properly initialize GRH TClass and FlowLabel in AHs
    - powerpc/pseries: Fix IBM_ARCH_VEC_NRCORES_OFFSET since POWER8NVL was added
    - powerpc/tm: Always reclaim in start_thread() for exec() class syscalls
    - usb: dwc2: fix regression on big-endian PowerPC/ARM systems
    - USB: EHCI: declare hostpc register as zero-length array
    - usb: common: otg-fsm: add license to usb-otg-fsm
    - mnt: fs_fully_visible test the proper mount for MNT_LOCKED
    - mnt: Account for MS_RDONLY in fs_fully_visible
    - mnt: If fs_fully_visible fails call put_filesystem.
    - of: fix autoloading due to broken modalias with no 'compatible'
    - of: irq: fix of_irq_get[_byname]() kernel-doc
    - locking/ww_mutex: Report recursive ww_mutex locking early
    - locking/qspinlock: Fix spin_unlock_wait() some more
    - locking/static_key: Fix concurrent static_key_slow_inc()
    - x86, build: copy ldlinux.c32 to image.iso
    - kprobes/x86: Clear TF bit in fault on single-stepping
    - x86/amd_nb: Fix boot crash on non-AMD systems
    - Revert "gpiolib: Split GPIO flags parsing and GPIO configuration"
    - uvc: Forward compat ioctls to their handlers directly
    - thermal: cpu_cooling: fix improper order during initialization
    - writeback: use higher precision calculation in domain_dirty_limits()
    - nfsd4/rpc: move backchannel create logic into rpc code
    - nfsd: Always lock state exclusively.
    - nfsd: Extend the mutex holding region around in nfsd4_process_open2()
    - posix_acl: Add set_posix_acl
    - nfsd: check permissions when setting ACLs
    - make nfs_atomic_open() call d_drop() on all ->open_context() errors.
    - NFS: Fix another OPEN_DOWNGRADE bug
    - ARM: imx6ul: Fix Micrel PHY mask
    - ARM: 8578/1: mm: ensure pmd_present only checks the valid bit
    - ARM: 8579/1: mm: Fix definition of pmd_mknotpresent
    - MIPS: KVM: Fix modular KVM under QEMU
    - mm: Export migrate_page_move_mapping and migrate_page_copy
    - UBIFS: Implement ->migratepage()
    - sched/fair: Fix cfs_rq avg tracking underflow
    - packet: Use symmetric hash for PACKET_FANOUT_HASH.
    - net_sched: fix mirrored packets checksum
    - cdc_ncm: workaround for EM7455 "silent" data interface
    - ipv6: Fix mem leak in rt6i_pcpu
    - ARCv2: Check for LL-SC livelock only if LLSC is enabled
    - ARCv2: LLSC: software backoff is NOT needed starting HS2.1c
    - kvm: Fix irq route entries exceeding KVM_MAX_IRQ_ROUTES
    - KVM: nVMX: VMX instructions: fix segment checks when L1 is in long mode.
    - HID: elo: kill not flush the work
    - HID: hiddev: validate num_values for HIDIOCGUSAGES, HIDIOCSUSAGES commands
    - tracing: Handle NULL formats in hold_module_trace_bprintk_format()
    - base: make module_create_drivers_dir race-free
    - iommu/arm-smmu: Wire up map_sg for arm-smmu-v3
    - iommu/vt-d: Enable QI on all IOMMUs before setting root entry
    - iommu/amd: Fix unity mapping initialization race
    - drm/mgag200: Black screen fix for G200e rev 4
    - ipmi: Remove smi_msg from waiting_rcv_msgs list before handle_one_recv_msg()
    - arm64: Rework valid_user_regs
    - vfs: add d_real_inode() helper
    - af_unix: fix hard linked sockets on overlay
    - btrfs: account for non-CoW'd blocks in btrfs_abort_transaction
    - drm/radeon: fix asic initialization for virtualized environments
    - drm/amdgpu/gfx7: fix broken condition check
    - ubi: Make recover_peb power cut aware
    - drm/amdkfd: unbind only existing processes
    - drm/amdkfd: destroy dbgmgr in notifier release
    - drm/dp/mst: Always clear proposed vcpi table for port.
    - drm/nouveau/disp/sor/gf119: both links use the same training register
    - drm/nouveau/gr/gf100-: update sm error decoding from gk20a nvgpu headers
    - drm/nouveau/fbcon: fix out-of-bounds memory accesses
    - drm/nouveau: fix for disabled fbdev emulation
    - drm/nouveau/disp/sor/gf119: select correct sor when poking training pattern
    - drm/i915/ilk: Don't disable SSC source if it's in use
    - drm/i915: Refresh cached DP port register value on resume
    - drm/i915: Update ifdeffery for mutex->owner
    - drm/i915: Update CDCLK_FREQ register on BDW after changing cdclk frequency
    - drm: add missing drm_mode_set_crtcinfo call
    - drm: make drm_atomic_set_mode_prop_for_crtc() more reliable
    - drm: atmel-hlcdc: actually disable scaling when no scaling is required
    - drm/ttm: Make ttm_bo_mem_compat available
    - drm/vmwgfx: Add an option to change assumed FB bpp
    - drm/vmwgfx: Work around mode set failure in 2D VMs
    - drm/vmwgfx: Check pin count before attempting to move a buffer
    - drm/vmwgfx: Delay pinning fbdev framebuffer until after mode set
    - drm/vmwgfx: Fix error paths when mapping framebuffer
    - memory: omap-gpmc: Fix omap gpmc EXTRADELAY timing
    - perf/x86: Fix undefined shift on 32-bit kernels
    - xen/balloon: Fix declared-but-not-defined warning
    - iio: Fix error handling in iio_trigger_attach_poll_func
    - iio:st_pressure: fix sampling gains (bring inline with ABI)
    - iio: light apds9960: Add the missing dev.parent
    - iio: proximity: as3935: correct IIO_CHAN_INFO_RAW output
    - iio: proximity: as3935: remove triggered buffer processing
    - iio: proximity: as3935: fix buffer stack trashing
    - iio: humidity: hdc100x: correct humidity integration time mask
    - iio: humidity: hdc100x: fix IIO_TEMP channel reporting
    - iio: hudmidity: hdc100x: fix incorrect shifting and scaling
    - staging: iio: accel: fix error check
    - iio: accel: kxsd9: fix the usage of spi_w8r8()
    - iio:ad7266: Fix broken regulator error handling
    - iio:ad7266: Fix support for optional regulators
    - iio:ad7266: Fix probe deferral for vref
    - tty/vt/keyboard: fix OOB access in do_compute_shiftstate()
    - hwmon: (dell-smm) Restrict fan control and serial number to CAP_SYS_ADMIN by
      default
    - hwmon: (dell-smm) Disallow fan_type() calls on broken machines
    - hwmon: (dell-smm) Cache fan_type() calls and change fan detection
    - ALSA: dummy: Fix a use-after-free at closing
    - ALSA: hda - Fix the headset mic jack detection on Dell machine
    - ALSA: hda / realtek - add two more Thinkpad IDs (5050,5053) for tpt460 fixup
    - ALSA: au88x0: Fix calculation in vortex_wtdma_bufshift()
    - ALSA: echoaudio: Fix memory allocation
    - ALSA: timer: Fix negative queue usage by racy accesses
    - ALSA: hda/realtek: Add Lenovo L460 to docking unit fixup
    - ALSA: hda - Add PCI ID for Kabylake-H
    - ALSA: hda - fix read before array start
    - ALSA: hda/realtek - add new pin definition in alc225 pin quirk table
    - ALSA: pcm: Free chmap at PCM free callback, too
    - ALSA: ctl: Stop notification after disconnection
    - ALSA: hda - fix use-after-free after module unload
    - ALSA: hda: add AMD Stoney PCI ID with proper driver caps
    - ARM: sunxi/dt: make the CHIP inherit from allwinner,sun5i-a13
    - ARM: dts: armada-38x: fix MBUS_ID for crypto SRAM on Armada 385 Linksys
    - ARM: mvebu: fix HW I/O coherency related deadlocks
    - ovl: Copy up underlying inode's ->i_mode to overlay inode
    - ovl: verify upper dentry in ovl_remove_and_whiteout()
    - scsi: fix race between simultaneous decrements of ->host_failed
    - 53c700: fix BUG on untagged commands
    - Fix reconnect to not defer smb3 session reconnect long after socket
      reconnect
    - cifs: dynamic allocation of ntlmssp blob
    - File names with trailing period or space need special case conversion
    - xen/acpi: allow xen-acpi-processor driver to load on Xen 4.7
    - crypto: qat - make qat_asym_algs.o depend on asn1 headers
    - tmpfs: don't undo fallocate past its last page
    - tmpfs: fix regression hang in fallocate undo
    - drm/i915: Revert DisplayPort fast link training feature
    - ovl: verify upper dentry before unlink and rename
    - Linux 4.4.16

  * Regression caused by `fuse: Add support for pid namespaces` in 4.4.0-6.21
    (LP: #1605344)
    - SAUCE: (namespace) fuse: Permit requests from other pid namespaces

  * CVE-2016-5400
    - media: fix airspy usb probe error path

  * Cannot mount proc in unprivileged containers if /proc/xen is mounted
    (LP: #1607374)
    - SAUCE: xenbus: Use proc_create_mount_point() to create /proc/xen

  * Mic mute key does not work for Ideapad laptops (LP: #1607153)
    - ideapad_laptop: Add an event for mic mute hotkey

  * NVMe stress test fails after 12 hours on Ubuntu 16.04 (LP: #1604995)
    - block: atari: Return early for unsupported sector size

  * Console extremely slow with 4.4 kernels for servers with Matrox G200er2 or
    similar (LP: #1605662)
    - SAUCE: vesafb: Set mtrr:3 (write-combining) as default

  * Ubuntu 16.04 - Full EEH Recovery Support for NVMe devices (LP: #1602724)
    - nvme: use a work item to submit async event requests
    - nvme: don't poll the CQ from the kthread
    - nvme: replace the kthread with a per-device watchdog timer
    - NVMe: Fix reset/remove race
    - nvme: Avoid reset work on watchdog timer function during error recovery
    - NVMe: Always use MSI/MSI-x interrupts

  * [LTC-Test] - NMI watchdog Bug and call traces when trinity is executed.
    (LP: #1602524)
    - ext4: factor out determining of hole size
    - ext4: return hole from ext4_map_blocks()
    - ext4: more efficient SEEK_DATA implementation

  * changelog: add CVEs as first class citizens (LP: #1604344)
    - avoid duplicate CVE numbers in changelog

  * [LTCTest][Opal][OP820] Machine crashed with Oops: Kernel access of bad area,
    sig: 11 [#1] while executing Froze PE Error injection (LP: #1603449)
    - powerpc/eeh: Fix invalid cached PE primary bus

  * Hotplug remove and re-add adds PCI adapter to next PCI domain (PCI)
    (LP: #1603574)
    - powerpc/pci: Assign fixed PHB number based on device-tree properties

  * nvme - reset_controller is not working after adapter's firmware upgrade
    (adapter quirk is needed) (LP: #1602726)
    - NVMe: Create discard zero quirk white list
    - nvme/quirk: Add a delay before checking for adapter readiness

  * ovs nat: conntrack netlink event are missing (LP: #1603468)
    - openvswitch: fix conntrack netlink event delivery

  * FlashGT - In Tuleta 8284-22A with card in card slot P1-C9, system Fails to
    boot operating system (LP: #1602785)
    - cxl: Ignore CAPI adapters misplaced in switched slots

  * CVE-2016-5728
    - misc: mic: Fix for double fetch security bug in VOP driver

  * CVE-2016-5244 (LP: #1589041)
    - rds: fix an infoleak in rds_inc_info_copy

  * Miscellaneous Ubuntu changes
    - Added Snapcraft files
    - SAUCE: snapcraft: cleanup and remove unnecessary elements

 -- Stefan Bader <stefan.bader@xxxxxxxxxxxxx>  Thu, 11 Aug 2016 17:34:14
+0200

** Changed in: linux (Ubuntu Xenial)
       Status: Fix Committed => Fix Released

** CVE added: http://www.cve.mitre.org/cgi-
bin/cvename.cgi?name=2016-5244

** CVE added: http://www.cve.mitre.org/cgi-
bin/cvename.cgi?name=2016-5400

** CVE added: http://www.cve.mitre.org/cgi-
bin/cvename.cgi?name=2016-5696

** CVE added: http://www.cve.mitre.org/cgi-
bin/cvename.cgi?name=2016-5728

-- 
You received this bug notification because you are a member of नेपाली
भाषा समायोजकहरुको समूह, which is subscribed to Xenial.
Matching subscriptions: Ubuntu 16.04 Bugs
https://bugs.launchpad.net/bugs/1597974

Title:
  ISST-LTE:pVM:monklp5:Ubuntu16.04.1:system crashed at
  lpfc_sli4_scmd_to_wqidx_distr

Status in linux package in Ubuntu:
  Fix Committed
Status in linux source package in Xenial:
  Fix Released
Status in linux source package in Yakkety:
  Fix Committed

Bug description:
  ---Problem Description---
  We have Ubuntu16.04.1 installed on our system and run DLPAR test for ZR1 adapter  after some time it crashes at lpfc_sli4_scmd_to_wqidx_distr+0x30/0x100
    
  Machine Type = 9119-MME*1085AE7 
   
  ---Debugger Data---
  e:mon> e
  cpu 0xe: Vector: 300 (Data Access) at [c0000003d45335a0]
      pc: d000000003a374e0: lpfc_sli4_scmd_to_wqidx_distr+0x30/0x100 [lpfc]
      lr: d0000000039d749c: lpfc_sli_calc_ring.part.20+0xdc/0x100 [lpfc]
      sp: c0000003d4533820
     msr: 8000000100009033
     dar: 0
   dsisr: 40000000
    current = 0xc0000003e06c2a20
    paca    = 0xc000000007af8500   softe: 0        irq_happened: 0x01
      pid   = 17983, comm = scsi_eh_23
  e:mon> r
  R00 = d0000000039d749c   R16 = 0000000000000000
  R01 = c0000003d4533820   R17 = c0000003d4533cd0
  R02 = d000000003a84160   R18 = c0000003d4533cb8
  R03 = c0000003ee76a000   R19 = c0000003d87e5088
  R04 = c0000003dad6a800   R20 = c0000003d4533cb0
  R05 = c0000003dad6a870   R21 = 000000000000001e
  R06 = 0000000000000001   R22 = c0000000018aab78
  R07 = d000000003a84160   R23 = c0000003dad6a870
  R08 = d000000003a2f830   R24 = c0000003dad6a800
  R09 = 0000000000000004   R25 = c0000003d4533978
  R10 = 0000000000000000   R26 = 0000000000000001
  R11 = d000000003a59a50   R27 = 0000000000000000
  R12 = 0000000028533824   R28 = c0000003e841e000
  R13 = c000000007af8500   R29 = c0000003ee76a000
  R14 = c0000003d87e5000   R30 = c0000003dad6a800
  R15 = c0000003d4533cb8   R31 = c0000003ee76a000
  pc  = d000000003a374e0 lpfc_sli4_scmd_to_wqidx_distr+0x30/0x100 [lpfc]
  cfar= c000000000008468 slb_miss_realmode+0x50/0x78
  lr  = d0000000039d749c lpfc_sli_calc_ring.part.20+0xdc/0x100 [lpfc]
  msr = 8000000100009033   cr  = 28538828
  ctr = c000000000ae3cf0   xer = 0000000020000010   trap =  300
  dar = 0000000000000000   dsisr = 40000000
  e:mon> 
   
  Stack trace output:
   e:mon> t
  [c0000003d4533850] d0000000039d749c lpfc_sli_calc_ring.part.20+0xdc/0x100 [lpfc]
  [c0000003d4533890] d0000000039df680 lpfc_sli_issue_iocb+0xf0/0x330 [lpfc]
  [c0000003d45338f0] d0000000039e3824 lpfc_sli_issue_iocb_wait+0x264/0x680 [lpfc]
  [c0000003d45339d0] d000000003a32944 lpfc_send_taskmgmt+0x2d4/0x7d0 [lpfc]
  [c0000003d4533aa0] d000000003a33564 lpfc_device_reset_handler+0x114/0x210 [lpfc]
  [c0000003d4533b60] c00000000075843c scsi_eh_ready_devs+0x68c/0xee0
  [c0000003d4533c50] c00000000075a91c scsi_error_handler+0x6bc/0x9e0
  [c0000003d4533d80] c0000000000e61e0 kthread+0x110/0x130
  [c0000003d4533e30] c000000000009538 ret_from_kernel_thread+0x5c/0xa4
  --- Exception: 0  at 0000000000000000
  e:mon>
   
  e:mon> dl
  [10194.079284] sd 13:0:3:0: [sdaf] tag#0 FAILED Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
  [10194.079293] sd 13:0:3:0: [sdaf] tag#0 CDB: Read(10) 28 00 02 7f ff 80 00 00 80 00
  [10194.079297] blk_update_request: I/O error, dev sdaf, sector 41942912
  [10194.079313] device-mapper: multipath: Failing path 65:240.
  [10194.079351] sd 13:0:2:0: [sdab] tag#0 FAILED Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
  [10194.079360] sd 13:0:2:0: [sdab] tag#0 CDB: Read(10) 28 00 02 7f ff 80 00 00 80 00
  [10194.079364] blk_update_request: I/O error, dev sdab, sector 41942912
  [10194.079375] device-mapper: multipath: Failing path 65:176.
  [10194.102832] scsi 13:0:1:0: alua: Detached
  [10194.110320] sd 13:0:1:1: [sdh] tag#0 FAILED Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
  [10194.110324] sd 13:0:1:1: [sdh] tag#0 CDB: Read(10) 28 00 02 7f ff 80 00 00 80 00
  [10194.110326] blk_update_request: I/O error, dev sdh, sector 41942912
  [10194.110334] device-mapper: multipath: Failing path 8:112.
  [10194.110394] sd 13:0:2:1: [sdac] tag#0 FAILED Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
  [10194.110398] sd 13:0:2:1: [sdac] tag#0 CDB: Read(10) 28 00 02 7f ff 80 00 00 80 00
  [10194.110401] blk_update_request: I/O error, dev sdac, sector 41942912
  [10194.110407] device-mapper: multipath: Failing path 65:192.
  [10194.110439] sd 13:0:3:1: [sdag] tag#0 FAILED Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
  [10194.110448] sd 13:0:3:1: [sdag] tag#0 CDB: Read(10) 28 00 02 7f ff 80 00 00 80 00
  [10194.110452] blk_update_request: I/O error, dev sdag, sector 41942912
  [10194.110464] device-mapper: multipath: Failing path 66:0.
  [10194.118851] scsi 13:0:0:1: alua: Detached
  [10194.122868] sd 13:0:3:0: [sdaf] tag#0 FAILED Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
  [10194.122879] sd 13:0:3:0: [sdaf] tag#0 CDB: Read(10) 28 00 02 7f ff 80 00 00 80 00
  [10194.122887] blk_update_request: I/O error, dev sdaf, sector 41942912
  [10194.122911] device-mapper: multipath: Failing path 65:240.
  [10194.138865] scsi 13:0:2:0: alua: Detached
  [10194.158852] scsi 13:0:3:0: alua: Detached
  [10194.162199] sd 13:0:3:1: [sdag] tag#0 FAILED Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
  [10194.162204] sd 13:0:3:1: [sdag] tag#0 CDB: Read(10) 28 00 02 7f ff 80 00 00 80 00
  [10194.162207] blk_update_request: I/O error, dev sdag, sector 41942912
  [10194.162216] device-mapper: multipath: Failing path 66:0.
  [10194.162241] device-mapper: multipath: Failing path 65:192.
  [10194.194835] scsi 13:0:1:1: alua: Detached
  [10194.202301] device-mapper: multipath: Failing path 65:208.
  [10194.202323] device-mapper: multipath: Failing path 8:128.
  [10194.202359] device-mapper: multipath: Failing path 66:16.
  [10194.218852] scsi 13:0:0:2: alua: Detached
  [10194.222391] device-mapper: multipath: Failing path 66:0.
  [10194.250830] scsi 13:0:2:1: alua: Detached
  [10194.274829] scsi 13:0:3:1: alua: Detached
  [10194.278436] device-mapper: multipath: Failing path 66:16.
  [10194.278467] device-mapper: multipath: Failing path 65:208.
  [10194.298817] scsi 13:0:1:2: alua: Detached
  [10194.306356] device-mapper: multipath: Failing path 65:224.
  [10194.306383] device-mapper: multipath: Failing path 65:160.
  [10194.306424] device-mapper: multipath: Failing path 66:32.
  [10194.334838] scsi 13:0:0:3: alua: Detached
  [10194.338579] device-mapper: multipath: Failing path 66:16.
  [10194.354934] scsi 13:0:2:2: alua: Detached
  [10194.378850] scsi 13:0:3:2: alua: Detached
  [10194.382605] device-mapper: multipath: Failing path 66:32.
  [10194.382643] device-mapper: multipath: Failing path 65:224.
  [10194.406826] scsi 13:0:1:3: alua: Detached
  [10194.410973] device-mapper: multipath: Failing path 66:32.
  [10194.434908] scsi 13:0:2:3: alua: Detached
  [10194.462920] scsi 13:0:3:3: alua: Detached
  [10194.587776] iommu: Removing device 0007:01:00.0 from group 0
  [10204.593263] pci_bus 0007:01: busn_res: [bus 01-ff] is released
  [10204.593333] rpadlpar_io: slot PHB 21 removed
  [10849.383986] PCI host bridge /pci@800000020000015  ranges:
  [10849.383991]  MEM 0x00003fc600000000..0x00003fc67effffff -> 0x0000000080000000
  [10849.383993]  MEM 0x000030c000000000..0x000030cfffffffff -> 0x0003d0c000000000
  [10849.389303] PCI: I/O resource not set for host bridge /pci@800000020000015 (domain 8)
  [10849.389372] PCI host bridge to bus 0008:01
  [10849.389378] pci_bus 0008:01: root bus resource [mem 0x3fc600000000-0x3fc67effffff] (bus address [0x80000000-0xfeffffff])
  [10849.389384] pci_bus 0008:01: root bus resource [bus 01-ff]
  [10849.394162] pci 0008:01:00.1: reg 0x160: [mem 0x00000000-0x0000ffff 64bit pref]
  [10849.394165] pci 0008:01:00.1: VF(n) BAR0 space: [mem 0x00000000-0x0013ffff 64bit pref] (contains BAR0 for 20 VFs)
  [10849.405662] pci 0008:01:00.0: reg 0x160: [mem 0x00000000-0x0000ffff 64bit pref]
  [10849.405664] pci 0008:01:00.0: VF(n) BAR0 space: [mem 0x00000000-0x0013ffff 64bit pref] (contains BAR0 for 20 VFs)
  [10849.491175] iommu: Adding device 0008:01:00.1 to group 0
  [10849.491704] iommu: Adding device 0008:01:00.0 to group 0
  [10849.492196] PIAR: overlapping address range
  [10849.492198] PIAR: overlapping address range
  [10849.492199] PIAR: overlapping address range
  [10849.492199] PIAR: overlapping address range
  [10849.492200] PIAR: overlapping address range
  [10849.492441] lpfc 0008:01:00.1: enabling device (0140 -> 0142)
  [10849.495406] lpfc 0008:01:00.1: ibm,query-pe-dma-windows(53) 10000 8000000 20000015 returned 0
  [10849.542283] lpfc 0008:01:00.1: Using 64-bit direct DMA at offset 800000000000000
  [10849.675139] scsi host14: Emulex LPe16000 16Gb PCIe Fibre Channel Adapter on PCI bus 01 device 01 irq 505
  [10850.235317] lpfc 0008:01:00.0: enabling device (0140 -> 0142)
  [10850.239152] lpfc 0008:01:00.0: Using 64-bit direct DMA at offset 800000000000000
  [10850.399263] scsi host15: Emulex LPe16000 16Gb PCIe Fibre Channel Adapter on PCI bus 01 device 00 irq 504
  [10850.959301] rpaphp: Slot [U78CA.001.CSS003P-P1-C6-C1] registered
  [10850.959309] rpadlpar_io: slot PHB 21 added
  [10851.847229] lpfc 0008:01:00.0: 1:1303 Link Up Event x1 received Data: x1 x0 x80 x0 x0 x0 0
  [10852.026827] scsi 15:0:0:0: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [10852.027527] sd 15:0:0:0: alua: supports implicit TPGS
  [10852.027843] sd 15:0:0:0: alua: port group 00 rel port 230
  [10852.027890] sd 15:0:0:0: [sda] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [10852.028276] sd 15:0:0:0: alua: port group 00 state A preferred supports tolusnA
  [10852.028425] sd 15:0:0:0: [sda] Write Protect is off
  [10852.028431] sd 15:0:0:0: [sda] Mode Sense: f5 00 00 08
  [10852.028455] sd 15:0:0:0: Attached scsi generic sg0 type 0
  [10852.028711] sd 15:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [10852.029804] scsi 15:0:0:1: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [10852.030789] sd 15:0:0:1: alua: supports implicit TPGS
  [10852.031484] sd 15:0:0:1: alua: port group 00 rel port 230
  [10852.031522] sd 15:0:0:1: [sdb] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [10852.031994] sd 15:0:0:1: alua: port group 00 state A preferred supports tolusnA
  [10852.032153] sd 15:0:0:1: Attached scsi generic sg1 type 0
  [10852.032239] sd 15:0:0:1: [sdb] Write Protect is off
  [10852.032246] sd 15:0:0:1: [sdb] Mode Sense: ed 00 00 08
  [10852.032596] sd 15:0:0:1: [sdb] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [10852.033460] scsi 15:0:0:2: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [10852.034530] sd 15:0:0:2: alua: supports implicit TPGS
  [10852.034917] sd 15:0:0:2: alua: port group 00 rel port 230
  [10852.035000] sd 15:0:0:2: [sdd] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [10852.035294] sd 15:0:0:2: alua: port group 00 state A preferred supports tolusnA
  [10852.035568] sd 15:0:0:2: Attached scsi generic sg2 type 0
  [10852.036739] scsi 15:0:0:3: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [10852.037441] sd 15:0:0:3: alua: supports implicit TPGS
  [10852.037740] sd 15:0:0:3: alua: port group 00 rel port 230
  [10852.037798] sd 15:0:0:3: [sde] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [10852.038070] sd 15:0:0:3: alua: port group 00 state A preferred supports tolusnA
  [10852.038234] sd 15:0:0:3: Attached scsi generic sg3 type 0
  [10852.038349] sd 15:0:0:3: [sde] Write Protect is off
  [10852.038355] sd 15:0:0:3: [sde] Mode Sense: ed 00 00 08
  [10852.038683] sd 15:0:0:3: [sde] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [10852.039314] scsi 15:0:1:0: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [10852.039748]  sdb:
  [10852.040238] sd 15:0:1:0: alua: supports implicit TPGS
  [10852.040632] sd 15:0:1:0: alua: port group 00 rel port 30
  [10852.040708] sd 15:0:1:0: [sdg] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [10852.041053] sd 15:0:1:0: alua: port group 00 state A preferred supports tolusnA
  [10852.041481] sd 15:0:1:0: [sdg] Write Protect is off
  [10852.041496] sd 15:0:1:0: [sdg] Mode Sense: f5 00 00 08
  [10852.041550] sd 15:0:0:1: [sdb] Attached SCSI disk
  [10852.041786] sd 15:0:1:0: [sdg] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [10852.042390] sd 15:0:1:0: Attached scsi generic sg4 type 0
  [10852.044049] scsi 15:0:1:1: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [10852.044795] sd 15:0:1:1: alua: supports implicit TPGS
  [10852.045180] sd 15:0:1:1: alua: port group 00 rel port 30
  [10852.045226] sd 15:0:0:0: [sda] Attached SCSI disk
  [10852.045313] sd 15:0:1:1: [sdh] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [10852.045631] sd 15:0:1:1: alua: port group 00 state A preferred supports tolusnA
  [10852.045730] sd 15:0:1:1: Attached scsi generic sg5 type 0
  [10852.045942] sd 15:0:1:1: [sdh] Write Protect is off
  [10852.045949] sd 15:0:1:1: [sdh] Mode Sense: ed 00 00 08
  [10852.046318] sd 15:0:1:1: [sdh] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [10852.046813] scsi 15:0:1:2: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [10852.047808] sd 15:0:1:2: alua: supports implicit TPGS
  [10852.048133] sd 15:0:1:2: alua: port group 00 rel port 30
  [10852.048358] sd 15:0:1:2: [sdi] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [10852.048520] sd 15:0:1:2: alua: port group 00 state A preferred supports tolusnA
  [10852.048643] sd 15:0:1:2: Attached scsi generic sg6 type 0
  [10852.049296]  sdh:
  [10852.049299] sd 15:0:1:2: [sdi] Write Protect is off
  [10852.049308] sd 15:0:1:2: [sdi] Mode Sense: ed 00 00 08
  [10852.049634] sd 15:0:1:2: [sdi] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [10852.049667] scsi 15:0:1:3: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [10852.050354] sd 15:0:1:3: alua: supports implicit TPGS
  [10852.050853] sd 15:0:1:3: alua: port group 00 rel port 30
  [10852.050943] sd 15:0:1:3: [sdaa] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [10852.051214] sd 15:0:1:1: [sdh] Attached SCSI disk
  [10852.051287] sd 15:0:1:3: alua: port group 00 state A preferred supports tolusnA
  [10852.051426] sd 15:0:1:3: Attached scsi generic sg7 type 0
  [10852.051646] sd 15:0:1:3: [sdaa] Write Protect is off
  [10852.051656] sd 15:0:1:3: [sdaa] Mode Sense: ed 00 00 08
  [10852.051967] sd 15:0:1:3: [sdaa] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [10852.052323] scsi 15:0:2:0: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [10852.052973] sd 15:0:2:0: alua: supports implicit TPGS
  [10852.053314] sd 15:0:2:0: alua: port group 00 rel port 100
  [10852.053406] sd 15:0:2:0: [sdab] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [10852.053686] sd 15:0:2:0: alua: port group 00 state A preferred supports tolusnA
  [10852.053892] sd 15:0:2:0: Attached scsi generic sg8 type 0
  [10852.054069] sd 15:0:2:0: [sdab] Write Protect is off
  [10852.054078] sd 15:0:2:0: [sdab] Mode Sense: f5 00 00 08
  [10852.054391] sd 15:0:2:0: [sdab] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [10852.055081] scsi 15:0:2:1: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [10852.056159] sd 15:0:2:1: alua: supports implicit TPGS
  [10852.056493] sd 15:0:2:1: alua: port group 00 rel port 100
  [10852.056574] sd 15:0:2:1: [sdac] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [10852.056997] sd 15:0:2:1: alua: port group 00 state A preferred supports tolusnA
  [10852.057274] sd 15:0:2:1: [sdac] Write Protect is off
  [10852.057280] sd 15:0:2:1: Attached scsi generic sg29 type 0
  [10852.057290] sd 15:0:2:1: [sdac] Mode Sense: ed 00 00 08
  [10852.057578] sd 15:0:2:1: [sdac] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [10852.058491] scsi 15:0:2:2: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [10852.059132]  sde:
  [10852.059173]  sdaa:
  [10852.060148] sd 15:0:2:2: alua: supports implicit TPGS
  [10852.060723] sd 15:0:2:2: alua: port group 00 rel port 100
  [10852.060814] sd 15:0:0:3: [sde] Attached SCSI disk
  [10852.060858] sd 15:0:1:3: [sdaa] Attached SCSI disk
  [10852.060942] sd 15:0:2:2: alua: rtpg failed with 8000002
  [10852.061167]  sdac:
  [10852.061313] sd 15:0:2:2: [sdad] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [10852.061363] sd 15:0:2:2: alua: port group 00 state A preferred supports tolusnA
  [10852.061615] sd 15:0:2:2: Attached scsi generic sg30 type 0
  [10852.062278] sd 15:0:2:2: [sdad] Write Protect is off
  [10852.062291] sd 15:0:2:2: [sdad] Mode Sense: ed 00 00 08
  [10852.062841] scsi 15:0:2:3: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [10852.062902] sd 15:0:2:2: [sdad] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [10852.063740] sd 15:0:2:1: [sdac] Attached SCSI disk
  [10852.063965] sd 15:0:2:3: alua: supports implicit TPGS
  [10852.064330] sd 15:0:2:0: [sdab] Attached SCSI disk
  [10852.064437] sd 15:0:2:3: alua: port group 00 rel port 100
  [10852.064507] sd 15:0:2:3: [sdae] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [10852.064927] sd 15:0:2:3: alua: port group 00 state A preferred supports tolusnA
  [10852.065231] sd 15:0:2:3: Attached scsi generic sg31 type 0
  [10852.065348] sd 15:0:2:3: [sdae] Write Protect is off
  [10852.065358] sd 15:0:2:3: [sdae] Mode Sense: ed 00 00 08
  [10852.065859] sd 15:0:2:3: [sdae] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [10852.065872]  sdad:
  [10852.065959]  sdi:
  [10852.066310] scsi 15:0:3:0: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [10852.067721] sd 15:0:2:2: [sdad] Attached SCSI disk
  [10852.067796] sd 15:0:1:2: [sdi] Attached SCSI disk
  [10852.067876] sd 15:0:3:0: alua: supports implicit TPGS
  [10852.068387] sd 15:0:3:0: [sdaf] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [10852.068426] sd 15:0:3:0: alua: port group 00 rel port 300
  [10852.068904] sd 15:0:3:0: alua: port group 00 state A preferred supports tolusnA
  [10852.069151]  sdae:
  [10852.069204] sd 15:0:3:0: Attached scsi generic sg32 type 0
  [10852.069657] sd 15:0:3:0: [sdaf] Write Protect is off
  [10852.069664] sd 15:0:3:0: [sdaf] Mode Sense: f5 00 00 08
  [10852.070344] sd 15:0:3:0: [sdaf] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [10852.070954] sd 15:0:2:3: [sdae] Attached SCSI disk
  [10852.074942] sd 15:0:3:0: [sdaf] Attached SCSI disk
  [10852.074954] scsi 15:0:3:1: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [10852.076026] sd 15:0:3:1: alua: supports implicit TPGS
  [10852.076714] sd 15:0:3:1: alua: port group 00 rel port 300
  [10852.076745] sd 15:0:3:1: [sdag] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [10852.077546] sd 15:0:3:1: alua: port group 00 state A preferred supports tolusnA
  [10852.077885] sd 15:0:3:1: Attached scsi generic sg33 type 0
  [10852.078100] sd 15:0:3:1: [sdag] Write Protect is off
  [10852.078109] sd 15:0:3:1: [sdag] Mode Sense: ed 00 00 08
  [10852.078403] sd 15:0:3:1: [sdag] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [10852.080875]  sdag:
  [10852.083603] sd 15:0:3:1: [sdag] Attached SCSI disk
  [10852.086470] scsi 15:0:3:2: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [10852.087290] sd 15:0:3:2: alua: supports implicit TPGS
  [10852.087630] sd 15:0:3:2: alua: port group 00 rel port 300
  [10852.087790] sd 15:0:3:2: [sdah] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [10852.087984] sd 15:0:3:2: alua: port group 00 state A preferred supports tolusnA
  [10852.088145] sd 15:0:3:2: Attached scsi generic sg34 type 0
  [10852.088392] sd 15:0:3:2: [sdah] Write Protect is off
  [10852.088411] sd 15:0:3:2: [sdah] Mode Sense: ed 00 00 08
  [10852.088687] sd 15:0:3:2: [sdah] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [10852.089078] scsi 15:0:3:3: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [10852.089911] sd 15:0:3:3: alua: supports implicit TPGS
  [10852.090323] sd 15:0:3:3: alua: port group 00 rel port 300
  [10852.090360] sd 15:0:3:3: [sdai] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [10852.090737] sd 15:0:3:3: alua: port group 00 state A preferred supports tolusnA
  [10852.091050] sd 15:0:3:3: Attached scsi gen0
  [12126.399029] scsi host17: Emulex LPe16000 16Gb PCIe Fibre Channel Adapter on PCI bus 01 device 00 irq 504
  [12126.959169] rpaphp: Slot [U78CA.001.CSS003P-P1-C6-C1] registered
  [12126.959176] rpadlpar_io: slot PHB 21 added
  [12127.844158] lpfc 0009:01:00.0: 1:1303 Link Up Event x1 received Data: x1 x0 x80 x0 x0 x0 0
  [12128.043085] scsi 17:0:0:0: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [12128.043688] sd 17:0:0:0: alua: supports implicit TPGS
  [12128.043969] sd 17:0:0:0: alua: port group 00 rel port 300
  [12128.044190] sd 17:0:0:0: [sda] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [12128.044311] sd 17:0:0:0: alua: port group 00 state A preferred supports tolusnA
  [12128.044430] sd 17:0:0:0: Attached scsi generic sg0 type 0
  [12128.044896] sd 17:0:0:0: [sda] Write Protect is off
  [12128.044903] sd 17:0:0:0: [sda] Mode Sense: f5 00 00 08
  [12128.045179] sd 17:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or [sdaf] Mode Sense: f5 00 00 08
  [12128.088966] sd 17:0:3:0: Attached scsi generic sg32 type 0
  [12128.089258] sd 17:0:3:0: [sdaf] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [12128.089877] sd 17:0:2:3: [sdae] Attached SCSI disk
  [12128.090349] scsi 17:0:3:1: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [12128.092029] sd 17:0:3:1: alua: supports implicit TPGS
  [12128.092495] sd 17:0:3:1: alua: port group 00 rel port 30
  [12128.092548] sd 17:0:3:1: [sdag] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [12128.092885] sd 17:0:3:1: alua: port group 00 state A preferred supports tolusnA
  [12128.093103] sd 17:0:3:1: Attached scsi generic sg33 type 0
  [12128.093172] sd 17:0:3:1: [sdag] Write Protect is off
  [12128.093183] sd 17:0:3:1: [sdag] Mode Sense: ed 00 00 08
  [12128.093503] sd 17:0:3:1: [sdag] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [12128.094183] scsi 17:0:3:2: Direct-Access     IBM    th 65:224.
  [12749.073922] device-mapper: multipath: Failing path 66:32.
  [12749.073953] device-mapper: multipath: Failing path 65:160.
  [12749.090309] scsi 17:0:0:3: alua: Detached
  [12749.094400] device-mapper: multipath: Failing path 66:16.
  [12749.118398] scsi 17:0:2:2: alua: Detached
  [12749.138317] scsi 17:0:3:2: alua: Detached
  [12749.141437] device-mapper: multipath: Failing path 65:224.
  [12749.141467] device-mapper: multipath: Failing path 66:32.
  [12749.162316] scsi 17:0:1:3: alua: Detached
  [12749.165448] device-mapper: multipath: Failing path 66:32.
  [12749.202489] scsi 17:0:2:3: alua: Detached
  [12749.238436] scsi 17:0:3:3: alua: Detached
  [12749.371720] iommu: Removing device 0009:01:00.0 from group 0
  [12759.378488] pci_bus 0009:01: busn_res: [bus 01-ff] is released
  [12759.378559] rpadlpar_io: slot PHB 21 removed
  [13405.725246] PCI host bridge /pci@800000020000015  ranges:
  [13405.725253]  MEM 0x00003fc600000000..0x00003fc67effffff -> 0x0000000080000000 l port 100
  [13408.460796] sd 19:0:2:1: [sdac] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [13408.461979] sd 19:0:2:1: [sdac] Write Protect is off
  [13408.461987] sd 19:0:2:1: [sdac] Mode Sense: ed 00 00 08
  [13408.462292] sd 19:0:2:1: [sdac] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [13408.463890] sd 19:0:2:0: [sdab] Attached SCSI disk
  [13408.465140]  sdac:
  [13408.467041] sd 19:0:2:1: [sdac] Attached SCSI disk
  [13408.569203]  sdd:
  [13408.569287]  sdi:
  [13408.570556] sd 19:0:0:2: [sdd] Attached SCSI disk
  [13408.570631] sd 19:0:1:2: [sdi] Attached SCSI disk
  [13438.697588] sd 19:0:0:1: [sdb] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [13439.326030]  rport-19:0-9: blocked FC remote port time out: removing rport
  [13453.426174] sd 19:0:0:1: [sdb] Write Protect is off
  [13453.426184] sd 19:0:0:1: [sdb] Mode Sense: ed 00 00 08
  [13453.426711] sd 19:0:0:1: [sdb] Write cache: disabled, read cache: enabled, doesn't support Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [14683.037860] scsi 21:0:0:2: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [14683.039069] sd 21:0:0:2: alua: supports implicit TPGS
  [14683.039472] sd 21:0:0:2: alua: port group 00 rel port 300
  [14683.039557] sd 21:0:0:2: [sdd] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [14683.039882] sd 21:0:0:2: alua: port group 00 state A preferred supports tolusnA
  [14683.040143] sd 21:0:0:2: Attached scsi generic sg2 type 0
  [14683.040221] sd 21:0:0:2: [sdd] Write Protect is off
  [14683.040231] sd 21:0:0:2: [sdd] Mode Sense: ed 00 00 08
  [14683.040517] sd 21:0:0:2: [sdd] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [14683.041295] scsi 21:0:0:3: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [14683.042497] sd 21:0:0:3: alua: supports implicit TPGS
  [14683.042870] sd 21:0:0:3: [sde] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [1468er: multipath: Failing path 8:16.
  [15299.302590] iommu: Removing device 000b:01:00.1 from group 0
  [15299.317740] scsi 21:0:0:0: alua: Detached
  [15299.325260] sd 21:0:2:0: [sdab] tag#0 FAILED Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
  [15299.325265] sd 21:0:2:0: [sdab] tag#0 CDB: Read(10) 28 00 02 7f ff 80 00 00 80 00
  [15299.325267] blk_update_request: I/O error, dev sdab, sector 41942912
  [15299.325275] device-mapper: multipath: Failing path 65:176.
  [15299.325296] sd 21:0:3:0: [sdae] tag#0 FAILED Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
  [15299.325298] sd 21:0:3:0: [sdae] tag#0 CDB: Read(10) 28 00 02 7f ff 80 00 00 80 00
  [15299.325301] blk_update_request: I/O error, dev sdae, sector 41942912
  [15299.325309] device-mapper: multipath: Failing path 65:224.
  [15299.353733] scsi 21:0:1:0: alua: Detached
  [15299.361265] sd 21:0:3:1: [sdaf] tag#0 FAILED Result: hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK
  [15299.361269] sd 21:0:3:1: [sdaf] tag#0 CDB958.559922] scsi 23:0:1:1: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [15958.560721] sd 23:0:1:1: alua: supports implicit TPGS
  [15958.561066] sd 23:0:1:1: alua: port group 00 rel port 100
  [15958.561092] sd 23:0:1:1: [sdh] 41943040 512-byte logical blocks: (21.5 GB/20.0 GiB)
  [15958.561697] sd 23:0:1:1: alua: port group 00 state A preferred supports tolusnA
  [15958.561880] sd 23:0:1:1: Attached scsi generic sg5 type 0
  [15958.561951] sd 23:0:1:1: [sdh] Write Protect is off
  [15958.561958] sd 23:0:1:1: [sdh] Mode Sense: ed 00 00 08
  [15958.562239] sd 23:0:1:1: [sdh] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
  [15958.562942] scsi 23:0:1:2: Direct-Access     IBM      2107900          .149 PQ: 0 ANSI: 5
  [15958.563577] sd 23:0:1:0: [sdg] Attached SCSI disk
  [15958.563921] sd 23:0:1:2: alua: supports implicit TPGS
  [15958.564270] sd 23:0:1:2: alua: port group 00 rel port 100
  [15958.564443] sd 23:0:1:2: [sdi] 41943040 512-byte
  d000000003a37530  7c6307b4      extsw   r3,r3
  d000000003a37534  38210030      addi    r1,r1,48
  d000000003a37538  e8010010      ld      r0,16(r1)
  d000000003a3753c  ebc1fff0      ld      r30,-16(r1)
  d000000003a37540  ebe1fff8      ld      r31,-8(r1)
  d000000003a37544  7c0803a6      mtlr    r0
  d000000003a37548  4e800020      blr
  d000000003a3754c  60420000      ori     r2,r2,0
  d000000003a37550  813f0b14      lwz     r9,2836(r31)
  d000000003a37554  2b890001      cmplwi  cr7,r9,1
  d000000003a37558  409dffa8      ble     cr7,d000000003a37500    # lpfc_sli4_scmd_to_wqidx_distr+0x50/0x100 [lpfc]
  d000000003a3755c  a14d0008      lhz     r10,8(r13)
  d000000003a37560  a13f0572      lhz     r9,1394(r31)
  d000000003a37564  7f895000      cmpw    cr7,r9,r10
  d000000003a37568  409dff98      ble     cr7,d000000003a37500    # lpfc_sli4_scmd_to_wqidx_distr+0x50/0x100 [lpfc]
  d000000003a3756c  e93f0568      ld      r9,1384(r31)
  e:mon>

  ---Steps to Reproduce---
   cd /kte/tools
  ./setup dlar
  cd /kte/tools/dlpar 
  ./start.dlpar -d 0

  Doing some analysis without a crashdump, as it's taking too long.

  This looks like a NULL pointer dereference, 
  if my assembly reading/matching to C is correct.

  Would need to understand why/how this
  '(struct scsi_cmnd *cmnd)->device' field is NULL.

  Analysis
  --------

  From xmon:
  	pc: <...>: lpfc_sli4_scmd_to_wqidx_distr+0x30/0x100 [lpfc]

          R10 = 0000000000000000   R26 = 0000000000000001

  From 'objdump -d /usr/lib/debug/<...>/lpfc.ko',

  	lpfc_sli4_scmd_to_wqidx_distr      = 674b0
  	lpfc_sli4_scmd_to_wqidx_distr+0x30 = 674e0 (crash)

          and

  	00000000000674b0 <lpfc_sli4_scmd_to_wqidx_distr>:
  	<...>
  	   674cc:       78 23 9e 7c     mr      r30,r4
  	   674d0:       78 1b 7f 7c     mr      r31,r3
  	<...>
  	   674dc:       10 00 5e e9     ld      r10,16(r30)
  	   674e0:       00 00 2a e9     ld      r9,0(r10)  <<-- (crash)
  	   674e4:       00 00 29 e9     ld      r9,0(r9)
  	<...>

  This is the relevant snippet of code:

  From Ubuntu 16.04 kernel 4.4.0-22.40 [1]

  	int lpfc_sli4_scmd_to_wqidx_distr(struct lpfc_hba *phba,
  					  struct lpfc_scsi_buf *lpfc_cmd)
  	{
  		struct scsi_cmnd *cmnd = lpfc_cmd->pCmd;
  	<...>
  		if (shost_use_blk_mq(cmnd->device->host)) {
  	<...>

  
  So, back to the assembly, this seems the 2 function parameters
  passed by register (r4, r3), loaded into other registers (r30, r31).

  	   674cc:       78 23 9e 7c     mr      r30,r4
  	   674d0:       78 1b 7f 7c     mr      r31,r3

  Per the load below, r10 is *cmnd, and r30 is *lpfc_cmd;  it loads
  lpfc_cmd->pCmd, which has offset 16 bytes into struct lpfc_cmd [2]
  (after 2 pointers * 8-bytes each, from list_head list [3])

             674dc:       10 00 5e e9     ld      r10,16(r30)

  	struct lpfc_scsi_buf {
  		struct list_head list;
  		struct scsi_cmnd *pCmd;
  	<...>

  	struct list_head {
  		struct list_head *next, *prev;
  	};

  And the load below hits the crash, because it dereferences r10 (*cmnd)
  which is zero:

             674e0:       00 00 2a e9     ld      r9,0(r10)  <<--
  (crash)

          From xmon:

          R10 = 0000000000000000   R26 = 0000000000000001

  That deference was for cmnd->device; you can see the load instruction immediately
  afterward would further dereference the cmnd->device pointer, for the device->host field,
  which has offset 0 into struct scsi_device [4]:
   --- this is a confirmantion that the assembly/C matching looks correct.

                  if (shost_use_blk_mq(cmnd->device->host)) {

  	struct scsi_device {
  		struct Scsi_Host *host;
  	<...>

  
  [1] http://kernel.ubuntu.com/git/ubuntu/ubuntu-xenial.git/tree/drivers/scsi/lpfc/lpfc_scsi.c?h=Ubuntu-4.4.0-22.40
  [2] http://kernel.ubuntu.com/git/ubuntu/ubuntu-xenial.git/tree/drivers/scsi/lpfc/lpfc_scsi.h?h=Ubuntu-4.4.0-22.40#n130
  [3] http://kernel.ubuntu.com/git/ubuntu/ubuntu-xenial.git/tree/include/linux/types.h?h=Ubuntu-4.4.0-22.40#n185
  [4] http://kernel.ubuntu.com/git/ubuntu/ubuntu-xenial.git/tree/include/scsi/scsi_device.h?h=Ubuntu-4.4.0-22.40#n77

  One detail missing..

  (In reply to comment #16)
  > And the load below hits the crash, because it dereferences r10 (*cmnd) which
  > is zero:
  > 
  > 	   674e0:       00 00 2a e9     ld      r9,0(r10)  <<-- (crash)
  > 
  > 	From xmon:
  > 
  > 	R10 = 0000000000000000   R26 = 0000000000000001
  > 
  > That deference was for cmnd->device;  [snip]

  which has offset zero into struct scsi_cmnd [5]

          if (shost_use_blk_mq(cmnd->device->host)) {

          struct scsi_cmnd {
  	        struct scsi_device *device;
          <...>

  [5] http://kernel.ubuntu.com/git/ubuntu/ubuntu-
  xenial.git/tree/include/scsi/scsi_cmnd.h?h=Ubuntu-4.4.0-22.40#n59

  > Would need to understand why/how this
  > '(struct scsi_cmnd *cmnd)->device' field is NULL.

  Checking this again today, it occurred to me that the problem is
  actually *cmnd == NULL, and dereferencing *cmnd (for cmnd->device)
  hits the crash.

  Fix submitted upstream:
      http://marc.info/?l=linux-scsi&m=146534119707379&w=2

  I didn't provide a test kernel because the system was running
  regression tests over weekend, and it takes long to reproduce the
  problem w/ DLPAR operations -- but the same problem could be
  reproduced w/ simpler test-cases (see commit), so I worked it in the
  background.

  If you keep hitting this, let me know and I'll provide a test kernel.

  Hi Mauricio,
  Installed the fix and verified. Now  am not finding the issue.

  Hi Canonical,

  Can you consider picking up this fix that has not yet made the
  upstream kernel?

  It's fairly obvious and trivial, very documented in the commit message
  (w/ test-cases), and has been tested successfully here in IBM (also
  see commit msg).

  http://marc.info/?l=linux-scsi&m=146534119707379&w=2

  The adapter vendor's team has not yet reviewed it on the mailing list
  (and no other patches for lpfc), so I guess it'll take some time until
  this makes in.

  Is that possible?

  Thanks

  Mauricio

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1597974/+subscriptions