kernel-packages team mailing list archive

[Joseph Salisbury <joseph.salisbury@xxxxxxxxxxxxx>]

I confirmed that the reproduce is repeatable.  I can reproduce the bug
within 30 to 60 minutes.  The bug can be reproduce with the following
steps:


1. Create a Virtual machine with 1 cpu and 2048M of memory.

2. Install Ubuntu on the VM.  I installed 15.10(Wily).

3. Install the linux-cloud-tools package to get the VSS Snapshot daemon:
	'sudo apt-get update'
	'sudo apt-get install linux-cloud-tools'

4. Create a backup schedule for this VM to run every 30 minutes.  In the
advance settings I set the VSS Settings to "VSS copy Backup".  I also
configured the backup to backup to it's own hard disk.  The machine I'm
using only has two hard disks.  One dedicated to Windows Server 2012 R2
and now the other for backups.

5. Download the tiobench .deb file from:
	http://packages.ubuntu.com/precise/amd64/tiobench/download

6. Install tiobench with 'sudo dpkg -i tiobench_0.3.3-5_amd64.deb'

7. Download the wrapper script that runs tiotest with the right IO pattern from bug report:
	https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+attachment/4529139/+files/run_tiotest.sh
	
8. Create a working directory for tiotest in the same directory where the run_tiotest.sh script was saved and will be run from:
	'mkdir tiotest-working-dir'

9.Ensure run_tiotest.sh script is executable:
	'chmod 755 run_tiotest.sh'

10. Start tiotest by running wrapper script: ./run_tiotest.sh


The bug will is reproduce when one of the backups is running.  To speed things up, you could also try running "Backup once", otherwise just wait for the backups to start on their own.

The wrapper script is starting three instances of tiotest with different
block sizes.  My test machine has only once hard disk, so three
instances of tiotest will cause the disk to be 60% to 100% utilized.
This can be seen with iostat, which you can get by installing the
sysstat package.  If your test machine has more than once disk, more
instances of tiotest may be needed in the wrapper script to increase the
IO load.

Just let me know if none of the steps are clear.  Now that I can
reproduce the bug, I'll dig deeper and see if this is caused by a
specific kernel.

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1470250

Title:
  [Hyper-V] Ubuntu 14.04.2 LTS Generation 2 SCSI Errors on VSS Based
  Backups

Status in linux package in Ubuntu:
  In Progress
Status in linux source package in Trusty:
  In Progress
Status in linux source package in Utopic:
  In Progress
Status in linux source package in Vivid:
  In Progress
Status in linux source package in Wily:
  In Progress

Bug description:
  Customers have reported running various versions of Ubuntu 14.04.2 LTS
  on Generation 2 Hyper-V Hosts.    On a random Basis, the file system
  will be mounted Read-Only due to a "disk error" (which really isn't
  the case here).    As a result, they must reboot the Ubuntu guest to
  get the file system to mount RW again.

  The Error seen are the following:
  Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968142] storvsc: Sense Key : Unit Attention [current] 
  Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968145] storvsc: Add. Sense: Changed operating definition
  Apr 30 00:02:01 balticnetworkstraining kernel: [640153.968161] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed. The Linux SCSI layer does not automatically adjust these parameters.
  Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584164] hv_storvsc vmbus_0_4: cmd 0x2a scsi status 0x2 srb status 0x82
  Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584178] hv_storvsc vmbus_0_4: stor pkt ffff88006eb6c700 autosense data valid - len 18
  Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584180] storvsc: Sense Key : Unit Attention [current] 
  Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584183] storvsc: Add. Sense: Changed operating definition
  Apr 30 01:23:26 balticnetworkstraining kernel: [645039.584198] sd 0:0:0:0: Warning! Received an indication that the operating parameters on this target have changed.  The Linux SCSI layer does not automatically adjust these parameters.

  This relates to the VSS "Windows Server Backup" process that kicks off at midnight on the host and finishes an hour and half later.   
  Yes, we do have hv_vss_daemon and hv_kvp_daemon running for the correct kernel version we have.   We're currently running kernel version 3.13.0-49-generic #83 on one system and 3.16.0-34-generic #37 on the other. -- We see the same errors on both.
  As a result, we've been hesitant to drop any more ubuntu guests on our 2012R2 hyper-v system because of this.   We can stop the backup process and all is good, but we need nightly backups to image all of our VM's.   All the windows guests have no issues of course.   We also have some CentOS based guests running without issues from what we've seen.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1470250/+subscriptions