kernel-packages team mailing list archive

[Stefan Bader <stefan.bader@xxxxxxxxxxxxx>]

Writing up my findings so far:
- this looks to be a m1.small instance (1 vcpu, 1.7GB memory)
- this instance type is a PV guest

So sched_clock (which is used by printk) will be using
paravirt_sched_clock which is set to xen_clocksource_read. The time info
is kept in a per-cpu variable (vcpu_info). This is initially part of the
hypervisor shared info but get changed later.

start_kernel()
  ...
  setup_per_cpu_areas();
  /* printks from above are ok */
  smp_prepare_boot_cpu();
    xen_smp_prepare_boot_cpu();
      xen_setup_vcpu_info_placement();
      /* vcpu_info changed */
      xen_init_spinlocks();
      /* printks from above call jumped in time */

So it looks like for some yet unknown reason, registering the vcpu_info
with the hypervisor is messing up the clocktime data.

@Joe, was this always happening (with the same instance type maybe) or
about how often. I am trying to get a feeling for whether it is some
race or maybe related to xen hypervisor version. Also will try to play
around with this on a local Xen host (but rather tomorrow).

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1349883

Title:
  dmesg time wildly incorrect on paravirtual EC2 instances.

Status in “linux” package in Ubuntu:
  Confirmed

Bug description:
  On Ubuntu 14.04 EC2 instances, during startup the dmesg time goes from
  0 seconds to thousands of seconds. This causes kernel logs to be
  recorded for months in the future.

  This appears to be related to the kernel trusting the TSC clock on Xen servers, as described here:
  https://forums.aws.amazon.com/thread.jspa?threadID=59753

  But I am not sure that is the cause.

  ProblemType: Bug
  DistroRelease: Ubuntu 14.04
  Package: linux-image-3.13.0-32-generic 3.13.0-32.57
  ProcVersionSignature: Ubuntu 3.13.0-32.57-generic 3.13.11.4
  Uname: Linux 3.13.0-32-generic x86_64
  AlsaDevices:
   total 0
   crw-rw---- 1 root audio 116,  1 Jul 29 14:41 seq
   crw-rw---- 1 root audio 116, 33 Jul 29 14:41 timer
  AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
  ApportVersion: 2.14.1-0ubuntu3.2
  Architecture: amd64
  ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
  AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
  CRDA: Error: [Errno 2] No such file or directory: 'iw'
  CurrentDmesg: [7562292.301737] init: plymouth-upstart-bridge main process ended, respawning
  Date: Tue Jul 29 14:47:54 2014
  Ec2AMI: ami-00dc1368
  Ec2AMIManifest: (unknown)
  Ec2AvailabilityZone: us-east-1c
  Ec2InstanceType: m1.small
  Ec2Kernel: aki-919dcaf8
  Ec2Ramdisk: unavailable
  IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
  Lspci:
   
  Lsusb: Error: [Errno 2] No such file or directory: 'lsusb'
  PciMultimedia:
   
  ProcEnviron:
   TERM=xterm
   PATH=(custom, no user)
   XDG_RUNTIME_DIR=<set>
   LANG=en_US.UTF-8
   SHELL=/bin/bash
  ProcFB:
   
  ProcKernelCmdLine: root=LABEL=cloudimg-rootfs ro console=hvc0
  RelatedPackageVersions:
   linux-restricted-modules-3.13.0-32-generic N/A
   linux-backports-modules-3.13.0-32-generic  N/A
   linux-firmware                             N/A
  RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
  SourcePackage: linux
  UpgradeStatus: No upgrade log present (probably fresh install)
  WifiSyslog:
  --- 
  AlsaDevices:
   total 0
   crw-rw---- 1 root audio 116,  1 Jul 29 14:41 seq
   crw-rw---- 1 root audio 116, 33 Jul 29 14:41 timer
  AplayDevices: Error: [Errno 2] No such file or directory
  ApportVersion: 2.14.1-0ubuntu3.2
  Architecture: amd64
  ArecordDevices: Error: [Errno 2] No such file or directory
  AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
  CRDA: Error: [Errno 2] No such file or directory
  CurrentDmesg: [7562292.301737] init: plymouth-upstart-bridge main process ended, respawning
  DistroRelease: Ubuntu 14.04
  Ec2AMI: ami-00dc1368
  Ec2AMIManifest: (unknown)
  Ec2AvailabilityZone: us-east-1c
  Ec2InstanceType: m1.small
  Ec2Kernel: aki-919dcaf8
  Ec2Ramdisk: unavailable
  IwConfig: Error: [Errno 2] No such file or directory
  Lspci:
   
  Lsusb: Error: [Errno 2] No such file or directory
  Package: linux (not installed)
  PciMultimedia:
   
  ProcEnviron:
   TERM=xterm
   PATH=(custom, no user)
   XDG_RUNTIME_DIR=<set>
   LANG=en_US.UTF-8
   SHELL=/bin/bash
  ProcFB:
   
  ProcKernelCmdLine: root=LABEL=cloudimg-rootfs ro console=hvc0
  ProcVersionSignature: Ubuntu 3.13.0-32.57-generic 3.13.11.4
  RelatedPackageVersions:
   linux-restricted-modules-3.13.0-32-generic N/A
   linux-backports-modules-3.13.0-32-generic  N/A
   linux-firmware                             N/A
  RfKill: Error: [Errno 2] No such file or directory
  Tags:  trusty ec2-images
  Uname: Linux 3.13.0-32-generic x86_64
  UpgradeStatus: No upgrade log present (probably fresh install)
  UserGroups: adm admin ssh sudo
  WifiSyslog:
   
  _MarkForUpload: True

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1349883/+subscriptions