← Back to team overview

kernel-packages team mailing list archive

[Bug 1349883] Re: dmesg time wildly incorrect on paravirtual EC2 instances.

 

When inspecting the vcpu info on a local PV guest and a EC2 guest provided by Joe, I found that:
 
 time = {
    version = 33456, 
    pad0 = 0, 
    tsc_timestamp = 86813831140545, 
    system_time = 37805107510066, 
    tsc_to_system_mul = 3743284562, 
    tsc_shift = -1 '\377', 
    flags = 0 '\000', 
    pad = "\000"
}

So system_time in both cases seems to be the uptime in nanoseconds of
the Xen host. My machine was up for about 10hrs. In the PV guest the
dmesg timestamps however are somehow corrected to be in agreement of the
guests uptime.

time = {
    version = 19934,
    pad0 = 0,
    tsc_timestamp = 8365090813767560,
    system_time = 4647273524058147,
    tsc_to_system_mul = 2386093295,
    tsc_shift = 0 '\000',
    flags = 0 '\000',
    pad = "\000"
}

The structure on the ec2 instance also seems to contain the uptime of
the Xen host (which is around 53 days). Though here, the timestamps in
dmesg and the value of system_time are the same. So the guest suddenly
got the uptime of the Xen host.

The only way this makes sense is that the delta (which is calculated
from (rdtsc - tsc_timestamp) with the shift and mul values should
normally be offset host to guest. Not sure how yet. Oh and one piece of
information from the irc conversation: apparently this may not happen on
first boot but on reboot, but once it happened it will always happen. So
far only observed with Intel CPUs.

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1349883

Title:
  dmesg time wildly incorrect on paravirtual EC2 instances.

Status in “linux” package in Ubuntu:
  Confirmed

Bug description:
  On Ubuntu 14.04 EC2 instances, during startup the dmesg time goes from
  0 seconds to thousands of seconds. This causes kernel logs to be
  recorded for months in the future.

  This appears to be related to the kernel trusting the TSC clock on Xen servers, as described here:
  https://forums.aws.amazon.com/thread.jspa?threadID=59753

  But I am not sure that is the cause.

  ProblemType: Bug
  DistroRelease: Ubuntu 14.04
  Package: linux-image-3.13.0-32-generic 3.13.0-32.57
  ProcVersionSignature: Ubuntu 3.13.0-32.57-generic 3.13.11.4
  Uname: Linux 3.13.0-32-generic x86_64
  AlsaDevices:
   total 0
   crw-rw---- 1 root audio 116,  1 Jul 29 14:41 seq
   crw-rw---- 1 root audio 116, 33 Jul 29 14:41 timer
  AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
  ApportVersion: 2.14.1-0ubuntu3.2
  Architecture: amd64
  ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
  AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
  CRDA: Error: [Errno 2] No such file or directory: 'iw'
  CurrentDmesg: [7562292.301737] init: plymouth-upstart-bridge main process ended, respawning
  Date: Tue Jul 29 14:47:54 2014
  Ec2AMI: ami-00dc1368
  Ec2AMIManifest: (unknown)
  Ec2AvailabilityZone: us-east-1c
  Ec2InstanceType: m1.small
  Ec2Kernel: aki-919dcaf8
  Ec2Ramdisk: unavailable
  IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
  Lspci:
   
  Lsusb: Error: [Errno 2] No such file or directory: 'lsusb'
  PciMultimedia:
   
  ProcEnviron:
   TERM=xterm
   PATH=(custom, no user)
   XDG_RUNTIME_DIR=<set>
   LANG=en_US.UTF-8
   SHELL=/bin/bash
  ProcFB:
   
  ProcKernelCmdLine: root=LABEL=cloudimg-rootfs ro console=hvc0
  RelatedPackageVersions:
   linux-restricted-modules-3.13.0-32-generic N/A
   linux-backports-modules-3.13.0-32-generic  N/A
   linux-firmware                             N/A
  RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
  SourcePackage: linux
  UpgradeStatus: No upgrade log present (probably fresh install)
  WifiSyslog:
  --- 
  AlsaDevices:
   total 0
   crw-rw---- 1 root audio 116,  1 Jul 29 14:41 seq
   crw-rw---- 1 root audio 116, 33 Jul 29 14:41 timer
  AplayDevices: Error: [Errno 2] No such file or directory
  ApportVersion: 2.14.1-0ubuntu3.2
  Architecture: amd64
  ArecordDevices: Error: [Errno 2] No such file or directory
  AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
  CRDA: Error: [Errno 2] No such file or directory
  CurrentDmesg: [7562292.301737] init: plymouth-upstart-bridge main process ended, respawning
  DistroRelease: Ubuntu 14.04
  Ec2AMI: ami-00dc1368
  Ec2AMIManifest: (unknown)
  Ec2AvailabilityZone: us-east-1c
  Ec2InstanceType: m1.small
  Ec2Kernel: aki-919dcaf8
  Ec2Ramdisk: unavailable
  IwConfig: Error: [Errno 2] No such file or directory
  Lspci:
   
  Lsusb: Error: [Errno 2] No such file or directory
  Package: linux (not installed)
  PciMultimedia:
   
  ProcEnviron:
   TERM=xterm
   PATH=(custom, no user)
   XDG_RUNTIME_DIR=<set>
   LANG=en_US.UTF-8
   SHELL=/bin/bash
  ProcFB:
   
  ProcKernelCmdLine: root=LABEL=cloudimg-rootfs ro console=hvc0
  ProcVersionSignature: Ubuntu 3.13.0-32.57-generic 3.13.11.4
  RelatedPackageVersions:
   linux-restricted-modules-3.13.0-32-generic N/A
   linux-backports-modules-3.13.0-32-generic  N/A
   linux-firmware                             N/A
  RfKill: Error: [Errno 2] No such file or directory
  Tags:  trusty ec2-images
  Uname: Linux 3.13.0-32-generic x86_64
  UpgradeStatus: No upgrade log present (probably fresh install)
  UserGroups: adm admin ssh sudo
  WifiSyslog:
   
  _MarkForUpload: True

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1349883/+subscriptions


References