yahoo-eng-team team mailing list archive

[Silvan Kaiser <silvan@xxxxxxxxxxx>]

Public bug reported:

The core issue are console.log files with ownership root:root and
permissions 600 during migration where a target hosts nova process is
unable to access these console.log files.

The setup is kilo version OpenStack with Quobyte backend running with
nas_secure_permissions and nas_secure_ownership set to true and libvirts
qemu.conf set to dynamic_ownership=0 and qemu running as nova:cinder
user:group.

After testing the different types migration (cold migration and live migration) i see two scenarios so far that produce a console.log file with the afore mentioned access issue:
1) cold migration on a stopped instance who's image resides in a (Quobyte) Cinder volume
2) live migrations of instances who's images reside either in ephemeral storage or in a (Quobyte) Cinder volume

On all occasions all files related to the instance are found to have
ownership nova:nova with permissions 644 with the sole exception of the
console.log file, e.g.:

[root@server06 log]# ls -lah /prod/openstack-nova/instances/0dfdfd1e-7f5e-47ea-a63c-06c4f84ae2d8*
total 22M
drwxr-xr-x. 1 nova nova   0 Feb 25 14:05 .
drwxrwxrwx. 1 root root   0 Feb 25 14:05 ..
-rw-------. 1 root root   0 Feb 25 14:05 console.log
-rw-r--r--. 1 nova nova 22M Feb 25 14:03 disk

Libvirt does not have a detection for Quobyte as a shared filesystem at
this point and thus believes the files to reside on a local filesystem.

Now i'm trying to find out 
1) Who creates console.log at which point (so i can check where the ownership & permissions for that are taken from)
2) Who manipulates console.log during migration (so i can check for the same as above)

General observation: libvirts qemu settings for dynamic ownership and
user:group do work as the other files are managed according to these
settings but console.log seems to be an exception (maybe not touched by
libvirt at all? Is Nova setting this?)

Any help appreciated.

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: cinder libvirt quobyte volume

** Tags added: cinder volume

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1549828

Title:
  permission denied on console.log during some migrations

Status in OpenStack Compute (nova):
  New

Bug description:
  The core issue are console.log files with ownership root:root and
  permissions 600 during migration where a target hosts nova process is
  unable to access these console.log files.

  The setup is kilo version OpenStack with Quobyte backend running with
  nas_secure_permissions and nas_secure_ownership set to true and
  libvirts qemu.conf set to dynamic_ownership=0 and qemu running as
  nova:cinder user:group.

  After testing the different types migration (cold migration and live migration) i see two scenarios so far that produce a console.log file with the afore mentioned access issue:
  1) cold migration on a stopped instance who's image resides in a (Quobyte) Cinder volume
  2) live migrations of instances who's images reside either in ephemeral storage or in a (Quobyte) Cinder volume

  On all occasions all files related to the instance are found to have
  ownership nova:nova with permissions 644 with the sole exception of
  the console.log file, e.g.:

  [root@server06 log]# ls -lah /prod/openstack-nova/instances/0dfdfd1e-7f5e-47ea-a63c-06c4f84ae2d8*
  total 22M
  drwxr-xr-x. 1 nova nova   0 Feb 25 14:05 .
  drwxrwxrwx. 1 root root   0 Feb 25 14:05 ..
  -rw-------. 1 root root   0 Feb 25 14:05 console.log
  -rw-r--r--. 1 nova nova 22M Feb 25 14:03 disk

  Libvirt does not have a detection for Quobyte as a shared filesystem
  at this point and thus believes the files to reside on a local
  filesystem.

  Now i'm trying to find out 
  1) Who creates console.log at which point (so i can check where the ownership & permissions for that are taken from)
  2) Who manipulates console.log during migration (so i can check for the same as above)

  General observation: libvirts qemu settings for dynamic ownership and
  user:group do work as the other files are managed according to these
  settings but console.log seems to be an exception (maybe not touched
  by libvirt at all? Is Nova setting this?)

  Any help appreciated.

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1549828/+subscriptions