yahoo-eng-team team mailing list archive

[OpenStack Infra <1834506@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/667976
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=8f975bc8287d980f3e6c5da601051cf626c081dd
Submitter: Zuul
Branch:    master

commit 8f975bc8287d980f3e6c5da601051cf626c081dd
Author: Miguel Herranz <miguel@xxxxxxxxxxxx>
Date:   Thu Jun 27 15:43:27 2019 +0200

    Add support for cloud-init on LXC instances
    
    Images that use cloud-init are not correctly initialized when using
    libvirt LXC nova driver.
    
    One way cloud-init checks if the OpenStack datasource should be used
    is by checking DMI data that is meaningful for virtual machines but
    not for containers.
    
    Another way cloud-init is using is to check if the 'product_name'
    env variable for init process (PID 1) is "OpenStack Nova" [1][2].
    
    This commit add that env variable to the instance when the driver
    is LXC.
    
    [1] https://cloudinit.readthedocs.io/en/latest/topics/datasources/openstack.html
    [2] https://git.launchpad.net/cloud-init/tree/tools/ds-identify#n974
    
    Closes-Bug: 1834506
    
    Change-Id: I2d0a4461081f5284d16df73a783cb7dae3ff0ef5
    Signed-off-by: Miguel Herranz <miguel@xxxxxxxxxxxx>


** Changed in: nova
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1834506

Title:
  cloud-init based images not working on LXC instances

Status in OpenStack Compute (nova):
  Fix Released

Bug description:
  Description
  ===========

  Apparently cloud-init based images are unable to correctly initialize
  in a OpenStack compute using LXC.

  Steps to reproduce:
  ==================

  Install a OpenStack using Libvirt + LXC (nova-compute-lxc package in
  Ubuntu) and launch a cloud-init based instance.

  This has been tested with an Ubuntu cloud image (after setting a
  default root password to allow console login, see [2]):

  ```
    wget http://uec-images.ubuntu.com/releases/18.04/release/ubuntu-18.04-server-cloudimg-amd64.tar.gz # md5sum f90bf979eab74f18bd4400a7355157dc, downloaded on 2019-06-18
    tar zxfv ubuntu-18.04-server-cloudimg-amd64.tar.gz
    virt-customize -a bionic-server-cloudimg-amd64.img --root-password password:gocubsgo
    openstack image create "ubuntu_with_root_password" --file bionic-server-cloudimg-amd64.img --disk-format raw --container-format bare --public
  ```

  NOTE: since cloud-init usually check the DMI product name (see [1]),
  in case that the host is an OpenStack compute host or instance itself,
  the DMI product name may be passed to the LXC instance, so cloud-init
  would use OpenStack datasource as expected, but for wrong reasons. To
  avoid that posibility, it may be better to reproduce in a non
  OpenStack based host.

  Environment info:

    Host: AWS instance (t2.large)
    OS: Ubuntu 18.04.2 LTS
    Kernel: Linux 4.15.0-1041-aws #43-Ubuntu SMP x86_64
    Openstack: queens

    Related packages versions:

    nova-api: Installed: 2:17.0.9-0ubuntu3
    python-nova: Installed: 2:17.0.9-0ubuntu3
    nova-compute: Installed: 2:17.0.9-0ubuntu3
    nova-compute-lxc: Installed: 2:17.0.9-0ubuntu3

  Expected result
  ===============
  The instance should have a working network configuration and be accesible, etc.

  Actual result
  =============

  The instance is created and is running, but is not accessible by
  network (neither using the router network namespace nor by assigning
  an floating IP).

  It can be accessed with 'virsh -c lxc:/// console instance-<number>'
  using root/gocubsgo previously setted:

  ```
  Ubuntu 18.04.2 LTS ubuntu console

  ubuntu login: root
  Password: 
  run-parts: /etc/update-motd.d/98-fsck-at-reboot exited with return code 1

  The programs included with the Ubuntu system are free software;
  the exact distribution terms for each program are described in the
  individual files in /usr/share/doc/*/copyright.

  Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
  applicable law.

  root@ubuntu:~# ip addr
  1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
      link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
      inet 127.0.0.1/8 scope host lo
         valid_lft forever preferred_lft forever
      inet6 ::1/128 scope host 
         valid_lft forever preferred_lft forever
  19: eth0@if20: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
      link/ether fa:16:3e:b3:25:0c brd ff:ff:ff:ff:ff:ff link-netnsid 0
  root@ubuntu:~# 
  ```

  It can  be seen that the network is not configured.

  The logs of cloud-init show that no datasource was found so it
  disabled cloud-init start.

  Logs & Configs
  ==============

  ```
  root@ubuntu:~# cat /run/cloud-init/.ds-identify.result 
  1
  ```

  ```
  root@ubuntu:~# cat /run/cloud-init/ds-identify.log 
  [up 22612.42s] ds-identify 
  policy loaded: mode=search report=false found=all maybe=all notfound=disabled
  /etc/cloud/cloud.cfg.d/90_dpkg.cfg set datasource_list: [ NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, None ]
  DMI_PRODUCT_NAME=HVM domU
  DMI_SYS_VENDOR=Xen
  DMI_PRODUCT_SERIAL=ec21a6db-7988-521e-8fe1-eaed834f9b54
  DMI_PRODUCT_UUID=EC21A6DB-7988-521E-8FE1-EAED834F9B54
  PID_1_PRODUCT_NAME=unavailable
  DMI_CHASSIS_ASSET_TAG=
  FS_LABELS=unavailable:container
  ISO9660_DEVS=unavailable:container
  KERNEL_CMDLINE=/sbin/init 
  VIRT=lxc-libvirt
  UNAME_KERNEL_NAME=Linux
  UNAME_KERNEL_RELEASE=4.15.0-1041-aws
  UNAME_KERNEL_VERSION=#43-Ubuntu SMP Thu Jun 6 13:39:11 UTC 2019
  UNAME_MACHINE=x86_64
  UNAME_NODENAME=ubuntu
  UNAME_OPERATING_SYSTEM=GNU/Linux
  DSNAME=
  DSLIST=NoCloud ConfigDrive OpenNebula DigitalOcean Azure AltCloud OVF MAAS GCE OpenStack CloudSigma SmartOS Bigstep Scaleway AliYun Ec2 CloudStack Hetzner IBMCloud None
  MODE=search
  ON_FOUND=all
  ON_MAYBE=all
  ON_NOTFOUND=disabled
  pid=24 ppid=5
  is_container=true
  is_ds_enabled(IBMCloud) = true.
  is_ds_enabled(IBMCloud) = true.
  No ds found [mode=search, notfound=disabled]. Disabled cloud-init [1]
  [up 22612.47s] returning 1
  root@ubuntu:~# cat /run/cloud-init/
  .ds-identify.result       cloud.cfg                 
  cloud-init-generator.log  ds-identify.log           
  ```

  ```
  root@ubuntu:~# cat /run/cloud-init/cloud.cfg
  di_report:
    datasource_list: [  ]
    # reporting not found result. notfound=disabled.
  ```

  ```
  root@ubuntu:~# cat /run/cloud-init/cloud-init-generator.log 
  /lib/systemd/system-generators/cloud-init-generator normal=/run/systemd/generator early=/run/systemd/generator.early late=/run/systemd/generator.late
  kernel command line (container[lxc-libvirt]: pid 1 cmdline not available): 
  kernel_cmdline found unset
  etc_file found unset
  default found enabled
  checking for datasource
  ds-identify rc=1
  ds-identify _RET=notfound
  cloud-init is enabled but no datasource found, disabling
  already disabled: no change needed [no /run/systemd/generator.early/multi-user.target.wants/cloud-init.target]
  ```

  Related bugs:
  =============

    https://bugs.launchpad.net/nova/+bug/1693524

  References
  ==========

  [1] https://cloudinit.readthedocs.io/en/latest/topics/datasources/openstack.html)
  [2] https://serverascode.com/2018/06/26/using-cloud-images.html

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1834506/+subscriptions