yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #79073
[Bug 1834506] [NEW] cloud-init based images not working on LXC instances
Public bug reported:
Description
===========
Apparently cloud-init based images are unable to correctly initialize in
a OpenStack compute using LXC.
Steps to reproduce:
==================
Install a OpenStack using Libvirt + LXC (nova-compute-lxc package in
Ubuntu) and launch a cloud-init based instance.
This has been tested with an Ubuntu cloud image (after setting a default
root password to allow console login, see [2]):
```
wget http://uec-images.ubuntu.com/releases/18.04/release/ubuntu-18.04-server-cloudimg-amd64.tar.gz # md5sum f90bf979eab74f18bd4400a7355157dc, downloaded on 2019-06-18
tar zxfv ubuntu-18.04-server-cloudimg-amd64.tar.gz
virt-customize -a bionic-server-cloudimg-amd64.img --root-password password:gocubsgo
openstack image create "ubuntu_with_root_password" --file bionic-server-cloudimg-amd64.img --disk-format raw --container-format bare --public
```
NOTE: since cloud-init usually check the DMI product name (see [1]), in
case that the host is an OpenStack compute host or instance itself, the
DMI product name may be passed to the LXC instance, so cloud-init would
use OpenStack datasource as expected, but for wrong reasons. To avoid
that posibility, it may be better to reproduce in a non OpenStack based
host.
Environment info:
Host: AWS instance (t2.large)
OS: Ubuntu 18.04.2 LTS
Kernel: Linux 4.15.0-1041-aws #43-Ubuntu SMP x86_64
Openstack: queens
Related packages versions:
nova-api: Installed: 2:17.0.9-0ubuntu3
python-nova: Installed: 2:17.0.9-0ubuntu3
nova-compute: Installed: 2:17.0.9-0ubuntu3
nova-compute-lxc: Installed: 2:17.0.9-0ubuntu3
Expected result
===============
The instance should have a working network configuration and be accesible, etc.
Actual result
=============
The instance is created and is running, but is not accessible by network
(neither using the router network namespace nor by assigning an floating
IP).
It can be accessed with 'virsh -c lxc:/// console instance-<number>'
using root/gocubsgo previously setted:
```
Ubuntu 18.04.2 LTS ubuntu console
ubuntu login: root
Password:
run-parts: /etc/update-motd.d/98-fsck-at-reboot exited with return code 1
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
root@ubuntu:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
19: eth0@if20: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether fa:16:3e:b3:25:0c brd ff:ff:ff:ff:ff:ff link-netnsid 0
root@ubuntu:~#
```
It can be seen that the network is not configured.
The logs of cloud-init show that no datasource was found so it disabled
cloud-init start.
Logs & Configs
==============
```
root@ubuntu:~# cat /run/cloud-init/.ds-identify.result
1
```
```
root@ubuntu:~# cat /run/cloud-init/ds-identify.log
[up 22612.42s] ds-identify
policy loaded: mode=search report=false found=all maybe=all notfound=disabled
/etc/cloud/cloud.cfg.d/90_dpkg.cfg set datasource_list: [ NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, None ]
DMI_PRODUCT_NAME=HVM domU
DMI_SYS_VENDOR=Xen
DMI_PRODUCT_SERIAL=ec21a6db-7988-521e-8fe1-eaed834f9b54
DMI_PRODUCT_UUID=EC21A6DB-7988-521E-8FE1-EAED834F9B54
PID_1_PRODUCT_NAME=unavailable
DMI_CHASSIS_ASSET_TAG=
FS_LABELS=unavailable:container
ISO9660_DEVS=unavailable:container
KERNEL_CMDLINE=/sbin/init
VIRT=lxc-libvirt
UNAME_KERNEL_NAME=Linux
UNAME_KERNEL_RELEASE=4.15.0-1041-aws
UNAME_KERNEL_VERSION=#43-Ubuntu SMP Thu Jun 6 13:39:11 UTC 2019
UNAME_MACHINE=x86_64
UNAME_NODENAME=ubuntu
UNAME_OPERATING_SYSTEM=GNU/Linux
DSNAME=
DSLIST=NoCloud ConfigDrive OpenNebula DigitalOcean Azure AltCloud OVF MAAS GCE OpenStack CloudSigma SmartOS Bigstep Scaleway AliYun Ec2 CloudStack Hetzner IBMCloud None
MODE=search
ON_FOUND=all
ON_MAYBE=all
ON_NOTFOUND=disabled
pid=24 ppid=5
is_container=true
is_ds_enabled(IBMCloud) = true.
is_ds_enabled(IBMCloud) = true.
No ds found [mode=search, notfound=disabled]. Disabled cloud-init [1]
[up 22612.47s] returning 1
root@ubuntu:~# cat /run/cloud-init/
.ds-identify.result cloud.cfg
cloud-init-generator.log ds-identify.log
```
```
root@ubuntu:~# cat /run/cloud-init/cloud.cfg
di_report:
datasource_list: [ ]
# reporting not found result. notfound=disabled.
```
```
root@ubuntu:~# cat /run/cloud-init/cloud-init-generator.log
/lib/systemd/system-generators/cloud-init-generator normal=/run/systemd/generator early=/run/systemd/generator.early late=/run/systemd/generator.late
kernel command line (container[lxc-libvirt]: pid 1 cmdline not available):
kernel_cmdline found unset
etc_file found unset
default found enabled
checking for datasource
ds-identify rc=1
ds-identify _RET=notfound
cloud-init is enabled but no datasource found, disabling
already disabled: no change needed [no /run/systemd/generator.early/multi-user.target.wants/cloud-init.target]
```
Related bugs:
=============
https://bugs.launchpad.net/nova/+bug/1693524
References
==========
[1] https://cloudinit.readthedocs.io/en/latest/topics/datasources/openstack.html)
[2] https://serverascode.com/2018/06/26/using-cloud-images.html
** Affects: nova
Importance: Undecided
Assignee: Miguel Ángel Herranz Trillo (maherranzt)
Status: In Progress
** Tags: compute libvirt lxc
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1834506
Title:
cloud-init based images not working on LXC instances
Status in OpenStack Compute (nova):
In Progress
Bug description:
Description
===========
Apparently cloud-init based images are unable to correctly initialize
in a OpenStack compute using LXC.
Steps to reproduce:
==================
Install a OpenStack using Libvirt + LXC (nova-compute-lxc package in
Ubuntu) and launch a cloud-init based instance.
This has been tested with an Ubuntu cloud image (after setting a
default root password to allow console login, see [2]):
```
wget http://uec-images.ubuntu.com/releases/18.04/release/ubuntu-18.04-server-cloudimg-amd64.tar.gz # md5sum f90bf979eab74f18bd4400a7355157dc, downloaded on 2019-06-18
tar zxfv ubuntu-18.04-server-cloudimg-amd64.tar.gz
virt-customize -a bionic-server-cloudimg-amd64.img --root-password password:gocubsgo
openstack image create "ubuntu_with_root_password" --file bionic-server-cloudimg-amd64.img --disk-format raw --container-format bare --public
```
NOTE: since cloud-init usually check the DMI product name (see [1]),
in case that the host is an OpenStack compute host or instance itself,
the DMI product name may be passed to the LXC instance, so cloud-init
would use OpenStack datasource as expected, but for wrong reasons. To
avoid that posibility, it may be better to reproduce in a non
OpenStack based host.
Environment info:
Host: AWS instance (t2.large)
OS: Ubuntu 18.04.2 LTS
Kernel: Linux 4.15.0-1041-aws #43-Ubuntu SMP x86_64
Openstack: queens
Related packages versions:
nova-api: Installed: 2:17.0.9-0ubuntu3
python-nova: Installed: 2:17.0.9-0ubuntu3
nova-compute: Installed: 2:17.0.9-0ubuntu3
nova-compute-lxc: Installed: 2:17.0.9-0ubuntu3
Expected result
===============
The instance should have a working network configuration and be accesible, etc.
Actual result
=============
The instance is created and is running, but is not accessible by
network (neither using the router network namespace nor by assigning
an floating IP).
It can be accessed with 'virsh -c lxc:/// console instance-<number>'
using root/gocubsgo previously setted:
```
Ubuntu 18.04.2 LTS ubuntu console
ubuntu login: root
Password:
run-parts: /etc/update-motd.d/98-fsck-at-reboot exited with return code 1
The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.
root@ubuntu:~# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
19: eth0@if20: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
link/ether fa:16:3e:b3:25:0c brd ff:ff:ff:ff:ff:ff link-netnsid 0
root@ubuntu:~#
```
It can be seen that the network is not configured.
The logs of cloud-init show that no datasource was found so it
disabled cloud-init start.
Logs & Configs
==============
```
root@ubuntu:~# cat /run/cloud-init/.ds-identify.result
1
```
```
root@ubuntu:~# cat /run/cloud-init/ds-identify.log
[up 22612.42s] ds-identify
policy loaded: mode=search report=false found=all maybe=all notfound=disabled
/etc/cloud/cloud.cfg.d/90_dpkg.cfg set datasource_list: [ NoCloud, ConfigDrive, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Hetzner, IBMCloud, None ]
DMI_PRODUCT_NAME=HVM domU
DMI_SYS_VENDOR=Xen
DMI_PRODUCT_SERIAL=ec21a6db-7988-521e-8fe1-eaed834f9b54
DMI_PRODUCT_UUID=EC21A6DB-7988-521E-8FE1-EAED834F9B54
PID_1_PRODUCT_NAME=unavailable
DMI_CHASSIS_ASSET_TAG=
FS_LABELS=unavailable:container
ISO9660_DEVS=unavailable:container
KERNEL_CMDLINE=/sbin/init
VIRT=lxc-libvirt
UNAME_KERNEL_NAME=Linux
UNAME_KERNEL_RELEASE=4.15.0-1041-aws
UNAME_KERNEL_VERSION=#43-Ubuntu SMP Thu Jun 6 13:39:11 UTC 2019
UNAME_MACHINE=x86_64
UNAME_NODENAME=ubuntu
UNAME_OPERATING_SYSTEM=GNU/Linux
DSNAME=
DSLIST=NoCloud ConfigDrive OpenNebula DigitalOcean Azure AltCloud OVF MAAS GCE OpenStack CloudSigma SmartOS Bigstep Scaleway AliYun Ec2 CloudStack Hetzner IBMCloud None
MODE=search
ON_FOUND=all
ON_MAYBE=all
ON_NOTFOUND=disabled
pid=24 ppid=5
is_container=true
is_ds_enabled(IBMCloud) = true.
is_ds_enabled(IBMCloud) = true.
No ds found [mode=search, notfound=disabled]. Disabled cloud-init [1]
[up 22612.47s] returning 1
root@ubuntu:~# cat /run/cloud-init/
.ds-identify.result cloud.cfg
cloud-init-generator.log ds-identify.log
```
```
root@ubuntu:~# cat /run/cloud-init/cloud.cfg
di_report:
datasource_list: [ ]
# reporting not found result. notfound=disabled.
```
```
root@ubuntu:~# cat /run/cloud-init/cloud-init-generator.log
/lib/systemd/system-generators/cloud-init-generator normal=/run/systemd/generator early=/run/systemd/generator.early late=/run/systemd/generator.late
kernel command line (container[lxc-libvirt]: pid 1 cmdline not available):
kernel_cmdline found unset
etc_file found unset
default found enabled
checking for datasource
ds-identify rc=1
ds-identify _RET=notfound
cloud-init is enabled but no datasource found, disabling
already disabled: no change needed [no /run/systemd/generator.early/multi-user.target.wants/cloud-init.target]
```
Related bugs:
=============
https://bugs.launchpad.net/nova/+bug/1693524
References
==========
[1] https://cloudinit.readthedocs.io/en/latest/topics/datasources/openstack.html)
[2] https://serverascode.com/2018/06/26/using-cloud-images.html
To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1834506/+subscriptions
Follow ups
