yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1622578] [NEW] cloud init metadata injection failed while booting VM

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: sutefun <1622578@xxxxxxxxxxxxxxxxxx>
Date: Mon, 12 Sep 2016 11:44:43 -0000
Reply-to: Bug 1622578 <1622578@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

Hello,

when we start a VM the process of cloud init failed with:

Inside the VM
url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [62/120s]: bad status code [500]

The VM is reachable and you can curl http://169.254.169.254 and get a
valid result but when you curl http://169.254.169.254/latest we get:
Remote metadata server experienced an internal server error.

In nova-api log I see: Unauthorized: The request you have made requires
authentication. (HTTP 401)


Environment

OS:
Ubuntu 16.04.1 LTS

packages on nova:
ii  nova-api                           2:13.1.0-0ubuntu1                     all          OpenStack Compute - API frontend
ii  nova-common                        2:13.1.0-0ubuntu1                     all          OpenStack Compute - common files
ii  nova-conductor                     2:13.1.0-0ubuntu1                     all          OpenStack Compute - conductor service
ii  nova-consoleauth                   2:13.1.0-0ubuntu1                     all          OpenStack Compute - Console Authenticator
ii  nova-novncproxy                    2:13.1.0-0ubuntu1                     all          OpenStack Compute - NoVNC proxy
ii  nova-scheduler                     2:13.1.0-0ubuntu1                     all          OpenStack Compute - virtual machine scheduler
ii  nova-spiceproxy                    2:13.1.0-0ubuntu1                     all          OpenStack Compute - spice html5 proxy
ii  python-nova                        2:13.1.0-0ubuntu1                     all          OpenStack Compute Python libraries
ii  python-novaclient                  2:3.3.1-2                             all          client library for OpenStack Compute API - Python 2.7
ii  python-neutronclient               1:4.1.1-2                             all          client API library for Neutron - Python 2.7


packages on neutron:

ii  neutron-common                     2:8.1.2-0ubuntu1                all          Neutron is a virtual network service for Openstack - common
ii  neutron-dhcp-agent                 2:8.1.2-0ubuntu1                all          Neutron is a virtual network service for Openstack - DHCP agent
ii  neutron-l3-agent                   2:8.1.2-0ubuntu1                all          Neutron is a virtual network service for Openstack - l3 agent
ii  neutron-metadata-agent             2:8.1.2-0ubuntu1                all          Neutron is a virtual network service for Openstack - metadata agent
ii  neutron-openvswitch-agent          2:8.1.2-0ubuntu1                all          Neutron is a virtual network service for Openstack - Open vSwitch plugin agent
ii  neutron-plugin-ml2                 2:8.1.2-0ubuntu1                all          Neutron is a virtual network service for Openstack - ML2 plugin
ii  neutron-plugin-openvswitch-agent   2:8.1.2-0ubuntu1                all          Transitional package for neutron-openvswitch-agent
ii  neutron-server                     2:8.1.2-0ubuntu1                all          Neutron is a virtual network service for Openstack - server
ii  python-neutron                     2:8.1.2-0ubuntu1                all          Neutron is a virtual network service for Openstack - Python library
ii  python-neutron-fwaas               1:8.0.0-0ubuntu1                all          Firewall-as-a-Service driver for OpenStack Neutron
ii  python-neutron-lib                 0.0.2-2                         all          Neutron shared routines and utilities - Python 2.7
ii  python-neutronclient               1:4.1.1-2                       all          client API library for Neutron - Python 2.7

Hypervisor is KVM

ii  nova-compute-kvm                     2:13.1.0-0ubuntu1                   all          OpenStack Compute - compute node (KVM)
ii  libvirt-bin                          1.3.1-1ubuntu10.1                   amd64        programs for the libvirt library
ii  libvirt0:amd64                       1.3.1-1ubuntu10.1                   amd64        library for interfacing with different virtualization systems
ii  nova-compute-libvirt                 2:13.1.0-0ubuntu1                   all          OpenStack Compute - compute node libvirt support
ii  python-libvirt                       1.3.1-1ubuntu1                      amd64        libvirt Python bindings

Storage is CEPH
root@openstack11:~# ceph version
ceph version 10.2.2 

Network is Neutron with openvswitch

I tried a lot to get around this issue, I hope I didn't oversee
something.

Maybe someone can help or fix this bug.

Thanks in advance

** Affects: nova
     Importance: Undecided
         Status: New

** Attachment added: "logs.zip"
   https://bugs.launchpad.net/bugs/1622578/+attachment/4739277/+files/logs.zip

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1622578

Title:
  cloud init metadata injection failed while booting VM

Status in OpenStack Compute (nova):
  New

Bug description:
  Hello,

  when we start a VM the process of cloud init failed with:

  Inside the VM
  url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [62/120s]: bad status code [500]

  The VM is reachable and you can curl http://169.254.169.254 and get a
  valid result but when you curl http://169.254.169.254/latest we get:
  Remote metadata server experienced an internal server error.

  In nova-api log I see: Unauthorized: The request you have made
  requires authentication. (HTTP 401)

  
  Environment

  OS:
  Ubuntu 16.04.1 LTS

  packages on nova:
  ii  nova-api                           2:13.1.0-0ubuntu1                     all          OpenStack Compute - API frontend
  ii  nova-common                        2:13.1.0-0ubuntu1                     all          OpenStack Compute - common files
  ii  nova-conductor                     2:13.1.0-0ubuntu1                     all          OpenStack Compute - conductor service
  ii  nova-consoleauth                   2:13.1.0-0ubuntu1                     all          OpenStack Compute - Console Authenticator
  ii  nova-novncproxy                    2:13.1.0-0ubuntu1                     all          OpenStack Compute - NoVNC proxy
  ii  nova-scheduler                     2:13.1.0-0ubuntu1                     all          OpenStack Compute - virtual machine scheduler
  ii  nova-spiceproxy                    2:13.1.0-0ubuntu1                     all          OpenStack Compute - spice html5 proxy
  ii  python-nova                        2:13.1.0-0ubuntu1                     all          OpenStack Compute Python libraries
  ii  python-novaclient                  2:3.3.1-2                             all          client library for OpenStack Compute API - Python 2.7
  ii  python-neutronclient               1:4.1.1-2                             all          client API library for Neutron - Python 2.7

  
  packages on neutron:

  ii  neutron-common                     2:8.1.2-0ubuntu1                all          Neutron is a virtual network service for Openstack - common
  ii  neutron-dhcp-agent                 2:8.1.2-0ubuntu1                all          Neutron is a virtual network service for Openstack - DHCP agent
  ii  neutron-l3-agent                   2:8.1.2-0ubuntu1                all          Neutron is a virtual network service for Openstack - l3 agent
  ii  neutron-metadata-agent             2:8.1.2-0ubuntu1                all          Neutron is a virtual network service for Openstack - metadata agent
  ii  neutron-openvswitch-agent          2:8.1.2-0ubuntu1                all          Neutron is a virtual network service for Openstack - Open vSwitch plugin agent
  ii  neutron-plugin-ml2                 2:8.1.2-0ubuntu1                all          Neutron is a virtual network service for Openstack - ML2 plugin
  ii  neutron-plugin-openvswitch-agent   2:8.1.2-0ubuntu1                all          Transitional package for neutron-openvswitch-agent
  ii  neutron-server                     2:8.1.2-0ubuntu1                all          Neutron is a virtual network service for Openstack - server
  ii  python-neutron                     2:8.1.2-0ubuntu1                all          Neutron is a virtual network service for Openstack - Python library
  ii  python-neutron-fwaas               1:8.0.0-0ubuntu1                all          Firewall-as-a-Service driver for OpenStack Neutron
  ii  python-neutron-lib                 0.0.2-2                         all          Neutron shared routines and utilities - Python 2.7
  ii  python-neutronclient               1:4.1.1-2                       all          client API library for Neutron - Python 2.7

  Hypervisor is KVM

  ii  nova-compute-kvm                     2:13.1.0-0ubuntu1                   all          OpenStack Compute - compute node (KVM)
  ii  libvirt-bin                          1.3.1-1ubuntu10.1                   amd64        programs for the libvirt library
  ii  libvirt0:amd64                       1.3.1-1ubuntu10.1                   amd64        library for interfacing with different virtualization systems
  ii  nova-compute-libvirt                 2:13.1.0-0ubuntu1                   all          OpenStack Compute - compute node libvirt support
  ii  python-libvirt                       1.3.1-1ubuntu1                      amd64        libvirt Python bindings

  Storage is CEPH
  root@openstack11:~# ceph version
  ceph version 10.2.2 

  Network is Neutron with openvswitch

  I tried a lot to get around this issue, I hope I didn't oversee
  something.

  Maybe someone can help or fix this bug.

  Thanks in advance

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1622578/+subscriptions
Follow ups

[Bug 1622578] Re: cloud init metadata injection failed while booting VM
From: Sylvain Bauza, 2016-09-13