yahoo-eng-team team mailing list archive

[Thierry Carrez <thierry.carrez+lp@xxxxxxxxx>]

** Changed in: nova
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1259267

Title:
  Nova Docker: Metadata service doesn't work

Status in OpenStack Compute (Nova):
  Fix Released

Bug description:
  I was playing around with cloud-init. I wanted to use cloud init as substitute for the missing environment variables feature.
  The basic idea is to define variables in the user data and inject the them before starting the service.

  Following:
  docker run -e "MY_Variable=MyValue" -d centos

  would look like in openstack:

  nova boot --image centos:latest --user-data "MY_Variable=MyValue"
  myinstance

  Unfortunately does the metadata service not work inside of a docker
  container. After some testing I figured out that the reason for that
  is that a container uses as default gateway the docker network (docker
  ip address). The metadata service simple rejects the call since the IP
  address of docker container is not associated with the nova instance.

  Note: The metadata service itself can be accessed
  (http://169.254.169.254) but it is not possible to access the actual
  data (http://169.254.169.254/2009-04-04 - Status 404)

  I was able to work around the issue by simply changing the route
  inside the container:

  # Hack: In order to receive data from the metadata service we must make sure we resolve the data via our nova network.
  #
  # A docker container in openstack has two NICs.
  # - eth0 has a IP address on the docker0 bridge which is usually an e.g. 172.0.0.0 IP address.
  # - pvnetXXXX is a IP address assigned by nova.
  #
  # Extract the NIC name of the nova network.
  #
  NOVA_NIC=$(ip a | grep pvnet | head -n 1 | cut -d: -f2)

  while [ "$NOVA_NIC" == "" ] ; do
     echo "Find nova NIC..."
     sleep 1
     NOVA_NIC=$(ip a | grep pvnet | head -n 1 | cut -d: -f2)
  done

  echo "Device $NOVA_NIC found. Wait until ready."
  sleep 3

  # Setup a network route to insure we use the nova network.
  #
  echo "[INFO] Create default route for $NOVA_NIC. Gateway 10.0.0.1"
  ip r r default via 10.0.0.1 dev $NOVA_NIC

  # Shutdown eth0 since icps will fetch enabled enterface for streaming.
  ip l set down dev eth0

  This approach is obviously a poor solution since it has certain
  expectation of the network.

  Another solution might be extend the docker driver to add a firewall
  rule which will masquerade requests on 169.254.169.254 with the actual
  nova network IP address

  I third solution would need improvements in docker. If docker would
  have a network mode which allows to assign the IP from outside this
  issue would be solved. That of course is a just which must be accepted
  by the docker community.

  Network Threads:
  - https://groups.google.com/forum/#!topic/docker-dev/YfCeX8TBweA



  
  Simple script to test metadata inside the container:

  #!/bin/bash
  status=$(curl -I -s -o /dev/null -w "%{http_code}" http://169.254.169.254/2009-04-04)

  while [ $status != '200' ]; do
     echo "Cannot access metadata, status: '$status', try again..."
     date # easier to see in docker logs that loop is still running.
     status=$(curl -I -s -o /dev/null -w "%{http_code}" http://169.254.169.254/2009-04-04)
     sleep 1
  done

  echo "Yes we got some user data:"
  curl http://169.254.169.254/2009-04-04

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1259267/+subscriptions