yahoo-eng-team team mailing list archive

[Marcus Klein <kleini76@xxxxxxxxx>]

Public bug reported:

It sometimes happens, when we spawn VMs, that the requests of cloud-init
inside the VM to the metadata agent are refused. This seems to be a
timing problem as this happens with fast booting images more often than
with slowly booting images. Error message for the request is "Connection
refused". Some seconds later the exact same request works without any
problems.

Our deployment is just upgraded from Ocata to Pike and neutron-ns-
metadata-proxy was replaced with haproxy. Since this change, the problem
occurs. Our setup uses Open vSwitch, self service networks and network
nodes for L3 router, metadata agent, dhcp agent are separated from
compute nodes and controller nodes. We use Ubuntu Cloud Archive
repositories to install on Ubuntu 16.04 LTS.

15:57:12.780152 IP (tos 0x0, ttl 64, id 7253, offset 0, flags [DF], proto TCP (6), length 60)
    192.168.5.3.59378 > 169.254.169.254.http: Flags [S], cksum 0xebec (correct), seq 4230673254, win 29200, options [mss 1460,sackOK,TS val 2933213616 ecr 0,nop,wscale 7], length 0
15:57:12.780208 IP (tos 0x0, ttl 64, id 6932, offset 0, flags [DF], proto TCP (6), length 40)
    169.254.169.254.http > 192.168.5.3.59378: Flags [R.], cksum 0xbe52 (correct), seq 0, ack 4230673255, win 0, length 0

The TCP SYN package to the metadata IP is replied with a TCP RST,ACK
package. The image does not try again to connect to the metadata agent
and is from then on not usable due to missing injection of public SSH
key.

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1818682

Title:
  HAproxy for metadata refuses connection from VM cloud-init

Status in neutron:
  New

Bug description:
  It sometimes happens, when we spawn VMs, that the requests of cloud-
  init inside the VM to the metadata agent are refused. This seems to be
  a timing problem as this happens with fast booting images more often
  than with slowly booting images. Error message for the request is
  "Connection refused". Some seconds later the exact same request works
  without any problems.

  Our deployment is just upgraded from Ocata to Pike and neutron-ns-
  metadata-proxy was replaced with haproxy. Since this change, the
  problem occurs. Our setup uses Open vSwitch, self service networks and
  network nodes for L3 router, metadata agent, dhcp agent are separated
  from compute nodes and controller nodes. We use Ubuntu Cloud Archive
  repositories to install on Ubuntu 16.04 LTS.

  15:57:12.780152 IP (tos 0x0, ttl 64, id 7253, offset 0, flags [DF], proto TCP (6), length 60)
      192.168.5.3.59378 > 169.254.169.254.http: Flags [S], cksum 0xebec (correct), seq 4230673254, win 29200, options [mss 1460,sackOK,TS val 2933213616 ecr 0,nop,wscale 7], length 0
  15:57:12.780208 IP (tos 0x0, ttl 64, id 6932, offset 0, flags [DF], proto TCP (6), length 40)
      169.254.169.254.http > 192.168.5.3.59378: Flags [R.], cksum 0xbe52 (correct), seq 0, ack 4230673255, win 0, length 0

  The TCP SYN package to the metadata IP is replied with a TCP RST,ACK
  package. The image does not try again to connect to the metadata agent
  and is from then on not usable due to missing injection of public SSH
  key.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1818682/+subscriptions