yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #82139
[Bug 1870228] [NEW] cloud-init metadata fallback broken
Public bug reported:
I came across an issue today for a user that was experiencing issues
connecting to metadata at 169.254.169.254. For a long time, cloud-init
has had a fallback mechanism to that allowed it to contact the metadata
service at http://<dhcp server ip>/latest/meta-data if
http://169.254.169.254/latest/meta-data were unavailable, like so:
[ 157.574921] cloud-init[1313]: 2020-03-31 09:53:24,158 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [50/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x7fa9e07c5890>, 'Connection to 169.254.169.254 timed out. (connect timeout=50.0)'))]
[ 208.629083] cloud-init[1313]: 2020-03-31 09:54:15,214 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [101/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x7fa9e07c5350>, 'Connection to 169.254.169.254 timed out. (connect timeout=50.0)'))]
[ 226.639267] cloud-init[1313]: 2020-03-31 09:54:33,224 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [119/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x7fa9e07c5250>, 'Connection to 169.254.169.254 timed out. (connect timeout=17.0)'))]
[ 227.640812] cloud-init[1313]: 2020-03-31 09:54:34,225 - DataSourceEc2.py[CRITICAL]: Giving up on md from ['http://169.254.169.254/2009-04-04/meta-data/instance-id'] after 120 seconds
[ 227.651134] cloud-init[1313]: 2020-03-31 09:54:34,236 - url_helper.py[WARNING]: Calling 'http://10.19.48.2/latest/meta-data/instance-id' failed [0/120s]: request error [('Connection aborted.', error(111, 'Connection refused'))]
[ 228.655226] cloud-init[1313]: 2020-03-31 09:54:35,240 - url_helper.py[WARNING]: Calling 'http://10.19.48.2/latest/meta-data/instance-id' failed [1/120s]: request error [('Connection aborted.', error(111, 'Connection refused'))]
In this Stein environment, isolated metadata is enabled, and the qdhcp
namespace has a listener at 169.254.169.254:80. Previous versions of
Neutron had the listener on 0.0.0.0:80, which helped facilitate the
fallback mechanism described above. The bug/patch where this was changed
is here:
[1] https://bugs.launchpad.net/neutron/+bug/1745618
Having this functionality back would be nice. Thoughts?
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1870228
Title:
cloud-init metadata fallback broken
Status in neutron:
New
Bug description:
I came across an issue today for a user that was experiencing issues
connecting to metadata at 169.254.169.254. For a long time, cloud-init
has had a fallback mechanism to that allowed it to contact the
metadata service at http://<dhcp server ip>/latest/meta-data if
http://169.254.169.254/latest/meta-data were unavailable, like so:
[ 157.574921] cloud-init[1313]: 2020-03-31 09:53:24,158 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [50/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x7fa9e07c5890>, 'Connection to 169.254.169.254 timed out. (connect timeout=50.0)'))]
[ 208.629083] cloud-init[1313]: 2020-03-31 09:54:15,214 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [101/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x7fa9e07c5350>, 'Connection to 169.254.169.254 timed out. (connect timeout=50.0)'))]
[ 226.639267] cloud-init[1313]: 2020-03-31 09:54:33,224 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [119/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Max retries exceeded with url: /2009-04-04/meta-data/instance-id (Caused by ConnectTimeoutError(<requests.packages.urllib3.connection.HTTPConnection object at 0x7fa9e07c5250>, 'Connection to 169.254.169.254 timed out. (connect timeout=17.0)'))]
[ 227.640812] cloud-init[1313]: 2020-03-31 09:54:34,225 - DataSourceEc2.py[CRITICAL]: Giving up on md from ['http://169.254.169.254/2009-04-04/meta-data/instance-id'] after 120 seconds
[ 227.651134] cloud-init[1313]: 2020-03-31 09:54:34,236 - url_helper.py[WARNING]: Calling 'http://10.19.48.2/latest/meta-data/instance-id' failed [0/120s]: request error [('Connection aborted.', error(111, 'Connection refused'))]
[ 228.655226] cloud-init[1313]: 2020-03-31 09:54:35,240 - url_helper.py[WARNING]: Calling 'http://10.19.48.2/latest/meta-data/instance-id' failed [1/120s]: request error [('Connection aborted.', error(111, 'Connection refused'))]
In this Stein environment, isolated metadata is enabled, and the qdhcp
namespace has a listener at 169.254.169.254:80. Previous versions of
Neutron had the listener on 0.0.0.0:80, which helped facilitate the
fallback mechanism described above. The bug/patch where this was
changed is here:
[1] https://bugs.launchpad.net/neutron/+bug/1745618
Having this functionality back would be nice. Thoughts?
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1870228/+subscriptions