yahoo-eng-team team mailing list archive

[Michael Johnson <johnsomor@xxxxxxxxx>]

Public bug reported:

In the older neutron-ns-metadata-proxy, in the newton release, the
number of threads is fixed at 100.  This is a drop from the previous
default setting of 1000 as a side effect of changing the number of wsgi
threads [1].

This is causing failures at sites with a large number of instances using deployment tools (instance cloud-init logs):
2017-08-01 15:44:36,773 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [119/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Request timed out. (timeout=17.0)]
2017-08-01 15:44:37,775 - DataSourceEc2.py[CRITICAL]: Giving up on md from ['http://169.254.169.254/2009-04-04/meta-data/instance-id'] after 120 seconds

Setting the value, for neutron-ns-metadata-proxy only, back up to 1000 resolves this issue.
It should also be noted that in the Ocata forward version of the neutron-ns-metadata-proxy the default value is 1024 [2].

I am going to propose a patch for stable/newton that sets the default
thread count for the neutron-ns-metadata-proxy back up to 1000.

[1] https://github.com/openstack/neutron/blob/master/releasenotes/notes/config-wsgi-pool-size-a4c06753b79fee6d.yaml
[2] https://github.com/openstack/neutron/blob/master/neutron/agent/metadata/driver.py#L44

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1711215

Title:
  Default neutron-ns-metadata-proxy threads setting is too low (newton)

Status in neutron:
  New

Bug description:
  In the older neutron-ns-metadata-proxy, in the newton release, the
  number of threads is fixed at 100.  This is a drop from the previous
  default setting of 1000 as a side effect of changing the number of
  wsgi threads [1].

  This is causing failures at sites with a large number of instances using deployment tools (instance cloud-init logs):
  2017-08-01 15:44:36,773 - url_helper.py[WARNING]: Calling 'http://169.254.169.254/2009-04-04/meta-data/instance-id' failed [119/120s]: request error [HTTPConnectionPool(host='169.254.169.254', port=80): Request timed out. (timeout=17.0)]
  2017-08-01 15:44:37,775 - DataSourceEc2.py[CRITICAL]: Giving up on md from ['http://169.254.169.254/2009-04-04/meta-data/instance-id'] after 120 seconds

  Setting the value, for neutron-ns-metadata-proxy only, back up to 1000 resolves this issue.
  It should also be noted that in the Ocata forward version of the neutron-ns-metadata-proxy the default value is 1024 [2].

  I am going to propose a patch for stable/newton that sets the default
  thread count for the neutron-ns-metadata-proxy back up to 1000.

  [1] https://github.com/openstack/neutron/blob/master/releasenotes/notes/config-wsgi-pool-size-a4c06753b79fee6d.yaml
  [2] https://github.com/openstack/neutron/blob/master/neutron/agent/metadata/driver.py#L44

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1711215/+subscriptions