← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #66943

[Bug 1713144] [NEW] nova metadata service breaks on hostname

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

For more than a year, i have deployed  multiple clusters using openstack-ansible.
This particular case is using FQDN for the internal and external endpoint.

[DEFAULT]
...

# Metadata
metadata_host = my-int.XXX.com
metadata_port = 8775
metadata_workers = 3
...

Recently, this breaks nova-metadata.

2017-08-25 21:19:02.928 18130 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports.
2017-08-25 21:19:02.931 18130 INFO nova.network.driver [-] Loading network driver 'nova.network.linux_net'
2017-08-25 21:19:03.078 18130 CRITICAL nova [-] AddrFormatError: failed to detect a valid IP address from 'my-int.XXX.com'
2017-08-25 21:19:03.078 18130 ERROR nova Traceback (most recent call last):
2017-08-25 21:19:03.078 18130 ERROR nova   File "/openstack/venvs/nova-15.1.8/bin/nova-api-metadata", line 11, in <module>
2017-08-25 21:19:03.078 18130 ERROR nova     sys.exit(main())
2017-08-25 21:19:03.078 18130 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/cmd/api_metadata.py", line 48, in main
2017-08-25 21:19:03.078 18130 ERROR nova     server = service.WSGIService('metadata', use_ssl=should_use_ssl)
2017-08-25 21:19:03.078 18130 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/service.py", line 309, in __init__
2017-08-25 21:19:03.078 18130 ERROR nova     self.manager = self._get_manager()
2017-08-25 21:19:03.078 18130 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/service.py", line 364, in _get_manager
2017-08-25 21:19:03.078 18130 ERROR nova     return manager_class()
2017-08-25 21:19:03.078 18130 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/api/manager.py", line 30, in __init__
2017-08-25 21:19:03.078 18130 ERROR nova     self.network_driver.metadata_accept()
2017-08-25 21:19:03.078 18130 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/network/linux_net.py", line 599, in metadata_accept
2017-08-25 21:19:03.078 18130 ERROR nova     (CONF.metadata_port, _iptables_dest(CONF.metadata_host)))
2017-08-25 21:19:03.078 18130 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/network/linux_net.py", line 588, in _iptables_dest
2017-08-25 21:19:03.078 18130 ERROR nova     if ((netaddr.IPAddress(ip).version == 4 and ip == '127.0.0.1')
2017-08-25 21:19:03.078 18130 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/netaddr/ip/__init__.py", line 306, in __init__
2017-08-25 21:19:03.078 18130 ERROR nova     'address from %r' % addr)
2017-08-25 21:19:03.078 18130 ERROR nova AddrFormatError: failed to detect a valid IP address from 'my-int.XXX.com'
2017-08-25 21:19:03.078 18130 ERROR nova
2017-08-25 21:19:09.833 18137 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports.
2017-08-25 21:19:09.836 18137 INFO nova.network.driver [-] Loading network driver 'nova.network.linux_net'
2017-08-25 21:19:10.040 18137 CRITICAL nova [-] AddrFormatError: failed to detect a valid IP address from 'my-int.XXX.com'
2017-08-25 21:19:10.040 18137 ERROR nova Traceback (most recent call last):
2017-08-25 21:19:10.040 18137 ERROR nova   File "/openstack/venvs/nova-15.1.8/bin/nova-api-metadata", line 11, in <module>
2017-08-25 21:19:10.040 18137 ERROR nova     sys.exit(main())
2017-08-25 21:19:10.040 18137 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/cmd/api_metadata.py", line 48, in main
2017-08-25 21:19:10.040 18137 ERROR nova     server = service.WSGIService('metadata', use_ssl=should_use_ssl)
2017-08-25 21:19:10.040 18137 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/service.py", line 309, in __init__
2017-08-25 21:19:10.040 18137 ERROR nova     self.manager = self._get_manager()
2017-08-25 21:19:10.040 18137 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/service.py", line 364, in _get_manager
2017-08-25 21:19:10.040 18137 ERROR nova     return manager_class()
2017-08-25 21:19:10.040 18137 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/api/manager.py", line 30, in __init__
2017-08-25 21:19:10.040 18137 ERROR nova     self.network_driver.metadata_accept()
2017-08-25 21:19:10.040 18137 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/network/linux_net.py", line 599, in metadata_accept
2017-08-25 21:19:10.040 18137 ERROR nova     (CONF.metadata_port, _iptables_dest(CONF.metadata_host)))
2017-08-25 21:19:10.040 18137 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/network/linux_net.py", line 588, in _iptables_dest
2017-08-25 21:19:10.040 18137 ERROR nova     if ((netaddr.IPAddress(ip).version == 4 and ip == '127.0.0.1')
2017-08-25 21:19:10.040 18137 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/netaddr/ip/__init__.py", line 306, in __init__
2017-08-25 21:19:10.040 18137 ERROR nova     'address from %r' % addr)
2017-08-25 21:19:10.040 18137 ERROR nova AddrFormatError: failed to detect a valid IP address from 'my-int.XXX.com'
2017-08-25 21:19:10.040 18137 ERROR nova

Changing hostname to IP address fixes it.

# Metadata
metadata_host = 172.16.236.2
metadata_port = 8775
metadata_workers = 3
...

Is this a bug? or a new change? or should we now should not use FQDN in
host ?

Thanks,
Shashi

** Affects: nova
     Importance: Undecided
         Status: New


** Tags: metadata

** Project changed: openstack-ansible => nova

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1713144

Title:
  nova metadata service breaks on hostname

Status in OpenStack Compute (nova):
  New

Bug description:
  For more than a year, i have deployed  multiple clusters using openstack-ansible.
  This particular case is using FQDN for the internal and external endpoint.

  [DEFAULT]
  ...

  # Metadata
  metadata_host = my-int.XXX.com
  metadata_port = 8775
  metadata_workers = 3
  ...

  Recently, this breaks nova-metadata.

  2017-08-25 21:19:02.928 18130 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports.
  2017-08-25 21:19:02.931 18130 INFO nova.network.driver [-] Loading network driver 'nova.network.linux_net'
  2017-08-25 21:19:03.078 18130 CRITICAL nova [-] AddrFormatError: failed to detect a valid IP address from 'my-int.XXX.com'
  2017-08-25 21:19:03.078 18130 ERROR nova Traceback (most recent call last):
  2017-08-25 21:19:03.078 18130 ERROR nova   File "/openstack/venvs/nova-15.1.8/bin/nova-api-metadata", line 11, in <module>
  2017-08-25 21:19:03.078 18130 ERROR nova     sys.exit(main())
  2017-08-25 21:19:03.078 18130 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/cmd/api_metadata.py", line 48, in main
  2017-08-25 21:19:03.078 18130 ERROR nova     server = service.WSGIService('metadata', use_ssl=should_use_ssl)
  2017-08-25 21:19:03.078 18130 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/service.py", line 309, in __init__
  2017-08-25 21:19:03.078 18130 ERROR nova     self.manager = self._get_manager()
  2017-08-25 21:19:03.078 18130 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/service.py", line 364, in _get_manager
  2017-08-25 21:19:03.078 18130 ERROR nova     return manager_class()
  2017-08-25 21:19:03.078 18130 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/api/manager.py", line 30, in __init__
  2017-08-25 21:19:03.078 18130 ERROR nova     self.network_driver.metadata_accept()
  2017-08-25 21:19:03.078 18130 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/network/linux_net.py", line 599, in metadata_accept
  2017-08-25 21:19:03.078 18130 ERROR nova     (CONF.metadata_port, _iptables_dest(CONF.metadata_host)))
  2017-08-25 21:19:03.078 18130 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/network/linux_net.py", line 588, in _iptables_dest
  2017-08-25 21:19:03.078 18130 ERROR nova     if ((netaddr.IPAddress(ip).version == 4 and ip == '127.0.0.1')
  2017-08-25 21:19:03.078 18130 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/netaddr/ip/__init__.py", line 306, in __init__
  2017-08-25 21:19:03.078 18130 ERROR nova     'address from %r' % addr)
  2017-08-25 21:19:03.078 18130 ERROR nova AddrFormatError: failed to detect a valid IP address from 'my-int.XXX.com'
  2017-08-25 21:19:03.078 18130 ERROR nova
  2017-08-25 21:19:09.833 18137 WARNING oslo_reports.guru_meditation_report [-] Guru meditation now registers SIGUSR1 and SIGUSR2 by default for backward compatibility. SIGUSR1 will no longer be registered in a future release, so please use SIGUSR2 to generate reports.
  2017-08-25 21:19:09.836 18137 INFO nova.network.driver [-] Loading network driver 'nova.network.linux_net'
  2017-08-25 21:19:10.040 18137 CRITICAL nova [-] AddrFormatError: failed to detect a valid IP address from 'my-int.XXX.com'
  2017-08-25 21:19:10.040 18137 ERROR nova Traceback (most recent call last):
  2017-08-25 21:19:10.040 18137 ERROR nova   File "/openstack/venvs/nova-15.1.8/bin/nova-api-metadata", line 11, in <module>
  2017-08-25 21:19:10.040 18137 ERROR nova     sys.exit(main())
  2017-08-25 21:19:10.040 18137 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/cmd/api_metadata.py", line 48, in main
  2017-08-25 21:19:10.040 18137 ERROR nova     server = service.WSGIService('metadata', use_ssl=should_use_ssl)
  2017-08-25 21:19:10.040 18137 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/service.py", line 309, in __init__
  2017-08-25 21:19:10.040 18137 ERROR nova     self.manager = self._get_manager()
  2017-08-25 21:19:10.040 18137 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/service.py", line 364, in _get_manager
  2017-08-25 21:19:10.040 18137 ERROR nova     return manager_class()
  2017-08-25 21:19:10.040 18137 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/api/manager.py", line 30, in __init__
  2017-08-25 21:19:10.040 18137 ERROR nova     self.network_driver.metadata_accept()
  2017-08-25 21:19:10.040 18137 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/network/linux_net.py", line 599, in metadata_accept
  2017-08-25 21:19:10.040 18137 ERROR nova     (CONF.metadata_port, _iptables_dest(CONF.metadata_host)))
  2017-08-25 21:19:10.040 18137 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/nova/network/linux_net.py", line 588, in _iptables_dest
  2017-08-25 21:19:10.040 18137 ERROR nova     if ((netaddr.IPAddress(ip).version == 4 and ip == '127.0.0.1')
  2017-08-25 21:19:10.040 18137 ERROR nova   File "/openstack/venvs/nova-15.1.8/lib/python2.7/site-packages/netaddr/ip/__init__.py", line 306, in __init__
  2017-08-25 21:19:10.040 18137 ERROR nova     'address from %r' % addr)
  2017-08-25 21:19:10.040 18137 ERROR nova AddrFormatError: failed to detect a valid IP address from 'my-int.XXX.com'
  2017-08-25 21:19:10.040 18137 ERROR nova

  Changing hostname to IP address fixes it.

  # Metadata
  metadata_host = 172.16.236.2
  metadata_port = 8775
  metadata_workers = 3
  ...

  Is this a bug? or a new change? or should we now should not use FQDN
  in host ?

  Thanks,
  Shashi

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1713144/+subscriptions

Follow ups

  Thread Previous • Date Previous • • Thread Next