yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1527274] [NEW] Neutron-metering-agent failed to add rule on router without gateway

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Sergey Belous <sbelous@xxxxxxxxxxxx>
Date: Thu, 17 Dec 2015 14:06:20 -0000
Reply-to: Bug 1527274 <1527274@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

If we trying to create meter-label-rule and there is a router without
external gateway, then the metering agent will raise an error:

2015-12-17 08:56:44.659 ERROR oslo_messaging.rpc.dispatcher [req-732f65fb-9a4e-4883-b545-3cb080c8cdae admin f8267bb3db654ca2a26a07d9757ec280] Exception during message handling: cannot concatenate 'str' and 'NoneType' objects
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     executor_callback))
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     executor_callback)
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     result = func(ctxt, **new_args)
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/neutron/neutron/services/metering/agents/metering_agent.py", line 222, in add_metering_label_rule
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     'add_metering_label_rule')
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 271, in inner
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     return f(*args, **kwargs)
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/neutron/neutron/services/metering/agents/metering_agent.py", line 176, in _invoke_driver
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     return getattr(self.metering_driver, func_name)(context, meterings)
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_log/helpers.py", line 46, in wrapper
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     return method(*args, **kwargs)
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/neutron/neutron/services/metering/drivers/iptables/iptables_driver.py", line 259, in add_metering_label_rule
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     self._add_metering_label_rule(router)
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/neutron/neutron/services/metering/drivers/iptables/iptables_driver.py", line 272, in _add_metering_label_rule
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     self._process_metering_rule_action(router, 'create')
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/neutron/neutron/services/metering/drivers/iptables/iptables_driver.py", line 281, in _process_metering_rule_action
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     ext_dev = self.get_external_device_name(rm.router['gw_port_id'])
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/neutron/neutron/services/metering/drivers/iptables/iptables_driver.py", line 132, in get_external_device_name
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     return (EXTERNAL_DEV_PREFIX + port_id)[:self.driver.DEV_NAME_LEN]
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher TypeError: cannot concatenate 'str' and 'NoneType' objects
2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher

Steps to reproduce:
1. Create internal net, subnet, router. Add interface to router for created net. Boot vm in created net.
2. Create neutron-meter-label
3. Try to create meter-label-rule
4. Rule not appear in iptables on router-namespace and after that we can see traces in neutron-meter-agent's logs like above.

** Affects: neutron
     Importance: Undecided
     Assignee: Sergey Belous (sbelous)
         Status: New


** Tags: metering

** Changed in: neutron
     Assignee: (unassigned) => Sergey Belous (sbelous)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1527274

Title:
  Neutron-metering-agent failed to add rule on router without gateway

Status in neutron:
  New

Bug description:
  If we trying to create meter-label-rule and there is a router without
  external gateway, then the metering agent will raise an error:

  2015-12-17 08:56:44.659 ERROR oslo_messaging.rpc.dispatcher [req-732f65fb-9a4e-4883-b545-3cb080c8cdae admin f8267bb3db654ca2a26a07d9757ec280] Exception during message handling: cannot concatenate 'str' and 'NoneType' objects
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher Traceback (most recent call last):
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     executor_callback))
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     executor_callback)
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     result = func(ctxt, **new_args)
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/neutron/neutron/services/metering/agents/metering_agent.py", line 222, in add_metering_label_rule
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     'add_metering_label_rule')
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 271, in inner
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     return f(*args, **kwargs)
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/neutron/neutron/services/metering/agents/metering_agent.py", line 176, in _invoke_driver
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     return getattr(self.metering_driver, func_name)(context, meterings)
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/usr/local/lib/python2.7/dist-packages/oslo_log/helpers.py", line 46, in wrapper
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     return method(*args, **kwargs)
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/neutron/neutron/services/metering/drivers/iptables/iptables_driver.py", line 259, in add_metering_label_rule
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     self._add_metering_label_rule(router)
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/neutron/neutron/services/metering/drivers/iptables/iptables_driver.py", line 272, in _add_metering_label_rule
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     self._process_metering_rule_action(router, 'create')
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/neutron/neutron/services/metering/drivers/iptables/iptables_driver.py", line 281, in _process_metering_rule_action
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     ext_dev = self.get_external_device_name(rm.router['gw_port_id'])
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher   File "/opt/stack/neutron/neutron/services/metering/drivers/iptables/iptables_driver.py", line 132, in get_external_device_name
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher     return (EXTERNAL_DEV_PREFIX + port_id)[:self.driver.DEV_NAME_LEN]
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher TypeError: cannot concatenate 'str' and 'NoneType' objects
  2015-12-17 08:56:44.659 TRACE oslo_messaging.rpc.dispatcher

  Steps to reproduce:
  1. Create internal net, subnet, router. Add interface to router for created net. Boot vm in created net.
  2. Create neutron-meter-label
  3. Try to create meter-label-rule
  4. Rule not appear in iptables on router-namespace and after that we can see traces in neutron-meter-agent's logs like above.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1527274/+subscriptions
Follow ups

[Bug 1527274] Re: Neutron-metering-agent failed to add rule on router without gateway
From: OpenStack Infra, 2016-04-21