yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1617248] [NEW] Metering-agent fail to create iptables rule for add_metering_label_rule

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Zhengwei Gao <gaozhengwei1@xxxxxxxx>
Date: Fri, 26 Aug 2016 10:03:13 -0000
Reply-to: Bug 1617248 <1617248@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Public bug reported:

When metering-agent process add_metering_label_rule from neutron-server,
it may fail to create iptables rule.

2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I neutron-meter-r-00599199-632 1 -i qg-f0732f6f-8e -d 192.168.10.0/24 -j neutron-meter-l-00599199-632
2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent COMMIT
2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent # Completed by iptables_manager
2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent # Generated by iptables_manager
2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent *raw
2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I OUTPUT 1 -j neutron-meter-OUTPUT
2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D OUTPUT 3
2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I PREROUTING 1 -j neutron-meter-PREROUTING
2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D PREROUTING 3
2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent COMMIT
2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent # Completed by iptables_manager
2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent ; Stdout: ; Stderr: iptables-restore v1.4.21: Couldn't load target `neutron-meter-l-00599199-632':No such file or directory
2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent
2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent Error occurred at line: 17
2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent Try `iptables-restore -h' or 'iptables-restore --help' for more information.
2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent
2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent
2016-08-24 18:59:52.395 50507 DEBUG oslo_concurrency.lockutils [req-60259460-b6d3-4c75-8ccf-a09ac4a6f3c8 5c7ad3780e3e4a8393cc0668d7ae594d cce3835f1f07431788f2df68c685a1b7 - - -] Lock "metering-agent" released by "neutron.services.metering.agents.metering_agent._invoke_driver" :: held 0.142s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:282

** Affects: neutron
     Importance: Undecided
     Assignee: Zhengwei Gao (multi-task)
         Status: New

** Description changed:

- When metering-agent process add_metering_label_rule from neutron-server,
- it may fail to create iptables rule.
- 
- 2016-08-24 18:59:52.392 50507 ERROR neutron.agent.linux.utils [req-60259460-b6d3-4c75-8ccf-a09ac4a6f3c8 5c7ad3780e3e4a8393cc0668d7ae594d cce3835f1f07431788f2df68c685a1b7 - - -] Exit code: 2; Stdin: # Generated by iptables_manager
- *filter
- :neutron-meter-neutron-met - [0:0]
- :neutron-meter-r-00599199-632 - [0:0]
- -I FORWARD 2 -j neutron-meter-FORWARD
- -D FORWARD 4
- -I INPUT 1 -j neutron-meter-INPUT
- -D INPUT 3
- -I OUTPUT 2 -j neutron-meter-OUTPUT
- -D OUTPUT 4
- -I neutron-filter-top 1 -j neutron-meter-local
- -D neutron-filter-top 3
- -D neutron-meter-l-00e4e019-099 1
- -I neutron-meter-l-00e4e019-099 1
- -D neutron-meter-l-01e4e019-099 1
- -I neutron-meter-l-01e4e019-099 1
- -I neutron-meter-r-00599199-632 1 -i qg-f0732f6f-8e -d 192.168.10.0/24 -j neutron-meter-l-00599199-632
- COMMIT
- # Completed by iptables_manager
- # Generated by iptables_manager
- *raw
- -I OUTPUT 1 -j neutron-meter-OUTPUT
- -D OUTPUT 3
- -I PREROUTING 1 -j neutron-meter-PREROUTING
- -D PREROUTING 3
- COMMIT
- # Completed by iptables_manager
- ; Stdout: ; Stderr: iptables-restore v1.4.21: Couldn't load target `neutron-meter-l-00599199-632':No such file or directory
- 
- Error occurred at line: 17
- Try `iptables-restore -h' or 'iptables-restore --help' for more information.
- 
- 2016-08-24 18:59:52.393 50507 ERROR neutron.agent.linux.iptables_manager [req-60259460-b6d3-4c75-8ccf-a09ac4a6f3c8 5c7ad3780e3e4a8393cc0668d7ae594d cce3835f1f07431788f2df68c685a1b7 - - -] IPTablesManager.apply failed to apply the following set of iptables rules:
-       1. # Generated by iptables_manager
-       2. *filter
-       3. :neutron-meter-neutron-met - [0:0]
-       4. :neutron-meter-r-00599199-632 - [0:0]
-       5. -I FORWARD 2 -j neutron-meter-FORWARD
-       6. -D FORWARD 4
-       7. -I INPUT 1 -j neutron-meter-INPUT
-       8. -D INPUT 3
-       9. -I OUTPUT 2 -j neutron-meter-OUTPUT
-      10. -D OUTPUT 4
-      11. -I neutron-filter-top 1 -j neutron-meter-local
-      12. -D neutron-filter-top 3
-      13. -D neutron-meter-l-00e4e019-099 1
-      14. -I neutron-meter-l-00e4e019-099 1
-      15. -D neutron-meter-l-01e4e019-099 1
-      16. -I neutron-meter-l-01e4e019-099 1
-      17. -I neutron-meter-r-00599199-632 1 -i qg-f0732f6f-8e -d 192.168.10.0/24 -j neutron-meter-l-00599199-632
-      18. COMMIT
-      19. # Completed by iptables_manager
-      20. # Generated by iptables_manager
-      21. *raw
-      22. -I OUTPUT 1 -j neutron-meter-OUTPUT
-      23. -D OUTPUT 3
-      24. -I PREROUTING 1 -j neutron-meter-PREROUTING
-      25. -D PREROUTING 3
-      26. COMMIT
-      27. # Completed by iptables_manager
-      28.
+ When metering-agent process add_metering_label_rule from neutron-server, it may fail to create iptables rule.
  2016-08-24 18:59:52.393 50507 DEBUG oslo_concurrency.lockutils [req-60259460-b6d3-4c75-8ccf-a09ac4a6f3c8 5c7ad3780e3e4a8393cc0668d7ae594d cce3835f1f07431788f2df68c685a1b7 - - -] Releasing semaphore "iptables-qrouter-a0e4e019-0992-4a2e-83ee-5ba241156e9d" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent [req-60259460-b6d3-4c75-8ccf-a09ac4a6f3c8 5c7ad3780e3e4a8393cc0668d7ae594d cce3835f1f07431788f2df68c685a1b7 - - -] Driver neutron.services.metering.drivers.iptables.iptables_driver.IptablesMeteringDriver:add_metering_label_rule runtime error
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent Traceback (most recent call last):
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/services/metering/agents/metering_agent.py", line 258, in _invoke_driver
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     return getattr(self.metering_driver, func_name)(context, meterings)
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/oslo_log/helpers.py", line 46, in wrapper
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     return method(*args, **kwargs)
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/services/metering/drivers/iptables/iptables_driver.py", line 272, in add_metering_label_rule
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     self._add_metering_label_rule(router)
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/services/metering/drivers/iptables/iptables_driver.py", line 285, in _add_metering_label_rule
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     self._process_metering_rule_action(router, 'create')
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/services/metering/drivers/iptables/iptables_driver.py", line 327, in _process_metering_rule_action
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     rules_chain)
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/services/metering/drivers/iptables/iptables_driver.py", line 58, in __exit__
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     self.im.apply()
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_manager.py", line 477, in apply
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     return self._apply()
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_manager.py", line 485, in _apply
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     return self._apply_synchronized()
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_manager.py", line 562, in _apply_synchronized
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     '\n'.join(log_lines))
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     self.force_reraise()
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     six.reraise(self.type_, self.value, self.tb)
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_manager.py", line 541, in _apply_synchronized
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     run_as_root=True)
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 140, in execute
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     raise RuntimeError(msg)
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent RuntimeError: Exit code: 2; Stdin: # Generated by iptables_manager
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent *filter
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent :neutron-meter-neutron-met - [0:0]
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent :neutron-meter-r-00599199-632 - [0:0]
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I FORWARD 2 -j neutron-meter-FORWARD
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D FORWARD 4
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I INPUT 1 -j neutron-meter-INPUT
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D INPUT 3
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I OUTPUT 2 -j neutron-meter-OUTPUT
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D OUTPUT 4
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I neutron-filter-top 1 -j neutron-meter-local
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D neutron-filter-top 3
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D neutron-meter-l-00e4e019-099 1
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I neutron-meter-l-00e4e019-099 1
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D neutron-meter-l-01e4e019-099 1
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I neutron-meter-l-01e4e019-099 1
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I neutron-meter-r-00599199-632 1 -i qg-f0732f6f-8e -d 192.168.10.0/24 -j neutron-meter-l-00599199-632
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent COMMIT
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent # Completed by iptables_manager
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent # Generated by iptables_manager
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent *raw
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I OUTPUT 1 -j neutron-meter-OUTPUT
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D OUTPUT 3
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I PREROUTING 1 -j neutron-meter-PREROUTING
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D PREROUTING 3
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent COMMIT
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent # Completed by iptables_manager
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent ; Stdout: ; Stderr: iptables-restore v1.4.21: Couldn't load target `neutron-meter-l-00599199-632':No such file or directory
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent Error occurred at line: 17
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent Try `iptables-restore -h' or 'iptables-restore --help' for more information.
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent
  2016-08-24 18:59:52.395 50507 DEBUG oslo_concurrency.lockutils [req-60259460-b6d3-4c75-8ccf-a09ac4a6f3c8 5c7ad3780e3e4a8393cc0668d7ae594d cce3835f1f07431788f2df68c685a1b7 - - -] Lock "metering-agent" released by "neutron.services.metering.agents.metering_agent._invoke_driver" :: held 0.142s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:282
  2016-08-24 18:59:52.413 50507 DEBUG oslo_messaging._drivers.amqpdriver [-] received message msg_id: None reply to None __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:201
  2016-08-24 18:59:52.414 50507 DEBUG oslo_concurrency.lockutils [req-60259460-b6d3-4c75-8ccf-a09ac4a6f3c8 5c7ad3780e3e4a8393cc0668d7ae594d cce3835f1f07431788f2df68c685a1b7 - - -] Lock "metering-agent" acquired by "neutron.services.metering.agents.metering_agent._invoke_driver" :: waited 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:270

** Description changed:

- When metering-agent process add_metering_label_rule from neutron-server, it may fail to create iptables rule.
- 2016-08-24 18:59:52.393 50507 DEBUG oslo_concurrency.lockutils [req-60259460-b6d3-4c75-8ccf-a09ac4a6f3c8 5c7ad3780e3e4a8393cc0668d7ae594d cce3835f1f07431788f2df68c685a1b7 - - -] Releasing semaphore "iptables-qrouter-a0e4e019-0992-4a2e-83ee-5ba241156e9d" lock /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:225
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent [req-60259460-b6d3-4c75-8ccf-a09ac4a6f3c8 5c7ad3780e3e4a8393cc0668d7ae594d cce3835f1f07431788f2df68c685a1b7 - - -] Driver neutron.services.metering.drivers.iptables.iptables_driver.IptablesMeteringDriver:add_metering_label_rule runtime error
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent Traceback (most recent call last):
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/services/metering/agents/metering_agent.py", line 258, in _invoke_driver
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     return getattr(self.metering_driver, func_name)(context, meterings)
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/oslo_log/helpers.py", line 46, in wrapper
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     return method(*args, **kwargs)
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/services/metering/drivers/iptables/iptables_driver.py", line 272, in add_metering_label_rule
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     self._add_metering_label_rule(router)
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/services/metering/drivers/iptables/iptables_driver.py", line 285, in _add_metering_label_rule
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     self._process_metering_rule_action(router, 'create')
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/services/metering/drivers/iptables/iptables_driver.py", line 327, in _process_metering_rule_action
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     rules_chain)
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/services/metering/drivers/iptables/iptables_driver.py", line 58, in __exit__
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     self.im.apply()
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_manager.py", line 477, in apply
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     return self._apply()
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_manager.py", line 485, in _apply
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     return self._apply_synchronized()
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_manager.py", line 562, in _apply_synchronized
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     '\n'.join(log_lines))
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     self.force_reraise()
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     six.reraise(self.type_, self.value, self.tb)
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/iptables_manager.py", line 541, in _apply_synchronized
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     run_as_root=True)
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent   File "/usr/lib/python2.7/site-packages/neutron/agent/linux/utils.py", line 140, in execute
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent     raise RuntimeError(msg)
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent RuntimeError: Exit code: 2; Stdin: # Generated by iptables_manager
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent *filter
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent :neutron-meter-neutron-met - [0:0]
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent :neutron-meter-r-00599199-632 - [0:0]
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I FORWARD 2 -j neutron-meter-FORWARD
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D FORWARD 4
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I INPUT 1 -j neutron-meter-INPUT
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D INPUT 3
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I OUTPUT 2 -j neutron-meter-OUTPUT
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D OUTPUT 4
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I neutron-filter-top 1 -j neutron-meter-local
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D neutron-filter-top 3
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D neutron-meter-l-00e4e019-099 1
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I neutron-meter-l-00e4e019-099 1
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D neutron-meter-l-01e4e019-099 1
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I neutron-meter-l-01e4e019-099 1
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I neutron-meter-r-00599199-632 1 -i qg-f0732f6f-8e -d 192.168.10.0/24 -j neutron-meter-l-00599199-632
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent COMMIT
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent # Completed by iptables_manager
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent # Generated by iptables_manager
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent *raw
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I OUTPUT 1 -j neutron-meter-OUTPUT
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D OUTPUT 3
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I PREROUTING 1 -j neutron-meter-PREROUTING
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D PREROUTING 3
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent COMMIT
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent # Completed by iptables_manager
- 2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent ; Stdout: ; Stderr: iptables-restore v1.4.21: Couldn't load target `neutron-meter-l-00599199-632':No such file or directory
+ When metering-agent process add_metering_label_rule from neutron-server,
+ it may fail to create iptables rule.
+ 
+ 
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent Error occurred at line: 17
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent Try `iptables-restore -h' or 'iptables-restore --help' for more information.
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent
  2016-08-24 18:59:52.395 50507 DEBUG oslo_concurrency.lockutils [req-60259460-b6d3-4c75-8ccf-a09ac4a6f3c8 5c7ad3780e3e4a8393cc0668d7ae594d cce3835f1f07431788f2df68c685a1b7 - - -] Lock "metering-agent" released by "neutron.services.metering.agents.metering_agent._invoke_driver" :: held 0.142s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:282
  2016-08-24 18:59:52.413 50507 DEBUG oslo_messaging._drivers.amqpdriver [-] received message msg_id: None reply to None __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:201
  2016-08-24 18:59:52.414 50507 DEBUG oslo_concurrency.lockutils [req-60259460-b6d3-4c75-8ccf-a09ac4a6f3c8 5c7ad3780e3e4a8393cc0668d7ae594d cce3835f1f07431788f2df68c685a1b7 - - -] Lock "metering-agent" acquired by "neutron.services.metering.agents.metering_agent._invoke_driver" :: waited 0.000s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:270

** Changed in: neutron
     Assignee: (unassigned) => Zhengwei Gao (multi-task)

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1617248

Title:
  Metering-agent fail to create iptables rule for
  add_metering_label_rule

Status in neutron:
  New

Bug description:
  When metering-agent process add_metering_label_rule from neutron-
  server, it may fail to create iptables rule.

  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I neutron-meter-r-00599199-632 1 -i qg-f0732f6f-8e -d 192.168.10.0/24 -j neutron-meter-l-00599199-632
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent COMMIT
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent # Completed by iptables_manager
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent # Generated by iptables_manager
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent *raw
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I OUTPUT 1 -j neutron-meter-OUTPUT
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D OUTPUT 3
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -I PREROUTING 1 -j neutron-meter-PREROUTING
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent -D PREROUTING 3
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent COMMIT
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent # Completed by iptables_manager
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent ; Stdout: ; Stderr: iptables-restore v1.4.21: Couldn't load target `neutron-meter-l-00599199-632':No such file or directory
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent Error occurred at line: 17
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent Try `iptables-restore -h' or 'iptables-restore --help' for more information.
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent
  2016-08-24 18:59:52.393 50507 ERROR neutron.services.metering.agents.metering_agent
  2016-08-24 18:59:52.395 50507 DEBUG oslo_concurrency.lockutils [req-60259460-b6d3-4c75-8ccf-a09ac4a6f3c8 5c7ad3780e3e4a8393cc0668d7ae594d cce3835f1f07431788f2df68c685a1b7 - - -] Lock "metering-agent" released by "neutron.services.metering.agents.metering_agent._invoke_driver" :: held 0.142s inner /usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py:282

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1617248/+subscriptions
Follow ups

[Bug 1617248] Re: Metering-agent fail to create iptables rule for add_metering_label_rule
From: OpenStack Infra, 2017-05-02