yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1097999] Re: multi-process metadata server runs iptables setup multiple times

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Thierry Carrez <thierry.carrez+lp@xxxxxxxxx>
Date: Wed, 22 Jan 2014 16:16:28 -0000
Reply-to: Bug 1097999 <1097999@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
** Changed in: nova
       Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1097999

Title:
  multi-process metadata server runs iptables  setup multiple times

Status in OpenStack Compute (Nova):
  Fix Released

Bug description:
  Using Devstack (latest master 1/9/2012)

  If I enable multi process metadata service (metadata_workers=5 in
  nova.conf)

  the iptables are modified multiple times:
  2013-01-10 00:25:16.0 25535 INFO nova.wsgi [-] metadata listening on 0.0.0.0:8775
  2013-01-10 00:25:16.1 25535 INFO nova.service [-] Starting 5 workers
  2013-01-10 00:25:16.3 25535 INFO nova.service [-] Started child 255442013-01-10 00:25:16.6 25535 INFO nova.service [-] Started child 255452013-01-10 00:25:16.8 25535 INFO nova.service [-] Started child 25546
  2013-01-10 00:25:16.9 25544 DEBUG nova.openstack.common.lockutils [-] Got semaphore "iptables" for method "_apply"... inner /opt/stack/nova/nova/openstack/common/lockutils.py:185
  2013-01-10 00:25:16.10 25544 DEBUG nova.openstack.common.lockutils [-] Attempting to grab file lock "iptables" for method "_apply"... inner /opt/stack/nova/nova/openstack/common/lockutils.py:189
  2013-01-10 00:25:16.11 25544 DEBUG nova.openstack.common.lockutils [-] Got file lock "iptables" at /opt/stack/data/nova/nova-iptables for method "_apply"... inner /opt/stack/nova/nova/openstack/common/lockutils.py:219
  2013-01-10 00:25:16.11 25535 INFO nova.service [-] Started child 25547
  2013-01-10 00:25:16.18 25544 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-save -c -t filter execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:16.20 25535 INFO nova.service [-] Started child 25548
  2013-01-10 00:25:16.16 25546 DEBUG nova.openstack.common.lockutils [-] Got semaphore "iptables" for method "_apply"... inner /opt/stack/nova/nova/openstack/common/lockutils.py:185
  2013-01-10 00:25:16.22 25546 DEBUG nova.openstack.common.lockutils [-] Attempting to grab file lock "iptables" for method "_apply"... inner /opt/stack/nova/nova/openstack/common/lockutils.py:189
  2013-01-10 00:25:16.27 25548 DEBUG nova.openstack.common.lockutils [-] Got semaphore "iptables" for method "_apply"... inner /opt/stack/nova/nova/openstack/common/lockutils.py:185
  2013-01-10 00:25:16.17 25545 DEBUG nova.openstack.common.lockutils [-] Got semaphore "iptables" for method "_apply"... inner /opt/stack/nova/nova/openstack/common/lockutils.py:185
  2013-01-10 00:25:16.33 25545 DEBUG nova.openstack.common.lockutils [-] Attempting to grab file lock "iptables" for method "_apply"... inner /opt/stack/nova/nova/openstack/common/lockutils.py:189
  2013-01-10 00:25:16.32 25547 DEBUG nova.openstack.common.lockutils [-] Got semaphore "iptables" for method "_apply"... inner /opt/stack/nova/nova/openstack/common/lockutils.py:185
  2013-01-10 00:25:16.33 25547 DEBUG nova.openstack.common.lockutils [-] Attempting to grab file lock "iptables" for method "_apply"... inner /opt/stack/nova/nova/openstack/common/lockutils.py:189
  2013-01-10 00:25:16.42 25548 DEBUG nova.openstack.common.lockutils [-] Attempting to grab file lock "iptables" for method "_apply"... inner /opt/stack/nova/nova/openstack/common/lockutils.py:189
  2013-01-10 00:25:16.214 25544 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:16.216 25544 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore -c execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:16.368 25544 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:16.370 25544 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-save -c -t mangle execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:16.535 25544 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:16.536 25544 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore -c execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:16.703 25544 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:16.704 25544 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-save -c -t nat execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:16.855 25544 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:16.856 25544 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore -c execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:17.18 25544 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:17.19 25544 DEBUG nova.network.linux_net [-] IPTablesManager.apply completed with success _apply /opt/stack/nova/nova/network/linux_net.py:385
  2013-01-10 00:25:17.20 25544 INFO nova.metadata.wsgi.server [-] (25544) wsgi starting up on http://0.0.0.0:8775/

  2013-01-10 00:25:17.25 25548 DEBUG nova.openstack.common.lockutils [-] Got file lock "iptables" at /opt/stack/data/nova/nova-iptables for method "_apply"... inner /opt/stack/nova/nova/openstack/common/lockutils.py:219
  2013-01-10 00:25:17.26 25548 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-save -c -t filter execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:17.180 25548 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:17.182 25548 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore -c execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:17.340 25548 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:17.342 25548 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-save -c -t mangle execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:17.510 25548 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:17.511 25548 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore -c execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:17.666 25548 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:17.667 25548 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-save -c -t nat execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:17.800 25548 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:17.801 25548 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore -c execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:17.952 25548 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:17.953 25548 DEBUG nova.network.linux_net [-] IPTablesManager.apply completed with success _apply /opt/stack/nova/nova/network/linux_net.py:385
  2013-01-10 00:25:17.955 25548 INFO nova.metadata.wsgi.server [-] (25548) wsgi starting up on http://0.0.0.0:8775/

  2013-01-10 00:25:17.962 25546 DEBUG nova.openstack.common.lockutils [-] Got file lock "iptables" at /opt/stack/data/nova/nova-iptables for method "_apply"... inner /opt/stack/nova/nova/openstack/common/lockutils.py:219
  2013-01-10 00:25:17.964 25546 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-save -c -t filter execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:18.120 25546 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:18.121 25546 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore -c execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:18.278 25546 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:18.279 25546 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-save -c -t mangle execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:18.439 25546 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:18.441 25546 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore -c execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:18.596 25546 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:18.598 25546 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-save -c -t nat execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:18.750 25546 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:18.751 25546 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore -c execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:18.932 25546 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:18.932 25546 DEBUG nova.network.linux_net [-] IPTablesManager.apply completed with success _apply /opt/stack/nova/nova/network/linux_net.py:385
  2013-01-10 00:25:18.934 25546 INFO nova.metadata.wsgi.server [-] (25546) wsgi starting up on http://0.0.0.0:8775/

  2013-01-10 00:25:18.941 25545 DEBUG nova.openstack.common.lockutils [-] Got file lock "iptables" at /opt/stack/data/nova/nova-iptables for method "_apply"... inner /opt/stack/nova/nova/openstack/common/lockutils.py:219
  2013-01-10 00:25:18.942 25545 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-save -c -t filter execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:19.94 25545 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:19.95 25545 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore -c execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:19.210 25545 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:19.210 25545 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-save -c -t mangle execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:19.328 25545 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:19.330 25545 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore -c execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:19.470 25545 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:19.471 25545 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-save -c -t nat execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:19.617 25545 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:19.619 25545 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore -c execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:19.735 25545 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:19.736 25545 DEBUG nova.network.linux_net [-] IPTablesManager.apply completed with success _apply /opt/stack/nova/nova/network/linux_net.py:385
  2013-01-10 00:25:19.737 25547 DEBUG nova.openstack.common.lockutils [-] Got file lock "iptables" at /opt/stack/data/nova/nova-iptables for method "_apply"... inner /opt/stack/nova/nova/openstack/common/lockutils.py:219
  2013-01-10 00:25:19.738 25547 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-save -c -t filter execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:19.739 25545 INFO nova.metadata.wsgi.server [-] (25545) wsgi starting up on http://0.0.0.0:8775/

  2013-01-10 00:25:19.854 25547 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:19.855 25547 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore -c execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:19.971 25547 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:19.972 25547 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-save -c -t mangle execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:20.88 25547 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:20.89 25547 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore -c execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:20.205 25547 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:20.206 25547 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-save -c -t nat execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:20.327 25547 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:20.328 25547 DEBUG nova.utils [-] Running cmd (subprocess): sudo nova-rootwrap /etc/nova/rootwrap.conf iptables-restore -c execute /opt/stack/nova/nova/utils.py:202
  2013-01-10 00:25:20.447 25547 DEBUG nova.utils [-] Result was 0 execute /opt/stack/nova/nova/utils.py:226
  2013-01-10 00:25:20.447 25547 DEBUG nova.network.linux_net [-] IPTablesManager.apply completed with success _apply /opt/stack/nova/nova/network/linux_net.py:385
  2013-01-10 00:25:20.449 25547 INFO nova.metadata.wsgi.server [-] (25547) wsgi starting up on http://0.0.0.0:8775/

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1097999/+subscriptions