← Back to team overview

yahoo-eng-team team mailing list archive

  1. yahoo-eng-team team
  2. Mailing list archive
  3. Message #93415

[Bug 2052367] [NEW] neutron-keepalived-state-change run neutron-rootwrap-daemon anyway regardless of config

  Thread Previous • Date Previous • • Thread Next 
Public bug reported:

neutron-keepalived-state-change run neutron-rootwrap-daemon anyway
regardless of config

https://review.opendev.org/q/topic:%22bug/1680183%22
It is the series of patches which replace "ip -o monitor" to pyroute2 python native process.

But we noticed that the neutron-keepalived-state-change run neutron-
rootwrap-daemon anyway. It is not configurable.

A example process structure is:
root     30474     1  0 10:23 ?        00:00:00 /usr/bin/python /bin/neutron-keepalived-state-change --router_id=0a3f589a-40f7-400c-8cb7-e915c255dec1 --namespace=qrouter-0a3f589a-40f7-400c-8cb7-e915c255dec1 --conf_dir=/var/lib/neutron/ha_confs/0a3f589a-40f7-400c-8cb7-e915c255dec1 --log-file=/var/lib/neutron/ha_confs/0a3f589a-40f7-400c-8cb7-e915c255dec1/neutron-keepalived-state-change.log --monitor_interface=ha-1c854712-f1 --monitor_cidr=169.254.0.167/24 --pid_file=/var/lib/neutron/external/pids/0a3f589a-40f7-400c-8cb7-e915c255dec1.monitor.pid.neutron-keepalived-state-change-monitor --state_path=/var/lib/neutron --user=993 --group=990
root     30478 30474  0 10:23 ?        00:00:00 sudo neutron-rootwrap-daemon /etc/neutron/rootwrap.conf
root     30482 30478  0 10:23 ?        00:00:00 /usr/bin/python /bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf
root     30479 30474  0 10:23 ?        00:00:00 /usr/bin/python /bin/neutron-keepalived-state-change --router_id=0a3f589a-40f7-400c-8cb7-e915c255dec1 --namespace=qrouter-0a3f589a-40f7-400c-8cb7-e915c255dec1 --conf_dir=/var/lib/neutron/ha_confs/0a3f589a-40f7-400c-8cb7-e915c255dec1 --log-file=/var/lib/neutron/ha_confs/0a3f589a-40f7-400c-8cb7-e915c255dec1/neutron-keepalived-state-change.log --monitor_interface=ha-1c854712-f1 --monitor_cidr=169.254.0.167/24 --pid_file=/var/lib/neutron/external/pids/0a3f589a-40f7-400c-8cb7-e915c255dec1.monitor.pid.neutron-keepalived-state-change-monitor --state_path=/var/lib/neutron --user=993 --group=990

We can see that the neutron-rootwrap-daemon did not spawn the "neutron-
keepalived-state-change" sub-process.

And that "sudo neutron-rootwrap-daemon /etc/neutron/rootwrap.conf" will
run into a zombie process finally.

neutron-keepalived-state-change.log LOGs:

2024-02-04 10:23:21.022 30449 INFO neutron.common.config [-] Logging enabled!
2024-02-04 10:23:21.023 30449 INFO neutron.common.config [-] /bin/neutron-keepalived-state-change version 13.7.2.dev20240202170237
2024-02-04 10:23:21.023 30449 DEBUG neutron.common.config [-] command line: /bin/neutron-keepalived-state-change --router_id=0a3f589a-40f7-400c-8cb7-e915c255dec1 --namespace=qrouter-0a3f589a-40f7-400c-8cb7-e915c255dec1 --conf_dir=/var/lib/neutron/ha_confs/0a3f589a-40f7-400c-8cb7-e915c255dec1 --log-file=/var/lib/neutron/ha_confs/0a3f589a-40f7-400c-8cb7-e915c255dec1/neutron-keepalived-state-change.log --monitor_interface=ha-1c854712-f1 --monitor_cidr=169.254.0.167/24 --pid_file=/var/lib/neutron/external/pids/0a3f589a-40f7-400c-8cb7-e915c255dec1.monitor.pid.neutron-keepalived-state-change-monitor --state_path=/var/lib/neutron --user=993 --group=990 setup_logging /usr/lib/python/site-packages/neutron/common/config.py:106
2024-02-04 10:23:21.033 30474 DEBUG neutron.agent.linux.utils [-] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-0a3f589a-40f7-400c-8cb7-e915c255dec1', 'ip', 'addr', 'show', 'ha-1c854712-f1'] execute_rootwrap_daemon /usr/lib/python/site-packages/neutron/agent/linux/utils.py:103
2024-02-04 10:23:21.043 30474 DEBUG oslo_rootwrap.client [-] Popen for ['sudo', 'neutron-rootwrap-daemon', '/etc/neutron/rootwrap.conf'] command has been instantiated _initialize /usr/lib/python/site-packages/oslo_rootwrap/client.py:73
2024-02-04 10:23:21.185 30474 INFO oslo_rootwrap.client [-] Spawned new rootwrap daemon process with pid=30478
2024-02-04 10:23:21.232 30474 DEBUG neutron.agent.l3.keepalived_state_change [-] Wrote router 0a3f589a-40f7-400c-8cb7-e915c255dec1 state master write_state_change /usr/lib/python/site-packages/neutron/agent/l3/keepalived_state_change.py:123
2024-02-04 10:23:21.235 30474 DEBUG neutron.agent.l3.keepalived_state_change [-] Notified agent router 0a3f589a-40f7-400c-8cb7-e915c255dec1, state master notify_agent /usr/lib/python/site-packages/neutron/agent/l3/keepalived_state_change.py:137
2024-02-04 10:23:21.236 30474 DEBUG neutron.agent.l3.keepalived_state_change [-] Initial status of router 0a3f589a-40f7-400c-8cb7-e915c255dec1 is master handle_initial_state /usr/lib/python/site-packages/neutron/agent/l3/keepalived_state_change.py:114


So the code of
https://github.com/openstack/neutron/blob/master/neutron/agent/l3/ha_router.py#L433-L444
should add some configurable options to decide whether run neutron-rootwrap-daemon the neutron-keepalived-state-change.

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2052367

Title:
  neutron-keepalived-state-change run neutron-rootwrap-daemon anyway
  regardless of config

Status in neutron:
  New

Bug description:
  neutron-keepalived-state-change run neutron-rootwrap-daemon anyway
  regardless of config

  https://review.opendev.org/q/topic:%22bug/1680183%22
  It is the series of patches which replace "ip -o monitor" to pyroute2 python native process.

  But we noticed that the neutron-keepalived-state-change run neutron-
  rootwrap-daemon anyway. It is not configurable.

  A example process structure is:
  root     30474     1  0 10:23 ?        00:00:00 /usr/bin/python /bin/neutron-keepalived-state-change --router_id=0a3f589a-40f7-400c-8cb7-e915c255dec1 --namespace=qrouter-0a3f589a-40f7-400c-8cb7-e915c255dec1 --conf_dir=/var/lib/neutron/ha_confs/0a3f589a-40f7-400c-8cb7-e915c255dec1 --log-file=/var/lib/neutron/ha_confs/0a3f589a-40f7-400c-8cb7-e915c255dec1/neutron-keepalived-state-change.log --monitor_interface=ha-1c854712-f1 --monitor_cidr=169.254.0.167/24 --pid_file=/var/lib/neutron/external/pids/0a3f589a-40f7-400c-8cb7-e915c255dec1.monitor.pid.neutron-keepalived-state-change-monitor --state_path=/var/lib/neutron --user=993 --group=990
  root     30478 30474  0 10:23 ?        00:00:00 sudo neutron-rootwrap-daemon /etc/neutron/rootwrap.conf
  root     30482 30478  0 10:23 ?        00:00:00 /usr/bin/python /bin/neutron-rootwrap-daemon /etc/neutron/rootwrap.conf
  root     30479 30474  0 10:23 ?        00:00:00 /usr/bin/python /bin/neutron-keepalived-state-change --router_id=0a3f589a-40f7-400c-8cb7-e915c255dec1 --namespace=qrouter-0a3f589a-40f7-400c-8cb7-e915c255dec1 --conf_dir=/var/lib/neutron/ha_confs/0a3f589a-40f7-400c-8cb7-e915c255dec1 --log-file=/var/lib/neutron/ha_confs/0a3f589a-40f7-400c-8cb7-e915c255dec1/neutron-keepalived-state-change.log --monitor_interface=ha-1c854712-f1 --monitor_cidr=169.254.0.167/24 --pid_file=/var/lib/neutron/external/pids/0a3f589a-40f7-400c-8cb7-e915c255dec1.monitor.pid.neutron-keepalived-state-change-monitor --state_path=/var/lib/neutron --user=993 --group=990

  We can see that the neutron-rootwrap-daemon did not spawn the
  "neutron-keepalived-state-change" sub-process.

  And that "sudo neutron-rootwrap-daemon /etc/neutron/rootwrap.conf"
  will run into a zombie process finally.

  neutron-keepalived-state-change.log LOGs:

  2024-02-04 10:23:21.022 30449 INFO neutron.common.config [-] Logging enabled!
  2024-02-04 10:23:21.023 30449 INFO neutron.common.config [-] /bin/neutron-keepalived-state-change version 13.7.2.dev20240202170237
  2024-02-04 10:23:21.023 30449 DEBUG neutron.common.config [-] command line: /bin/neutron-keepalived-state-change --router_id=0a3f589a-40f7-400c-8cb7-e915c255dec1 --namespace=qrouter-0a3f589a-40f7-400c-8cb7-e915c255dec1 --conf_dir=/var/lib/neutron/ha_confs/0a3f589a-40f7-400c-8cb7-e915c255dec1 --log-file=/var/lib/neutron/ha_confs/0a3f589a-40f7-400c-8cb7-e915c255dec1/neutron-keepalived-state-change.log --monitor_interface=ha-1c854712-f1 --monitor_cidr=169.254.0.167/24 --pid_file=/var/lib/neutron/external/pids/0a3f589a-40f7-400c-8cb7-e915c255dec1.monitor.pid.neutron-keepalived-state-change-monitor --state_path=/var/lib/neutron --user=993 --group=990 setup_logging /usr/lib/python/site-packages/neutron/common/config.py:106
  2024-02-04 10:23:21.033 30474 DEBUG neutron.agent.linux.utils [-] Running command (rootwrap daemon): ['ip', 'netns', 'exec', 'qrouter-0a3f589a-40f7-400c-8cb7-e915c255dec1', 'ip', 'addr', 'show', 'ha-1c854712-f1'] execute_rootwrap_daemon /usr/lib/python/site-packages/neutron/agent/linux/utils.py:103
  2024-02-04 10:23:21.043 30474 DEBUG oslo_rootwrap.client [-] Popen for ['sudo', 'neutron-rootwrap-daemon', '/etc/neutron/rootwrap.conf'] command has been instantiated _initialize /usr/lib/python/site-packages/oslo_rootwrap/client.py:73
  2024-02-04 10:23:21.185 30474 INFO oslo_rootwrap.client [-] Spawned new rootwrap daemon process with pid=30478
  2024-02-04 10:23:21.232 30474 DEBUG neutron.agent.l3.keepalived_state_change [-] Wrote router 0a3f589a-40f7-400c-8cb7-e915c255dec1 state master write_state_change /usr/lib/python/site-packages/neutron/agent/l3/keepalived_state_change.py:123
  2024-02-04 10:23:21.235 30474 DEBUG neutron.agent.l3.keepalived_state_change [-] Notified agent router 0a3f589a-40f7-400c-8cb7-e915c255dec1, state master notify_agent /usr/lib/python/site-packages/neutron/agent/l3/keepalived_state_change.py:137
  2024-02-04 10:23:21.236 30474 DEBUG neutron.agent.l3.keepalived_state_change [-] Initial status of router 0a3f589a-40f7-400c-8cb7-e915c255dec1 is master handle_initial_state /usr/lib/python/site-packages/neutron/agent/l3/keepalived_state_change.py:114

  
  So the code of
  https://github.com/openstack/neutron/blob/master/neutron/agent/l3/ha_router.py#L433-L444
  should add some configurable options to decide whether run neutron-rootwrap-daemon the neutron-keepalived-state-change.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2052367/+subscriptions
  Thread Previous • Date Previous • • Thread Next