yahoo-eng-team team mailing list archive

Thread
Date
[Bug 2026122] [NEW] neutron-l3-agent permissin denied when execute ( list_network_namespaces)

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Hristo Ispirov <2026122@xxxxxxxxxxxxxxxxxx>
Date: Wed, 05 Jul 2023 12:29:26 -0000
Reply-to: Bug 2026122 <2026122@xxxxxxxxxxxxxxxxxx>
Sender: noreply@xxxxxxxxxxxxx
Public bug reported:

I have neutron manual installation on ubuntu server 22.04.
Everything was working fine before I rebooted the machine.
Now I have this error: 

2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task [None req-835bfb24-967b-44a4-849a-c661efe0c7e5 - - - - - -] Error during L3NATAgentWithStateReport.periodic_sync_routers_task: PermissionError: [Errno 13] Permission denied
2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task Traceback (most recent call last):
2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task   File "/usr/lib/python3/dist-packages/oslo_service/periodic_task.py", line 216, in run_periodic_tasks
2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task     task(self, context)
2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task   File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 887, in periodic_sync_routers_task
2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task     with self.namespaces_manager as ns_manager:
2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task   File "/usr/lib/python3/dist-packages/neutron/agent/l3/namespace_manager.py", line 71, in __enter__
2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task     self._all_namespaces = self.list_all()
2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task   File "/usr/lib/python3/dist-packages/neutron/agent/l3/namespace_manager.py", line 117, in list_all
2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task     namespaces = ip_lib.list_network_namespaces()
2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task   File "/usr/lib/python3/dist-packages/neutron/agent/linux/ip_lib.py", line 963, in list_network_namespaces
2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task     return privileged.list_netns(**kwargs)
2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task   File "/usr/lib/python3/dist-packages/oslo_privsep/priv_context.py", line 271, in _wrap
2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task     return self.channel.remote_call(name, args, kwargs,
2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task   File "/usr/lib/python3/dist-packages/oslo_privsep/daemon.py", line 215, in remote_call
2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task     raise exc_type(*result[2])
2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task PermissionError: [Errno 13] Permission denied
2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task

My /etc/sudoers.d/neutron_sudoers fail contains :

Defaults:neutron !requiretty

neutron ALL = (root) NOPASSWD: /usr/bin/neutron-rootwrap
/etc/neutron/rootwrap.conf *


My /etc/neutron/rootwrap.d/rootwrap.filters fail contains:

# Command filters to allow privsep daemon to be started via rootwrap.
#
# This file should be owned by (and only-writeable by) the root user

[Filters]

# By installing the following, the local admin is asserting that:
#
# 1. The python module load path used by privsep-helper
#    command as root (as started by sudo/rootwrap) is trusted.
# 2. Any oslo.config files matching the --config-file
#    arguments below are trusted.
# 3. Users allowed to run sudo/rootwrap with this configuration(*) are
#    also allowed to invoke python "entrypoint" functions from
#    --privsep_context with the additional (possibly root) privileges
#    configured for that context.
#
# (*) ie: the user is allowed by /etc/sudoers to run rootwrap as root
#
# In particular, the oslo.config and python module path must not
# be writeable by the unprivileged user.

# PRIVSEP
# oslo.privsep default neutron context
privsep: PathFilter, privsep-helper, root,
 --config-file, /etc/(?!\.\.).*,
 --privsep_context, neutron.privileged.default,
 --privsep_sock_path, /

# NOTE: A second `--config-file` arg can also be added above. Since
# many neutron components are installed like that (eg: by devstack).
# Adjust to suit local requirements.

# DEBUG
sleep: RegExpFilter, sleep, root, sleep, \d+

# EXECUTE COMMANDS IN A NAMESPACE
ip: IpFilter, ip, root
ip_exec: IpNetnsExecFilter, ip, root

# METADATA PROXY
haproxy: RegExpFilter, haproxy, root, haproxy, -f, .*
haproxy_env: EnvFilter, env, root, PROCESS_TAG=, haproxy, -f, .*

# DHCP
dnsmasq: CommandFilter, dnsmasq, root
dnsmasq_env: EnvFilter, env, root, PROCESS_TAG=, dnsmasq

# DIBBLER
dibbler-client: CommandFilter, dibbler-client, root
dibbler-client_env: EnvFilter, env, root, PROCESS_TAG=, dibbler-client

# L3
radvd: CommandFilter, radvd, root
radvd_env: EnvFilter, env, root, PROCESS_TAG=, radvd
keepalived: CommandFilter, keepalived, root
keepalived_env: EnvFilter, env, root, PROCESS_TAG=, keepalived
keepalived_state_change: CommandFilter, neutron-keepalived-state-change, root
keepalived_state_change_env: EnvFilter, env, root, PROCESS_TAG=, neutron-keepalived-state-change

# OPEN VSWITCH
ovs-ofctl: CommandFilter, ovs-ofctl, root
ovsdb-client: CommandFilter, ovsdb-client, root


My /etc/neutron/rootwrap.conf fail contains :
# Configuration for neutron-rootwrap
# This file should be owned by (and only-writeable by) the root user

[DEFAULT]
# List of directories to load filter definitions from (separated by ',').
# These directories MUST all be only writeable by root !
filters_path=/etc/neutron/rootwrap.d,/usr/share/neutron/rootwrap

# List of directories to search executables in, in case filters do not
# explicitely specify a full path (separated by ',')
# If not specified, defaults to system PATH environment variable.
# These directories MUST all be only writeable by root !
exec_dirs=/sbin,/usr/sbin,/bin,/usr/bin,/usr/local/bin,/usr/local/sbin,/etc/neutron/kill_scripts

# Enable logging to syslog
# Default value is False
use_syslog=False

# Which syslog facility to use.
# Valid values include auth, authpriv, syslog, local0, local1...
# Default value is 'syslog'
syslog_log_facility=syslog

# Which messages to log.
# INFO means log all usage
# ERROR means only log unsuccessful attempts
syslog_log_level=ERROR

# Rootwrap daemon exits after this seconds of inactivity
daemon_timeout=600

# Rootwrap daemon limits itself to that many file descriptors (Linux only)
rlimit_nofile=1024


In neutron config I have this :
[agent]
root_helper = "sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf"

I am not sure if it is neutron error or oslo.privsep error.

** Affects: neutron
     Importance: Undecided
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2026122

Title:
  neutron-l3-agent permissin denied when execute (
  list_network_namespaces)

Status in neutron:
  New

Bug description:
  I have neutron manual installation on ubuntu server 22.04.
  Everything was working fine before I rebooted the machine.
  Now I have this error: 

  2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task [None req-835bfb24-967b-44a4-849a-c661efe0c7e5 - - - - - -] Error during L3NATAgentWithStateReport.periodic_sync_routers_task: PermissionError: [Errno 13] Permission denied
  2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task Traceback (most recent call last):
  2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task   File "/usr/lib/python3/dist-packages/oslo_service/periodic_task.py", line 216, in run_periodic_tasks
  2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task     task(self, context)
  2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task   File "/usr/lib/python3/dist-packages/neutron/agent/l3/agent.py", line 887, in periodic_sync_routers_task
  2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task     with self.namespaces_manager as ns_manager:
  2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task   File "/usr/lib/python3/dist-packages/neutron/agent/l3/namespace_manager.py", line 71, in __enter__
  2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task     self._all_namespaces = self.list_all()
  2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task   File "/usr/lib/python3/dist-packages/neutron/agent/l3/namespace_manager.py", line 117, in list_all
  2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task     namespaces = ip_lib.list_network_namespaces()
  2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task   File "/usr/lib/python3/dist-packages/neutron/agent/linux/ip_lib.py", line 963, in list_network_namespaces
  2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task     return privileged.list_netns(**kwargs)
  2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task   File "/usr/lib/python3/dist-packages/oslo_privsep/priv_context.py", line 271, in _wrap
  2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task     return self.channel.remote_call(name, args, kwargs,
  2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task   File "/usr/lib/python3/dist-packages/oslo_privsep/daemon.py", line 215, in remote_call
  2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task     raise exc_type(*result[2])
  2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task PermissionError: [Errno 13] Permission denied
  2023-07-05 12:20:55.449 48258 ERROR oslo_service.periodic_task

  My /etc/sudoers.d/neutron_sudoers fail contains :

  Defaults:neutron !requiretty

  neutron ALL = (root) NOPASSWD: /usr/bin/neutron-rootwrap
  /etc/neutron/rootwrap.conf *

  
  My /etc/neutron/rootwrap.d/rootwrap.filters fail contains:

  # Command filters to allow privsep daemon to be started via rootwrap.
  #
  # This file should be owned by (and only-writeable by) the root user

  [Filters]

  # By installing the following, the local admin is asserting that:
  #
  # 1. The python module load path used by privsep-helper
  #    command as root (as started by sudo/rootwrap) is trusted.
  # 2. Any oslo.config files matching the --config-file
  #    arguments below are trusted.
  # 3. Users allowed to run sudo/rootwrap with this configuration(*) are
  #    also allowed to invoke python "entrypoint" functions from
  #    --privsep_context with the additional (possibly root) privileges
  #    configured for that context.
  #
  # (*) ie: the user is allowed by /etc/sudoers to run rootwrap as root
  #
  # In particular, the oslo.config and python module path must not
  # be writeable by the unprivileged user.

  # PRIVSEP
  # oslo.privsep default neutron context
  privsep: PathFilter, privsep-helper, root,
   --config-file, /etc/(?!\.\.).*,
   --privsep_context, neutron.privileged.default,
   --privsep_sock_path, /

  # NOTE: A second `--config-file` arg can also be added above. Since
  # many neutron components are installed like that (eg: by devstack).
  # Adjust to suit local requirements.

  # DEBUG
  sleep: RegExpFilter, sleep, root, sleep, \d+

  # EXECUTE COMMANDS IN A NAMESPACE
  ip: IpFilter, ip, root
  ip_exec: IpNetnsExecFilter, ip, root

  # METADATA PROXY
  haproxy: RegExpFilter, haproxy, root, haproxy, -f, .*
  haproxy_env: EnvFilter, env, root, PROCESS_TAG=, haproxy, -f, .*

  # DHCP
  dnsmasq: CommandFilter, dnsmasq, root
  dnsmasq_env: EnvFilter, env, root, PROCESS_TAG=, dnsmasq

  # DIBBLER
  dibbler-client: CommandFilter, dibbler-client, root
  dibbler-client_env: EnvFilter, env, root, PROCESS_TAG=, dibbler-client

  # L3
  radvd: CommandFilter, radvd, root
  radvd_env: EnvFilter, env, root, PROCESS_TAG=, radvd
  keepalived: CommandFilter, keepalived, root
  keepalived_env: EnvFilter, env, root, PROCESS_TAG=, keepalived
  keepalived_state_change: CommandFilter, neutron-keepalived-state-change, root
  keepalived_state_change_env: EnvFilter, env, root, PROCESS_TAG=, neutron-keepalived-state-change

  # OPEN VSWITCH
  ovs-ofctl: CommandFilter, ovs-ofctl, root
  ovsdb-client: CommandFilter, ovsdb-client, root

  
  My /etc/neutron/rootwrap.conf fail contains :
  # Configuration for neutron-rootwrap
  # This file should be owned by (and only-writeable by) the root user

  [DEFAULT]
  # List of directories to load filter definitions from (separated by ',').
  # These directories MUST all be only writeable by root !
  filters_path=/etc/neutron/rootwrap.d,/usr/share/neutron/rootwrap

  # List of directories to search executables in, in case filters do not
  # explicitely specify a full path (separated by ',')
  # If not specified, defaults to system PATH environment variable.
  # These directories MUST all be only writeable by root !
  exec_dirs=/sbin,/usr/sbin,/bin,/usr/bin,/usr/local/bin,/usr/local/sbin,/etc/neutron/kill_scripts

  # Enable logging to syslog
  # Default value is False
  use_syslog=False

  # Which syslog facility to use.
  # Valid values include auth, authpriv, syslog, local0, local1...
  # Default value is 'syslog'
  syslog_log_facility=syslog

  # Which messages to log.
  # INFO means log all usage
  # ERROR means only log unsuccessful attempts
  syslog_log_level=ERROR

  # Rootwrap daemon exits after this seconds of inactivity
  daemon_timeout=600

  # Rootwrap daemon limits itself to that many file descriptors (Linux only)
  rlimit_nofile=1024

  
  In neutron config I have this :
  [agent]
  root_helper = "sudo /usr/bin/neutron-rootwrap /etc/neutron/rootwrap.conf"

  I am not sure if it is neutron error or oslo.privsep error.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2026122/+subscriptions