yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #95392
[Bug 2099911] [NEW] Castellan as config source fails with "greenlet.error: cannot switch to a different thread"
Public bug reported:
1) Setup Castellan as config source
2) Start Neutron
3) DHCP and L3 agents fail to start
Output:
Traceback (most recent call last):
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
timer()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/timer.py", line 59, in __call__
cb(*args, **kw)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 147, in _do_acquire
waiter.switch()
greenlet.error: cannot switch to a different thread
Log:
2025-02-18 15:22:28.969 153057 INFO castellan._config_driver [-] option '[DEFAULT] default_log_levels' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:28.973 153057 INFO castellan._config_driver [-] option '[DEFAULT] rate_limit_burst' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:28.974 153057 INFO neutron.common.config [-] Logging enabled!
2025-02-18 15:22:28.974 153057 INFO neutron.common.config [-] /var/lib/kolla/venv/bin/neutron-l3-agent version 24.1.1.dev11
2025-02-18 15:22:28.975 153057 INFO castellan._config_driver [-] option '[AGENT] report_interval' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:28.975 153057 INFO castellan._config_driver [-] option '[DEFAULT] host' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:28.976 153057 INFO castellan._config_driver [-] option '[DEFAULT] periodic_interval' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:28.976 153057 INFO castellan._config_driver [-] option '[DEFAULT] periodic_fuzzy_delay' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.499 153057 INFO castellan._config_driver [-] option '[DEFAULT] cleanup_on_shutdown' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.500 153057 INFO castellan._config_driver [-] option '[DEFAULT] ipv6_gateway' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.500 153057 INFO castellan._config_driver [-] option '[AGENT] check_child_processes_interval' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.501 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_default_rule' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.501 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_file' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.502 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_file' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.503 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_dirs' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.539 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[OVS] of_inactivity_probe' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.545 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[DEFAULT] ovs_use_veth' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.547 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_messaging_metrics] metrics_socket_file' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.547 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_messaging_metrics] metrics_buffer_size' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.548 153057 INFO castellan._config_driver [-] option '[oslo_messaging_metrics] metrics_thread_stop_timeout' not present in '[castellan_configsource] mapping_file'
4) press Ctrl+C.
Output:
^C^CError in sys.excepthook:
Traceback (most recent call last):
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 115, in acquire
hubs.get_hub().switch()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 310, in switch
return self.greenlet.switch()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 362, in run
self.wait(sleep_time)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/poll.py", line 77, in wait
time.sleep(seconds)
KeyboardInterrupt
Original exception was:
Traceback (most recent call last):
File "/var/lib/kolla/venv/bin/neutron-l3-agent", line 8, in <module>
sys.exit(main())
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/cmd/eventlet/agents/l3.py", line 24, in main
l3_agent.main()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3_agent.py", line 56, in main
server = neutron_service.Service.create(
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/service.py", line 421, in create
service_obj = cls(host, binary, topic, manager,
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/service.py", line 356, in __init__
self.manager = manager_class(host=host, *args, **kwargs)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3/agent.py", line 1002, in __init__
super(L3NATAgentWithStateReport, self).__init__(host=host, conf=conf)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3/agent.py", line 307, in __init__
self.plugin_rpc.get_host_ha_router_count(self.context))
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_utils/timeutils.py", line 333, in wrapper
result = func(*args, **kwargs)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/common/utils.py", line 962, in wrapper
ret = f(*args, **kwargs)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3/agent.py", line 199, in get_host_ha_router_count
return cctxt.call(context, 'get_host_ha_router_count', host=self.host)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron_lib/rpc.py", line 149, in call
return self._original_context.call(ctxt, method, **kwargs)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_messaging/rpc/client.py", line 186, in call
with metrics.get_collector(self.conf, "rpc_client",
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_messaging/_metrics/client.py", line 82, in __enter__
if not self.conf.metrics_enabled:
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_config/cfg.py", line 3172, in __getattr__
return self._conf._get(name, self._group)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_config/cfg.py", line 2664, in _get
value, loc = self._do_get(name, group, namespace)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_config/cfg.py", line 2752, in _do_get
val = source.get(group_name, name, opt)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/castellan/_config_driver.py", line 126, in get
LOG.info("option '[%s] %s' not present in '[%s] mapping_file'",
File "/usr/lib64/python3.9/logging/__init__.py", line 1806, in info
self.log(INFO, msg, *args, **kwargs)
File "/usr/lib64/python3.9/logging/__init__.py", line 1844, in log
self.logger.log(level, msg, *args, **kwargs)
File "/usr/lib64/python3.9/logging/__init__.py", line 1512, in log
self._log(level, msg, args, **kwargs)
File "/usr/lib64/python3.9/logging/__init__.py", line 1589, in _log
self.handle(record)
File "/usr/lib64/python3.9/logging/__init__.py", line 1599, in handle
self.callHandlers(record)
File "/usr/lib64/python3.9/logging/__init__.py", line 1661, in callHandlers
hdlr.handle(record)
File "/usr/lib64/python3.9/logging/__init__.py", line 950, in handle
self.acquire()
File "/usr/lib64/python3.9/logging/__init__.py", line 901, in acquire
self.lock.acquire()
File "/usr/lib64/python3.9/threading.py", line 160, in acquire
rc = self._block.acquire(blocking, timeout)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 115, in acquire
hubs.get_hub().switch()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 310, in switch
return self.greenlet.switch()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 362, in run
self.wait(sleep_time)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/poll.py", line 77, in wait
time.sleep(seconds)
KeyboardInterrupt
^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib64/python3.9/logging/__init__.py", line 2140, in shutdown
h.acquire()
File "/usr/lib64/python3.9/logging/__init__.py", line 901, in acquire
self.lock.acquire()
File "/usr/lib64/python3.9/threading.py", line 160, in acquire
rc = self._block.acquire(blocking, timeout)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 115, in acquire
hubs.get_hub().switch()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 310, in switch
return self.greenlet.switch()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 362, in run
self.wait(sleep_time)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/poll.py", line 77, in wait
time.sleep(seconds)
KeyboardInterrupt
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.9/logging/__init__.py", line 2150, in shutdown
h.release()
File "/usr/lib64/python3.9/logging/__init__.py", line 908, in release
self.lock.release()
File "/usr/lib64/python3.9/threading.py", line 185, in release
raise RuntimeError("cannot release un-acquired lock")
RuntimeError: cannot release un-acquired lock
** Affects: neutron
Importance: Undecided
Assignee: Mitya Eremeev (mitos)
Status: In Progress
** Description changed:
1) Setup Castellan as config source
2) Start Neutron
3) DHCP and L3 agents fail to start
Output:
Traceback (most recent call last):
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
- timer()
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/timer.py", line 59, in __call__
- cb(*args, **kw)
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 147, in _do_acquire
- waiter.switch()
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
+ timer()
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/timer.py", line 59, in __call__
+ cb(*args, **kw)
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 147, in _do_acquire
+ waiter.switch()
greenlet.error: cannot switch to a different thread
-
press Ctrl+C:
^C^CError in sys.excepthook:
Traceback (most recent call last):
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 115, in acquire
- hubs.get_hub().switch()
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 310, in switch
- return self.greenlet.switch()
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 362, in run
- self.wait(sleep_time)
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/poll.py", line 77, in wait
- time.sleep(seconds)
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 115, in acquire
+ hubs.get_hub().switch()
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 310, in switch
+ return self.greenlet.switch()
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 362, in run
+ self.wait(sleep_time)
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/poll.py", line 77, in wait
+ time.sleep(seconds)
KeyboardInterrupt
Original exception was:
Traceback (most recent call last):
- File "/var/lib/kolla/venv/bin/neutron-l3-agent", line 8, in <module>
- sys.exit(main())
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/cmd/eventlet/agents/l3.py", line 24, in main
- l3_agent.main()
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3_agent.py", line 56, in main
- server = neutron_service.Service.create(
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/service.py", line 421, in create
- service_obj = cls(host, binary, topic, manager,
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/service.py", line 356, in __init__
- self.manager = manager_class(host=host, *args, **kwargs)
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3/agent.py", line 1002, in __init__
- super(L3NATAgentWithStateReport, self).__init__(host=host, conf=conf)
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3/agent.py", line 307, in __init__
- self.plugin_rpc.get_host_ha_router_count(self.context))
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_utils/timeutils.py", line 333, in wrapper
- result = func(*args, **kwargs)
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/common/utils.py", line 962, in wrapper
- ret = f(*args, **kwargs)
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3/agent.py", line 199, in get_host_ha_router_count
- return cctxt.call(context, 'get_host_ha_router_count', host=self.host)
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron_lib/rpc.py", line 149, in call
- return self._original_context.call(ctxt, method, **kwargs)
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_messaging/rpc/client.py", line 186, in call
- with metrics.get_collector(self.conf, "rpc_client",
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_messaging/_metrics/client.py", line 82, in __enter__
- if not self.conf.metrics_enabled:
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_config/cfg.py", line 3172, in __getattr__
- return self._conf._get(name, self._group)
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_config/cfg.py", line 2664, in _get
- value, loc = self._do_get(name, group, namespace)
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_config/cfg.py", line 2752, in _do_get
- val = source.get(group_name, name, opt)
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/castellan/_config_driver.py", line 126, in get
- LOG.info("option '[%s] %s' not present in '[%s] mapping_file'",
- File "/usr/lib64/python3.9/logging/__init__.py", line 1806, in info
- self.log(INFO, msg, *args, **kwargs)
- File "/usr/lib64/python3.9/logging/__init__.py", line 1844, in log
- self.logger.log(level, msg, *args, **kwargs)
- File "/usr/lib64/python3.9/logging/__init__.py", line 1512, in log
- self._log(level, msg, args, **kwargs)
- File "/usr/lib64/python3.9/logging/__init__.py", line 1589, in _log
- self.handle(record)
- File "/usr/lib64/python3.9/logging/__init__.py", line 1599, in handle
- self.callHandlers(record)
- File "/usr/lib64/python3.9/logging/__init__.py", line 1661, in callHandlers
- hdlr.handle(record)
- File "/usr/lib64/python3.9/logging/__init__.py", line 950, in handle
- self.acquire()
- File "/usr/lib64/python3.9/logging/__init__.py", line 901, in acquire
- self.lock.acquire()
- File "/usr/lib64/python3.9/threading.py", line 160, in acquire
- rc = self._block.acquire(blocking, timeout)
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 115, in acquire
- hubs.get_hub().switch()
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 310, in switch
- return self.greenlet.switch()
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 362, in run
- self.wait(sleep_time)
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/poll.py", line 77, in wait
- time.sleep(seconds)
+ File "/var/lib/kolla/venv/bin/neutron-l3-agent", line 8, in <module>
+ sys.exit(main())
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/cmd/eventlet/agents/l3.py", line 24, in main
+ l3_agent.main()
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3_agent.py", line 56, in main
+ server = neutron_service.Service.create(
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/service.py", line 421, in create
+ service_obj = cls(host, binary, topic, manager,
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/service.py", line 356, in __init__
+ self.manager = manager_class(host=host, *args, **kwargs)
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3/agent.py", line 1002, in __init__
+ super(L3NATAgentWithStateReport, self).__init__(host=host, conf=conf)
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3/agent.py", line 307, in __init__
+ self.plugin_rpc.get_host_ha_router_count(self.context))
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_utils/timeutils.py", line 333, in wrapper
+ result = func(*args, **kwargs)
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/common/utils.py", line 962, in wrapper
+ ret = f(*args, **kwargs)
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3/agent.py", line 199, in get_host_ha_router_count
+ return cctxt.call(context, 'get_host_ha_router_count', host=self.host)
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron_lib/rpc.py", line 149, in call
+ return self._original_context.call(ctxt, method, **kwargs)
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_messaging/rpc/client.py", line 186, in call
+ with metrics.get_collector(self.conf, "rpc_client",
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_messaging/_metrics/client.py", line 82, in __enter__
+ if not self.conf.metrics_enabled:
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_config/cfg.py", line 3172, in __getattr__
+ return self._conf._get(name, self._group)
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_config/cfg.py", line 2664, in _get
+ value, loc = self._do_get(name, group, namespace)
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_config/cfg.py", line 2752, in _do_get
+ val = source.get(group_name, name, opt)
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/castellan/_config_driver.py", line 126, in get
+ LOG.info("option '[%s] %s' not present in '[%s] mapping_file'",
+ File "/usr/lib64/python3.9/logging/__init__.py", line 1806, in info
+ self.log(INFO, msg, *args, **kwargs)
+ File "/usr/lib64/python3.9/logging/__init__.py", line 1844, in log
+ self.logger.log(level, msg, *args, **kwargs)
+ File "/usr/lib64/python3.9/logging/__init__.py", line 1512, in log
+ self._log(level, msg, args, **kwargs)
+ File "/usr/lib64/python3.9/logging/__init__.py", line 1589, in _log
+ self.handle(record)
+ File "/usr/lib64/python3.9/logging/__init__.py", line 1599, in handle
+ self.callHandlers(record)
+ File "/usr/lib64/python3.9/logging/__init__.py", line 1661, in callHandlers
+ hdlr.handle(record)
+ File "/usr/lib64/python3.9/logging/__init__.py", line 950, in handle
+ self.acquire()
+ File "/usr/lib64/python3.9/logging/__init__.py", line 901, in acquire
+ self.lock.acquire()
+ File "/usr/lib64/python3.9/threading.py", line 160, in acquire
+ rc = self._block.acquire(blocking, timeout)
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 115, in acquire
+ hubs.get_hub().switch()
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 310, in switch
+ return self.greenlet.switch()
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 362, in run
+ self.wait(sleep_time)
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/poll.py", line 77, in wait
+ time.sleep(seconds)
KeyboardInterrupt
^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
- File "/usr/lib64/python3.9/logging/__init__.py", line 2140, in shutdown
- h.acquire()
- File "/usr/lib64/python3.9/logging/__init__.py", line 901, in acquire
- self.lock.acquire()
- File "/usr/lib64/python3.9/threading.py", line 160, in acquire
- rc = self._block.acquire(blocking, timeout)
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 115, in acquire
- hubs.get_hub().switch()
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 310, in switch
- return self.greenlet.switch()
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 362, in run
- self.wait(sleep_time)
- File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/poll.py", line 77, in wait
- time.sleep(seconds)
+ File "/usr/lib64/python3.9/logging/__init__.py", line 2140, in shutdown
+ h.acquire()
+ File "/usr/lib64/python3.9/logging/__init__.py", line 901, in acquire
+ self.lock.acquire()
+ File "/usr/lib64/python3.9/threading.py", line 160, in acquire
+ rc = self._block.acquire(blocking, timeout)
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 115, in acquire
+ hubs.get_hub().switch()
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 310, in switch
+ return self.greenlet.switch()
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 362, in run
+ self.wait(sleep_time)
+ File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/poll.py", line 77, in wait
+ time.sleep(seconds)
KeyboardInterrupt
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
- File "/usr/lib64/python3.9/logging/__init__.py", line 2150, in shutdown
- h.release()
- File "/usr/lib64/python3.9/logging/__init__.py", line 908, in release
- self.lock.release()
- File "/usr/lib64/python3.9/threading.py", line 185, in release
- raise RuntimeError("cannot release un-acquired lock")
+ File "/usr/lib64/python3.9/logging/__init__.py", line 2150, in shutdown
+ h.release()
+ File "/usr/lib64/python3.9/logging/__init__.py", line 908, in release
+ self.lock.release()
+ File "/usr/lib64/python3.9/threading.py", line 185, in release
+ raise RuntimeError("cannot release un-acquired lock")
RuntimeError: cannot release un-acquired lock
-
Log:
2025-02-18 15:22:28.969 153057 INFO castellan._config_driver [-] option '[DEFAULT] default_log_levels' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:28.973 153057 INFO castellan._config_driver [-] option '[DEFAULT] rate_limit_burst' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:28.974 153057 INFO neutron.common.config [-] Logging enabled!
2025-02-18 15:22:28.974 153057 INFO neutron.common.config [-] /var/lib/kolla/venv/bin/neutron-l3-agent version 24.1.1.dev11
2025-02-18 15:22:28.975 153057 INFO castellan._config_driver [-] option '[AGENT] report_interval' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:28.975 153057 INFO castellan._config_driver [-] option '[DEFAULT] host' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:28.976 153057 INFO castellan._config_driver [-] option '[DEFAULT] periodic_interval' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:28.976 153057 INFO castellan._config_driver [-] option '[DEFAULT] periodic_fuzzy_delay' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.499 153057 INFO castellan._config_driver [-] option '[DEFAULT] cleanup_on_shutdown' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.500 153057 INFO castellan._config_driver [-] option '[DEFAULT] ipv6_gateway' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.500 153057 INFO castellan._config_driver [-] option '[AGENT] check_child_processes_interval' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.501 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_default_rule' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.501 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_file' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.502 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_file' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.503 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_dirs' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.539 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[OVS] of_inactivity_probe' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.545 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[DEFAULT] ovs_use_veth' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.547 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_messaging_metrics] metrics_socket_file' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.547 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_messaging_metrics] metrics_buffer_size' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.548 153057 INFO castellan._config_driver [-] option '[oslo_messaging_metrics] metrics_thread_stop_timeout' not present in '[castellan_configsource] mapping_file'
** Summary changed:
- Castellan as config source failes with "greenlet.error: cannot switch to a different thread"
+ Castellan as config source fails with "greenlet.error: cannot switch to a different thread"
** Description changed:
1) Setup Castellan as config source
2) Start Neutron
3) DHCP and L3 agents fail to start
Output:
Traceback (most recent call last):
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
timer()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/timer.py", line 59, in __call__
cb(*args, **kw)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 147, in _do_acquire
waiter.switch()
greenlet.error: cannot switch to a different thread
- press Ctrl+C:
+ Log:
+
+ 2025-02-18 15:22:28.969 153057 INFO castellan._config_driver [-] option '[DEFAULT] default_log_levels' not present in '[castellan_configsource] mapping_file'
+ 2025-02-18 15:22:28.973 153057 INFO castellan._config_driver [-] option '[DEFAULT] rate_limit_burst' not present in '[castellan_configsource] mapping_file'
+ 2025-02-18 15:22:28.974 153057 INFO neutron.common.config [-] Logging enabled!
+ 2025-02-18 15:22:28.974 153057 INFO neutron.common.config [-] /var/lib/kolla/venv/bin/neutron-l3-agent version 24.1.1.dev11
+ 2025-02-18 15:22:28.975 153057 INFO castellan._config_driver [-] option '[AGENT] report_interval' not present in '[castellan_configsource] mapping_file'
+ 2025-02-18 15:22:28.975 153057 INFO castellan._config_driver [-] option '[DEFAULT] host' not present in '[castellan_configsource] mapping_file'
+ 2025-02-18 15:22:28.976 153057 INFO castellan._config_driver [-] option '[DEFAULT] periodic_interval' not present in '[castellan_configsource] mapping_file'
+ 2025-02-18 15:22:28.976 153057 INFO castellan._config_driver [-] option '[DEFAULT] periodic_fuzzy_delay' not present in '[castellan_configsource] mapping_file'
+ 2025-02-18 15:22:29.499 153057 INFO castellan._config_driver [-] option '[DEFAULT] cleanup_on_shutdown' not present in '[castellan_configsource] mapping_file'
+ 2025-02-18 15:22:29.500 153057 INFO castellan._config_driver [-] option '[DEFAULT] ipv6_gateway' not present in '[castellan_configsource] mapping_file'
+ 2025-02-18 15:22:29.500 153057 INFO castellan._config_driver [-] option '[AGENT] check_child_processes_interval' not present in '[castellan_configsource] mapping_file'
+ 2025-02-18 15:22:29.501 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_default_rule' not present in '[castellan_configsource] mapping_file'
+ 2025-02-18 15:22:29.501 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_file' not present in '[castellan_configsource] mapping_file'
+ 2025-02-18 15:22:29.502 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_file' not present in '[castellan_configsource] mapping_file'
+ 2025-02-18 15:22:29.503 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_dirs' not present in '[castellan_configsource] mapping_file'
+ 2025-02-18 15:22:29.539 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[OVS] of_inactivity_probe' not present in '[castellan_configsource] mapping_file'
+ 2025-02-18 15:22:29.545 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[DEFAULT] ovs_use_veth' not present in '[castellan_configsource] mapping_file'
+ 2025-02-18 15:22:29.547 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_messaging_metrics] metrics_socket_file' not present in '[castellan_configsource] mapping_file'
+ 2025-02-18 15:22:29.547 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_messaging_metrics] metrics_buffer_size' not present in '[castellan_configsource] mapping_file'
+ 2025-02-18 15:22:29.548 153057 INFO castellan._config_driver [-] option '[oslo_messaging_metrics] metrics_thread_stop_timeout' not present in '[castellan_configsource] mapping_file'
+
+
+ 4) press Ctrl+C.
+
+ Output:
^C^CError in sys.excepthook:
Traceback (most recent call last):
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 115, in acquire
hubs.get_hub().switch()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 310, in switch
return self.greenlet.switch()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 362, in run
self.wait(sleep_time)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/poll.py", line 77, in wait
time.sleep(seconds)
KeyboardInterrupt
Original exception was:
Traceback (most recent call last):
File "/var/lib/kolla/venv/bin/neutron-l3-agent", line 8, in <module>
sys.exit(main())
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/cmd/eventlet/agents/l3.py", line 24, in main
l3_agent.main()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3_agent.py", line 56, in main
server = neutron_service.Service.create(
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/service.py", line 421, in create
service_obj = cls(host, binary, topic, manager,
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/service.py", line 356, in __init__
self.manager = manager_class(host=host, *args, **kwargs)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3/agent.py", line 1002, in __init__
super(L3NATAgentWithStateReport, self).__init__(host=host, conf=conf)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3/agent.py", line 307, in __init__
self.plugin_rpc.get_host_ha_router_count(self.context))
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_utils/timeutils.py", line 333, in wrapper
result = func(*args, **kwargs)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/common/utils.py", line 962, in wrapper
ret = f(*args, **kwargs)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3/agent.py", line 199, in get_host_ha_router_count
return cctxt.call(context, 'get_host_ha_router_count', host=self.host)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron_lib/rpc.py", line 149, in call
return self._original_context.call(ctxt, method, **kwargs)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_messaging/rpc/client.py", line 186, in call
with metrics.get_collector(self.conf, "rpc_client",
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_messaging/_metrics/client.py", line 82, in __enter__
if not self.conf.metrics_enabled:
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_config/cfg.py", line 3172, in __getattr__
return self._conf._get(name, self._group)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_config/cfg.py", line 2664, in _get
value, loc = self._do_get(name, group, namespace)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_config/cfg.py", line 2752, in _do_get
val = source.get(group_name, name, opt)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/castellan/_config_driver.py", line 126, in get
LOG.info("option '[%s] %s' not present in '[%s] mapping_file'",
File "/usr/lib64/python3.9/logging/__init__.py", line 1806, in info
self.log(INFO, msg, *args, **kwargs)
File "/usr/lib64/python3.9/logging/__init__.py", line 1844, in log
self.logger.log(level, msg, *args, **kwargs)
File "/usr/lib64/python3.9/logging/__init__.py", line 1512, in log
self._log(level, msg, args, **kwargs)
File "/usr/lib64/python3.9/logging/__init__.py", line 1589, in _log
self.handle(record)
File "/usr/lib64/python3.9/logging/__init__.py", line 1599, in handle
self.callHandlers(record)
File "/usr/lib64/python3.9/logging/__init__.py", line 1661, in callHandlers
hdlr.handle(record)
File "/usr/lib64/python3.9/logging/__init__.py", line 950, in handle
self.acquire()
File "/usr/lib64/python3.9/logging/__init__.py", line 901, in acquire
self.lock.acquire()
File "/usr/lib64/python3.9/threading.py", line 160, in acquire
rc = self._block.acquire(blocking, timeout)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 115, in acquire
hubs.get_hub().switch()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 310, in switch
return self.greenlet.switch()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 362, in run
self.wait(sleep_time)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/poll.py", line 77, in wait
time.sleep(seconds)
KeyboardInterrupt
^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib64/python3.9/logging/__init__.py", line 2140, in shutdown
h.acquire()
File "/usr/lib64/python3.9/logging/__init__.py", line 901, in acquire
self.lock.acquire()
File "/usr/lib64/python3.9/threading.py", line 160, in acquire
rc = self._block.acquire(blocking, timeout)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 115, in acquire
hubs.get_hub().switch()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 310, in switch
return self.greenlet.switch()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 362, in run
self.wait(sleep_time)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/poll.py", line 77, in wait
time.sleep(seconds)
KeyboardInterrupt
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.9/logging/__init__.py", line 2150, in shutdown
h.release()
File "/usr/lib64/python3.9/logging/__init__.py", line 908, in release
self.lock.release()
File "/usr/lib64/python3.9/threading.py", line 185, in release
raise RuntimeError("cannot release un-acquired lock")
RuntimeError: cannot release un-acquired lock
-
- Log:
-
- 2025-02-18 15:22:28.969 153057 INFO castellan._config_driver [-] option '[DEFAULT] default_log_levels' not present in '[castellan_configsource] mapping_file'
- 2025-02-18 15:22:28.973 153057 INFO castellan._config_driver [-] option '[DEFAULT] rate_limit_burst' not present in '[castellan_configsource] mapping_file'
- 2025-02-18 15:22:28.974 153057 INFO neutron.common.config [-] Logging enabled!
- 2025-02-18 15:22:28.974 153057 INFO neutron.common.config [-] /var/lib/kolla/venv/bin/neutron-l3-agent version 24.1.1.dev11
- 2025-02-18 15:22:28.975 153057 INFO castellan._config_driver [-] option '[AGENT] report_interval' not present in '[castellan_configsource] mapping_file'
- 2025-02-18 15:22:28.975 153057 INFO castellan._config_driver [-] option '[DEFAULT] host' not present in '[castellan_configsource] mapping_file'
- 2025-02-18 15:22:28.976 153057 INFO castellan._config_driver [-] option '[DEFAULT] periodic_interval' not present in '[castellan_configsource] mapping_file'
- 2025-02-18 15:22:28.976 153057 INFO castellan._config_driver [-] option '[DEFAULT] periodic_fuzzy_delay' not present in '[castellan_configsource] mapping_file'
- 2025-02-18 15:22:29.499 153057 INFO castellan._config_driver [-] option '[DEFAULT] cleanup_on_shutdown' not present in '[castellan_configsource] mapping_file'
- 2025-02-18 15:22:29.500 153057 INFO castellan._config_driver [-] option '[DEFAULT] ipv6_gateway' not present in '[castellan_configsource] mapping_file'
- 2025-02-18 15:22:29.500 153057 INFO castellan._config_driver [-] option '[AGENT] check_child_processes_interval' not present in '[castellan_configsource] mapping_file'
- 2025-02-18 15:22:29.501 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_default_rule' not present in '[castellan_configsource] mapping_file'
- 2025-02-18 15:22:29.501 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_file' not present in '[castellan_configsource] mapping_file'
- 2025-02-18 15:22:29.502 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_file' not present in '[castellan_configsource] mapping_file'
- 2025-02-18 15:22:29.503 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_dirs' not present in '[castellan_configsource] mapping_file'
- 2025-02-18 15:22:29.539 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[OVS] of_inactivity_probe' not present in '[castellan_configsource] mapping_file'
- 2025-02-18 15:22:29.545 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[DEFAULT] ovs_use_veth' not present in '[castellan_configsource] mapping_file'
- 2025-02-18 15:22:29.547 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_messaging_metrics] metrics_socket_file' not present in '[castellan_configsource] mapping_file'
- 2025-02-18 15:22:29.547 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_messaging_metrics] metrics_buffer_size' not present in '[castellan_configsource] mapping_file'
- 2025-02-18 15:22:29.548 153057 INFO castellan._config_driver [-] option '[oslo_messaging_metrics] metrics_thread_stop_timeout' not present in '[castellan_configsource] mapping_file'
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2099911
Title:
Castellan as config source fails with "greenlet.error: cannot switch
to a different thread"
Status in neutron:
In Progress
Bug description:
1) Setup Castellan as config source
2) Start Neutron
3) DHCP and L3 agents fail to start
Output:
Traceback (most recent call last):
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 471, in fire_timers
timer()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/timer.py", line 59, in __call__
cb(*args, **kw)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 147, in _do_acquire
waiter.switch()
greenlet.error: cannot switch to a different thread
Log:
2025-02-18 15:22:28.969 153057 INFO castellan._config_driver [-] option '[DEFAULT] default_log_levels' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:28.973 153057 INFO castellan._config_driver [-] option '[DEFAULT] rate_limit_burst' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:28.974 153057 INFO neutron.common.config [-] Logging enabled!
2025-02-18 15:22:28.974 153057 INFO neutron.common.config [-] /var/lib/kolla/venv/bin/neutron-l3-agent version 24.1.1.dev11
2025-02-18 15:22:28.975 153057 INFO castellan._config_driver [-] option '[AGENT] report_interval' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:28.975 153057 INFO castellan._config_driver [-] option '[DEFAULT] host' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:28.976 153057 INFO castellan._config_driver [-] option '[DEFAULT] periodic_interval' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:28.976 153057 INFO castellan._config_driver [-] option '[DEFAULT] periodic_fuzzy_delay' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.499 153057 INFO castellan._config_driver [-] option '[DEFAULT] cleanup_on_shutdown' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.500 153057 INFO castellan._config_driver [-] option '[DEFAULT] ipv6_gateway' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.500 153057 INFO castellan._config_driver [-] option '[AGENT] check_child_processes_interval' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.501 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_default_rule' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.501 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_file' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.502 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_file' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.503 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_policy] policy_dirs' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.539 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[OVS] of_inactivity_probe' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.545 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[DEFAULT] ovs_use_veth' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.547 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_messaging_metrics] metrics_socket_file' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.547 153057 INFO castellan._config_driver [req-5f029160-6cff-4834-9c7f-ac5d418d8356 - - - - - -] option '[oslo_messaging_metrics] metrics_buffer_size' not present in '[castellan_configsource] mapping_file'
2025-02-18 15:22:29.548 153057 INFO castellan._config_driver [-] option '[oslo_messaging_metrics] metrics_thread_stop_timeout' not present in '[castellan_configsource] mapping_file'
4) press Ctrl+C.
Output:
^C^CError in sys.excepthook:
Traceback (most recent call last):
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 115, in acquire
hubs.get_hub().switch()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 310, in switch
return self.greenlet.switch()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 362, in run
self.wait(sleep_time)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/poll.py", line 77, in wait
time.sleep(seconds)
KeyboardInterrupt
Original exception was:
Traceback (most recent call last):
File "/var/lib/kolla/venv/bin/neutron-l3-agent", line 8, in <module>
sys.exit(main())
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/cmd/eventlet/agents/l3.py", line 24, in main
l3_agent.main()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3_agent.py", line 56, in main
server = neutron_service.Service.create(
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/service.py", line 421, in create
service_obj = cls(host, binary, topic, manager,
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/service.py", line 356, in __init__
self.manager = manager_class(host=host, *args, **kwargs)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3/agent.py", line 1002, in __init__
super(L3NATAgentWithStateReport, self).__init__(host=host, conf=conf)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3/agent.py", line 307, in __init__
self.plugin_rpc.get_host_ha_router_count(self.context))
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_utils/timeutils.py", line 333, in wrapper
result = func(*args, **kwargs)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/common/utils.py", line 962, in wrapper
ret = f(*args, **kwargs)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron/agent/l3/agent.py", line 199, in get_host_ha_router_count
return cctxt.call(context, 'get_host_ha_router_count', host=self.host)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/neutron_lib/rpc.py", line 149, in call
return self._original_context.call(ctxt, method, **kwargs)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_messaging/rpc/client.py", line 186, in call
with metrics.get_collector(self.conf, "rpc_client",
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_messaging/_metrics/client.py", line 82, in __enter__
if not self.conf.metrics_enabled:
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_config/cfg.py", line 3172, in __getattr__
return self._conf._get(name, self._group)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_config/cfg.py", line 2664, in _get
value, loc = self._do_get(name, group, namespace)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/oslo_config/cfg.py", line 2752, in _do_get
val = source.get(group_name, name, opt)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/castellan/_config_driver.py", line 126, in get
LOG.info("option '[%s] %s' not present in '[%s] mapping_file'",
File "/usr/lib64/python3.9/logging/__init__.py", line 1806, in info
self.log(INFO, msg, *args, **kwargs)
File "/usr/lib64/python3.9/logging/__init__.py", line 1844, in log
self.logger.log(level, msg, *args, **kwargs)
File "/usr/lib64/python3.9/logging/__init__.py", line 1512, in log
self._log(level, msg, args, **kwargs)
File "/usr/lib64/python3.9/logging/__init__.py", line 1589, in _log
self.handle(record)
File "/usr/lib64/python3.9/logging/__init__.py", line 1599, in handle
self.callHandlers(record)
File "/usr/lib64/python3.9/logging/__init__.py", line 1661, in callHandlers
hdlr.handle(record)
File "/usr/lib64/python3.9/logging/__init__.py", line 950, in handle
self.acquire()
File "/usr/lib64/python3.9/logging/__init__.py", line 901, in acquire
self.lock.acquire()
File "/usr/lib64/python3.9/threading.py", line 160, in acquire
rc = self._block.acquire(blocking, timeout)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 115, in acquire
hubs.get_hub().switch()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 310, in switch
return self.greenlet.switch()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 362, in run
self.wait(sleep_time)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/poll.py", line 77, in wait
time.sleep(seconds)
KeyboardInterrupt
^CError in atexit._run_exitfuncs:
Traceback (most recent call last):
File "/usr/lib64/python3.9/logging/__init__.py", line 2140, in shutdown
h.acquire()
File "/usr/lib64/python3.9/logging/__init__.py", line 901, in acquire
self.lock.acquire()
File "/usr/lib64/python3.9/threading.py", line 160, in acquire
rc = self._block.acquire(blocking, timeout)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/semaphore.py", line 115, in acquire
hubs.get_hub().switch()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 310, in switch
return self.greenlet.switch()
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/hub.py", line 362, in run
self.wait(sleep_time)
File "/var/lib/kolla/venv/lib64/python3.9/site-packages/eventlet/hubs/poll.py", line 77, in wait
time.sleep(seconds)
KeyboardInterrupt
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib64/python3.9/logging/__init__.py", line 2150, in shutdown
h.release()
File "/usr/lib64/python3.9/logging/__init__.py", line 908, in release
self.lock.release()
File "/usr/lib64/python3.9/threading.py", line 185, in release
raise RuntimeError("cannot release un-acquired lock")
RuntimeError: cannot release un-acquired lock
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2099911/+subscriptions
