yahoo-eng-team team mailing list archive

[OpenStack Infra <1680183@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.openstack.org/453838
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=757476983ca0832099f04b9f029617ab270de9fe
Submitter: Jenkins
Branch:    master

commit 757476983ca0832099f04b9f029617ab270de9fe
Author: Ihar Hrachyshka <ihrachys@xxxxxxxxxx>
Date:   Wed Apr 5 19:07:46 2017 +0000

    execute: don't call greenthread.sleep directly
    
    The module may be used in scope that is not using eventlet (f.e. from
    inside neutron-keepalived-state-change), in which case a call to
    greenthread.sleep from the main thread will result in AssertionError.
    
    Instead of that, call time.sleep that will redirect to greenthread.sleep
    in eventlet environment.
    
    This reflects what we do in oslo.concurrency execute implementation.
    
    Change-Id: Id73c0e5015feca2c559da4a68a70aeaaf5cec2a7
    Closes-Bug: #1680183


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1680183

Title:
  neutron-keepalived-state-change fails with "AssertionError: do not
  call blocking functions from the mainloop"

Status in neutron:
  Fix Released

Bug description:
  17:39:17.802 6173 CRITICAL neutron [-] AssertionError: do not call blocking functions from the mainloop
  17:39:17.802 6173 ERROR neutron Traceback (most recent call last):
  17:39:17.802 6173 ERROR neutron   File "/opt/stack/neutron/.tox/dsvm-functional/bin/neutron-keepalived-state-change", line 10, in <module>
  17:39:17.802 6173 ERROR neutron     sys.exit(main())
  17:39:17.802 6173 ERROR neutron   File "/opt/stack/neutron/neutron/cmd/keepalived_state_change.py", line 19, in main
  17:39:17.802 6173 ERROR neutron     keepalived_state_change.main()
  17:39:17.802 6173 ERROR neutron   File "/opt/stack/neutron/neutron/agent/l3/keepalived_state_change.py", line 157, in main
  17:39:17.802 6173 ERROR neutron     cfg.CONF.monitor_cidr).start()
  17:39:17.802 6173 ERROR neutron   File "/opt/stack/neutron/neutron/agent/linux/daemon.py", line 249, in start
  17:39:17.802 6173 ERROR neutron     self.run()
  17:39:17.802 6173 ERROR neutron   File "/opt/stack/neutron/neutron/agent/l3/keepalived_state_change.py", line 70, in run
  17:39:17.802 6173 ERROR neutron     for iterable in self.monitor:
  17:39:17.802 6173 ERROR neutron   File "/opt/stack/neutron/neutron/agent/linux/async_process.py", line 256, in _iter_queue
  17:39:17.802 6173 ERROR neutron     yield queue.get(block=block)
  17:39:17.802 6173 ERROR neutron   File "/opt/stack/neutron/.tox/dsvm-functional/lib/python2.7/site-packages/eventlet/queue.py", line 313, in get
  17:39:17.802 6173 ERROR neutron     return waiter.wait()
  17:39:17.802 6173 ERROR neutron   File "/opt/stack/neutron/.tox/dsvm-functional/lib/python2.7/site-packages/eventlet/queue.py", line 141, in wait
  17:39:17.802 6173 ERROR neutron     return get_hub().switch()
  17:39:17.802 6173 ERROR neutron   File "/opt/stack/neutron/.tox/dsvm-functional/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 294, in switch
  17:39:17.802 6173 ERROR neutron     return self.greenlet.switch()
  17:39:17.802 6173 ERROR neutron   File "/opt/stack/neutron/.tox/dsvm-functional/lib/python2.7/site-packages/eventlet/hubs/hub.py", line 346, in run
  17:39:17.802 6173 ERROR neutron     self.wait(sleep_time)
  17:39:17.802 6173 ERROR neutron   File "/opt/stack/neutron/.tox/dsvm-functional/lib/python2.7/site-packages/eventlet/hubs/poll.py", line 85, in wait
  17:39:17.802 6173 ERROR neutron     presult = self.do_poll(seconds)
  17:39:17.802 6173 ERROR neutron   File "/opt/stack/neutron/.tox/dsvm-functional/lib/python2.7/site-packages/eventlet/hubs/epolls.py", line 62, in do_poll
  17:39:17.802 6173 ERROR neutron     return self.poll.poll(seconds)
  17:39:17.802 6173 ERROR neutron   File "/opt/stack/neutron/neutron/agent/l3/keepalived_state_change.py", line 134, in handle_sigterm
  17:39:17.802 6173 ERROR neutron     self._kill_monitor()
  17:39:17.802 6173 ERROR neutron   File "/opt/stack/neutron/neutron/agent/l3/keepalived_state_change.py", line 131, in _kill_monitor
  17:39:17.802 6173 ERROR neutron     run_as_root=True)
  17:39:17.802 6173 ERROR neutron   File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 221, in kill_process
  17:39:17.802 6173 ERROR neutron     execute(['kill', '-%d' % signal, pid], run_as_root=run_as_root)
  17:39:17.802 6173 ERROR neutron   File "/opt/stack/neutron/neutron/agent/linux/utils.py", line 155, in execute
  17:39:17.802 6173 ERROR neutron     greenthread.sleep(0)
  17:39:17.802 6173 ERROR neutron   File "/opt/stack/neutron/.tox/dsvm-functional/lib/python2.7/site-packages/eventlet/greenthread.py", line 31, in sleep
  17:39:17.802 6173 ERROR neutron     assert hub.greenlet is not current, 'do not call blocking functions from the mainloop'
  17:39:17.802 6173 ERROR neutron AssertionError: do not call blocking functions from the mainloop
  17:39:17.802 6173 ERROR neutron

  This is what I see when running fullstack l3ha tests, once I enable
  syslog logging for the helper process.

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1680183/+subscriptions