yahoo-eng-team team mailing list archive

[OpenStack Infra <1958363@xxxxxxxxxxxxxxxxxx>]

Reviewed:  https://review.opendev.org/c/openstack/neutron/+/827331
Committed: https://opendev.org/openstack/neutron/commit/507989fc622a6633926c4288827f482bae6054dd
Submitter: "Zuul (22348)"
Branch:    master

commit 507989fc622a6633926c4288827f482bae6054dd
Author: Rodolfo Alonso Hernandez <ralonsoh@xxxxxxxxxx>
Date:   Sat Jan 29 10:10:22 2022 +0000

    Use a thread local variable to store the Nova Notifier enable flag
    
    The Nova Notifier can be called simultaneously by several RPC
    callbacks from the agents (DHCP, L2), trying to update the provisioning
    status of a port. In order to handle each context notifier enable flag,
    a thread local variable is used. This will isolate the flag update
    if two entities inform at the same time and one RPC callback is attended
    during the processing of the other one.
    
    This patch also removes the debug messages added to debug this issue.
    
    Closes-Bug: #1958363
    Change-Id: Ie670fba4b3afe427747732d2c3948d92311e960e


** Changed in: neutron
       Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1958363

Title:
  Notifications to nova disabled causing tests failures

Status in neutron:
  Fix Released

Bug description:
  For some reason (I have no idea why) during the scenario tests neutron
  stops sending notifications about port status change to nova. In the
  logs of neutron server there is info that notifications are disabled,
  like:

  Jan 18 12:05:45.415318 ubuntu-focal-inmotion-iad3-0028053821 neutron-server[71111]: DEBUG neutron.db.provisioning_blocks [None req-b33edd99-de3e-4c86-9fb8-405862418a59 None None] Provisioning complete for port 52a33477-c818-456c-b47f-a323e04cce42 triggered by entity L2. {{(pid=71111) provisioning_complete /opt/stack/neutron/neutron/db/provisioning_blocks.py:139}}
  Jan 18 12:05:45.415822 ubuntu-focal-inmotion-iad3-0028053821 neutron-server[71111]: DEBUG neutron_lib.callbacks.manager [None req-b33edd99-de3e-4c86-9fb8-405862418a59 None None] Publish callbacks ['neutron.plugins.ml2.plugin.Ml2Plugin._port_provisioned-1745609'] for port (52a33477-c818-456c-b47f-a323e04cce42), provisioning_complete {{(pid=71111) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:176}}
  Jan 18 12:05:45.483625 ubuntu-focal-inmotion-iad3-0028053821 neutron-server[71111]: DEBUG neutron.plugins.ml2.plugin [None req-b33edd99-de3e-4c86-9fb8-405862418a59 None None] Current status of the port 52a33477-c818-456c-b47f-a323e04cce42 is: DOWN; New status is: ACTIVE {{(pid=71111) _update_individual_port_db_status /opt/stack/neutron/neutron/plugins/ml2/plugin.py:2213}}
  Jan 18 12:05:45.484006 ubuntu-focal-inmotion-iad3-0028053821 neutron-server[71111]: DEBUG neutron_lib.callbacks.manager [None req-b33edd99-de3e-4c86-9fb8-405862418a59 None None] Publish callbacks ['neutron.plugins.ml2.plugin.SecurityGroupDbMixin._ensure_default_security_group_handler-1688939', 'neutron.services.qos.qos_plugin.QoSPlugin._check_port_for_placement_allocation_change-102324'] for port (52a33477-c818-456c-b47f-a323e04cce42), before_update {{(pid=71111) _notify_loop /usr/local/lib/python3.8/dist-packages/neutron_lib/callbacks/manager.py:176}}
  Jan 18 12:05:45.514465 ubuntu-focal-inmotion-iad3-0028053821 neutron-server[71111]: DEBUG neutron.notifiers.nova [None req-b33edd99-de3e-4c86-9fb8-405862418a59 None None] Nova notifier disabled {{(pid=71111) _can_notify /opt/stack/neutron/neutron/notifiers/nova.py:180}}

  Example of failure:
  https://0cc0fb9d5c802f1c7b9c-6aa1154e79cc29d864ed6f661bf68125.ssl.cf5.rackcdn.com/824982/2/check/neutron-
  ovs-tempest-multinode-full/1e135d1/testr_results.html

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1958363/+subscriptions