yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #46303
[Bug 1399249] Re: Neutron openvswitch-agent doesn't recover ports from binding_failed status
Reviewed: https://review.openstack.org/162260
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=63fe3a418c685ca90077f1dd4c35fd9ccf586fca
Submitter: Jenkins
Branch: master
commit 63fe3a418c685ca90077f1dd4c35fd9ccf586fca
Author: Ihar Hrachyshka <ihrachys@xxxxxxxxxx>
Date: Mon Mar 9 18:05:18 2015 +0100
Add the rebinding chance in _bind_port_if_needed
Make function _bind_port_if_needed to bind at least one time when the port's
binding status passed in is already in binding_failed.
This is the second attempt to introduce the patch (the first one was
reverted due to regression that broke Ironic), now with proper
notification sent even when binding attempt failed.
The patch also fixes several cases when we attempted to notify with a
binding context that was not committed into database.
The patch changes _attempt_binding to call _commit_port_binding only
with the binding final state:
1. Successful binding: will just call _commit_port_binding.
2. Unsuccessful binding: will call _commit_port_binding at the final
attempt to bind the port.
This is in order to refrain from reverts, with will really complicate
things even more.
Co-Authored-By: Yalei Wang <yalei.wang@xxxxxxxxx>
Co-Authored-By: Nir Magnezi <nmagnezi@xxxxxxxxxx>
Co-Authored-By: John Schwarz <jschwarz@xxxxxxxxxx>
Change-Id: I437290affd8eb87177d0626bf7935a165859cbdd
Closes-Bug: #1399249
** Changed in: neutron
Status: In Progress => Fix Released
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1399249
Title:
Neutron openvswitch-agent doesn't recover ports from binding_failed
status
Status in neutron:
Fix Released
Bug description:
Ports created when neutron-openvswitch-agent is down are in status
down and "binding:vif_type=binding_failed" which is as it should be.
When the agent is rebooted it should be able to recreate the ports
according to the DB, but instead it logs a WARNING and creates the
port with status DOWN. only solution is to delete the port and create
it again
From agent log:
2014-12-04 16:53:00.559 16319 WARNING neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-2dcc9141-7439-450a-bb2a-fe31ab577f47 None] Device 3dc73917-93b1-4f6d-a2e1-90c74cea6de7 not defined on plugin
Recreation steps:
shut down ovs-agent and wait for neutron to notice:
[root@RHEL7Server ~]# systemctl stop neutron-openvswitch-agent.service
[root@RHEL7Server ~(keystone_admin)]# neutron agent-list | grep open
| 2d97bbd1-b937-4b19-8205-4167bbcb659d | Open vSwitch agent | node_29 | xxx | True | neutron-openvswitch-agent |
create router and attach it to network
[root@RHEL7Server ~(keystone_admin)]# neutron router-create myrouter --ha False
Created a new router:
+-----------------------+--------------------------------------+
| Field | Value |
+-----------------------+--------------------------------------+
| admin_state_up | True |
| distributed | False |
| external_gateway_info | |
| ha | False |
| id | 8210f453-2a17-400e-ae32-74aa1503d0a5 |
| name | myrouter |
| routes | |
| status | ACTIVE |
| tenant_id | 183611eb84204b839e43d97c081973c0 |
+-----------------------+--------------------------------------+
[root@RHEL7Server ~(keystone_admin)]# neutron router-interface-add myrouter private
Added interface 3dc73917-93b1-4f6d-a2e1-90c74cea6de7 to router myrouter.
[root@RHEL7Server ~(keystone_admin)]# neutron l3-agent-list-hosting-router myrouter
+--------------------------------------+---------+----------------+-------+
| id | host | admin_state_up | alive |
+--------------------------------------+---------+----------------+-------+
| 0110d49c-59dd-496c-a2a3-549a2ad4ba4d | node_29 | True | :-) |
+--------------------------------------+---------+----------------+-------+
Port will show status DOWN, and "binding_failed"
[root@RHEL7Server ~(keystone_admin)]# neutron port-show 3dc73917-93b1-4f6d-a2e1-90c74cea6de7
+-----------------------+---------------------------------------------------------------------------------+
| Field | Value |
+-----------------------+---------------------------------------------------------------------------------+
| admin_state_up | True |
| allowed_address_pairs | |
| binding:host_id | node_29 |
| binding:profile | {} |
| binding:vif_details | {} |
| binding:vif_type | binding_failed |
| binding:vnic_type | normal |
| device_id | 8210f453-2a17-400e-ae32-74aa1503d0a5 |
| device_owner | network:router_interface |
| extra_dhcp_opts | |
| fixed_ips | {"subnet_id": "d8881a14-bd8b-4595-b497-8da6587a46c1", "ip_address": "10.0.0.1"} |
| id | 3dc73917-93b1-4f6d-a2e1-90c74cea6de7 |
| mac_address | fa:16:3e:db:0f:9b |
| name | |
| network_id | 6091abc0-4fdf-402d-aaf0-3a955fabd6b7 |
| security_groups | |
| status | DOWN |
| tenant_id | 183611eb84204b839e43d97c081973c0 |
+-----------------------+---------------------------------------------------------------------------------+
start ovs-agent
systemctl start neutron-openvswitch-agent.service
[root@RHEL7Server ~(keystone_admin)]# neutron agent-list | grep open
| 2d97bbd1-b937-4b19-8205-4167bbcb659d | Open vSwitch agent | node_29 | :-) | True | neutron-openvswitch-agent |
Port will be forever down and even when restarting the agent won't
bring it up
From agent log:
2014-12-04 16:53:00.559 16319 WARNING neutron.plugins.openvswitch.agent.ovs_neutron_agent [req-2dcc9141-7439-450a-bb2a-fe31ab577f47 None] Device 3dc73917-93b1-4f6d-a2e1-90c74cea6de7 not defined on plugin
Workarounds:
1) Recreate the resource (VM, DHCP or router port)
2) Update DB and restart OVS agent as detailed in https://bugs.launchpad.net/neutron/+bug/1399249/comments/10.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1399249/+subscriptions
References
