yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #72166
[Bug 1761555] [NEW] [dvr][fast-exit] router add/remove subnet operations are not idempotent
Public bug reported:
OpenStack Queens from UCA (xenial, GA kernel), 2 external subnets (one
routed provider network), 2 tenant subnets, all subnets in the same
address scope to trigger "fast exit" logic.
Tenant subnet cidr: 192.168.100.0/24
Other tenant subnet cidr: 192.168.200.0/24
Relevant agent configs:
http://paste.openstack.org/show/718514/
Commands and outputs:
http://paste.openstack.org/show/JFYmGJwF1pdtliQOfXgd/
Overall, a similar situation as with
https://bugs.launchpad.net/neutron/+bug/1759956 but with one tenant
subnet at first for which routes and rules do not get deleted at all.
Problem description:
* router add subnet tenantsubnet
* routes in fip namespace and rules in qrouter namespace get created and a distributed port gets created for DVR;
* router remove subnet tenantsubnet
* routes are still there, no new logged events in DVR l3 agent logs
If two networks are added then removing one of them triggers removal of
routes and rules and new messages are logged in l3 agent log (the rules
removed are affected by pad.lv/1759956).
A sequence of add subnet/remove subnet commands may result in errors
logged in l3 agent logs: http://paste.openstack.org/show/718511/
Sometimes after re-adding a tenantsubnet in presence of othertenantsubnet a proper route is added for a few seconds but then removed:
# just do some operations
(openstack) router add subnet pubrouter tenantsubnet
(openstack) router add subnet pubrouter othertenantsubnet
(openstack) router add subnet pubrouter tenantsubnet
(openstack) router add subnet pubrouter tenantsubnet
(openstack) router remove subnet pubrouter tenantsubnet
# lots of errors, see http://paste.openstack.org/show/718511/
# try again without restarting agents
(openstack) router add subnet pubrouter tenantsubnet # ran client command
# ... got 192.168.100.0/24 here for a few seconds while l3 agent was doing something
10.232.16.0/21 dev fg-7f42af4f-ad proto kernel scope link src 10.232.17.5
169.254.106.114/31 dev fpr-3182a7c6-b proto kernel scope link src 169.254.106.115
192.168.100.0/24 via 169.254.106.114 dev fpr-3182a7c6-b
192.168.200.0/24 via 169.254.106.114 dev fpr-3182a7c6-b
# finished server and l3 agent finished processing "router add subnet pubrouter tenantsubnet"
# route got deleted
root@ipotane:~# ip netns exec fip-64ab1ec3-4927-4f09-87f9-804e7f4f8748 ip r
10.232.16.0/21 dev fg-7f42af4f-ad proto kernel scope link src 10.232.17.5
169.254.106.114/31 dev fpr-3182a7c6-b proto kernel scope link src 169.254.106.115
192.168.200.0/24 via 169.254.106.114 dev fpr-3182a7c6-b
There is something wrong with how tenant network add/remove
notifications are sent it seems because on first removal of a tenant
network nothing is logged in l3 agent logs but there is activity in
neutron server logs.
** Affects: neutron
Importance: Undecided
Status: New
** Tags: cpe-onsite
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1761555
Title:
[dvr][fast-exit] router add/remove subnet operations are not
idempotent
Status in neutron:
New
Bug description:
OpenStack Queens from UCA (xenial, GA kernel), 2 external subnets (one
routed provider network), 2 tenant subnets, all subnets in the same
address scope to trigger "fast exit" logic.
Tenant subnet cidr: 192.168.100.0/24
Other tenant subnet cidr: 192.168.200.0/24
Relevant agent configs:
http://paste.openstack.org/show/718514/
Commands and outputs:
http://paste.openstack.org/show/JFYmGJwF1pdtliQOfXgd/
Overall, a similar situation as with
https://bugs.launchpad.net/neutron/+bug/1759956 but with one tenant
subnet at first for which routes and rules do not get deleted at all.
Problem description:
* router add subnet tenantsubnet
* routes in fip namespace and rules in qrouter namespace get created and a distributed port gets created for DVR;
* router remove subnet tenantsubnet
* routes are still there, no new logged events in DVR l3 agent logs
If two networks are added then removing one of them triggers removal
of routes and rules and new messages are logged in l3 agent log (the
rules removed are affected by pad.lv/1759956).
A sequence of add subnet/remove subnet commands may result in errors
logged in l3 agent logs: http://paste.openstack.org/show/718511/
Sometimes after re-adding a tenantsubnet in presence of othertenantsubnet a proper route is added for a few seconds but then removed:
# just do some operations
(openstack) router add subnet pubrouter tenantsubnet
(openstack) router add subnet pubrouter othertenantsubnet
(openstack) router add subnet pubrouter tenantsubnet
(openstack) router add subnet pubrouter tenantsubnet
(openstack) router remove subnet pubrouter tenantsubnet
# lots of errors, see http://paste.openstack.org/show/718511/
# try again without restarting agents
(openstack) router add subnet pubrouter tenantsubnet # ran client command
# ... got 192.168.100.0/24 here for a few seconds while l3 agent was doing something
10.232.16.0/21 dev fg-7f42af4f-ad proto kernel scope link src 10.232.17.5
169.254.106.114/31 dev fpr-3182a7c6-b proto kernel scope link src 169.254.106.115
192.168.100.0/24 via 169.254.106.114 dev fpr-3182a7c6-b
192.168.200.0/24 via 169.254.106.114 dev fpr-3182a7c6-b
# finished server and l3 agent finished processing "router add subnet pubrouter tenantsubnet"
# route got deleted
root@ipotane:~# ip netns exec fip-64ab1ec3-4927-4f09-87f9-804e7f4f8748 ip r
10.232.16.0/21 dev fg-7f42af4f-ad proto kernel scope link src 10.232.17.5
169.254.106.114/31 dev fpr-3182a7c6-b proto kernel scope link src 169.254.106.115
192.168.200.0/24 via 169.254.106.114 dev fpr-3182a7c6-b
There is something wrong with how tenant network add/remove
notifications are sent it seems because on first removal of a tenant
network nothing is logged in l3 agent logs but there is activity in
neutron server logs.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1761555/+subscriptions