yahoo-eng-team team mailing list archive
-
yahoo-eng-team team
-
Mailing list archive
-
Message #92506
[Bug 2024251] [NEW] Adding a static route to router returns internal server error
Public bug reported:
In a multi-tenant setup, with a shared provider network, adding a static
route to a router returns internal server error. Note: IP addresses are
obfucated
openstack router set --route destination='300.300.0.0/16',gateway='172.31.96.1' tickets_private_router
HttpException: 500: Server Error for url: https://controller01:9696/v2.0/routers/78c02b65-9819-48d9-9700-f045a28909e0, Request Failed: internal server error while processing your request.
Openstack release: Antelope (2023.1)
Neutron release: 22.0.0-1
OVN: ovn22.09
Distribution: Centos 9 Streams
Manual Installation.
The router:
# openstack router show tickets_private_router
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | |
| created_at | 2023-06-14T17:49:52Z |
| description | |
| enable_ndp_proxy | None |
| external_gateway_info | {"network_id": "b0c5df5c-8d5c-4fc2-babf-829421243017", "external_fixed_ips": [{"subnet_id": "4b9540d4-e3d4-403d-9918-98f111d8fedd", "ip_address": "172.31.97.226"}], "enable_snat": true} |
| flavor_id | None |
| id | 78c02b65-9819-48d9-9700-f045a28909e0 |
| interfaces_info | [{"port_id": "7f628431-87bf-460e-88a0-b0426040a07c", "ip_address": "172.31.100.1", "subnet_id": "dd636d3d-7dc9-472e-b340-095ff85db1c2"}] |
| name | tickets_private_router |
| project_id | d9e5cee8a9e94a1883376ce48e676ca4 |
| revision_number | 10 |
| routes | |
| status | ACTIVE |
| tags | |
| tenant_id | d9e5cee8a9e94a1883376ce48e676ca4 |
| updated_at | 2023-06-16T16:58:52Z |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
In the logs for neutron-server:
2023-06-16 13:17:39.769 10 ERROR ovsdbapp.backend.ovs_idl.transaction [None req-beb7e61c-a6da-4055-b41f-2bc728061394 - - - - - -] OVSDB Error: {"details":"Table Logical_Router column static_routes row 3fae901a-c2e3-468b-b8af-4343b6c3b937 references nonexistent row 57183168-288c-4700-b651-a87e3f2bf4eb in table Logical_Router_Static_Route.","error":"referential integrity violation"}
neutron-server_1 | 2023-06-16 13:17:39.769 10 ERROR ovsdbapp.backend.ovs_idl.transaction [None req-79ee6c1d-3c7b-4578-aedb-9af1dbb13e60 13d6e02fd8b74011b0e5d600d5c64300 cbadbe9f0cce4066b722422b3844a09e - - default default] Traceback (most recent call last):
neutron-server_1 | File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 118, in run
neutron-server_1 | txn.results.put(txn.do_commit())
neutron-server_1 | File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
neutron-server_1 | raise RuntimeError(msg)
neutron-server_1 | RuntimeError: OVSDB Error: {"details":"Table Logical_Router column static_routes row 3fae901a-c2e3-468b-b8af-4343b6c3b937 references nonexistent row 57183168-288c-4700-b651-a87e3f2bf4eb in table Logical_Router_Static_Route.","error":"referential integrity violation"}
neutron-server_1 |
neutron-server_1 | 2023-06-16 13:17:39.769 10 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-79ee6c1d-3c7b-4578-aedb-9af1dbb13e60 13d6e02fd8b74011b0e5d600d5c64300 cbadbe9f0cce4066b722422b3844a09e - - default default] Unable to update router 78c02b65-9819-48d9-9700-f045a28909e0. Error: OVSDB Error: {"details":"Table Logical_Router column static_routes row 3fae901a-c2e3-468b-b8af-4343b6c3b937 references nonexistent row 57183168-288c-4700-b651-a87e3f2bf4eb in table Logical_Router_Static_Route.","error":"referential integrity violation"}: RuntimeError: OVSDB Error: {"details":"Table Logical_Router column static_routes row 3fae901a-c2e3-468b-b8af-4343b6c3b937 references nonexistent row 57183168-288c-4700-b651-a87e3f2bf4eb in table Logical_Router_Static_Route.","error":"referential integrity violation"}
neutron-server_1 | 2023-06-16 13:17:39.770 10 ERROR neutron.services.ovn_l3.plugin [None req-79ee6c1d-3c7b-4578-aedb-9af1dbb13e60 13d6e02fd8b74011b0e5d600d5c64300 cbadbe9f0cce4066b722422b3844a09e - - default default] Unable to update lrouter for 78c02b65-9819-48d9-9700-f045a28909e0: RuntimeError: OVSDB Error: {"details":"Table Logical_Router column static_routes row 3fae901a-c2e3-468b-b8af-4343b6c3b937 references nonexistent row 57183168-288c-4700-b651-a87e3f2bf4eb in table Logical_Router_Static_Route.","error":"referential integrity violation"}
Stack trace follows.
This will ocassionaly succeed on other routers after a delay of 10+
minutes, but some routers refuse to accept the staic route.
** Affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/2024251
Title:
Adding a static route to router returns internal server error
Status in neutron:
New
Bug description:
In a multi-tenant setup, with a shared provider network, adding a
static route to a router returns internal server error. Note: IP
addresses are obfucated
openstack router set --route destination='300.300.0.0/16',gateway='172.31.96.1' tickets_private_router
HttpException: 500: Server Error for url: https://controller01:9696/v2.0/routers/78c02b65-9819-48d9-9700-f045a28909e0, Request Failed: internal server error while processing your request.
Openstack release: Antelope (2023.1)
Neutron release: 22.0.0-1
OVN: ovn22.09
Distribution: Centos 9 Streams
Manual Installation.
The router:
# openstack router show tickets_private_router
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Field | Value |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| admin_state_up | UP |
| availability_zone_hints | |
| availability_zones | |
| created_at | 2023-06-14T17:49:52Z |
| description | |
| enable_ndp_proxy | None |
| external_gateway_info | {"network_id": "b0c5df5c-8d5c-4fc2-babf-829421243017", "external_fixed_ips": [{"subnet_id": "4b9540d4-e3d4-403d-9918-98f111d8fedd", "ip_address": "172.31.97.226"}], "enable_snat": true} |
| flavor_id | None |
| id | 78c02b65-9819-48d9-9700-f045a28909e0 |
| interfaces_info | [{"port_id": "7f628431-87bf-460e-88a0-b0426040a07c", "ip_address": "172.31.100.1", "subnet_id": "dd636d3d-7dc9-472e-b340-095ff85db1c2"}] |
| name | tickets_private_router |
| project_id | d9e5cee8a9e94a1883376ce48e676ca4 |
| revision_number | 10 |
| routes | |
| status | ACTIVE |
| tags | |
| tenant_id | d9e5cee8a9e94a1883376ce48e676ca4 |
| updated_at | 2023-06-16T16:58:52Z |
+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
In the logs for neutron-server:
2023-06-16 13:17:39.769 10 ERROR ovsdbapp.backend.ovs_idl.transaction [None req-beb7e61c-a6da-4055-b41f-2bc728061394 - - - - - -] OVSDB Error: {"details":"Table Logical_Router column static_routes row 3fae901a-c2e3-468b-b8af-4343b6c3b937 references nonexistent row 57183168-288c-4700-b651-a87e3f2bf4eb in table Logical_Router_Static_Route.","error":"referential integrity violation"}
neutron-server_1 | 2023-06-16 13:17:39.769 10 ERROR ovsdbapp.backend.ovs_idl.transaction [None req-79ee6c1d-3c7b-4578-aedb-9af1dbb13e60 13d6e02fd8b74011b0e5d600d5c64300 cbadbe9f0cce4066b722422b3844a09e - - default default] Traceback (most recent call last):
neutron-server_1 | File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/connection.py", line 118, in run
neutron-server_1 | txn.results.put(txn.do_commit())
neutron-server_1 | File "/usr/lib/python3.9/site-packages/ovsdbapp/backend/ovs_idl/transaction.py", line 123, in do_commit
neutron-server_1 | raise RuntimeError(msg)
neutron-server_1 | RuntimeError: OVSDB Error: {"details":"Table Logical_Router column static_routes row 3fae901a-c2e3-468b-b8af-4343b6c3b937 references nonexistent row 57183168-288c-4700-b651-a87e3f2bf4eb in table Logical_Router_Static_Route.","error":"referential integrity violation"}
neutron-server_1 |
neutron-server_1 | 2023-06-16 13:17:39.769 10 ERROR neutron.plugins.ml2.drivers.ovn.mech_driver.ovsdb.ovn_client [None req-79ee6c1d-3c7b-4578-aedb-9af1dbb13e60 13d6e02fd8b74011b0e5d600d5c64300 cbadbe9f0cce4066b722422b3844a09e - - default default] Unable to update router 78c02b65-9819-48d9-9700-f045a28909e0. Error: OVSDB Error: {"details":"Table Logical_Router column static_routes row 3fae901a-c2e3-468b-b8af-4343b6c3b937 references nonexistent row 57183168-288c-4700-b651-a87e3f2bf4eb in table Logical_Router_Static_Route.","error":"referential integrity violation"}: RuntimeError: OVSDB Error: {"details":"Table Logical_Router column static_routes row 3fae901a-c2e3-468b-b8af-4343b6c3b937 references nonexistent row 57183168-288c-4700-b651-a87e3f2bf4eb in table Logical_Router_Static_Route.","error":"referential integrity violation"}
neutron-server_1 | 2023-06-16 13:17:39.770 10 ERROR neutron.services.ovn_l3.plugin [None req-79ee6c1d-3c7b-4578-aedb-9af1dbb13e60 13d6e02fd8b74011b0e5d600d5c64300 cbadbe9f0cce4066b722422b3844a09e - - default default] Unable to update lrouter for 78c02b65-9819-48d9-9700-f045a28909e0: RuntimeError: OVSDB Error: {"details":"Table Logical_Router column static_routes row 3fae901a-c2e3-468b-b8af-4343b6c3b937 references nonexistent row 57183168-288c-4700-b651-a87e3f2bf4eb in table Logical_Router_Static_Route.","error":"referential integrity violation"}
Stack trace follows.
This will ocassionaly succeed on other routers after a delay of 10+
minutes, but some routers refuse to accept the staic route.
To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/2024251/+subscriptions
Follow ups
