← Back to team overview

yahoo-eng-team team mailing list archive

[Bug 1588593] [NEW] If Neutron IPAM driver is setted,using 'net-delete' command to delete the network created when ipam_driver is not set,the command seems to cause dead loop.

 

Public bug reported:

In Mitaka,

When ipam_driver is not setted,created a network with a subnet,then using the reference implementation of 
Neutron IPAM driver by setting 'ipam_driver='internal'',and using 'net-delete' command to delete the 
network created when ipam_driver is not set,the command seems to cause dead loop.


1)Specifying ‘ipam_driver = <None>’ in the neutron.conf file,created a network with a subnet
[root@localhost devstack]# neutron net-create net_vlan_01 --provider:network_type vlan --provider:physical_network physnet1 --provider:segmentation_id  2 
Created a new network:
+---------------------------+--------------------------------------+
| Field                     | Value                                |
+---------------------------+--------------------------------------+
| admin_state_up            | True                                 |
| availability_zone_hints   |                                      |
| availability_zones        |                                      |
| created_at                | 2016-06-03T02:42:50                  |
| description               |                                      |
| id                        | 666f8a6a-e3e3-4183-84b3-a43c92b050f5 |
| ipv4_address_scope        |                                      |
| ipv6_address_scope        |                                      |
| mtu                       | 1500                                 |
| name                      | net_vlan_01                          |
| port_security_enabled     | True                                 |
| provider:network_type     | vlan                                 |
| provider:physical_network | physnet1                             |
| provider:segmentation_id  | 2                                    |
| qos_policy_id             |                                      |
| router:external           | False                                |
| shared                    | False                                |
| status                    | ACTIVE                               |
| subnets                   |                                      |
| tags                      |                                      |
| tenant_id                 | 69fa49e368d340679ab3d05de3426bfa     |
| updated_at                | 2016-06-03T02:42:50                  |
+---------------------------+--------------------------------------+
[root@localhost devstack]# neutron subnet-create net_vlan_01 --name subnet_vlan_01 101.1.1.0/24
Created a new subnet:
+-------------------+----------------------------------------------+
| Field             | Value                                        |
+-------------------+----------------------------------------------+
| allocation_pools  | {"start": "101.1.1.2", "end": "101.1.1.254"} |
| cidr              | 101.1.1.0/24                                 |
| created_at        | 2016-06-03T02:42:56                          |
| description       |                                              |
| dns_nameservers   |                                              |
| enable_dhcp       | True                                         |
| gateway_ip        | 101.1.1.1                                    |
| host_routes       |                                              |
| id                | 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad         |
| ip_version        | 4                                            |
| ipv6_address_mode |                                              |
| ipv6_ra_mode      |                                              |
| name              | subnet_vlan_01                               |
| network_id        | 666f8a6a-e3e3-4183-84b3-a43c92b050f5         |
| subnetpool_id     |                                              |
| tenant_id         | 69fa49e368d340679ab3d05de3426bfa             |
| updated_at        | 2016-06-03T02:42:56                          |
+-------------------+----------------------------------------------+
[root@localhost devstack]# neutron net-list
+--------------------------------------+-------------+----------------------------------------------------------+
| id                                   | name        | subnets                                                  |
+--------------------------------------+-------------+----------------------------------------------------------+
| 666f8a6a-e3e3-4183-84b3-a43c92b050f5 | net_vlan_01 | 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad 101.1.1.0/24        |
+--------------------------------------+-------------+----------------------------------------------------------+
[root@localhost devstack]# neutron port-list 
| e759c9df-db2d-477b-94e9-02d80844f7f9 |      | fa:16:3e:2d:e5:6e | {"subnet_id": "1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad", "ip_address": "101.1.1.2"}    

2)Modified ‘ipam_driver = ‘internal’’ in the neutron.conf file,and restarted neutron-server service                        |
[root@localhost devstack]# vi /etc/neutron/neutron.conf 

# Neutron IPAM (IP address management) driver to use. If ipam_driver is not set
# (default behavior), no IPAM driver is used. In order to use the reference
# implementation of Neutron IPAM driver, use 'internal'. (string value)
#ipam_driver = <None>
ipam_driver = internal

"/etc/neutron/neutron.conf" 1538L, 54626C written

[root@localhost devstack]# neutron net-list
+--------------------------------------+-------------+----------------------------------------------------------+
| id                                   | name        | subnets                                                  |
+--------------------------------------+-------------+----------------------------------------------------------+
| 666f8a6a-e3e3-4183-84b3-a43c92b050f5 | net_vlan_01 | 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad 101.1.1.0/24        |
+--------------------------------------+-------------+----------------------------------------------------------+
[root@localhost devstack]# neutron port-list |grep 101
| e759c9df-db2d-477b-94e9-02d80844f7f9 |      | fa:16:3e:2d:e5:6e | {"subnet_id": "1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad", "ip_address": "101.1.1.2"}  

3)Deleting the network returns none,and dead., 
the usage of neutron-server is above 80%,and neutron-server's log is repeating “ERROR neutron.ipam.drivers.neutrondb_ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3
 referenced to Neutron subnet 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad does not exist”             |
[root@localhost devstack]# neutron net-delete net_vlan_01

Last login: Thu Jun  2 22:38:22 2016 from 10.43.165.22
[root@localhost ~]# top
top - 22:46:35 up 22:54, 33 users,  load average: 0.98, 0.99, 0.80
Tasks: 562 total,   4 running, 558 sleeping,   0 stopped,   0 zombie
%Cpu(s):  6.2 us,  0.4 sy,  0.0 ni, 93.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 13189404+total, 10021431+free,  6880860 used, 24798868 buff/cache
KiB Swap:  4194300 total,  4194300 free,        0 used. 12443545+avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                 
16279 stack     20   0  285740 100868   2152 R  85.4  0.1   2:02.06 neutron-server                                                          
 6529 mysql     20   0 6192992 352368  13120 S   4.3  0.3   4:58.96 mysqld                                                                  
11472 root      20   0   78952   6656   2904 S   4.3  0.0   2:05.32 ssh                                                                     
 4351 stack     20   0  291656  85224   6568 S   3.0  0.1  24:39.79 cinder-schedule                                                         
14363 stack     20   0  237336  61796   5412 R   2.6  0.0  29:18.80 neutron-openvsw                                                         
20050 stack     20   0 2048392 150276  12168 S   2.6  0.1  35:49.49 nova-compute                                                            
 1131 stack     20   0  276828  92148   2452 S   2.3  0.1  25:45.10 nova-conductor                                                          
 1134 stack     20   0  278832  94184   2460 S   2.3  0.1  25:46.85 nova-conductor                                                          
 1135 stack     20   0  278288  93632   2460 S   2.3  0.1  25:47.77 nova-conductor                                                          
 1655 rabbitmq  20   0 4096148 212828   3136 S   2.3  0.2  45:15.30 beam.smp                                                                
16280 stack     20   0  268268  83180   2056 S   2.3  0.1   0:03.72 neutron-server                                                          
27445 stack     20   0  230736  55112   5420 S   2.3  0.0  24:20.15 neutron-dhcp-ag                                                         
 1130 stack     20   0  280308  95736   2468 S   2.0  0.1  25:47.58 nova-conductor                                                          
 1132 stack     20   0  280084  95476   2468 S   2.0  0.1  25:48.68 nova-conductor                                                          
 1133 stack     20   0  276848  92148   2452 S   2.0  0.1  25:43.78 nova-conductor                                                          
 5094 stack     20   0  288528  77764   2240 S   2.0  0.1  26:00.90 cinder-volume                                                           
16281 stack     20   0  267496  82240   2056 S   2.0  0.1   0:03.67 neutron-server                                                          
20207 stack     20   0  265884  84360   5628 S   2.0  0.1  23:32.15 nova-consoleaut                                                         
22228 stack     20   0  266268  84764   5628 S   2.0  0.1  23:45.42 nova-scheduler                                                          
24333 stack     20   0  231164  70956   2168 S   2.0  0.1  22:29.85 glance-api                                                              
24335 stack     20   0  225508  65548   2488 S   2.0  0.0  21:41.44 glance-api                                                              
24340 stack     20   0  231676  71900   2680 S   2.0  0.1  23:21.55 glance-api                                                              
24343 stack     20   0  231164  70864   2168 S   2.0  0.1  22:29.88 glance-api                                                              
25713 stack     20   0  355648 147884   4252 S   2.0  0.1  22:36.73 nova-api                                                                
25718 stack     20   0  316728 129704   2168 S   2.0  0.1   0:35.72 nova-api                                                                
27962 stack     20   0  230892  55400   5412 S   2.0  0.0  23:37.71 neutron-l3-agen                                                         
28333 stack     20   0  227604  51948   5408 S   2.0  0.0  23:21.07 neutron-metadat                                                         
11471 root      20   0   52748   2176   1492 S   1.7  0.0   0:47.99 scp                                                                     
 3835 stack     20   0  299164  92840   6568 S   1.3  0.1  12:56.21 cinder-api                                                              
25581 stack     20   0  298928 115296   5616 S   1.3  0.1  12:49.65 nova-api                                                                
32121 stack     20   0  252552  72948   5560 S   1.3  0.1  12:41.01 nova-conductor                                                          
 4875 stack     20   0  286764  80432   6564 S   1.0  0.1  12:46.46 cinder-volume                                                           
24206 stack     20   0  224384  67540   5596 S   1.0  0.1  12:17.30 glance-api                                                              
   34 root      20   0       0      0      0 S   0.7  0.0   2:17.17 rcu_sched                                                               
[root@localhost ~]# 

2016-06-02 22:44:18.315 ERROR neutron.ipam.drivers.neutrondb_ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3
 referenced to Neutron subnet 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad does not exist
2016-06-02 22:44:18.316 DEBUG neutron.db.ipam_pluggable_backend [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426
 on ip deallocation: <neutron.db.models_v2.IPAllocation[object at 6588b10] {port_id=u'e759c9df-db2d-477b-94e9-02d80844f7f9', ip_address=u'101
dbd7-ae1e-4d7c-a767-ec3106cc62ad', network_id=u'666f8a6a-e3e3-4183-84b3-a43c92b050f5'}> from (pid=16279) _ipam_deallocate_ips /opt/stack/neut
able_backend.py:73
2016-06-02 22:44:18.330 DEBUG neutron.callbacks.manager [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Not
fter_delete from (pid=16279) _notify_loop /opt/stack/neutron/neutron/callbacks/manager.py:138
2016-06-02 22:44:18.331 DEBUG neutron.callbacks.manager [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Cal
l3_db._notify_routers_callback from (pid=16279) _notify_loop /opt/stack/neutron/neutron/callbacks/manager.py:145
2016-06-02 22:44:18.331 DEBUG neutron.callbacks.manager [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Cal
l3_dvrscheduler_db._notify_port_delete from (pid=16279) _notify_loop /opt/stack/neutron/neutron/callbacks/manager.py:145
2016-06-02 22:44:18.409 DEBUG neutron.db.l3_dvrscheduler_db [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa]
DVR port e759c9df-db2d-477b-94e9-02d80844f7f9 on host localhost.localdomain from (pid=16279) get_dvr_routers_to_remove /opt/stack/neutron/neu
b.py:162
2016-06-02 22:44:18.428 DEBUG oslo_messaging._drivers.amqpdriver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de342
c18736292407fa506c49404a98c88 FANOUT topic 'q-agent-notifier-port-delete' from (pid=16279) _send /usr/lib/python2.7/site-packages/oslo_messag
y:439
2016-06-02 22:44:18.435 DEBUG oslo_messaging._drivers.amqpdriver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de342
72b12fefd43aab74b1acdee75acca FANOUT topic 'q-agent-notifier-security_group-update' from (pid=16279) _send /usr/lib/python2.7/site-packages/o
qpdriver.py:439
2016-06-02 22:44:18.437 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
1e-4d7c-a767-ec3106cc62ad from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:959
2016-06-02 22:44:18.505 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Po
] from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:984
2016-06-02 22:44:18.508 DEBUG neutron.callbacks.manager [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Not
 before_delete from (pid=16279) _notify_loop /opt/stack/neutron/neutron/callbacks/manager.py:138
2016-06-02 22:44:18.616 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
 (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1023
2016-06-02 22:44:18.621 DEBUG neutron.ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Loading i
 (pid=16279) get_instance /opt/stack/neutron/neutron/ipam/driver.py:51
2016-06-02 22:44:18.697 ERROR neutron.ipam.drivers.neutrondb_ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3
 referenced to Neutron subnet 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad does not exist
2016-06-02 22:44:18.714 INFO neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Sub
67-ec3106cc62ad was deleted concurrently
2016-06-02 22:44:18.739 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Po
om (pid=16279) delete_network /opt/stack/neutron/neutron/plugins/ml2/plugin.py:846
2016-06-02 22:44:18.744 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Su
eutron.db.models_v2.Subnet[object at 659a750] {tenant_id=u'69fa49e368d340679ab3d05de3426bfa', id=u'1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad', nam
ork_id=u'666f8a6a-e3e3-4183-84b3-a43c92b050f5', segment_id=None, subnetpool_id=None, ip_version=4, cidr=u'101.1.1.0/24', gateway_ip=u'101.1.1
v6_ra_mode=None, ipv6_address_mode=None, standard_attr_id=49}>] from (pid=16279) delete_network /opt/stack/neutron/neutron/plugins/ml2/plugin
2016-06-02 22:44:18.745 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
1e-4d7c-a767-ec3106cc62ad from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:959
2016-06-02 22:44:18.822 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Po
] from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:984
2016-06-02 22:44:18.825 DEBUG neutron.callbacks.manager [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Not
 before_delete from (pid=16279) _notify_loop /opt/stack/neutron/neutron/callbacks/manager.py:138
2016-06-02 22:44:19.072 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
 (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1023
2016-06-02 22:44:19.075 DEBUG neutron.ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Loading i
 (pid=16279) get_instance /opt/stack/neutron/neutron/ipam/driver.py:51
2016-06-02 22:44:19.121 ERROR neutron.ipam.drivers.neutrondb_ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3
 referenced to Neutron subnet 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad does not exist
2016-06-02 22:44:19.136 INFO neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Sub
67-ec3106cc62ad was deleted concurrently
2016-06-02 22:44:19.154 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Po
om (pid=16279) delete_network /opt/stack/neutron/neutron/plugins/ml2/plugin.py:846
2016-06-02 22:44:19.159 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Su
eutron.db.models_v2.Subnet[object at 659a750] {tenant_id=u'69fa49e368d340679ab3d05de3426bfa', id=u'1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad', nam
ork_id=u'666f8a6a-e3e3-4183-84b3-a43c92b050f5', segment_id=None, subnetpool_id=None, ip_version=4, cidr=u'101.1.1.0/24', gateway_ip=u'101.1.1
v6_ra_mode=None, ipv6_address_mode=None, standard_attr_id=49}>] from (pid=16279) delete_network /opt/stack/neutron/neutron/plugins/ml2/plugin
2016-06-02 22:44:19.160 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
1e-4d7c-a767-ec3106cc62ad from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:959
2016-06-02 22:44:19.233 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Po
] from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:984
2016-06-02 22:44:19.237 DEBUG neutron.callbacks.manager [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Not
 before_delete from (pid=16279) _notify_loop /opt/stack/neutron/neutron/callbacks/manager.py:138
2016-06-02 22:44:19.317 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
 (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1023
2016-06-02 22:44:19.320 DEBUG neutron.ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Loading i
 (pid=16279) get_instance /opt/stack/neutron/neutron/ipam/driver.py:51
2016-06-02 22:44:19.328 ERROR neutron.ipam.drivers.neutrondb_ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3
 referenced to Neutron subnet 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad does not exist
2016-06-02 22:44:19.341 INFO neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Sub
67-ec3106cc62ad was deleted concurrently
2016-06-02 22:44:19.360 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Po
om (pid=16279) delete_network /opt/stack/neutron/neutron/plugins/ml2/plugin.py:846
2016-06-02 22:44:19.364 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Su
eutron.db.models_v2.Subnet[object at 659a750] {tenant_id=u'69fa49e368d340679ab3d05de3426bfa', id=u'1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad', nam
ork_id=u'666f8a6a-e3e3-4183-84b3-a43c92b050f5', segment_id=None, subnetpool_id=None, ip_version=4, cidr=u'101.1.1.0/24', gateway_ip=u'101.1.1
v6_ra_mode=None, ipv6_address_mode=None, standard_attr_id=49}>] from (pid=16279) delete_network /opt/stack/neutron/neutron/plugins/ml2/plugin
2016-06-02 22:44:19.365 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
1e-4d7c-a767-ec3106cc62ad from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:959
2016-06-02 22:44:19.438 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Po
] from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:984
2016-06-02 22:44:19.441 DEBUG neutron.callbacks.manager [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Not
 before_delete from (pid=16279) _notify_loop /opt/stack/neutron/neutron/callbacks/manager.py:138
2016-06-02 22:44:19.524 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
 (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1023
2016-06-02 22:44:19.527 DEBUG neutron.ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Loading i
 (pid=16279) get_instance /opt/stack/neutron/neutron/ipam/driver.py:51
2016-06-02 22:44:19.537 ERROR neutron.ipam.drivers.neutrondb_ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3
 referenced to Neutron subnet 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad does not exist
2016-06-02 22:44:19.553 INFO neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Sub
67-ec3106cc62ad was deleted concurrently
2016-06-02 22:44:19.578 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Po
om (pid=16279) delete_network /opt/stack/neutron/neutron/plugins/ml2/plugin.py:846
2016-06-02 22:44:19.582 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Su
eutron.db.models_v2.Subnet[object at 659a750] {tenant_id=u'69fa49e368d340679ab3d05de3426bfa', id=u'1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad', nam
ork_id=u'666f8a6a-e3e3-4183-84b3-a43c92b050f5', segment_id=None, subnetpool_id=None, ip_version=4, cidr=u'101.1.1.0/24', gateway_ip=u'101.1.1
v6_ra_mode=None, ipv6_address_mode=None, standard_attr_id=49}>] from (pid=16279) delete_network /opt/stack/neutron/neutron/plugins/ml2/plugin
2016-06-02 22:44:19.584 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
1e-4d7c-a767-ec3106cc62ad from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:959
2016-06-02 22:44:19.651 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Po
] from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:984
2016-06-02 22:44:19.654 DEBUG neutron.callbacks.manager [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Not
 before_delete from (pid=16279) _notify_loop /opt/stack/neutron/neutron/callbacks/manager.py:138
2016-06-02 22:44:19.716 DEBUG neutron.wsgi [-] (16278) accepted ('192.168.1.177', 60806) from (pid=16278) server /usr/lib/python2.7/site-pack
2016-06-02 22:44:19.734 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
 (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1023
2016-06-02 22:44:19.736 DEBUG neutron.ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Loading i
 (pid=16279) get_instance /opt/stack/neutron/neutron/ipam/driver.py:51
2016-06-02 22:44:19.744 ERROR neutron.ipam.drivers.neutrondb_ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3
 referenced to Neutron subnet 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad does not exist

** Affects: neutron
     Importance: Undecided
     Assignee: Yan Songming (songmingyan)
         Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1588593

Title:
  If Neutron IPAM driver is setted,using 'net-delete' command to delete
  the network created when ipam_driver is not set,the command seems to
  cause dead loop.

Status in neutron:
  New

Bug description:
  In Mitaka,

  When ipam_driver is not setted,created a network with a subnet,then using the reference implementation of 
  Neutron IPAM driver by setting 'ipam_driver='internal'',and using 'net-delete' command to delete the 
  network created when ipam_driver is not set,the command seems to cause dead loop.

  
  1)Specifying ‘ipam_driver = <None>’ in the neutron.conf file,created a network with a subnet
  [root@localhost devstack]# neutron net-create net_vlan_01 --provider:network_type vlan --provider:physical_network physnet1 --provider:segmentation_id  2 
  Created a new network:
  +---------------------------+--------------------------------------+
  | Field                     | Value                                |
  +---------------------------+--------------------------------------+
  | admin_state_up            | True                                 |
  | availability_zone_hints   |                                      |
  | availability_zones        |                                      |
  | created_at                | 2016-06-03T02:42:50                  |
  | description               |                                      |
  | id                        | 666f8a6a-e3e3-4183-84b3-a43c92b050f5 |
  | ipv4_address_scope        |                                      |
  | ipv6_address_scope        |                                      |
  | mtu                       | 1500                                 |
  | name                      | net_vlan_01                          |
  | port_security_enabled     | True                                 |
  | provider:network_type     | vlan                                 |
  | provider:physical_network | physnet1                             |
  | provider:segmentation_id  | 2                                    |
  | qos_policy_id             |                                      |
  | router:external           | False                                |
  | shared                    | False                                |
  | status                    | ACTIVE                               |
  | subnets                   |                                      |
  | tags                      |                                      |
  | tenant_id                 | 69fa49e368d340679ab3d05de3426bfa     |
  | updated_at                | 2016-06-03T02:42:50                  |
  +---------------------------+--------------------------------------+
  [root@localhost devstack]# neutron subnet-create net_vlan_01 --name subnet_vlan_01 101.1.1.0/24
  Created a new subnet:
  +-------------------+----------------------------------------------+
  | Field             | Value                                        |
  +-------------------+----------------------------------------------+
  | allocation_pools  | {"start": "101.1.1.2", "end": "101.1.1.254"} |
  | cidr              | 101.1.1.0/24                                 |
  | created_at        | 2016-06-03T02:42:56                          |
  | description       |                                              |
  | dns_nameservers   |                                              |
  | enable_dhcp       | True                                         |
  | gateway_ip        | 101.1.1.1                                    |
  | host_routes       |                                              |
  | id                | 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad         |
  | ip_version        | 4                                            |
  | ipv6_address_mode |                                              |
  | ipv6_ra_mode      |                                              |
  | name              | subnet_vlan_01                               |
  | network_id        | 666f8a6a-e3e3-4183-84b3-a43c92b050f5         |
  | subnetpool_id     |                                              |
  | tenant_id         | 69fa49e368d340679ab3d05de3426bfa             |
  | updated_at        | 2016-06-03T02:42:56                          |
  +-------------------+----------------------------------------------+
  [root@localhost devstack]# neutron net-list
  +--------------------------------------+-------------+----------------------------------------------------------+
  | id                                   | name        | subnets                                                  |
  +--------------------------------------+-------------+----------------------------------------------------------+
  | 666f8a6a-e3e3-4183-84b3-a43c92b050f5 | net_vlan_01 | 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad 101.1.1.0/24        |
  +--------------------------------------+-------------+----------------------------------------------------------+
  [root@localhost devstack]# neutron port-list 
  | e759c9df-db2d-477b-94e9-02d80844f7f9 |      | fa:16:3e:2d:e5:6e | {"subnet_id": "1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad", "ip_address": "101.1.1.2"}    

  2)Modified ‘ipam_driver = ‘internal’’ in the neutron.conf file,and restarted neutron-server service                        |
  [root@localhost devstack]# vi /etc/neutron/neutron.conf 

  # Neutron IPAM (IP address management) driver to use. If ipam_driver is not set
  # (default behavior), no IPAM driver is used. In order to use the reference
  # implementation of Neutron IPAM driver, use 'internal'. (string value)
  #ipam_driver = <None>
  ipam_driver = internal

  "/etc/neutron/neutron.conf" 1538L, 54626C written

  [root@localhost devstack]# neutron net-list
  +--------------------------------------+-------------+----------------------------------------------------------+
  | id                                   | name        | subnets                                                  |
  +--------------------------------------+-------------+----------------------------------------------------------+
  | 666f8a6a-e3e3-4183-84b3-a43c92b050f5 | net_vlan_01 | 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad 101.1.1.0/24        |
  +--------------------------------------+-------------+----------------------------------------------------------+
  [root@localhost devstack]# neutron port-list |grep 101
  | e759c9df-db2d-477b-94e9-02d80844f7f9 |      | fa:16:3e:2d:e5:6e | {"subnet_id": "1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad", "ip_address": "101.1.1.2"}  

  3)Deleting the network returns none,and dead., 
  the usage of neutron-server is above 80%,and neutron-server's log is repeating “ERROR neutron.ipam.drivers.neutrondb_ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3
   referenced to Neutron subnet 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad does not exist”             |
  [root@localhost devstack]# neutron net-delete net_vlan_01

  Last login: Thu Jun  2 22:38:22 2016 from 10.43.165.22
  [root@localhost ~]# top
  top - 22:46:35 up 22:54, 33 users,  load average: 0.98, 0.99, 0.80
  Tasks: 562 total,   4 running, 558 sleeping,   0 stopped,   0 zombie
  %Cpu(s):  6.2 us,  0.4 sy,  0.0 ni, 93.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
  KiB Mem : 13189404+total, 10021431+free,  6880860 used, 24798868 buff/cache
  KiB Swap:  4194300 total,  4194300 free,        0 used. 12443545+avail Mem 

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                 
  16279 stack     20   0  285740 100868   2152 R  85.4  0.1   2:02.06 neutron-server                                                          
   6529 mysql     20   0 6192992 352368  13120 S   4.3  0.3   4:58.96 mysqld                                                                  
  11472 root      20   0   78952   6656   2904 S   4.3  0.0   2:05.32 ssh                                                                     
   4351 stack     20   0  291656  85224   6568 S   3.0  0.1  24:39.79 cinder-schedule                                                         
  14363 stack     20   0  237336  61796   5412 R   2.6  0.0  29:18.80 neutron-openvsw                                                         
  20050 stack     20   0 2048392 150276  12168 S   2.6  0.1  35:49.49 nova-compute                                                            
   1131 stack     20   0  276828  92148   2452 S   2.3  0.1  25:45.10 nova-conductor                                                          
   1134 stack     20   0  278832  94184   2460 S   2.3  0.1  25:46.85 nova-conductor                                                          
   1135 stack     20   0  278288  93632   2460 S   2.3  0.1  25:47.77 nova-conductor                                                          
   1655 rabbitmq  20   0 4096148 212828   3136 S   2.3  0.2  45:15.30 beam.smp                                                                
  16280 stack     20   0  268268  83180   2056 S   2.3  0.1   0:03.72 neutron-server                                                          
  27445 stack     20   0  230736  55112   5420 S   2.3  0.0  24:20.15 neutron-dhcp-ag                                                         
   1130 stack     20   0  280308  95736   2468 S   2.0  0.1  25:47.58 nova-conductor                                                          
   1132 stack     20   0  280084  95476   2468 S   2.0  0.1  25:48.68 nova-conductor                                                          
   1133 stack     20   0  276848  92148   2452 S   2.0  0.1  25:43.78 nova-conductor                                                          
   5094 stack     20   0  288528  77764   2240 S   2.0  0.1  26:00.90 cinder-volume                                                           
  16281 stack     20   0  267496  82240   2056 S   2.0  0.1   0:03.67 neutron-server                                                          
  20207 stack     20   0  265884  84360   5628 S   2.0  0.1  23:32.15 nova-consoleaut                                                         
  22228 stack     20   0  266268  84764   5628 S   2.0  0.1  23:45.42 nova-scheduler                                                          
  24333 stack     20   0  231164  70956   2168 S   2.0  0.1  22:29.85 glance-api                                                              
  24335 stack     20   0  225508  65548   2488 S   2.0  0.0  21:41.44 glance-api                                                              
  24340 stack     20   0  231676  71900   2680 S   2.0  0.1  23:21.55 glance-api                                                              
  24343 stack     20   0  231164  70864   2168 S   2.0  0.1  22:29.88 glance-api                                                              
  25713 stack     20   0  355648 147884   4252 S   2.0  0.1  22:36.73 nova-api                                                                
  25718 stack     20   0  316728 129704   2168 S   2.0  0.1   0:35.72 nova-api                                                                
  27962 stack     20   0  230892  55400   5412 S   2.0  0.0  23:37.71 neutron-l3-agen                                                         
  28333 stack     20   0  227604  51948   5408 S   2.0  0.0  23:21.07 neutron-metadat                                                         
  11471 root      20   0   52748   2176   1492 S   1.7  0.0   0:47.99 scp                                                                     
   3835 stack     20   0  299164  92840   6568 S   1.3  0.1  12:56.21 cinder-api                                                              
  25581 stack     20   0  298928 115296   5616 S   1.3  0.1  12:49.65 nova-api                                                                
  32121 stack     20   0  252552  72948   5560 S   1.3  0.1  12:41.01 nova-conductor                                                          
   4875 stack     20   0  286764  80432   6564 S   1.0  0.1  12:46.46 cinder-volume                                                           
  24206 stack     20   0  224384  67540   5596 S   1.0  0.1  12:17.30 glance-api                                                              
     34 root      20   0       0      0      0 S   0.7  0.0   2:17.17 rcu_sched                                                               
  [root@localhost ~]# 

  2016-06-02 22:44:18.315 ERROR neutron.ipam.drivers.neutrondb_ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3
   referenced to Neutron subnet 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad does not exist
  2016-06-02 22:44:18.316 DEBUG neutron.db.ipam_pluggable_backend [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426
   on ip deallocation: <neutron.db.models_v2.IPAllocation[object at 6588b10] {port_id=u'e759c9df-db2d-477b-94e9-02d80844f7f9', ip_address=u'101
  dbd7-ae1e-4d7c-a767-ec3106cc62ad', network_id=u'666f8a6a-e3e3-4183-84b3-a43c92b050f5'}> from (pid=16279) _ipam_deallocate_ips /opt/stack/neut
  able_backend.py:73
  2016-06-02 22:44:18.330 DEBUG neutron.callbacks.manager [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Not
  fter_delete from (pid=16279) _notify_loop /opt/stack/neutron/neutron/callbacks/manager.py:138
  2016-06-02 22:44:18.331 DEBUG neutron.callbacks.manager [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Cal
  l3_db._notify_routers_callback from (pid=16279) _notify_loop /opt/stack/neutron/neutron/callbacks/manager.py:145
  2016-06-02 22:44:18.331 DEBUG neutron.callbacks.manager [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Cal
  l3_dvrscheduler_db._notify_port_delete from (pid=16279) _notify_loop /opt/stack/neutron/neutron/callbacks/manager.py:145
  2016-06-02 22:44:18.409 DEBUG neutron.db.l3_dvrscheduler_db [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa]
  DVR port e759c9df-db2d-477b-94e9-02d80844f7f9 on host localhost.localdomain from (pid=16279) get_dvr_routers_to_remove /opt/stack/neutron/neu
  b.py:162
  2016-06-02 22:44:18.428 DEBUG oslo_messaging._drivers.amqpdriver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de342
  c18736292407fa506c49404a98c88 FANOUT topic 'q-agent-notifier-port-delete' from (pid=16279) _send /usr/lib/python2.7/site-packages/oslo_messag
  y:439
  2016-06-02 22:44:18.435 DEBUG oslo_messaging._drivers.amqpdriver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de342
  72b12fefd43aab74b1acdee75acca FANOUT topic 'q-agent-notifier-security_group-update' from (pid=16279) _send /usr/lib/python2.7/site-packages/o
  qpdriver.py:439
  2016-06-02 22:44:18.437 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
  1e-4d7c-a767-ec3106cc62ad from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:959
  2016-06-02 22:44:18.505 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Po
  ] from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:984
  2016-06-02 22:44:18.508 DEBUG neutron.callbacks.manager [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Not
   before_delete from (pid=16279) _notify_loop /opt/stack/neutron/neutron/callbacks/manager.py:138
  2016-06-02 22:44:18.616 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
   (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1023
  2016-06-02 22:44:18.621 DEBUG neutron.ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Loading i
   (pid=16279) get_instance /opt/stack/neutron/neutron/ipam/driver.py:51
  2016-06-02 22:44:18.697 ERROR neutron.ipam.drivers.neutrondb_ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3
   referenced to Neutron subnet 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad does not exist
  2016-06-02 22:44:18.714 INFO neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Sub
  67-ec3106cc62ad was deleted concurrently
  2016-06-02 22:44:18.739 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Po
  om (pid=16279) delete_network /opt/stack/neutron/neutron/plugins/ml2/plugin.py:846
  2016-06-02 22:44:18.744 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Su
  eutron.db.models_v2.Subnet[object at 659a750] {tenant_id=u'69fa49e368d340679ab3d05de3426bfa', id=u'1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad', nam
  ork_id=u'666f8a6a-e3e3-4183-84b3-a43c92b050f5', segment_id=None, subnetpool_id=None, ip_version=4, cidr=u'101.1.1.0/24', gateway_ip=u'101.1.1
  v6_ra_mode=None, ipv6_address_mode=None, standard_attr_id=49}>] from (pid=16279) delete_network /opt/stack/neutron/neutron/plugins/ml2/plugin
  2016-06-02 22:44:18.745 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
  1e-4d7c-a767-ec3106cc62ad from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:959
  2016-06-02 22:44:18.822 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Po
  ] from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:984
  2016-06-02 22:44:18.825 DEBUG neutron.callbacks.manager [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Not
   before_delete from (pid=16279) _notify_loop /opt/stack/neutron/neutron/callbacks/manager.py:138
  2016-06-02 22:44:19.072 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
   (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1023
  2016-06-02 22:44:19.075 DEBUG neutron.ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Loading i
   (pid=16279) get_instance /opt/stack/neutron/neutron/ipam/driver.py:51
  2016-06-02 22:44:19.121 ERROR neutron.ipam.drivers.neutrondb_ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3
   referenced to Neutron subnet 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad does not exist
  2016-06-02 22:44:19.136 INFO neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Sub
  67-ec3106cc62ad was deleted concurrently
  2016-06-02 22:44:19.154 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Po
  om (pid=16279) delete_network /opt/stack/neutron/neutron/plugins/ml2/plugin.py:846
  2016-06-02 22:44:19.159 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Su
  eutron.db.models_v2.Subnet[object at 659a750] {tenant_id=u'69fa49e368d340679ab3d05de3426bfa', id=u'1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad', nam
  ork_id=u'666f8a6a-e3e3-4183-84b3-a43c92b050f5', segment_id=None, subnetpool_id=None, ip_version=4, cidr=u'101.1.1.0/24', gateway_ip=u'101.1.1
  v6_ra_mode=None, ipv6_address_mode=None, standard_attr_id=49}>] from (pid=16279) delete_network /opt/stack/neutron/neutron/plugins/ml2/plugin
  2016-06-02 22:44:19.160 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
  1e-4d7c-a767-ec3106cc62ad from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:959
  2016-06-02 22:44:19.233 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Po
  ] from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:984
  2016-06-02 22:44:19.237 DEBUG neutron.callbacks.manager [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Not
   before_delete from (pid=16279) _notify_loop /opt/stack/neutron/neutron/callbacks/manager.py:138
  2016-06-02 22:44:19.317 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
   (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1023
  2016-06-02 22:44:19.320 DEBUG neutron.ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Loading i
   (pid=16279) get_instance /opt/stack/neutron/neutron/ipam/driver.py:51
  2016-06-02 22:44:19.328 ERROR neutron.ipam.drivers.neutrondb_ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3
   referenced to Neutron subnet 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad does not exist
  2016-06-02 22:44:19.341 INFO neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Sub
  67-ec3106cc62ad was deleted concurrently
  2016-06-02 22:44:19.360 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Po
  om (pid=16279) delete_network /opt/stack/neutron/neutron/plugins/ml2/plugin.py:846
  2016-06-02 22:44:19.364 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Su
  eutron.db.models_v2.Subnet[object at 659a750] {tenant_id=u'69fa49e368d340679ab3d05de3426bfa', id=u'1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad', nam
  ork_id=u'666f8a6a-e3e3-4183-84b3-a43c92b050f5', segment_id=None, subnetpool_id=None, ip_version=4, cidr=u'101.1.1.0/24', gateway_ip=u'101.1.1
  v6_ra_mode=None, ipv6_address_mode=None, standard_attr_id=49}>] from (pid=16279) delete_network /opt/stack/neutron/neutron/plugins/ml2/plugin
  2016-06-02 22:44:19.365 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
  1e-4d7c-a767-ec3106cc62ad from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:959
  2016-06-02 22:44:19.438 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Po
  ] from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:984
  2016-06-02 22:44:19.441 DEBUG neutron.callbacks.manager [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Not
   before_delete from (pid=16279) _notify_loop /opt/stack/neutron/neutron/callbacks/manager.py:138
  2016-06-02 22:44:19.524 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
   (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1023
  2016-06-02 22:44:19.527 DEBUG neutron.ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Loading i
   (pid=16279) get_instance /opt/stack/neutron/neutron/ipam/driver.py:51
  2016-06-02 22:44:19.537 ERROR neutron.ipam.drivers.neutrondb_ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3
   referenced to Neutron subnet 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad does not exist
  2016-06-02 22:44:19.553 INFO neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Sub
  67-ec3106cc62ad was deleted concurrently
  2016-06-02 22:44:19.578 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Po
  om (pid=16279) delete_network /opt/stack/neutron/neutron/plugins/ml2/plugin.py:846
  2016-06-02 22:44:19.582 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Su
  eutron.db.models_v2.Subnet[object at 659a750] {tenant_id=u'69fa49e368d340679ab3d05de3426bfa', id=u'1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad', nam
  ork_id=u'666f8a6a-e3e3-4183-84b3-a43c92b050f5', segment_id=None, subnetpool_id=None, ip_version=4, cidr=u'101.1.1.0/24', gateway_ip=u'101.1.1
  v6_ra_mode=None, ipv6_address_mode=None, standard_attr_id=49}>] from (pid=16279) delete_network /opt/stack/neutron/neutron/plugins/ml2/plugin
  2016-06-02 22:44:19.584 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
  1e-4d7c-a767-ec3106cc62ad from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:959
  2016-06-02 22:44:19.651 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Po
  ] from (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:984
  2016-06-02 22:44:19.654 DEBUG neutron.callbacks.manager [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Not
   before_delete from (pid=16279) _notify_loop /opt/stack/neutron/neutron/callbacks/manager.py:138
  2016-06-02 22:44:19.716 DEBUG neutron.wsgi [-] (16278) accepted ('192.168.1.177', 60806) from (pid=16278) server /usr/lib/python2.7/site-pack
  2016-06-02 22:44:19.734 DEBUG neutron.plugins.ml2.plugin [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] De
   (pid=16279) delete_subnet /opt/stack/neutron/neutron/plugins/ml2/plugin.py:1023
  2016-06-02 22:44:19.736 DEBUG neutron.ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3d05de3426bfa] Loading i
   (pid=16279) get_instance /opt/stack/neutron/neutron/ipam/driver.py:51
  2016-06-02 22:44:19.744 ERROR neutron.ipam.drivers.neutrondb_ipam.driver [req-fd43f9e3-9423-41bc-8434-7787bab0f263 admin 69fa49e368d340679ab3
   referenced to Neutron subnet 1c60dbd7-ae1e-4d7c-a767-ec3106cc62ad does not exist

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1588593/+subscriptions


Follow ups