yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1261510] Re: Instance fails to spawn in tempest tests

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Matt Riedemann <mriedem@xxxxxxxxxx>
Date: Fri, 07 Feb 2014 14:33:36 -0000
Reply-to: Bug 1261510 <1261510@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Looking through the scheduler logs for instance
682be6a5-4b31-43a4-bb11-c884eecebc9b I'm not seeing any errors, the
scheduler is trying to place the instance.

>From the neutron server logs it looks like it's trying to setup ports
for the instance:

2013-12-16 18:54:31.362 7254 DEBUG keystoneclient.middleware.auth_token [-] Received request from user: 705f4d7e328b4e79916ec7f8e6c26d6c with project_id : 1967063381b84664b08a693b11d2e95f and roles: _member_,admin  _build_user_headers /opt/stack/new/python-keystoneclient/keystoneclient/middleware/auth_token.py:932
2013-12-16 18:54:31.363 7254 DEBUG routes.middleware [-] No route matched for GET /ports.json __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:97
2013-12-16 18:54:31.363 7254 DEBUG routes.middleware [-] Matched GET /ports.json __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:100
2013-12-16 18:54:31.363 7254 DEBUG routes.middleware [-] Route path: '/ports{.format}', defaults: {'action': u'index', 'controller': <wsgify at 79741008 wrapping <function resource at 0x4c08758>>} __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:102
2013-12-16 18:54:31.363 7254 DEBUG routes.middleware [-] Match dict: {'action': u'index', 'controller': <wsgify at 79741008 wrapping <function resource at 0x4c08758>>, 'format': u'json'} __call__ /usr/lib/python2.7/dist-packages/routes/middleware.py:103
2013-12-16 18:54:31.374 7254 DEBUG neutron.api.v2.base [req-c9f4e8cb-97e4-499b-80af-98189199b070 705f4d7e328b4e79916ec7f8e6c26d6c 1967063381b84664b08a693b11d2e95f] Policy rule:get_port:allowed_address_pairs not found. Assuming no authZ check is defined for allowed_address_pairs _is_visible /opt/stack/new/neutron/neutron/api/v2/base.py:149
2013-12-16 18:54:31.375 7254 DEBUG neutron.api.v2.base [req-c9f4e8cb-97e4-499b-80af-98189199b070 705f4d7e328b4e79916ec7f8e6c26d6c 1967063381b84664b08a693b11d2e95f] Policy rule:get_port:fixed_ips not found. Assuming no authZ check is defined for fixed_ips _is_visible /opt/stack/new/neutron/neutron/api/v2/base.py:149
2013-12-16 18:54:31.375 7254 DEBUG neutron.api.v2.base [req-c9f4e8cb-97e4-499b-80af-98189199b070 705f4d7e328b4e79916ec7f8e6c26d6c 1967063381b84664b08a693b11d2e95f] Policy rule:get_port:mac_address not found. Assuming no authZ check is defined for mac_address _is_visible /opt/stack/new/neutron/neutron/api/v2/base.py:149
2013-12-16 18:54:31.375 7254 INFO neutron.wsgi [req-c9f4e8cb-97e4-499b-80af-98189199b070 705f4d7e328b4e79916ec7f8e6c26d6c 1967063381b84664b08a693b11d2e95f] 127.0.0.1 - - [16/Dec/2013 18:54:31] "GET /v2.0/ports.json?tenant_id=cbe2baa37d104c71bd8db69f229a622e&device_id=682be6a5-4b31-43a4-bb11-c884eecebc9b HTTP/1.1" 200 826 0.014504

I'm seeing a lot of ovs errors in the neutron agent log but they don't
match the timestamps of the timeout:

Stderr: 'ovs-vsctl: no row "tapda8a8a73-be" in table Port\n'
2013-12-16 18:49:20.066 8305 ERROR neutron.agent.linux.ovs_lib [-] Unable to execute ['ovs-vsctl', '--timeout=2', 'set', 'Port', 'tapec1e8f0c-97', 'tag=53']. Exception: 
Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ovs-vsctl', '--timeout=2', 'set', 'Port', 'tapec1e8f0c-97', 'tag=53']
Exit code: 1
Stdout: ''
Stderr: 'ovs-vsctl: no row "tapec1e8f0c-97" in table Port\n'


I see one hit for the instance uuid in the neutron dhcp agent log:

http://logs.openstack.org/22/62322/2/check/check-tempest-dsvm-neutron-
isolated/cce7146/logs/screen-q-dhcp.txt.gz#_2013-12-16_18_24_17_427

That's when it's trying to setup the port on the network since it
doesn't yet exist:

DHCP port dhcp93c980cc-1156-52ae-8491-deb1feab8172-79336d59-d7ad-4319
-9c9e-60cdabd0e7d7 on network 79336d59-d7ad-4319-9c9e-60cdabd0e7d7 does
not yet exist. setup_dhcp_port
/opt/stack/new/neutron/neutron/agent/linux/dhcp.py:699

And then I'm seeing command failures in the dhcp agent log so it looks
like it's not able to setup the port:

2013-12-16 18:24:17.713 8327 DEBUG neutron.agent.linux.utils [req-65638ce1-62d1-46b0-bba5-307fe4b0e67b 705f4d7e328b4e79916ec7f8e6c26d6c 1967063381b84664b08a693b11d2e95f] 
Command: ['sudo', '/usr/local/bin/neutron-rootwrap', '/etc/neutron/rootwrap.conf', 'ip', 'netns', 'exec', 'qdhcp-79336d59-d7ad-4319-9c9e-60cdabd0e7d7', 'ip', '-o', 'link', 'show', 'tap224569f2-93']
Exit code: 1
Stdout: ''
Stderr: 'Cannot open network namespace: No such file or directory\n' execute /opt/stack/new/neutron/neutron/agent/linux/utils.py:73
2013-12-16 18:24:17.715 8327 DEBUG neutron.agent.linux.utils [req-65638ce1-62d1-46b0-bba5-307fe4b0e67b 705f4d7e328b4e79916ec7f8e6c26d6c 1967063381b84664b08a693b11d2e95f] Running command: ['ip', '-o', 'link', 'show', 'br-int'] create_process /opt/stack/new/neutron/neutron/agent/linux/utils.py:47
2013-12-16 18:24:17.732 8327 DEBUG neutron.agent.linux.utils [req-65638ce1-62d1-46b0-bba5-307fe4b0e67b 705f4d7e328b4e79916ec7f8e6c26d6c 1967063381b84664b08a693b11d2e95f] 
Command: ['ip', '-o', 'link', 'show', 'br-int']
Exit code: 0
Stdout: '7: br-int: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN \\    link/ether e2:56:48:0c:56:46 brd ff:ff:ff:ff:ff:ff\n'
Stderr: '' execute /opt/stack/new/neutron/neutron/agent/linux/utils.py:73

The last times you see the instance showing up in the nova compute logs
is when it's updating the network info cache:

2013-12-16 18:54:31.359 8385 DEBUG nova.network.neutronv2.api [-] get_instance_nw_info() for server-tempest-373999100 _get_instance_nw_info /opt/stack/new/nova/nova/network/neutronv2/api.py:456
2013-12-16 18:54:31.360 8385 DEBUG neutronclient.client [-] 
REQ: curl -i http://127.0.0.1:9696/v2.0/ports.json?tenant_id=cbe2baa37d104c71bd8db69f229a622e&device_id=682be6a5-4b31-43a4-bb11-c884eecebc9b -X GET -H "X-Auth-Token: MIITUAYJKoZIhvcNAQcCoIITQTCCEz0CAQExCTAHBgUrDgMCGjCCEaYGCSqGSIb3DQEHAaCCEZcEghGTeyJhY2Nlc3MiOiB7InRva2VuIjogeyJpc3N1ZWRfYXQiOiAiMjAxMy0xMi0xNlQxODoyMzoyOS4zNDgxMzciLCAiZXhwaXJlcyI6ICIyMDEzLTEyLTE3VDE4OjIzOjI5WiIsICJpZCI6ICJwbGFjZWhvbGRlciIsICJ0ZW5hbnQiOiB7ImRlc2NyaXB0aW9uIjogbnVsbCwgImVuYWJsZWQiOiB0cnVlLCAiaWQiOiAiMTk2NzA2MzM4MWI4NDY2NGIwOGE2OTNiMTFkMmU5NWYiLCAibmFtZSI6ICJzZXJ2aWNlIn19LCAic2VydmljZUNhdGFsb2ciOiBbeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTI3LjAuMC4xOjg3NzQvdjIvMTk2NzA2MzM4MWI4NDY2NGIwOGE2OTNiMTFkMmU5NWYiLCAicmVnaW9uIjogIlJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTI3LjAuMC4xOjg3NzQvdjIvMTk2NzA2MzM4MWI4NDY2NGIwOGE2OTNiMTFkMmU5NWYiLCAiaWQiOiAiYzMzNjQ0OGYzMzRlNDQ0ZGI1NTI1MTJmNmI0MWYxN2MiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMjcuMC4wLjE6ODc3NC92Mi8xOTY3MDYzMzgxYjg0NjY0YjA4YTY5M2IxMWQyZTk1ZiJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJjb21wdXRlIiwgIm5hbWUiOiAibm92YSJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xMjcuMC4wLjE6OTY5Ni8iLCAicmVnaW9uIjogIlJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTI3LjAuMC4xOjk2OTYvIiwgImlkIjogIjBhYmMyNjFjN2NhYzRhMTM4NzNlMTUwODY4MzU4NmZjIiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTI3LjAuMC4xOjk2OTYvIn1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogIm5ldHdvcmsiLCAibmFtZSI6ICJuZXV0cm9uIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEyNy4wLjAuMTo4Nzc2L3YyLzE5NjcwNjMzODFiODQ2NjRiMDhhNjkzYjExZDJlOTVmIiwgInJlZ2lvbiI6ICJSZWdpb25PbmUiLCAiaW50ZXJuYWxVUkwiOiAiaHR0cDovLzEyNy4wLjAuMTo4Nzc2L3YyLzE5NjcwNjMzODFiODQ2NjRiMDhhNjkzYjExZDJlOTVmIiwgImlkIjogIjc5NjMyYTdhZTI4YjQ2NzZiNDBiM2VhMGEwNDhkNTAzIiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTI3LjAuMC4xOjg3NzYvdjIvMTk2NzA2MzM4MWI4NDY2NGIwOGE2OTNiMTFkMmU5NWYifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAidm9sdW1ldjIiLCAibmFtZSI6ICJjaW5kZXIifSwgeyJlbmRwb2ludHMiOiBbeyJhZG1pblVSTCI6ICJodHRwOi8vMTI3LjAuMC4xOjg3NzQvdjMiLCAicmVnaW9uIjogIlJlZ2lvbk9uZSIsICJpbnRlcm5hbFVSTCI6ICJodHRwOi8vMTI3LjAuMC4xOjg3NzQvdjMiLCAiaWQiOiAiNjQ1NjcwOWY5YzEzNDk1Njg5NTBjOGNlMDFmNWRiN2UiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMjcuMC4wLjE6ODc3NC92MyJ9XSwgImVuZHBvaW50c19saW5rcyI6IFtdLCAidHlwZSI6ICJjb21wdXRldjMiLCAibmFtZSI6ICJub3ZhIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEyNy4wLjAuMTozMzMzIiwgInJlZ2lvbiI6ICJSZWdpb25PbmUiLCAiaW50ZXJuYWxVUkwiOiAiaHR0cDovLzEyNy4wLjAuMTozMzMzIiwgImlkIjogIjg1OGM1ZjlhZTBkNDQ2Zjk5OTU5MjYzZmI4YThlYTA3IiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTI3LjAuMC4xOjMzMzMifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiczMiLCAibmFtZSI6ICJzMyJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xMjcuMC4wLjE6OTI5MiIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMjcuMC4wLjE6OTI5MiIsICJpZCI6ICIwMTVkODE3MjNlMDY0YzVmYTk2MWY2MTc3YTg4ZDM4MSIsICJwdWJsaWNVUkwiOiAiaHR0cDovLzEyNy4wLjAuMTo5MjkyIn1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogImltYWdlIiwgIm5hbWUiOiAiZ2xhbmNlIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEyNy4wLjAuMTo4Nzc3IiwgInJlZ2lvbiI6ICJSZWdpb25PbmUiLCAiaW50ZXJuYWxVUkwiOiAiaHR0cDovLzEyNy4wLjAuMTo4Nzc3IiwgImlkIjogIjg3M2Q3MjRmODZjNjRmNjdhYjVjOTdiYWM3ZjExNmNmIiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTI3LjAuMC4xOjg3NzcifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAibWV0ZXJpbmciLCAibmFtZSI6ICJjZWlsb21ldGVyIn0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEyNy4wLjAuMTo4MDAwL3YxIiwgInJlZ2lvbiI6ICJSZWdpb25PbmUiLCAiaW50ZXJuYWxVUkwiOiAiaHR0cDovLzEyNy4wLjAuMTo4MDAwL3YxIiwgImlkIjogImM3ZWVmNjI2Njk4ODQ3MDJiNmQ3ZGVkOGZhOTQ3N2JjIiwgInB1YmxpY1VSTCI6ICJodHRwOi8vMTI3LjAuMC4xOjgwMDAvdjEifV0sICJlbmRwb2ludHNfbGlua3MiOiBbXSwgInR5cGUiOiAiY2xvdWRmb3JtYXRpb24iLCAibmFtZSI6ICJoZWF0LWNmbiJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xMjcuMC4wLjE6ODc3Ni92MS8xOTY3MDYzMzgxYjg0NjY0YjA4YTY5M2IxMWQyZTk1ZiIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMjcuMC4wLjE6ODc3Ni92MS8xOTY3MDYzMzgxYjg0NjY0YjA4YTY5M2IxMWQyZTk1ZiIsICJpZCI6ICIxNjNlNDI1MGZhZTc0YmY0OGE3ZjExNTQ2NmMzZTRlYSIsICJwdWJsaWNVUkwiOiAiaHR0cDovLzEyNy4wLjAuMTo4Nzc2L3YxLzE5NjcwNjMzODFiODQ2NjRiMDhhNjkzYjExZDJlOTVmIn1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogInZvbHVtZSIsICJuYW1lIjogImNpbmRlciJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xMjcuMC4wLjE6ODc3My9zZXJ2aWNlcy9BZG1pbiIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMjcuMC4wLjE6ODc3My9zZXJ2aWNlcy9DbG91ZCIsICJpZCI6ICIyZjhhMTFkMWVkMzY0YTY5OTVkMjczZDZlMTEyMTgzOSIsICJwdWJsaWNVUkwiOiAiaHR0cDovLzEyNy4wLjAuMTo4NzczL3NlcnZpY2VzL0Nsb3VkIn1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogImVjMiIsICJuYW1lIjogImVjMiJ9LCB7ImVuZHBvaW50cyI6IFt7ImFkbWluVVJMIjogImh0dHA6Ly8xMjcuMC4wLjE6ODAwNC92MS8xOTY3MDYzMzgxYjg0NjY0YjA4YTY5M2IxMWQyZTk1ZiIsICJyZWdpb24iOiAiUmVnaW9uT25lIiwgImludGVybmFsVVJMIjogImh0dHA6Ly8xMjcuMC4wLjE6ODAwNC92MS8xOTY3MDYzMzgxYjg0NjY0YjA4YTY5M2IxMWQyZTk1ZiIsICJpZCI6ICI1MWUwMTU1NzRkNGY0NjNiYmYxZmI0ZjUzNWIwYTQ5OSIsICJwdWJsaWNVUkwiOiAiaHR0cDovLzEyNy4wLjAuMTo4MDA0L3YxLzE5NjcwNjMzODFiODQ2NjRiMDhhNjkzYjExZDJlOTVmIn1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogIm9yY2hlc3RyYXRpb24iLCAibmFtZSI6ICJoZWF0In0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEyNy4wLjAuMTo4MDgwIiwgInJlZ2lvbiI6ICJSZWdpb25PbmUiLCAiaW50ZXJuYWxVUkwiOiAiaHR0cDovLzEyNy4wLjAuMTo4MDgwL3YxL0FVVEhfMTk2NzA2MzM4MWI4NDY2NGIwOGE2OTNiMTFkMmU5NWYiLCAiaWQiOiAiMDU4NzE2NGQyNDk5NDMyNzk2OWJiZWIzMjdlNjQxYzkiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMjcuMC4wLjE6ODA4MC92MS9BVVRIXzE5NjcwNjMzODFiODQ2NjRiMDhhNjkzYjExZDJlOTVmIn1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogIm9iamVjdC1zdG9yZSIsICJuYW1lIjogInN3aWZ0In0sIHsiZW5kcG9pbnRzIjogW3siYWRtaW5VUkwiOiAiaHR0cDovLzEyNy4wLjAuMTozNTM1Ny92Mi4wIiwgInJlZ2lvbiI6ICJSZWdpb25PbmUiLCAiaW50ZXJuYWxVUkwiOiAiaHR0cDovLzEyNy4wLjAuMTo1MDAwL3YyLjAiLCAiaWQiOiAiNWFmOTNjYTA1MjNmNDljMDgxZTczOTc4Y2E1OGJiYWIiLCAicHVibGljVVJMIjogImh0dHA6Ly8xMjcuMC4wLjE6NTAwMC92Mi4wIn1dLCAiZW5kcG9pbnRzX2xpbmtzIjogW10sICJ0eXBlIjogImlkZW50aXR5IiwgIm5hbWUiOiAia2V5c3RvbmUifV0sICJ1c2VyIjogeyJ1c2VybmFtZSI6ICJuZXV0cm9uIiwgInJvbGVzX2xpbmtzIjogW10sICJpZCI6ICI3MDVmNGQ3ZTMyOGI0ZTc5OTE2ZWM3ZjhlNmMyNmQ2YyIsICJyb2xlcyI6IFt7Im5hbWUiOiAiX21lbWJlcl8ifSwgeyJuYW1lIjogImFkbWluIn1dLCAibmFtZSI6ICJuZXV0cm9uIn0sICJtZXRhZGF0YSI6IHsiaXNfYWRtaW4iOiAwLCAicm9sZXMiOiBbIjlmZTJmZjllZTQzODRiMTg5NGE5MDg3OGQzZTkyYmFiIiwgImVhM2VhMWQwN2IzOTRiOGFhY2Y3MzU0NDllZDNiMDY0Il19fX0xggGBMIIBfQIBATBcMFcxCzAJBgNVBAYTAlVTMQ4wDAYDVQQIDAVVbnNldDEOMAwGA1UEBwwFVW5zZXQxDjAMBgNVBAoMBVVuc2V0MRgwFgYDVQQDDA93d3cuZXhhbXBsZS5jb20CAQEwBwYFKw4DAhowDQYJKoZIhvcNAQEBBQAEggEAA0g+dwBhmox7dabOIyveWw-G-GE0JI1FBPfx98e1A0WYc5O3Izm3tYscwqT2PODXdubKKIFiIkzgDJobi8-ycUTD6Sgv2WbpBuYR+x74NfWDGilBT41icfeoohLYS4kzeS9xpwzCg5qH1+1PsmUK9WZApaFkecJFSDWTkMkcAoUthLbYgaTebYjSP07XTgsTfrgVoQOq9pF312w0XbR8aMrzeptVOcCbHpgatfwkWNQu5+Y9dq79qaRwLT8XB5HjES2nc97eXJUaby4WEzW0rwRsf6EBmD8aJGN9SKmVXdPIVmfXFLe9W8qibdmbWVkgUSboGAR9i2h9Y89z-Yf+Ow==" -H "Content-Type: application/json" -H "Accept: application/json" -H "User-Agent: python-neutronclient"
 http_log_req /opt/stack/new/python-neutronclient/neutronclient/common/utils.py:176
2013-12-16 18:54:31.376 8385 DEBUG neutronclient.client [-] RESP:{'date': 'Mon, 16 Dec 2013 18:54:31 GMT', 'status': '200', 'content-length': '702', 'content-type': 'application/json; charset=UTF-8', 'content-location': 'http://127.0.0.1:9696/v2.0/ports.json?tenant_id=cbe2baa37d104c71bd8db69f229a622e&device_id=682be6a5-4b31-43a4-bb11-c884eecebc9b'} {"ports": [{"status": "DOWN", "binding:host_id": "devstack-precise-check-rax-iad-871005.slave.openstack.org", "name": "", "allowed_address_pairs": [], "admin_state_up": true, "network_id": "79336d59-d7ad-4319-9c9e-60cdabd0e7d7", "tenant_id": "cbe2baa37d104c71bd8db69f229a622e", "extra_dhcp_opts": [], "binding:vif_type": "ovs", "device_owner": "compute:None", "binding:capabilities": {"port_filter": true}, "mac_address": "fa:16:3e:93:38:7a", "fixed_ips": [{"subnet_id": "1e10be61-657b-4ac3-8e4e-e7231ed0d452", "ip_address": "10.100.0.2"}], "id": "d2e886f4-059f-4791-88d4-e50963cc45f2", "security_groups": ["a0d91903-6c9d-413c-99bc-93216d4b77d2"], "device_id": "682be6a5-4b31-43a4-bb11-c884eecebc9b"}]}
 http_log_resp /opt/stack/new/python-neutronclient/neutronclient/common/utils.py:182
2013-12-16 18:54:31.377 8385 DEBUG nova.objects.instance [-] Lazy-loading `info_cache' on Instance uuid 682be6a5-4b31-43a4-bb11-c884eecebc9b obj_load_attr /opt/stack/new/nova/nova/objects/instance.py:491

So it looks to me like this is a timeout due to the port never getting
created for the instance.  Probably need more neutron people to look at
this.

** Tags added: gate network

** Also affects: neutron
   Importance: Undecided
       Status: New

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1261510

Title:
  Instance fails to spawn in tempest tests

Status in OpenStack Neutron (virtual network service):
  New
Status in OpenStack Compute (Nova):
  New

Bug description:
  This happened only 3 times in the past 12 hours, so nothing to worry
  about so far.

  Logstash query for the exact failure in [1] available at [2]
  I am also seeing more "Timeout waiting for thing" errors (not the same condition as bug 1254890, which affects the large_ops job and is due to nova/neutron chatty interface). Logstash query for this at [3] (13 hits in past 12 hours). I think they might have the same root cause.

  
  [1] http://logs.openstack.org/22/62322/2/check/check-tempest-dsvm-neutron-isolated/cce7146
  [2] http://logstash.openstack.org/#eyJzZWFyY2giOiJcImZhaWxlZCB0byByZWFjaCBBQ1RJVkUgc3RhdHVzXCIgQU5EICBcIkN1cnJlbnQgc3RhdHVzOiBCVUlMRFwiIEFORCBcIkN1cnJlbnQgdGFzayBzdGF0ZTogc3Bhd25pbmdcIiIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiNDMyMDAiLCJncmFwaG1vZGUiOiJjb3VudCIsInRpbWUiOnsidXNlcl9pbnRlcnZhbCI6MH0sInN0YW1wIjoxMzg3MjIzNzQ0Mjk2fQ==
  [3] http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiRGV0YWlsczogVGltZWQgb3V0IHdhaXRpbmcgZm9yIHRoaW5nXCIiLCJmaWVsZHMiOltdLCJvZmZzZXQiOjAsInRpbWVmcmFtZSI6IjQzMjAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTM4NzIyMzg2Mjg1MH0=

To manage notifications about this bug go to:
https://bugs.launchpad.net/neutron/+bug/1261510/+subscriptions
References

[Bug 1261510] [NEW] Instance fails to spawn in tempest tests
From: Salvatore Orlando, 2013-12-16