yahoo-eng-team team mailing list archive

Thread
Date
[Bug 1323578] Re: randmly failing rebuild on ssh-key errors on a setup which is configured for non-migration resize

To: yahoo-eng-team@xxxxxxxxxxxxxxxxxxx
From: Daniel Berrange <1323578@xxxxxxxxxxxxxxxxxx>
Date: Tue, 08 Jul 2014 15:36:28 -0000
Reply-to: Bug 1323578 <1323578@xxxxxxxxxxxxxxxxxx>
Sender: bounces@xxxxxxxxxxxxx
Proposing a blueprint/nova-specs is only appropriate if the submitter
actually intends to provide a solution. We're not wanting nova-specs for
arbitrary user reported wish-list items.

For user wish list items like this, they should be status:
confirmed/triaged, importance: wishlist


** Changed in: nova
       Status: Invalid => Confirmed

-- 
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to OpenStack Compute (nova).
https://bugs.launchpad.net/bugs/1323578

Title:
  randmly failing rebuild on ssh-key errors on a setup which is
  configured for non-migration resize

Status in OpenStack Compute (Nova):
  Confirmed

Bug description:
  I created a setup with 3 computes but configured nova.conf with the following on each of the computes: 
  allow_resize_to_same_host=True
  scheduler_default_filters=AllHostsFilter 

  when I try to resize I failed constantly on ssh-key error so I added
  this on my hosts:

  [root@orange-vdsf ~(keystone_admin)]# cat /var/lib/nova/.ssh/config
  Host *
  StrictHostKeyChecking no
  #UserKnownHostsFile=/dev/null 

  after adding the config 
  it seems that sometimes we are trying to migrate an instance - which still fails migration and therefor we fail the resize and other time we resize locally and that succeeds. 
  I could not understand when we select to migrate and when we select to resize locally but it might be a scheduler issue. 

  [root@orange-vdsf ~(keystone_admin)]# egrep a400d2f2-80ce-4ebf-9847-8885b7742fec /var/log/nova/*
  /var/log/nova/nova-api.log:2014-05-20 13:45:02.892 22298 INFO nova.osapi_compute.wsgi.server [req-ba38a1d5-80e4-4051-a89b-7daeae20edbf c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] 10.35.160.73 "GET /v2/4ad766166539403189f2caca1ba306aa/servers/a400d2f2-80ce-4ebf-9847-8885b7742fec HTTP/1.1" status: 200 len: 1793 time: 0.1236081
  /var/log/nova/nova-api.log:2014-05-20 13:45:06.822 22295 INFO nova.osapi_compute.wsgi.server [req-ce846da7-5757-47da-99b2-cdc2d1dd15b7 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] 10.35.160.73 "GET /v2/4ad766166539403189f2caca1ba306aa/servers/a400d2f2-80ce-4ebf-9847-8885b7742fec HTTP/1.1" status: 200 len: 1923 time: 0.1979871
  /var/log/nova/nova-api.log:2014-05-20 13:45:09.193 22295 INFO nova.osapi_compute.wsgi.server [req-4b03f451-bfd1-49c2-ae41-9d2576f10a66 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] 10.35.160.73 "GET /v2/4ad766166539403189f2caca1ba306aa/servers/a400d2f2-80ce-4ebf-9847-8885b7742fec HTTP/1.1" status: 200 len: 1923 time: 0.1024799
  /var/log/nova/nova-api.log:2014-05-20 13:45:11.735 22295 INFO nova.osapi_compute.wsgi.server [req-f1db26ba-a097-4377-9a78-2e3b5b5f4a8a c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] 10.35.160.73 "GET /v2/4ad766166539403189f2caca1ba306aa/servers/a400d2f2-80ce-4ebf-9847-8885b7742fec HTTP/1.1" status: 200 len: 1923 time: 0.2173102
  /var/log/nova/nova-api.log:2014-05-20 13:45:14.081 22295 INFO nova.osapi_compute.wsgi.server [req-547bcbe4-974e-47d3-89a4-996a67a6be41 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] 10.35.160.73 "GET /v2/4ad766166539403189f2caca1ba306aa/servers/a400d2f2-80ce-4ebf-9847-8885b7742fec HTTP/1.1" status: 200 len: 1923 time: 0.1151090
  /var/log/nova/nova-api.log:2014-05-20 13:45:16.632 22295 INFO nova.osapi_compute.wsgi.server [req-2efc51c8-2cd1-4aa0-a7b6-c75e275f7657 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] 10.35.160.73 "GET /v2/4ad766166539403189f2caca1ba306aa/servers/a400d2f2-80ce-4ebf-9847-8885b7742fec HTTP/1.1" status: 200 len: 1940 time: 0.1693149
  /var/log/nova/nova-api.log:2014-05-20 13:46:15.071 22295 INFO nova.osapi_compute.wsgi.server [req-f2053fdc-be29-4c3d-bca0-647080997ea5 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] 10.35.160.73 "GET /v2/4ad766166539403189f2caca1ba306aa/servers/a400d2f2-80ce-4ebf-9847-8885b7742fec HTTP/1.1" status: 200 len: 1940 time: 0.0634232
  /var/log/nova/nova-api.log:2014-05-20 13:46:18.156 22295 INFO nova.osapi_compute.wsgi.server [req-4f8e2c79-c885-4805-8af2-72cfe6f41bee c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] 10.35.160.73 "GET /v2/4ad766166539403189f2caca1ba306aa/servers/a400d2f2-80ce-4ebf-9847-8885b7742fec HTTP/1.1" status: 200 len: 1940 time: 0.0809522
  /var/log/nova/nova-api.log:2014-05-20 13:46:18.696 22295 INFO nova.osapi_compute.wsgi.server [req-b51803d7-d892-47cd-8402-d9730a1ecfb5 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] 10.35.160.73 "POST /v2/4ad766166539403189f2caca1ba306aa/servers/a400d2f2-80ce-4ebf-9847-8885b7742fec/action HTTP/1.1" status: 202 len: 185 time: 0.5187571
  /var/log/nova/nova-api.log:2014-05-20 13:46:20.989 22295 INFO nova.osapi_compute.wsgi.server [req-f4a6b905-b66d-4373-9103-c97a99e96625 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] 10.35.160.73 "GET /v2/4ad766166539403189f2caca1ba306aa/servers/a400d2f2-80ce-4ebf-9847-8885b7742fec HTTP/1.1" status: 200 len: 2023 time: 0.0786262
  /var/log/nova/nova-api.log:2014-05-20 13:46:23.928 22295 INFO nova.osapi_compute.wsgi.server [req-26e28464-32c7-4119-b8ad-146ff9521c1c c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] 10.35.160.73 "GET /v2/4ad766166539403189f2caca1ba306aa/servers/a400d2f2-80ce-4ebf-9847-8885b7742fec HTTP/1.1" status: 200 len: 2018 time: 0.0815430
  /var/log/nova/nova-api.log:2014-05-20 13:46:29.433 22295 INFO nova.osapi_compute.wsgi.server [req-2c7dc057-a221-41f4-a53f-cde62a327077 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] 10.35.160.73 "GET /v2/4ad766166539403189f2caca1ba306aa/servers/a400d2f2-80ce-4ebf-9847-8885b7742fec HTTP/1.1" status: 200 len: 2018 time: 0.0772440
  /var/log/nova/nova-api.log:2014-05-20 13:46:37.515 22295 INFO nova.osapi_compute.wsgi.server [req-b8595e2e-3c66-457f-a333-9e7100af6a6a c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] 10.35.160.73 "GET /v2/4ad766166539403189f2caca1ba306aa/servers/a400d2f2-80ce-4ebf-9847-8885b7742fec HTTP/1.1" status: 200 len: 2018 time: 0.0671909
  /var/log/nova/nova-api.log:2014-05-20 13:46:48.033 22295 INFO nova.osapi_compute.wsgi.server [req-d9d546fe-de91-419c-9c5d-ebb971a03f67 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] 10.35.160.73 "GET /v2/4ad766166539403189f2caca1ba306aa/servers/a400d2f2-80ce-4ebf-9847-8885b7742fec HTTP/1.1" status: 200 len: 2018 time: 0.0703950
  /var/log/nova/nova-api.log:2014-05-20 13:47:01.065 22295 INFO nova.osapi_compute.wsgi.server [req-74cf9d33-42d1-4ba8-b16e-b2bba8284fd0 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] 10.35.160.73 "GET /v2/4ad766166539403189f2caca1ba306aa/servers/a400d2f2-80ce-4ebf-9847-8885b7742fec HTTP/1.1" status: 200 len: 2018 time: 0.0708179
  /var/log/nova/nova-api.log:2014-05-20 13:47:13.067 22295 INFO nova.osapi_compute.wsgi.server [req-31e66eb8-7672-44e6-8000-5d0414353032 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] 10.35.160.73 "POST /v2/4ad766166539403189f2caca1ba306aa/servers/a400d2f2-80ce-4ebf-9847-8885b7742fec/action HTTP/1.1" status: 204 len: 147 time: 0.0920320
  /var/log/nova/nova-api.log:2014-05-20 13:47:14.392 22295 INFO nova.osapi_compute.wsgi.server [req-f9eee129-f879-44cf-8766-b3802661e5ad c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] 10.35.160.73 "GET /v2/4ad766166539403189f2caca1ba306aa/servers/a400d2f2-80ce-4ebf-9847-8885b7742fec HTTP/1.1" status: 200 len: 2010 time: 0.0635741
  /var/log/nova/nova-compute.log:2014-05-20 13:45:01.734 22235 AUDIT nova.compute.manager [req-9ae6b6e0-2e77-48ea-8234-23f4691bfd5a c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Starting instance...
  /var/log/nova/nova-compute.log:2014-05-20 13:45:02.340 22235 AUDIT nova.compute.claims [req-9ae6b6e0-2e77-48ea-8234-23f4691bfd5a c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Attempting claim: memory 512 MB, disk 1 GB, VCPUs 1
  /var/log/nova/nova-compute.log:2014-05-20 13:45:02.340 22235 AUDIT nova.compute.claims [req-9ae6b6e0-2e77-48ea-8234-23f4691bfd5a c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Total memory: 15888 MB, used: 2048.00 MB
  /var/log/nova/nova-compute.log:2014-05-20 13:45:02.341 22235 AUDIT nova.compute.claims [req-9ae6b6e0-2e77-48ea-8234-23f4691bfd5a c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] memory limit not specified, defaulting to unlimited
  /var/log/nova/nova-compute.log:2014-05-20 13:45:02.341 22235 AUDIT nova.compute.claims [req-9ae6b6e0-2e77-48ea-8234-23f4691bfd5a c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Total disk: 59 GB, used: 3.00 GB
  /var/log/nova/nova-compute.log:2014-05-20 13:45:02.341 22235 AUDIT nova.compute.claims [req-9ae6b6e0-2e77-48ea-8234-23f4691bfd5a c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] disk limit not specified, defaulting to unlimited
  /var/log/nova/nova-compute.log:2014-05-20 13:45:02.341 22235 AUDIT nova.compute.claims [req-9ae6b6e0-2e77-48ea-8234-23f4691bfd5a c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Total CPUs: 4 VCPUs, used: 3.00 VCPUs
  /var/log/nova/nova-compute.log:2014-05-20 13:45:02.341 22235 AUDIT nova.compute.claims [req-9ae6b6e0-2e77-48ea-8234-23f4691bfd5a c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] CPUs limit not specified, defaulting to unlimited
  /var/log/nova/nova-compute.log:2014-05-20 13:45:02.342 22235 AUDIT nova.compute.claims [req-9ae6b6e0-2e77-48ea-8234-23f4691bfd5a c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Claim successful
  /var/log/nova/nova-compute.log:2014-05-20 13:45:05.120 22235 INFO nova.virt.libvirt.driver [req-9ae6b6e0-2e77-48ea-8234-23f4691bfd5a c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Creating image
  /var/log/nova/nova-compute.log:2014-05-20 13:45:10.942 22235 INFO nova.compute.manager [-] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] During sync_power_state the instance has a pending task. Skip.
  /var/log/nova/nova-compute.log:2014-05-20 13:45:11.030 22235 INFO nova.virt.libvirt.firewall [req-9ae6b6e0-2e77-48ea-8234-23f4691bfd5a c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Called setup_basic_filtering in nwfilter
  /var/log/nova/nova-compute.log:2014-05-20 13:45:11.031 22235 INFO nova.virt.libvirt.firewall [req-9ae6b6e0-2e77-48ea-8234-23f4691bfd5a c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Ensuring static filters
  /var/log/nova/nova-compute.log:2014-05-20 13:45:14.614 22235 INFO nova.virt.libvirt.driver [-] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Instance spawned successfully.
  /var/log/nova/nova-compute.log:2014-05-20 13:46:18.841 22235 AUDIT nova.compute.claims [req-b51803d7-d892-47cd-8402-d9730a1ecfb5 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Attempting claim: memory 512 MB, disk 10 GB, VCPUs 1
  /var/log/nova/nova-compute.log:2014-05-20 13:46:18.842 22235 AUDIT nova.compute.claims [req-b51803d7-d892-47cd-8402-d9730a1ecfb5 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Total memory: 15888 MB, used: 3072.00 MB
  /var/log/nova/nova-compute.log:2014-05-20 13:46:18.842 22235 AUDIT nova.compute.claims [req-b51803d7-d892-47cd-8402-d9730a1ecfb5 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] memory limit not specified, defaulting to unlimited
  /var/log/nova/nova-compute.log:2014-05-20 13:46:18.842 22235 AUDIT nova.compute.claims [req-b51803d7-d892-47cd-8402-d9730a1ecfb5 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Total disk: 59 GB, used: 5.00 GB
  /var/log/nova/nova-compute.log:2014-05-20 13:46:18.842 22235 AUDIT nova.compute.claims [req-b51803d7-d892-47cd-8402-d9730a1ecfb5 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] disk limit not specified, defaulting to unlimited
  /var/log/nova/nova-compute.log:2014-05-20 13:46:18.842 22235 AUDIT nova.compute.claims [req-b51803d7-d892-47cd-8402-d9730a1ecfb5 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Total CPUs: 4 VCPUs, used: 5.00 VCPUs
  /var/log/nova/nova-compute.log:2014-05-20 13:46:18.843 22235 AUDIT nova.compute.claims [req-b51803d7-d892-47cd-8402-d9730a1ecfb5 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] CPUs limit not specified, defaulting to unlimited
  /var/log/nova/nova-compute.log:2014-05-20 13:46:18.843 22235 AUDIT nova.compute.claims [req-b51803d7-d892-47cd-8402-d9730a1ecfb5 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Claim successful
  /var/log/nova/nova-compute.log:2014-05-20 13:46:18.868 22235 AUDIT nova.compute.resource_tracker [req-b51803d7-d892-47cd-8402-d9730a1ecfb5 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] Updating from migration a400d2f2-80ce-4ebf-9847-8885b7742fec
  /var/log/nova/nova-compute.log:2014-05-20 13:46:18.898 22235 AUDIT nova.compute.manager [req-b51803d7-d892-47cd-8402-d9730a1ecfb5 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Migrating
  /var/log/nova/nova-compute.log:2014-05-20 13:46:19.856 22235 INFO nova.virt.libvirt.driver [-] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Instance destroyed successfully.
  /var/log/nova/nova-compute.log:2014-05-20 13:46:21.616 22235 INFO nova.virt.libvirt.driver [req-b51803d7-d892-47cd-8402-d9730a1ecfb5 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Creating image
  /var/log/nova/nova-compute.log:2014-05-20 13:46:22.157 22235 INFO nova.virt.libvirt.firewall [req-b51803d7-d892-47cd-8402-d9730a1ecfb5 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Called setup_basic_filtering in nwfilter
  /var/log/nova/nova-compute.log:2014-05-20 13:46:22.157 22235 INFO nova.virt.libvirt.firewall [req-b51803d7-d892-47cd-8402-d9730a1ecfb5 c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Ensuring static filters
  /var/log/nova/nova-compute.log:2014-05-20 13:46:23.513 22235 INFO nova.virt.libvirt.driver [-] [instance: a400d2f2-80ce-4ebf-9847-8885b7742fec] Instance running successfully.
  /var/log/nova/nova-scheduler.log:2014-05-20 13:45:00.943 22289 INFO nova.scheduler.filter_scheduler [req-9ae6b6e0-2e77-48ea-8234-23f4691bfd5a c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] Attempting to build 10 instance(s) uuids: [u'777ae1d1-eb81-4268-b4fe-0af40cff473c', u'19c0b1b4-88d9-4bd5-baa7-22d5fd92be4f', u'632be5d1-4b3c-4e3d-b96c-65f6df357218', u'417830a6-cbe9-4597-b583-6997cbd44284', u'69cb24cf-2f3b-4e59-9a3f-0ab8f9b73b9f', u'fa5d6212-8723-4002-8313-8f7aa03d13e8', u'a400d2f2-80ce-4ebf-9847-8885b7742fec', u'af4bba03-bdcd-472f-b9c1-5dde8c34d0bf', u'4cdfbe99-f404-4975-9409-1362e2550487', u'c78eed88-db3b-4578-8f50-3119331ec4fb']
  /var/log/nova/nova-scheduler.log:2014-05-20 13:45:01.391 22289 INFO nova.scheduler.filter_scheduler [req-9ae6b6e0-2e77-48ea-8234-23f4691bfd5a c9062d562d9f41e4a1fdce36a4f176f6 4ad766166539403189f2caca1ba306aa] Choosing host WeighedHost [host: orange-vdsf.qa.lab.tlv.redhat.com, weight: 1.0] for instance a400d2f2-80ce-4ebf-9847-8885b7742fec
  egrep: /var/log/nova/tmp: Is a directory
  [root@orange-vdsf ~(keystone_admin)]# nova list 
  +--------------------------------------+--------------------------------------------+--------+------------+-------------+---------------------------+
  | ID                                   | Name                                       | Status | Task State | Power State | Networks                  |
  +--------------------------------------+--------------------------------------------+--------+------------+-------------+---------------------------+
  | 19c0b1b4-88d9-4bd5-baa7-22d5fd92be4f | dafna-19c0b1b4-88d9-4bd5-baa7-22d5fd92be4f | ACTIVE | -          | Running     | novanetwork=192.168.32.2  |
  | 417830a6-cbe9-4597-b583-6997cbd44284 | dafna-417830a6-cbe9-4597-b583-6997cbd44284 | ACTIVE | -          | Running     | novanetwork=192.168.32.3  |
  | 4cdfbe99-f404-4975-9409-1362e2550487 | dafna-4cdfbe99-f404-4975-9409-1362e2550487 | ERROR  | -          | Running     | novanetwork=192.168.32.10 |
  | 632be5d1-4b3c-4e3d-b96c-65f6df357218 | dafna-632be5d1-4b3c-4e3d-b96c-65f6df357218 | ACTIVE | -          | Running     | novanetwork=192.168.32.9  |
  | 69cb24cf-2f3b-4e59-9a3f-0ab8f9b73b9f | dafna-69cb24cf-2f3b-4e59-9a3f-0ab8f9b73b9f | ACTIVE | -          | Running     | novanetwork=192.168.32.8  |
  | 777ae1d1-eb81-4268-b4fe-0af40cff473c | dafna-777ae1d1-eb81-4268-b4fe-0af40cff473c | ACTIVE | -          | Running     | novanetwork=192.168.32.5  |
  | a400d2f2-80ce-4ebf-9847-8885b7742fec | dafna-a400d2f2-80ce-4ebf-9847-8885b7742fec | ACTIVE | -          | Running     | novanetwork=192.168.32.11 |
  | af4bba03-bdcd-472f-b9c1-5dde8c34d0bf | dafna-af4bba03-bdcd-472f-b9c1-5dde8c34d0bf | ERROR  | -          | Running     | novanetwork=192.168.32.7  |
  | c78eed88-db3b-4578-8f50-3119331ec4fb | dafna-c78eed88-db3b-4578-8f50-3119331ec4fb | ERROR  | -          | Running     | novanetwork=192.168.32.6  |
  | fa5d6212-8723-4002-8313-8f7aa03d13e8 | dafna-fa5d6212-8723-4002-8313-8f7aa03d13e8 | ACTIVE | -          | Running     | novanetwork=192.168.32.4  |
  +--------------------------------------+--------------------------------------------+--------+------------+-------------+---------------------------+
  [root@orange-vdsf ~(keystone_admin)]#

To manage notifications about this bug go to:
https://bugs.launchpad.net/nova/+bug/1323578/+subscriptions
References

[Bug 1323578] [NEW] randmly failing rebuild on ssh-key errors on a setup which is configured for non-migration resize
From: Dafna Ron, 2014-05-27