openstack team mailing list archive

[Jay Pipes <jaypipes@xxxxxxxxx>]

On 03/20/2012 11:19 AM, David Kranz wrote:
> In a diablo/kvm cluster that has been running for a long time, a user
> reported problems with some vms, tried rebooting them and eventually
> deleted them. I recently noticed messages in the nova compute log like:
> Found 13 in the database and 10 on the hypervisor.
>
> Looking at the source code I understand that this means the instances
> have been deleted as far as the hypervisor is concerned, but nova still
> thinks they are there.

Correct.

> I found the offending instances in the database and they were still
> listed as in the active state even though they
> had a deletion date recorded. I tried to delete them but was unable due
> to a foreign key error with virtual_interfaces. I could play around with
> deleting various things from the database but there are real users. Is
> their a documented way to "clean up" the state of the nova database in
> such situations? It seems like a bug that the database could get into
> this state.

I also ran into this issue on Diablo/KVM...

> Also, it seems that deleted instances are never removed from the
> database. Is that a bug?

No, it's not a bug. But was is a bug is ACTIVE status instances that are 
deleted.

Another issue right now is that Essex has moved on and much of the code 
involved in this stuff is substantially different in trunk. We first 
need to identify whether this issue exists in trunk today, and if so, 
patch trunk and then backport a patch to Diablo...

Best,
-jay