openstack team mailing list archive
-
openstack team
-
Mailing list archive
-
Message #08858
Re: Scalability issue in nova-dhcpbridge
Hi Vish,
> I believe it is safe to ignore the old leases. If nova-network has
> been down for a while it could potentially be nice to refresh all of
> the leases that it knows about, but I don't think it will harm
> anything if you remove it.
>
> Are you running flatdhcp with a single network host on a large
> install? I would think that multi_host would be a better choice in
> that case.
>
> There is also a potentially nasty performance issue in linux_net
> where it creates all of the leases. It is a very expensive operation
> and needs to be reoptimized after the foreign keys were removed from
> the network tables. Currently it is doing 2 database for every active
> instance in the db.
Thanks for making these changes so quickly. Initial testing shows a
huge improvement. If I push things harder we do slow down somewhat
because each call of nova-dhcpbridge takes time just in the import code:
# cat slow
#!/usr/bin/python
from nova.network import linux_net
# time ./slow
0.295s
So 500 "old" leases will take 150 seconds to process even though we
are just dropping them. dnsmasq has a dbus interface which I was going
to look at, but for now I just wrapped nova-dhcpbridge with a stupid
shell script:
#!/bin/sh
if [ "$1" == "old" ]; then
exit
fi
exec /opt/stack/nova/bin/nova-dhcpbridge.orig $@
Also "nova list" with 1000 instances is a bit slow (13 seconds on my
setup). It drops to 6.5 seconds if we avoid doing all the queries to
get the hypervisor hostname:
def _get_hypervisor_hostname(self, context, instance):
+ return "junk"
compute_node = db.compute_node_get_by_host(context,
I wonder if there is a way we can do this via a join.
Anton
References