openstack team mailing list archive

[Thomas Kärgel <kaergel@xxxxxxxxxxxxx>]

Hi Johanna,

sure, I'll report my results, but this can take a while. The last time
this issue occurred on my environment is May 10th. It occurs quite
sporadic and i don't know how to manually reproduce this issue yet (i
have to be very careful with my experiments since the environment is
running productive already). Maybe you should also give dnsmasq 2.66 a
try, because you wrote that you can manually reproduce this behavior.


Best Regards,
Thomas


Am 20.05.2013 12:25, schrieb Heinonen, Johanna (NSN - FI/Espoo):
> Hi,
> 
> You are right. Manually doing kill -HUP <PID> has just the effect you mentioned. In the syslog you can see
> 
> May 20 13:09:41 grizzly-236 dnsmasq[6170]: cleared cache
> May 20 13:09:41 grizzly-236 dnsmasq-dhcp[6170]: read /var/lib/quantum/dhcp/d5879bbb-ada6-4323-a7b1-87b5db244513/host
> May 20 13:09:41 grizzly-236 dnsmasq-dhcp[6170]: read /var/lib/quantum/dhcp/d5879bbb-ada6-4323-a7b1-87b5db244513/opts
> 
> 
> But the problem stays. Only 'service quantum-dhcp-agent restart' fixes the problem.
> If you try the newer version of the qnsmasq, I'd be interested to hear the results.
> 
> BR
> Johanna
> 
> 
> 
> -----Original Message-----
> From: ext Thomas Kärgel [mailto:kaergel@xxxxxxxxxxxxx] 
> Sent: Monday, May 20, 2013 11:37 AM
> To: Heinonen, Johanna (NSN - FI/Espoo)
> Cc: openstack@xxxxxxxxxxxxxxxxxxx
> Subject: Re: [Openstack] DHCP problem in grizzly
> 
> Hi,
> 
> thx for the interesting info, Johanna: dnsmasq version is also 2.59 on
> my environment. Can you can confirm that manually sigHUPing all running
> dnsmasq processes has no effect?
> (dnsmasq claims to have reread the configs in syslog, but still refuses
> to deliver the new addresses.)
> Maybe updating dnsmasq to a more recent release would solve our problem.
> Current stable is 2.66. I'll give it a try when i get back to office
> tomorrow.
> 
> Best regards
> 
> Thomas
> 
> 
> Am 20.05.2013 10:21, schrieb Heinonen, Johanna (NSN - FI/Espoo):
>> Hi Thomas,
>>
>> I am using Ubuntu12.04 and the dnsmasq version is 2.59
>>
>> BR
>> Johanna
>>
>>
>> -----Original Message-----
>> From: Openstack [mailto:openstack-bounces+johanna.heinonen=nsn.com@xxxxxxxxxxxxxxxxxxx] On Behalf Of ext Thomas Kärgel
>> Sent: Monday, May 20, 2013 10:09 AM
>> To: openstack@xxxxxxxxxxxxxxxxxxx
>> Subject: Re: [Openstack] DHCP problem in grizzly
>>
>> Hi Johanna,
>>
>> I'm facing the same behavior on a Folsom-installation on SLES11SP2. I
>> noticed that new hosts have correct entries in the dnsmasq config-files.
>> The dnsmasq processes get a HUP signal by DHCP-Agent, but simply refuse
>> to deliver the new address. Instead dnsmasq logs claim "no address
>> available". Exactly like in your description.
>> What distrubtion and exact dnsmasq-versions are in use on your environment?
>> I assume dnsmasq is not rereading its configs correctly on signal HUP.
>> dnsmasq logs claim it has reread configs, but it still does not deliver
>> the new adresses in host-file.
>> Manually trying to sigHUP dnsmasq had no effect. The only way to get out
>> of this state seems to be restarting DHCP-agent.
>>
>> Best regards
>> Thomas
>>
>> Am 20.05.2013 08:51, schrieb Heinonen, Johanna (NSN - FI/Espoo):
>>> Hi,
>>>  
>>> I have installed grizzly with quantum and ovs-plugin. It seems that
>>> grizzly allocates the third address of each subnet for dhcp. (In folsom
>>> it was the second address). This means that the VMs will get addresses
>>> .2, .4, .5, .
>>>  
>>> In my setup the first VM always boots fine and gets the address x.x.x.2.
>>> This can be seen in the syslog:
>>>  
>>> May 20 08:29:23 grizzly-236 dnsmasq-dhcp[2190]:
>>> DHCPDISCOVER(tapdbcef145-f5) 10.0.2.15 fa:16:3e:2d:0d:e0
>>> May 20 08:29:23 grizzly-236 dnsmasq-dhcp[2190]:
>>> DHCPOFFER(tapdbcef145-f5) 10.20.30.2 fa:16:3e:2d:0d:e0
>>> May 20 08:29:23 grizzly-236 dnsmasq-dhcp[2190]:
>>> DHCPREQUEST(tapdbcef145-f5) 10.20.30.2 fa:16:3e:2d:0d:e0
>>> May 20 08:29:23 grizzly-236 dnsmasq-dhcp[2190]: DHCPACK(tapdbcef145-f5)
>>> 10.20.30.2 fa:16:3e:2d:0d:e0 10-20-30-2
>>>  
>>> The problem comes when I start the second VM. Nova shows that the
>>> x.x.x.4 is allocated
>>>  
>>> root@grizzly-236:~# nova list
>>> +--------------------------------------+----------+--------+------------------------+
>>> | ID                                   | Name     | Status |
>>> Networks               |
>>> +--------------------------------------+----------+--------+------------------------+
>>> | c112ccbb-5039-4d05-b414-b53a1eafa2d8 | q-test   | ACTIVE |
>>> tenant1-net=10.20.30.2 |
>>> | 4f26975c-995d-403d-88b0-e7bbf189baad | q-test-2 | ACTIVE |
>>> tenant1-net=10.20.30.4 |
>>> +--------------------------------------+----------+--------+------------------------+
>>>  
>>> But from syslog I see that the answer to the DHCPDISCOVER is "no address
>>> available"
>>>  
>>> May 20 08:33:34 grizzly-236 dnsmasq-dhcp[2190]:
>>> DHCPDISCOVER(tapdbcef145-f5) 10.0.2.15 fa:16:3e:fc:1f:9a no address
>>> available
>>> May 20 08:33:52 grizzly-236 dnsmasq-dhcp[2190]:
>>> DHCPDISCOVER(tapdbcef145-f5) 10.0.2.15 fa:16:3e:fc:1f:9a no address
>>> available
>>>  
>>> When I restart the quantum-dhcp-server the problem disappears. This can
>>> be seen from the syslog:
>>>  
>>> May 20 09:01:40 grizzly-236 dnsmasq[7235]: started, version 2.59
>>> cachesize 150
>>> May 20 09:01:40 grizzly-236 dnsmasq[7235]: compile time options: IPv6
>>> GNU-getopt DBus i18n DHCP TFTP conntrack IDN
>>> May 20 09:01:40 grizzly-236 dnsmasq[7235]: warning: no upstream servers
>>> configured
>>> May 20 09:01:40 grizzly-236 dnsmasq-dhcp[7235]: DHCP, static leases only
>>> on 10.20.30.0, lease time 2m
>>> May 20 09:01:40 grizzly-236 dnsmasq[7235]: cleared cache
>>> May 20 09:01:40 grizzly-236 dnsmasq-dhcp[7235]: read
>>> /var/lib/quantum/dhcp/e6f27330-be41-478c-b4d2-49ed4ce0af00/host
>>> May 20 09:01:40 grizzly-236 dnsmasq-dhcp[7235]: read
>>> /var/lib/quantum/dhcp/e6f27330-be41-478c-b4d2-49ed4ce0af00/opts
>>> May 20 09:02:19 grizzly-236 dnsmasq-dhcp[7235]:
>>> DHCPREQUEST(tapdbcef145-f5) 10.20.30.2 fa:16:3e:2d:0d:e0
>>> May 20 09:02:19 grizzly-236 dnsmasq-dhcp[7235]: DHCPNAK(tapdbcef145-f5)
>>> 10.20.30.2 fa:16:3e:2d:0d:e0 lease not found
>>> May 20 09:02:19 grizzly-236 dnsmasq-dhcp[7235]:
>>> DHCPDISCOVER(tapdbcef145-f5) fa:16:3e:2d:0d:e0
>>> May 20 09:02:19 grizzly-236 dnsmasq-dhcp[7235]:
>>> DHCPOFFER(tapdbcef145-f5) 10.20.30.2 fa:16:3e:2d:0d:e0
>>> May 20 09:02:19 grizzly-236 dnsmasq-dhcp[7235]:
>>> DHCPREQUEST(tapdbcef145-f5) 10.20.30.2 fa:16:3e:2d:0d:e0
>>> May 20 09:02:19 grizzly-236 dnsmasq-dhcp[7235]: DHCPACK(tapdbcef145-f5)
>>> 10.20.30.2 fa:16:3e:2d:0d:e0 10-20-30-2
>>>  
>>> *May 20 09:03:39 grizzly-236 dnsmasq-dhcp[7235]:
>>> DHCPDISCOVER(tapdbcef145-f5) fa:16:3e:fc:1f:9a*
>>> *May 20 09:03:39 grizzly-236 dnsmasq-dhcp[7235]:
>>> DHCPOFFER(tapdbcef145-f5) 10.20.30.4 fa:16:3e:fc:1f:9a*
>>> *May 20 09:03:39 grizzly-236 dnsmasq-dhcp[7235]:
>>> DHCPREQUEST(tapdbcef145-f5) 10.20.30.4 fa:16:3e:fc:1f:9a*
>>> *May 20 09:03:39 grizzly-236 dnsmasq-dhcp[7235]: DHCPACK(tapdbcef145-f5)
>>> 10.20.30.4 fa:16:3e:fc:1f:9a 10-20-30-4*
>>>  
>>>  
>>> What could be the problem? Have you seen similar behavior? If yes, how
>>> did you fix this?
>>>  
>>> Best regards,
>>> Johanna
>>>  
>>>
>>>
>>> _______________________________________________
>>> Mailing list: https://launchpad.net/~openstack
>>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~openstack
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>
>>
> 
> 


-- 
Thomas Kärgel
Linux Consultant
Mail: kaergel@xxxxxxxxxxxxx
B1 Systems GmbH
Osterfeldstraße 7 / 85088 Vohburg / http://www.b1-systems.de
GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt,HRB 3537

Attachment: signature.asc
Description: OpenPGP digital signature