openstack team mailing list archive

[Marco Colombo <mail.colombomarco@xxxxxxxxx>]

Hi Thomas,
i have upgrade dnsmasq to version 2.66, but the problem persist.
If you need more log/information, i'm able to reproduce the problem.

Thanks
Best Regards


2013/5/22 Thomas Kärgel <kaergel@xxxxxxxxxxxxx>

> Hi Marco,
>
> can you also reproduce it?
>
> If so i would suggest to give dnsmasq 2.66 a try. Since i'm not able to
> reproduce this behavior on the environment i'm woking on it will take
> weeks to get to an really conclusive result.
>
> Best regards
> thomas
>
>
> Am 22.05.2013 09:55, schrieb Marco Colombo:
> > Hi All,
> > i have the same problem with Grizzly, Ubuntu 12.04 and Dnsmasq 2.59
> > "killall dnsmasq && service quantum-dhcp-agent restart" fixes the
> > problem temporarily
> >
> > Best Regards,
> > Marco
> >
> >
> >
> >
> > 2013/5/22 Thomas Kärgel <kaergel@xxxxxxxxxxxxx
> > <mailto:kaergel@xxxxxxxxxxxxx>>
> >
> >     Hi Johanna,
> >
> >     sure, I'll report my results, but this can take a while. The last
> time
> >     this issue occurred on my environment is May 10th. It occurs quite
> >     sporadic and i don't know how to manually reproduce this issue yet (i
> >     have to be very careful with my experiments since the environment is
> >     running productive already). Maybe you should also give dnsmasq 2.66
> a
> >     try, because you wrote that you can manually reproduce this behavior.
> >
> >
> >     Best Regards,
> >     Thomas
> >
> >
> >     Am 20.05.2013 12:25, schrieb Heinonen, Johanna (NSN - FI/Espoo):
> >     > Hi,
> >     >
> >     > You are right. Manually doing kill -HUP <PID> has just the effect
> >     you mentioned. In the syslog you can see
> >     >
> >     > May 20 13:09:41 grizzly-236 dnsmasq[6170]: cleared cache
> >     > May 20 13:09:41 grizzly-236 dnsmasq-dhcp[6170]: read
> >     /var/lib/quantum/dhcp/d5879bbb-ada6-4323-a7b1-87b5db244513/host
> >     > May 20 13:09:41 grizzly-236 dnsmasq-dhcp[6170]: read
> >     /var/lib/quantum/dhcp/d5879bbb-ada6-4323-a7b1-87b5db244513/opts
> >     >
> >     >
> >     > But the problem stays. Only 'service quantum-dhcp-agent restart'
> >     fixes the problem.
> >     > If you try the newer version of the qnsmasq, I'd be interested to
> >     hear the results.
> >     >
> >     > BR
> >     > Johanna
> >     >
> >     >
> >     >
> >     > -----Original Message-----
> >     > From: ext Thomas Kärgel [mailto:kaergel@xxxxxxxxxxxxx
> >     <mailto:kaergel@xxxxxxxxxxxxx>]
> >     > Sent: Monday, May 20, 2013 11:37 AM
> >     > To: Heinonen, Johanna (NSN - FI/Espoo)
> >     > Cc: openstack@xxxxxxxxxxxxxxxxxxx
> >     <mailto:openstack@xxxxxxxxxxxxxxxxxxx>
> >     > Subject: Re: [Openstack] DHCP problem in grizzly
> >     >
> >     > Hi,
> >     >
> >     > thx for the interesting info, Johanna: dnsmasq version is also
> 2.59 on
> >     > my environment. Can you can confirm that manually sigHUPing all
> >     running
> >     > dnsmasq processes has no effect?
> >     > (dnsmasq claims to have reread the configs in syslog, but still
> >     refuses
> >     > to deliver the new addresses.)
> >     > Maybe updating dnsmasq to a more recent release would solve our
> >     problem.
> >     > Current stable is 2.66. I'll give it a try when i get back to
> office
> >     > tomorrow.
> >     >
> >     > Best regards
> >     >
> >     > Thomas
> >     >
> >     >
> >     > Am 20.05.2013 10:21, schrieb Heinonen, Johanna (NSN - FI/Espoo):
> >     >> Hi Thomas,
> >     >>
> >     >> I am using Ubuntu12.04 and the dnsmasq version is 2.59
> >     >>
> >     >> BR
> >     >> Johanna
> >     >>
> >     >>
> >     >> -----Original Message-----
> >     >> From: Openstack [mailto:openstack-bounces+johanna.heinonen
> >     <mailto:openstack-bounces%2Bjohanna.heinonen>=
> nsn.com@xxxxxxxxxxxxxxxxxxx
> >     <mailto:nsn.com@xxxxxxxxxxxxxxxxxxx>] On Behalf Of ext Thomas Kärgel
> >     >> Sent: Monday, May 20, 2013 10:09 AM
> >     >> To: openstack@xxxxxxxxxxxxxxxxxxx
> >     <mailto:openstack@xxxxxxxxxxxxxxxxxxx>
> >     >> Subject: Re: [Openstack] DHCP problem in grizzly
> >     >>
> >     >> Hi Johanna,
> >     >>
> >     >> I'm facing the same behavior on a Folsom-installation on
> SLES11SP2. I
> >     >> noticed that new hosts have correct entries in the dnsmasq
> >     config-files.
> >     >> The dnsmasq processes get a HUP signal by DHCP-Agent, but simply
> >     refuse
> >     >> to deliver the new address. Instead dnsmasq logs claim "no address
> >     >> available". Exactly like in your description.
> >     >> What distrubtion and exact dnsmasq-versions are in use on your
> >     environment?
> >     >> I assume dnsmasq is not rereading its configs correctly on signal
> >     HUP.
> >     >> dnsmasq logs claim it has reread configs, but it still does not
> >     deliver
> >     >> the new adresses in host-file.
> >     >> Manually trying to sigHUP dnsmasq had no effect. The only way to
> >     get out
> >     >> of this state seems to be restarting DHCP-agent.
> >     >>
> >     >> Best regards
> >     >> Thomas
> >     >>
> >     >> Am 20.05.2013 08:51, schrieb Heinonen, Johanna (NSN - FI/Espoo):
> >     >>> Hi,
> >     >>>
> >     >>> I have installed grizzly with quantum and ovs-plugin. It seems
> that
> >     >>> grizzly allocates the third address of each subnet for dhcp. (In
> >     folsom
> >     >>> it was the second address). This means that the VMs will get
> >     addresses
> >     >>> .2, .4, .5, .
> >     >>>
> >     >>> In my setup the first VM always boots fine and gets the address
> >     x.x.x.2.
> >     >>> This can be seen in the syslog:
> >     >>>
> >     >>> May 20 08:29:23 grizzly-236 dnsmasq-dhcp[2190]:
> >     >>> DHCPDISCOVER(tapdbcef145-f5) 10.0.2.15 fa:16:3e:2d:0d:e0
> >     >>> May 20 08:29:23 grizzly-236 dnsmasq-dhcp[2190]:
> >     >>> DHCPOFFER(tapdbcef145-f5) 10.20.30.2 fa:16:3e:2d:0d:e0
> >     >>> May 20 08:29:23 grizzly-236 dnsmasq-dhcp[2190]:
> >     >>> DHCPREQUEST(tapdbcef145-f5) 10.20.30.2 fa:16:3e:2d:0d:e0
> >     >>> May 20 08:29:23 grizzly-236 dnsmasq-dhcp[2190]:
> >     DHCPACK(tapdbcef145-f5)
> >     >>> 10.20.30.2 fa:16:3e:2d:0d:e0 10-20-30-2
> >     >>>
> >     >>> The problem comes when I start the second VM. Nova shows that the
> >     >>> x.x.x.4 is allocated
> >     >>>
> >     >>> root@grizzly-236:~# nova list
> >     >>>
> >
> +--------------------------------------+----------+--------+------------------------+
> >     >>> | ID                                   | Name     | Status |
> >     >>> Networks               |
> >     >>>
> >
> +--------------------------------------+----------+--------+------------------------+
> >     >>> | c112ccbb-5039-4d05-b414-b53a1eafa2d8 | q-test   | ACTIVE |
> >     >>> tenant1-net=10.20.30.2 |
> >     >>> | 4f26975c-995d-403d-88b0-e7bbf189baad | q-test-2 | ACTIVE |
> >     >>> tenant1-net=10.20.30.4 |
> >     >>>
> >
> +--------------------------------------+----------+--------+------------------------+
> >     >>>
> >     >>> But from syslog I see that the answer to the DHCPDISCOVER is "no
> >     address
> >     >>> available"
> >     >>>
> >     >>> May 20 08:33:34 grizzly-236 dnsmasq-dhcp[2190]:
> >     >>> DHCPDISCOVER(tapdbcef145-f5) 10.0.2.15 fa:16:3e:fc:1f:9a no
> address
> >     >>> available
> >     >>> May 20 08:33:52 grizzly-236 dnsmasq-dhcp[2190]:
> >     >>> DHCPDISCOVER(tapdbcef145-f5) 10.0.2.15 fa:16:3e:fc:1f:9a no
> address
> >     >>> available
> >     >>>
> >     >>> When I restart the quantum-dhcp-server the problem disappears.
> >     This can
> >     >>> be seen from the syslog:
> >     >>>
> >     >>> May 20 09:01:40 grizzly-236 dnsmasq[7235]: started, version 2.59
> >     >>> cachesize 150
> >     >>> May 20 09:01:40 grizzly-236 dnsmasq[7235]: compile time options:
> >     IPv6
> >     >>> GNU-getopt DBus i18n DHCP TFTP conntrack IDN
> >     >>> May 20 09:01:40 grizzly-236 dnsmasq[7235]: warning: no upstream
> >     servers
> >     >>> configured
> >     >>> May 20 09:01:40 grizzly-236 dnsmasq-dhcp[7235]: DHCP, static
> >     leases only
> >     >>> on 10.20.30.0, lease time 2m
> >     >>> May 20 09:01:40 grizzly-236 dnsmasq[7235]: cleared cache
> >     >>> May 20 09:01:40 grizzly-236 dnsmasq-dhcp[7235]: read
> >     >>> /var/lib/quantum/dhcp/e6f27330-be41-478c-b4d2-49ed4ce0af00/host
> >     >>> May 20 09:01:40 grizzly-236 dnsmasq-dhcp[7235]: read
> >     >>> /var/lib/quantum/dhcp/e6f27330-be41-478c-b4d2-49ed4ce0af00/opts
> >     >>> May 20 09:02:19 grizzly-236 dnsmasq-dhcp[7235]:
> >     >>> DHCPREQUEST(tapdbcef145-f5) 10.20.30.2 fa:16:3e:2d:0d:e0
> >     >>> May 20 09:02:19 grizzly-236 dnsmasq-dhcp[7235]:
> >     DHCPNAK(tapdbcef145-f5)
> >     >>> 10.20.30.2 fa:16:3e:2d:0d:e0 lease not found
> >     >>> May 20 09:02:19 grizzly-236 dnsmasq-dhcp[7235]:
> >     >>> DHCPDISCOVER(tapdbcef145-f5) fa:16:3e:2d:0d:e0
> >     >>> May 20 09:02:19 grizzly-236 dnsmasq-dhcp[7235]:
> >     >>> DHCPOFFER(tapdbcef145-f5) 10.20.30.2 fa:16:3e:2d:0d:e0
> >     >>> May 20 09:02:19 grizzly-236 dnsmasq-dhcp[7235]:
> >     >>> DHCPREQUEST(tapdbcef145-f5) 10.20.30.2 fa:16:3e:2d:0d:e0
> >     >>> May 20 09:02:19 grizzly-236 dnsmasq-dhcp[7235]:
> >     DHCPACK(tapdbcef145-f5)
> >     >>> 10.20.30.2 fa:16:3e:2d:0d:e0 10-20-30-2
> >     >>>
> >     >>> *May 20 09:03:39 grizzly-236 dnsmasq-dhcp[7235]:
> >     >>> DHCPDISCOVER(tapdbcef145-f5) fa:16:3e:fc:1f:9a*
> >     >>> *May 20 09:03:39 grizzly-236 dnsmasq-dhcp[7235]:
> >     >>> DHCPOFFER(tapdbcef145-f5) 10.20.30.4 fa:16:3e:fc:1f:9a*
> >     >>> *May 20 09:03:39 grizzly-236 dnsmasq-dhcp[7235]:
> >     >>> DHCPREQUEST(tapdbcef145-f5) 10.20.30.4 fa:16:3e:fc:1f:9a*
> >     >>> *May 20 09:03:39 grizzly-236 dnsmasq-dhcp[7235]:
> >     DHCPACK(tapdbcef145-f5)
> >     >>> 10.20.30.4 fa:16:3e:fc:1f:9a 10-20-30-4*
> >     >>>
> >     >>>
> >     >>> What could be the problem? Have you seen similar behavior? If
> >     yes, how
> >     >>> did you fix this?
> >     >>>
> >     >>> Best regards,
> >     >>> Johanna
> >     >>>
> >     >>>
> >     >>>
> >     >>> _______________________________________________
> >     >>> Mailing list: https://launchpad.net/~openstack
> >     >>> Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> >     <mailto:openstack@xxxxxxxxxxxxxxxxxxx>
> >     >>> Unsubscribe : https://launchpad.net/~openstack
> >     >>> More help   : https://help.launchpad.net/ListHelp
> >     >>>
> >     >>
> >     >>
> >     >
> >     >
> >
> >
> >     --
> >     Thomas Kärgel
> >     Linux Consultant
> >     Mail: kaergel@xxxxxxxxxxxxx <mailto:kaergel@xxxxxxxxxxxxx>
> >     B1 Systems GmbH
> >     Osterfeldstraße 7 / 85088 Vohburg / http://www.b1-systems.de
> >     GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt,HRB
> 3537
> >
> >
> >     _______________________________________________
> >     Mailing list: https://launchpad.net/~openstack
> >     Post to     : openstack@xxxxxxxxxxxxxxxxxxx
> >     <mailto:openstack@xxxxxxxxxxxxxxxxxxx>
> >     Unsubscribe : https://launchpad.net/~openstack
> >     More help   : https://help.launchpad.net/ListHelp
> >
> >
> >
> >
> > --
> > Marco Colombo
>
>
> --
> Thomas Kärgel
> Linux Consultant
> Tel.: +49 172 2037945
> Mail: kaergel@xxxxxxxxxxxxx
> B1 Systems GmbH
> Osterfeldstraße 7 / 85088 Vohburg / http://www.b1-systems.de
> GF: Ralph Dehner / Unternehmenssitz: Vohburg / AG: Ingolstadt,HRB 3537
>
>


-- 
Marco Colombo