← Back to team overview

fuel-dev team mailing list archive

Re: network verification (use case)

 

On Mon, Jan 27, 2014 at 12:51 AM, Andrew Woodward <xarses@xxxxxxxxx> wrote:

> wow, its interesting to know that the L3 isn't tested, in the verify
> networks task. That is probably part of the problem i'm seeing and IMHO is
> more important to verify than the VLAN range. We don't currently track
> gateways, so at the moment we don't know who we can ping during the
> pre-deployment checks.
>
> however my idea is to also add a clause in puppet to force it to fail if
> it can ping the primary controller, this will cause nodes after the first
> controller to fail more rapidly (right after or during exec of l23networks
> code there are already ping hooks there too) this will help spot nodes with
> errors for which proceeding further is useless anyways.
>

Typo: I'd assume you meant "fail if it cannot ping the primary controller"
here.

now with the research we've been doing for multiple L3/L2 domains, we will
> have to know gateways for each network, so we will have those available to
> ping in cases that multi-l3 is used.
>
> Andrew
>
>
> On Fri, Jan 24, 2014 at 5:30 AM, Miroslav Anashkin <manashkin@xxxxxxxxxxxx
> > wrote:
>
>> My 5c.
>>
>> 1. Full network verification may take hours in case there are hundreds of
>> VLANs configured. So, to avoid timeouts, divide the verification process
>> into smaller batches. Make timeout configurable - as I remember Cyan case,
>> their timeouts were 1+ minute to start up and calibrate connection for each
>> optical NIC.
>>
>> 2. Network verification is fully independent feature and even not
>> mandatory - so, make it background process. Let it show the current
>> verification progress and report errors as soon as it get some.
>>
>> 3. Add feature to allow network verification against selected node or
>> interface - with priority over common background verification. Make the
>> green/gray/red network port icon a button.
>>
>> 4. 95% of network misconfiguration errors can be found on single node
>> verification. Let us check network settings against single node first.
>>
>> Kind regards,
>> Miroslav
>>
>>
>> On Fri, Jan 24, 2014 at 2:56 PM, Mike Scherbakov <
>> mscherbakov@xxxxxxxxxxxx> wrote:
>>
>>> Network verification will fail, if any required interfaces are down.
>>>
>>> Let's discuss here first any improvements we could do for this feature,
>>> before creating the blueprint. I'm all in for finding out what we can do
>>> better here, as network issues looks to be the most frequent thing which
>>> happens in real deployments.
>>>
>>>
>>> On Fri, Jan 24, 2014 at 1:06 PM, Andrey Danin <adanin@xxxxxxxxxxxx>wrote:
>>>
>>>> Gleb, to get interfaces' states from DB you can do this:
>>>>> http://paste.openstack.org/show/61805/
>>>>>
>>>>>
>>>>> On Fri, Jan 24, 2014 at 4:30 AM, Roman Alekseenkov <
>>>>> ralekseenkov@xxxxxxxxxxxx> wrote:
>>>>>
>>>>>> Gleb - thanks for bringing this up. I like the proposal, actually.
>>>>>> Whatever makes the life of deployment engineers easier...
>>>>>>
>>>>>> Evgeny - the thing you mentioned cannot be a full solution for two
>>>>>> reasons. The first is scale (nobody will click on each node to check the
>>>>>> status of its NICs), the second is mass configuration (people are likely to
>>>>>> configure NICs for multiple nodes at once and again won't go into
>>>>>> individual nodes). You can imagine how bad it's going to be with 100+
>>>>>> nodes...
>>>>>>
>>>>>> Andrew Woodward and Alex Shaposhnikov also have some specific
>>>>>> suggestions on how to improve network verification and make it more
>>>>>> meaningful. Guys - please speak up.
>>>>>>
>>>>>> I'd like to see a consolidated blueprint on launchpad from you guys
>>>>>> (Gleb, Andrew, and Alex), which David and I can take and prioritize.
>>>>>>
>>>>>> Thanks,
>>>>>> Roman
>>>>>>
>>>>>>
>>>>>> On Thu, Jan 23, 2014 at 7:50 AM, Mike Scherbakov <
>>>>>> mscherbakov@xxxxxxxxxxxx> wrote:
>>>>>>
>>>>>>> Our verify network feature (activated by corresponding button on
>>>>>>> networks tab) verifies L2 connectivity of OpenStack networks. It configures
>>>>>>> desired networking on bootstrap nodes, and runs UDP packets on required
>>>>>>> interfaces.
>>>>>>> We also check for unwanted DHCP traffic. There are no checks on L3
>>>>>>> layer at the moment.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> On Jan 23, 2014 7:10 PM, "Evgeniy L" <eli@xxxxxxxxxxxx> wrote:
>>>>>>>
>>>>>>>> Hi Gleb,
>>>>>>>>
>>>>>>>> Regarding state of interfaces, we have such feature right now.
>>>>>>>> It was merged and should be available in 4.0 release
>>>>>>>>
>>>>>>>> https://github.com/stackforge/fuel-web/commit/0e60ff862d75d8d2ef37d2b0f8d834260f8349b6
>>>>>>>>
>>>>>>>> But as far as I know it doesn't work correctly in virtual box.
>>>>>>>>
>>>>>>>> [image: Inline image 2]
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Jan 23, 2014 at 12:05 PM, Gleb Galkin <ggalkin@xxxxxxxxxxxx
>>>>>>>> > wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Hello, all
>>>>>>>>>
>>>>>>>>> Right now I have a bunch of nodes and each has 4 network
>>>>>>>>> interfaces. How can I check that every interface on every node is UP,
>>>>>>>>> that all switch ports are configured properly and there are no
>>>>>>>>> connectivity problem?
>>>>>>>>>
>>>>>>>>> We have network verification in GUI but it can't provide me with
>>>>>>>>> detail information about all network issues, can it?
>>>>>>>>> I'd like to got detail information like
>>>>>>>>>
>>>>>>>>> on the node number X interface eth2 (xx:xx:xx:xx:xx:xx) has link
>>>>>>>>> status 'down'.
>>>>>>>>> or
>>>>>>>>> on the node number X interface eth3 (xx:xx:xx:xx:xx:xx) is up but
>>>>>>>>> it can't ping fuel node
>>>>>>>>>
>>>>>>>>> and so on
>>>>>>>>>
>>>>>>>>> It's good to have this information BEFORE you press Deploy button.
>>>>>>>>>
>>>>>>>>> Maybe we already have something like this? Maybe our network
>>>>>>>>> verification write some report about the network issues?
>>>>>>>>>
>>>>>>>>> If we don't have this feature we should consider it. It'll save a
>>>>>>>>> lot of time for deployers.
>>>>>>>>> We can use mcollective to up all network interfaces on all nodes
>>>>>>>>> and make arping the fuel node or something.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Best Regards,
>>>>>>>>> Gleb Galkin
>>>>>>>>> OpenStack Deployment Engineer
>>>>>>>>>
>>>>>>>>> Mirantis Inc.
>>>>>>>>> www.mirantis.com
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Mailing list: https://launchpad.net/~fuel-dev
>>>>>>>>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>>>>>>> Unsubscribe : https://launchpad.net/~fuel-dev
>>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Mailing list: https://launchpad.net/~fuel-dev
>>>>>>>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>>>>>> Unsubscribe : https://launchpad.net/~fuel-dev
>>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>>
>>>>>>>>
>>>>>>> --
>>>>>>> Mailing list: https://launchpad.net/~fuel-dev
>>>>>>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>>>>> Unsubscribe : https://launchpad.net/~fuel-dev
>>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> --
>>>>>> Mailing list: https://launchpad.net/~fuel-dev
>>>>>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>>>>> Unsubscribe : https://launchpad.net/~fuel-dev
>>>>>> More help   : https://help.launchpad.net/ListHelp
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Andrey Danin
>>>>> adanin@xxxxxxxxxxxx
>>>>> skype: gcon.monolake
>>>>>
>>>>
>>>
>>>
>>> --
>>> Mike Scherbakov
>>> #mihgen
>>>
>>> --
>>> Mailing list: https://launchpad.net/~fuel-dev
>>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>>> Unsubscribe : https://launchpad.net/~fuel-dev
>>> More help   : https://help.launchpad.net/ListHelp
>>>
>>>
>>
>>
>> --
>>
>> *Kind Regards*
>>
>> *Miroslav Anashkin**L2 support engineer**,*
>> *Mirantis Inc.*
>> *+7(495)640-4944 <%2B7%28495%29640-4944> (office receptionist)*
>> *+1(650)587-5200 <%2B1%28650%29587-5200> (office receptionist, call from
>> US)*
>> *35b, Bld. 3, Vorontsovskaya St.*
>> *Moscow**, Russia, 109147.*
>>
>> www.mirantis.com
>>
>> manashkin@xxxxxxxxxxxx
>>
>>
>> --
>> Mailing list: https://launchpad.net/~fuel-dev
>> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
>> Unsubscribe : https://launchpad.net/~fuel-dev
>> More help   : https://help.launchpad.net/ListHelp
>>
>>
>
>
> --
> If google has done it, Google did it right!
>
> --
> Mailing list: https://launchpad.net/~fuel-dev
> Post to     : fuel-dev@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~fuel-dev
> More help   : https://help.launchpad.net/ListHelp
>
>


-- 
Dmitry Borodaenko

PNG image


Follow ups

References