← Back to team overview

openstack team mailing list archive

Re: Grizzy,Quantum public network ports DOWN

 

On 06/19/2013 07:00 PM, Samuel Winchenbach wrote:
Here are ALL the warnings and errors I see when restarting nova and quantum: http://pastie.org/pastes/8059837/text Here are ALL the warnings and errors I see when creating both networks (public and internal), the router that bridges them, assigning the public network as the gateway, adding the internal interface to it, launching a VM, allocating a floating IP, and assigning it to the VM: http://pastie.org/pastes/8059850/text

I am not sure about the nbd stuff, but I think think that is just because the qemu disk image is not resized to the disk size yet. The ONLY warning that I think might effect me is:

Jun 19 11:53:32 test1 dnsmasq[28913]: warning: no upstream servers configured

Any idea if that could be causing my DOWN ports issue?

Sorry to chime in pretty late. This is a bug. In certain plugins when a port is created the status of the port is set to DOWN. This port is set to UP when the plugin receives an indication that the port is actually UP (for example when a agent detects that a tap device has been created). This is not treated well when dealing with gateway ports and flotaing IP's.

I'll take care of this.
Thanks
Gary


Thanks,
Sam


On Wed, Jun 19, 2013 at 11:03 AM, Samuel Winchenbach <swinchen@xxxxxxxxx <mailto:swinchen@xxxxxxxxx>> wrote:

    I
    did that, and I went one step further... I deleted and recreated
    ALL the databases to ensure that I would be starting from scratch.
     I also deleted all bridges, stopped openvswitch, deleted conf.db,
    started openvswitch and recreated all the bridges.

    This solved the reference to "test1-int" but the ports are still
    stuck in the down state :(

    Sam


    On Tue, Jun 18, 2013 at 4:39 PM, Filipe Manco
    <filipe.manco@xxxxxxxxx <mailto:filipe.manco@xxxxxxxxx>> wrote:

        Stop quantum agents (not quantum-api) and openvswitch service.
        Delete /etc/openvswitch/conf.dband delte all agents using
        quantum agent-delete <id>. Start openvswitch service and then
        quantum agents. If ports still down check quantum logs mainly
        quantum l3-agent. If you don't find anything interesting
        delete the networks and recreate them.

        If you still have references to "test1-int" check the quantum
        database ovs_tunnel_endpoints table and manually remove any
        reference.

        Filipe Manco
        http://about.me/fmanco


        2013/6/18 Samuel Winchenbach <swinchen@xxxxxxxxx
        <mailto:swinchen@xxxxxxxxx>>

            Hmmm I used both of those commands, but no matter what I
            do I can not remove references to "test1-int" in
            /etc/openvswitch/conf.db

            Should I just manually replace those with the IP?  Delete
            the file?


            On Tue, Jun 18, 2013 at 1:14 PM, Filipe Manco
            <filipe.manco@xxxxxxxxx <mailto:filipe.manco@xxxxxxxxx>>
            wrote:

                Honestly I'm not sure because I've always used IPs.
                But according to the logs it looks so. After changing
                configurations you should probably run
                quantum-netns-cleanup and quantum-ovs-cleanup before
                starting the services.

                Filipe Manco
                http://about.me/fmanco


                2013/6/18 Samuel Winchenbach <swinchen@xxxxxxxxx
                <mailto:swinchen@xxxxxxxxx>>

                    I think I may be onto something:
                    http://pastie.org/pastes/8056137/text

                    from syslog
                    Jun 18 12:57:26 test1 ovs-vsctl:
                    00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --
                    --may-exist add-port br-int qvo3eb6d144-07 -- set
                    Interface qvo3eb6d144-07
                    external-ids:iface-id=3eb6d144-077e-42cf-ad2e-57c50aa00399
                    external-ids:iface-status=active
                    external-ids:attached-mac=fa:16:3e:92:31:1e
                    external-ids:vm-uuid=add44e48-6f42-4ede-a646-f29e74ccc02d
                    Jun 18 12:57:26 test1 ovs-vswitchd:
                    03753|socket_util|ERR|"test1-int" is not a valid
                    IP address
                    Jun 18 12:57:26 test1 ovs-vswitchd:
                    03755|netdev_vport|ERR|gre-1: gre type requires
                    valid 'remote_ip' argument
                    Jun 18 12:57:30 test1 ovs-vsctl:
                    00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl
                    --timeout=2 set Port qvo3eb6d144-07 tag=1
                    Jun 18 12:57:30 test1 ovs-vswitchd:
                    03768|netdev_vport|ERR|gre-1: gre type requires
                    valid 'remote_ip' argument


                    Looks like you might not be able to use entries
                    from /etc/hosts in the config files?








                    On Tue, Jun 18, 2013 at 10:42 AM, Samuel
                    Winchenbach <swinchen@xxxxxxxxx
                    <mailto:swinchen@xxxxxxxxx>> wrote:

                        I have three agents running (Open vSwitch
                        agent, DHCP agent, and L3 agent):
                        http://pastie.org/pastes/8055658/text
                        The agents listed on test3 are there because
                        ubuntu starts them automatically.  L3 agent
                        will never run on test3 because it doesn't
                        even have an external interface.   Right now I
                        am just trying to limit it to one node.

                        Here is my l3_agent.ini:
                        http://pastie.org/pastes/8055674/text
                        Here are a list of bridges (eth1 is my
                        external interface)
                        http://pastie.org/pastes/8055678/text

                        Thanks again for all your help!


                        On Tue, Jun 18, 2013 at 10:32 AM, Filipe Manco
                        <filipe.manco@xxxxxxxxx
                        <mailto:filipe.manco@xxxxxxxxx>> wrote:

                            What is the status of quantum agent-list?
                            I see on your node test3 the agents are
                            down and you don't have openvswitch agent.
                            I would check for the logs of the l3
                            agent? Have you configured the external
                            network id on the l3 agent config file?

                            Filipe Manco
                            http://about.me/fmanco


                            2013/6/18 Samuel Winchenbach
                            <swinchen@xxxxxxxxx
                            <mailto:swinchen@xxxxxxxxx>>

                                Hi Filipe,

                                Thanks for the response.  I already
                                had the /etc/sudoers.d/quantum_sudoers
                                file.   On a whim I added "root_helper
                                = sudo quantum-rootwrap
                                /etc/quantum/rootwrap.conf" to
                                /etc/quantum/dhcp_agent.ini and that
                                took care of that problem.

                                I managed to remove the libvirt errors
                                by disabling apparmor.

                                All the ports on my public network are
                                still listed as "DOWN"   I have
                                managed to remove all of the errors
                                and warnings from quantum but those
                                ports will still not come up.   I
                                really am lost.

                                Thanks again for the post, I am not
                                sure what to try next :/

                                Sam


                                On Tue, Jun 18, 2013 at 10:13 AM,
                                Filipe Manco <filipe.manco@xxxxxxxxx
                                <mailto:filipe.manco@xxxxxxxxx>> wrote:

                                    From what I can see in the logs
                                    you must create the file
                                    /etc/sudoers.d/quantum_sudoers
                                    with the following contents:

                                    Defaults:quantum !requiretty
                                    quantum ALL = (root) NOPASSWD:
                                    /usr/bin/quantum-rootwrap


                                    About the libvirt error edit the
                                    file /etc/libvirt/qemu.conf and
                                    add the following:

                                    cgroup_device_acl = [
                                        "/dev/null", "/dev/full",
                                    "/dev/zero",
                                        "/dev/random", "/dev/urandom",
                                        "/dev/ptmx", "/dev/kvm",
                                    "/dev/kqemu",
                                        "/dev/rtc","/dev/hpet" ,
                                    "/dev/net/tun"
                                    ]

                                    Probably this won't fix all of
                                    your issues. The logs ofI don't
                                    the l3 agent will be helpful.

                                    Filipe Manco
                                    http://about.me/fmanco


                                    2013/6/18 Samuel Winchenbach
                                    <swinchen@xxxxxxxxx
                                    <mailto:swinchen@xxxxxxxxx>>

                                        I may have found the cause of
                                        my problem, but I am unsure of
                                        the solution.  In my libvirt
                                        log file I found many error
                                        messages similar to this:

                                        2013-06-18 13:12:19.812+0000:
                                        8353: warning :
                                        virAuditSend:135 : Failed to
                                        send audit message virt=kvm
                                        resrc=net reason=open
                                        vm="instance-00000033"
                                        uuid=bca8a09e-46aa-408b-81cd-2432068361c1
                                        net=FA:16:3E:71:7F:68
                                        path="/dev/net/tun"
                                        rdev=0A:C8: Operation not
                                        permitted

                                        Sam


                                        On Mon, Jun 17, 2013 at 8:52
                                        PM, Samuel Winchenbach
                                        <swinchen@xxxxxxxxx
                                        <mailto:swinchen@xxxxxxxxx>>
                                        wrote:

                                            Here is a bunch more
                                            information from quantum:
                                            http://pastie.org/pastes/8053820/text

                                            If anyone has any ideas I
                                            would really appreciate
                                            it.  Thanks!



                                            On Mon, Jun 17, 2013 at
                                            5:07 PM, Samuel
                                            Winchenbach
                                            <swinchen@xxxxxxxxx
                                            <mailto:swinchen@xxxxxxxxx>>
                                            wrote:

                                                Hi All,

                                                I have been stuck on a
                                                problem for a couple
                                                of days now.  I am
                                                using Grizzly on
                                                Ubuntu 12.04 LTS.  I
                                                can launch vms, create
                                                networks, subnets,
                                                routers, etc.  The
                                                problem is quantum
                                                reports that all fors
                                                on the public network
                                                are "DOWN"  for
                                                example:
                                                http://pastie.org/pastes/8053283/text

                                                Does anyone have any
                                                hints or tips on what
                                                might be causing this,
                                                or the errors listed
                                                below in the quantum
                                                logs?  Thanks!

                                                quantum configuration:
                                                http://pastie.org/pastes/8053100/text

                                                nova configuration:
                                                http://pastie.org/pastes/8043800/text

                                                quantum logs:
                                                http://pastie.org/pastes/8053269/text
                                                (this is everything
                                                logged during the
                                                creation the networks,
                                                launching of vm
                                                instances, allocating
                                                the floating ip and
                                                assigning it to a VM)




                                        _______________________________________________
                                        Mailing list:
                                        https://launchpad.net/~openstack
                                        <https://launchpad.net/%7Eopenstack>
                                        Post to     :
                                        openstack@xxxxxxxxxxxxxxxxxxx
                                        <mailto:openstack@xxxxxxxxxxxxxxxxxxx>
                                        Unsubscribe :
                                        https://launchpad.net/~openstack
                                        <https://launchpad.net/%7Eopenstack>
                                        More help   :
                                        https://help.launchpad.net/ListHelp













_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to     : openstack@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Follow ups

References