← Back to team overview

openstack team mailing list archive

Re: Grizzy,Quantum public network ports DOWN

 

On 06/20/2013 11:59 AM, Gary Kotton wrote:
On 06/19/2013 07:00 PM, Samuel Winchenbach wrote:
Here are ALL the warnings and errors I see when restarting nova and quantum: http://pastie.org/pastes/8059837/text Here are ALL the warnings and errors I see when creating both networks (public and internal), the router that bridges them, assigning the public network as the gateway, adding the internal interface to it, launching a VM, allocating a floating IP, and assigning it to the VM: http://pastie.org/pastes/8059850/text

I am not sure about the nbd stuff, but I think think that is just because the qemu disk image is not resized to the disk size yet. The ONLY warning that I think might effect me is:

Jun 19 11:53:32 test1 dnsmasq[28913]: warning: no upstream servers configured

Any idea if that could be causing my DOWN ports issue?

Sorry to chime in pretty late. This is a bug. In certain plugins when a port is created the status of the port is set to DOWN. This port is set to UP when the plugin receives an indication that the port is actually UP (for example when a agent detects that a tap device has been created). This is not treated well when dealing with gateway ports and flotaing IP's.

Please note that this is just a usability issue. The port is actually up and forwarding traffic.


I'll take care of this.
Thanks
Gary


Thanks,
Sam


On Wed, Jun 19, 2013 at 11:03 AM, Samuel Winchenbach <swinchen@xxxxxxxxx <mailto:swinchen@xxxxxxxxx>> wrote:

    I
    did that, and I went one step further... I deleted and recreated
    ALL the databases to ensure that I would be starting from
    scratch.  I also deleted all bridges, stopped openvswitch,
    deleted conf.db, started openvswitch and recreated all the bridges.

    This solved the reference to "test1-int" but the ports are still
    stuck in the down state :(

    Sam


    On Tue, Jun 18, 2013 at 4:39 PM, Filipe Manco
    <filipe.manco@xxxxxxxxx <mailto:filipe.manco@xxxxxxxxx>> wrote:

        Stop quantum agents (not quantum-api) and openvswitch
        service. Delete /etc/openvswitch/conf.dband delte all agents
        using quantum agent-delete <id>. Start openvswitch service
        and then quantum agents. If ports still down check quantum
        logs mainly quantum l3-agent. If you don't find anything
        interesting delete the networks and recreate them.

        If you still have references to "test1-int" check the quantum
        database ovs_tunnel_endpoints table and manually remove any
        reference.

        Filipe Manco
        http://about.me/fmanco


        2013/6/18 Samuel Winchenbach <swinchen@xxxxxxxxx
        <mailto:swinchen@xxxxxxxxx>>

            Hmmm I used both of those commands, but no matter what I
            do I can not remove references to "test1-int" in
            /etc/openvswitch/conf.db

            Should I just manually replace those with the IP?  Delete
            the file?


            On Tue, Jun 18, 2013 at 1:14 PM, Filipe Manco
            <filipe.manco@xxxxxxxxx <mailto:filipe.manco@xxxxxxxxx>>
            wrote:

                Honestly I'm not sure because I've always used IPs.
                But according to the logs it looks so. After changing
                configurations you should probably run
                quantum-netns-cleanup and quantum-ovs-cleanup before
                starting the services.

                Filipe Manco
                http://about.me/fmanco


                2013/6/18 Samuel Winchenbach <swinchen@xxxxxxxxx
                <mailto:swinchen@xxxxxxxxx>>

                    I think I may be onto something:
                    http://pastie.org/pastes/8056137/text

                    from syslog
                    Jun 18 12:57:26 test1 ovs-vsctl:
                    00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl --
                    --may-exist add-port br-int qvo3eb6d144-07 -- set
                    Interface qvo3eb6d144-07
                    external-ids:iface-id=3eb6d144-077e-42cf-ad2e-57c50aa00399
                    external-ids:iface-status=active
                    external-ids:attached-mac=fa:16:3e:92:31:1e
                    external-ids:vm-uuid=add44e48-6f42-4ede-a646-f29e74ccc02d
                    Jun 18 12:57:26 test1 ovs-vswitchd:
                    03753|socket_util|ERR|"test1-int" is not a valid
                    IP address
                    Jun 18 12:57:26 test1 ovs-vswitchd:
                    03755|netdev_vport|ERR|gre-1: gre type requires
                    valid 'remote_ip' argument
                    Jun 18 12:57:30 test1 ovs-vsctl:
                    00001|vsctl|INFO|Called as /usr/bin/ovs-vsctl
                    --timeout=2 set Port qvo3eb6d144-07 tag=1
                    Jun 18 12:57:30 test1 ovs-vswitchd:
                    03768|netdev_vport|ERR|gre-1: gre type requires
                    valid 'remote_ip' argument


                    Looks like you might not be able to use entries
                    from /etc/hosts in the config files?








                    On Tue, Jun 18, 2013 at 10:42 AM, Samuel
                    Winchenbach <swinchen@xxxxxxxxx
                    <mailto:swinchen@xxxxxxxxx>> wrote:

                        I have three agents running (Open vSwitch
                        agent, DHCP agent, and L3 agent):
                        http://pastie.org/pastes/8055658/text
                        The agents listed on test3 are there because
                        ubuntu starts them automatically.  L3 agent
                        will never run on test3 because it doesn't
                        even have an external interface.   Right now
                        I am just trying to limit it to one node.

                        Here is my l3_agent.ini:
                        http://pastie.org/pastes/8055674/text
                        Here are a list of bridges (eth1 is my
                        external interface)
                        http://pastie.org/pastes/8055678/text

                        Thanks again for all your help!


                        On Tue, Jun 18, 2013 at 10:32 AM, Filipe
                        Manco <filipe.manco@xxxxxxxxx
                        <mailto:filipe.manco@xxxxxxxxx>> wrote:

                            What is the status of quantum agent-list?
                            I see on your node test3 the agents are
                            down and you don't have openvswitch agent.
                            I would check for the logs of the l3
                            agent? Have you configured the external
                            network id on the l3 agent config file?

                            Filipe Manco
                            http://about.me/fmanco


                            2013/6/18 Samuel Winchenbach
                            <swinchen@xxxxxxxxx
                            <mailto:swinchen@xxxxxxxxx>>

                                Hi Filipe,

                                Thanks for the response.  I already
                                had the
                                /etc/sudoers.d/quantum_sudoers file.
                                  On a whim I added "root_helper =
                                sudo quantum-rootwrap
                                /etc/quantum/rootwrap.conf" to
                                /etc/quantum/dhcp_agent.ini and that
                                took care of that problem.

                                I managed to remove the libvirt
                                errors by disabling apparmor.

                                All the ports on my public network
                                are still listed as "DOWN"   I have
                                managed to remove all of the errors
                                and warnings from quantum but those
                                ports will still not come up.   I
                                really am lost.

                                Thanks again for the post, I am not
                                sure what to try next :/

                                Sam


                                On Tue, Jun 18, 2013 at 10:13 AM,
                                Filipe Manco <filipe.manco@xxxxxxxxx
                                <mailto:filipe.manco@xxxxxxxxx>> wrote:

                                    From what I can see in the logs
                                    you must create the file
                                    /etc/sudoers.d/quantum_sudoers
                                    with the following contents:

                                    Defaults:quantum !requiretty
                                    quantum ALL = (root) NOPASSWD:
                                    /usr/bin/quantum-rootwrap


                                    About the libvirt error edit the
                                    file /etc/libvirt/qemu.conf and
                                    add the following:

                                    cgroup_device_acl = [
                                        "/dev/null", "/dev/full",
                                    "/dev/zero",
                                        "/dev/random", "/dev/urandom",
                                        "/dev/ptmx", "/dev/kvm",
                                    "/dev/kqemu",
                                        "/dev/rtc","/dev/hpet" ,
                                    "/dev/net/tun"
                                    ]

                                    Probably this won't fix all of
                                    your issues. The logs ofI don't
                                    the l3 agent will be helpful.

                                    Filipe Manco
                                    http://about.me/fmanco


                                    2013/6/18 Samuel Winchenbach
                                    <swinchen@xxxxxxxxx
                                    <mailto:swinchen@xxxxxxxxx>>

                                        I may have found the cause of
                                        my problem, but I am unsure
                                        of the solution.  In my
                                        libvirt log file I found many
                                        error messages similar to this:

                                        2013-06-18 13:12:19.812+0000:
                                        8353: warning :
                                        virAuditSend:135 : Failed to
                                        send audit message virt=kvm
                                        resrc=net reason=open
                                        vm="instance-00000033"
                                        uuid=bca8a09e-46aa-408b-81cd-2432068361c1
                                        net=FA:16:3E:71:7F:68
                                        path="/dev/net/tun"
                                        rdev=0A:C8: Operation not
                                        permitted

                                        Sam


                                        On Mon, Jun 17, 2013 at 8:52
                                        PM, Samuel Winchenbach
                                        <swinchen@xxxxxxxxx
                                        <mailto:swinchen@xxxxxxxxx>>
                                        wrote:

                                            Here is a bunch more
                                            information from quantum:
                                            http://pastie.org/pastes/8053820/text

                                            If anyone has any ideas I
                                            would really appreciate
                                            it.  Thanks!



                                            On Mon, Jun 17, 2013 at
                                            5:07 PM, Samuel
                                            Winchenbach
                                            <swinchen@xxxxxxxxx
                                            <mailto:swinchen@xxxxxxxxx>>
                                            wrote:

                                                Hi All,

                                                I have been stuck on
                                                a problem for a
                                                couple of days now.
                                                 I am using Grizzly
                                                on Ubuntu 12.04 LTS.
                                                 I can launch vms,
                                                create networks,
                                                subnets, routers,
                                                etc.  The problem is
                                                quantum reports that
                                                all fors on the
                                                public network are
                                                "DOWN"  for example:
                                                http://pastie.org/pastes/8053283/text

                                                Does anyone have any
                                                hints or tips on what
                                                might be causing
                                                this, or the errors
                                                listed below in the
                                                quantum logs?  Thanks!

                                                quantum
                                                configuration:
                                                http://pastie.org/pastes/8053100/text

                                                nova configuration:
                                                http://pastie.org/pastes/8043800/text

                                                quantum logs:
                                                http://pastie.org/pastes/8053269/text
                                                (this is everything
                                                logged during the
                                                creation the
                                                networks, launching
                                                of vm instances,
                                                allocating the
                                                floating ip and
                                                assigning it to a VM)




                                        _______________________________________________
                                        Mailing list:
                                        https://launchpad.net/~openstack
                                        <https://launchpad.net/%7Eopenstack>
                                        Post to     :
                                        openstack@xxxxxxxxxxxxxxxxxxx
                                        <mailto:openstack@xxxxxxxxxxxxxxxxxxx>
                                        Unsubscribe :
                                        https://launchpad.net/~openstack
                                        <https://launchpad.net/%7Eopenstack>
                                        More help   :
                                        https://help.launchpad.net/ListHelp













_______________________________________________
Mailing list:https://launchpad.net/~openstack
Post to     :openstack@xxxxxxxxxxxxxxxxxxx
Unsubscribe :https://launchpad.net/~openstack
More help   :https://help.launchpad.net/ListHelp



_______________________________________________
Mailing list: https://launchpad.net/~openstack
Post to     : openstack@xxxxxxxxxxxxxxxxxxx
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Follow ups

References