openstack team mailing list archive

[David Kang <dkang@xxxxxxx>]

 It is strange.
The node is only for Quantum-{linuxbridge, dhcp, l3}-agent.
As far as I know, the quantum private network that is not associated with a quantum router
has only ns-xxx interface.
The quantum private network otherwise have both ns-xxx and qr-xxx interfaces.

 Thanks,
 David

----- Original Message -----
> Just some more notes.
> 
> It looks like you're running this system as both a network node and
> compute
> node, I think the pdf you found from Redhat assumed the system was a
> dedicated
> network node, i.e. it only had qr- and qg- interfaces, and not ns- as
> created by
> plug() when an instance is booted.
> 
> Multiple routes for the same destination, going out two different
> interfaces not
> connected to the same network, are going to cause you trouble. It's
> non-deterministic where a packet will go without ip rules.
> 
> I'm going to let you go and debug this some more on your own, as it
> looks like
> it's your iptables config causing it, you just need to get the correct
> rules in
> there.
> 
> -Brian
> 
> On 07/24/2013 11:34 AM, David Kang wrote:
> >
> >  Thanks, Brian.
> > My answers are put in your email with "-->".
> >
> >  David
> >
> > ----- Original Message -----
> >> On 07/24/2013 10:42 AM, David Kang wrote:
> >>>
> >>>   If I remove the following REJECT rules, it works perfectly.
> >>> -A INPUT -j REJECT --reject-with icmp-host-prohibited
> >>> -A FORWARD -j REJECT --reject-with icmp-host-prohibited
> >>>
> >>>  With them, it looks like that the packets are dropped at the
> >>>  bridge
> >>>  before they can be forwarded.
> >>
> >> So I'll keep asking - why can't you just remove them? It gets you
> >> running and
> >> if you're just kicking the tires it's a valid workaround.
> >>
> >
> > --> My sponsor STRONGLY wants to have the rules for security
> > purpose.
> >
> >>> I ran the iptables commands recommended by RedHat.
> >>>
> >>> When I ping 10.12.182.13 from a VM (192.168.3.3),
> >>> I cannot see any packets from qr-32411859-c0,
> >>> but I can see packets are dropped at brqf56b3f53-d3.
> >>> The outputs of tcpdump is shown below.
> >>>
> >>> $ brctl show
> >>> bridge name bridge id STP enabled interfaces
> >>> brq69f480ab-06 8000.001e675ba339 no eth2.82
> >>>                                                         tapd8bd73c9-3a
> >>> brqf56b3f53-d3 8000.001e675ba338 no eth1.2001
> >>>                                                         tap32411859-c0
> >>>                                                         tapfa6a1d01-16
> >>> $ route -n
> >>> Kernel IP routing table
> >>> Destination Gateway Genmask Flags Metric Ref Use Iface
> >>> 192.168.3.0 0.0.0.0 255.255.255.0 U 0 0 0 ns-fa6a1d01-16
> >>> 192.168.3.0 0.0.0.0 255.255.255.0 U 0 0 0 qr-32411859-c0
> >>
> >> Overlapping IP ranges? That could be a problem.
> >
> > --> Those are generated by quantum-linuxbridge-agent.
> >   If a quantum network is associated to a quantum l3 router, qr-xxx
> >   interface is added.
> >
> >>
> >>> 10.12.182.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2.182
> >>> 10.12.82.0 0.0.0.0 255.255.255.0 U 0 0 0 qg-d8bd73c9-3a
> >>> 0.0.0.0 10.12.82.1 0.0.0.0 UG 0 0 0 qg-d8bd73c9-3a
> >>
> >> Why is your default route going out this interface and not
> >> eth2.182?
> >
> > --> I didn't show it. Another default router of eth2.182 also exist
> > below 10.12.82.1.
> >   Quantum automatically made qg-d8bd73c9-3a as a default router.
> >   It is the interface to the gateway of the "external network" where
> >   floating IP is assigned.
> >
> >>
> >>> $ tcpdump -i qr-32411859-c0 -nn
> >>>    // nothing special
> >>
> >> What about ns-fa6a1d01-16? That overlapping IP range looks
> >> suspicious.
> >
> > --> It was made by quantum-linux-bridge.
> >
> >>
> >>> $ tcpdump -i brqf56b3f53-d3 -nn icmp
> >>> tcpdump: WARNING: brqf56b3f53-d3: no IPv4 address assigned
> >>> tcpdump: verbose output suppressed, use -v or -vv for full
> >>> protocol
> >>> decode
> >>> listening on brqf56b3f53-d3, link-type EN10MB (Ethernet), capture
> >>> size 65535 bytes
> >>> 13:48:46.892785 IP 192.168.3.3 > 10.12.182.13: ICMP echo request,
> >>> id
> >>> 46605, seq 1855, length 64
> >>> 13:48:46.892825 IP 192.168.3.2 > 192.168.3.3: ICMP host
> >>> 10.12.182.13
> >>> unreachable - admin prohibited, length 92
> >>
> >> This is the reject iptables rule firing, so those other rules are
> >> not
> >> matching.
> >> You need to look at 'iptables -L -v -n -x' to see if their
> >> packet/byte
> >> counts
> >> are increasing or not. If not, start using things like 'ip route
> >> get
> >> $dest' to
> >> figure out what interfaces the kernel is using for output, which
> >> will
> >> help you
> >> fix those rules to be correct.
> >
> > --> Thanks, I will try it.
> >
> >>
> >> -Brian
> >>
> >>> ----- Original Message -----
> >>>> On 07/23/2013 11:41 PM, David Kang wrote:
> >>>>>
> >>>>>  A Redhat manual suggests the following rule to enable
> >>>>>  forwarding
> >>>>>  packets
> >>>>> among VMs and external network.
> >>>>> https://access.redhat.com/site/documentation/en-US/Red_Hat_OpenStack/2/pdf/Release_Notes/Red_Hat_OpenStack-2-Release_Notes-en-US.pdf
> >>>>>
> >>>>> iptables -t filter -I FORWARD -i qr-+ -o qg-+ -j ACCEPT
> >>>>> iptables -t filter -I FORWARD -i qg-+ -o qr-+ -j ACCEPT
> >>>>> iptables -t filter -I FORWARD -i qr-+ -o qr-+ -j ACCEPT
> >>>>>
> >>>>>  But it doesn't work for me.
> >>>>
> >>>> Can you elaborate on what "it doesn't work" means?
> >>>>
> >>>> Do any of those rules show increased packet/byte counts,
> >>>> indicating
> >>>> they've been
> >>>> matched?
> >>>>
> >>>> Is IP forwarding enabled?
> >>>>
> >>>> Is there a mis-configuration in your bridge config? Use 'brctl
> >>>> show'
> >>>> to see
> >>>> where all the tap and other devices are attached.
> >>>>
> >>>> Deleting that one FORWARD rule causing all the trouble is going
> >>>> to
> >>>> be
> >>>> a much
> >>>> quicker solution.
> >>>>
> >>>> -Brian
> >>>>
> >>>>> ----- Original Message -----
> >>>>>> On 07/23/2013 12:22 PM, David Kang wrote:
> >>>>>>>
> >>>>>>>  Hi,
> >>>>>>>
> >>>>>>>   We are running OpenStack Folsom on CentOS 6.4.
> >>>>>>> Quantum-linuxbridge-agent is used.
> >>>>>>> By default, the Quantum node has the following entries in its
> >>>>>>> /etc/sysconfig/iptables file.
> >>>>>>>
> >>>>>>> -A INPUT -j REJECT --reject-with icmp-host-prohibited
> >>>>>>> -A FORWARD -j REJECT --reject-with icmp-host-prohibited
> >>>>>>>
> >>>>>>>  With those two lines, VM cannot get IP address from the DHCP
> >>>>>>>  server
> >>>>>>>  running on the Quantum node.
> >>>>>>> More specifically, the first line prevents a VM from getting
> >>>>>>> IP
> >>>>>>> address from DHCP server.
> >>>>>>> The second line prevents a VM from talking to other VMs and
> >>>>>>> external
> >>>>>>> worlds.
> >>>>>>> Is there a better way to make the Quantum network work well
> >>>>>>> than just commenting them out?
> >>>>>>
> >>>>>> Since Quantum isn't adding them, and you want the system to act
> >>>>>> as
> >>>>>> a
> >>>>>> DHCP server
> >>>>>> and gateway, I think you have two choices:
> >>>>>>
> >>>>>> 1. Delete them
> >>>>>> 2. Craft rules to sit above them (using -I) to allow certain
> >>>>>> packets
> >>>>>>
> >>>>>> #2 gets tricky as you'll need to account for DHCP, metadata,
> >>>>>> etc.
> >>>>>> in
> >>>>>> the INPUT
> >>>>>> chain, and in the FORWARD chain you could maybe start by
> >>>>>> allowing
> >>>>>> all
> >>>>>> traffic
> >>>>>> from your bridge. You would need to do some more work there.
> >>>>>>
> >>>>>> I believe any DHCP iptables rules will be on the compute hosts,
> >>>>>> and
> >>>>>> will be put
> >>>>>> in place for anti-spoofing. Since this is the network node you
> >>>>>> won't
> >>>>>> see them here.
> >>>>>>
> >>>>>> -Brian
> >>>>>
> >>>
> >

-- 
----------------------
Dr. Dong-In "David" Kang
Computer Scientist
USC/ISI