openstack team mailing list archive

[David Kang <dkang@xxxxxxx>]

 Thanks, Brian.
My answers are put in your email with "-->".

 David

----- Original Message -----
> On 07/24/2013 10:42 AM, David Kang wrote:
> >
> >   If I remove the following REJECT rules, it works perfectly.
> > -A INPUT -j REJECT --reject-with icmp-host-prohibited
> > -A FORWARD -j REJECT --reject-with icmp-host-prohibited
> >
> >  With them, it looks like that the packets are dropped at the bridge
> >  before they can be forwarded.
> 
> So I'll keep asking - why can't you just remove them? It gets you
> running and
> if you're just kicking the tires it's a valid workaround.
> 

--> My sponsor STRONGLY wants to have the rules for security purpose.

> > I ran the iptables commands recommended by RedHat.
> >
> > When I ping 10.12.182.13 from a VM (192.168.3.3),
> > I cannot see any packets from qr-32411859-c0,
> > but I can see packets are dropped at brqf56b3f53-d3.
> > The outputs of tcpdump is shown below.
> >
> > $ brctl show
> > bridge name bridge id STP enabled interfaces
> > brq69f480ab-06 8000.001e675ba339 no eth2.82
> >                                                         tapd8bd73c9-3a
> > brqf56b3f53-d3 8000.001e675ba338 no eth1.2001
> >                                                         tap32411859-c0
> >                                                         tapfa6a1d01-16
> > $ route -n
> > Kernel IP routing table
> > Destination Gateway Genmask Flags Metric Ref Use Iface
> > 192.168.3.0 0.0.0.0 255.255.255.0 U 0 0 0 ns-fa6a1d01-16
> > 192.168.3.0 0.0.0.0 255.255.255.0 U 0 0 0 qr-32411859-c0
> 
> Overlapping IP ranges? That could be a problem.

--> Those are generated by quantum-linuxbridge-agent.
  If a quantum network is associated to a quantum l3 router, qr-xxx interface is added.

> 
> > 10.12.182.0 0.0.0.0 255.255.255.0 U 0 0 0 eth2.182
> > 10.12.82.0 0.0.0.0 255.255.255.0 U 0 0 0 qg-d8bd73c9-3a
> > 0.0.0.0 10.12.82.1 0.0.0.0 UG 0 0 0 qg-d8bd73c9-3a
> 
> Why is your default route going out this interface and not eth2.182?

--> I didn't show it. Another default router of eth2.182 also exist below 10.12.82.1.
  Quantum automatically made qg-d8bd73c9-3a as a default router.
  It is the interface to the gateway of the "external network" where
  floating IP is assigned.

> 
> > $ tcpdump -i qr-32411859-c0 -nn
> >    // nothing special
> 
> What about ns-fa6a1d01-16? That overlapping IP range looks suspicious.

--> It was made by quantum-linux-bridge.

> 
> > $ tcpdump -i brqf56b3f53-d3 -nn icmp
> > tcpdump: WARNING: brqf56b3f53-d3: no IPv4 address assigned
> > tcpdump: verbose output suppressed, use -v or -vv for full protocol
> > decode
> > listening on brqf56b3f53-d3, link-type EN10MB (Ethernet), capture
> > size 65535 bytes
> > 13:48:46.892785 IP 192.168.3.3 > 10.12.182.13: ICMP echo request, id
> > 46605, seq 1855, length 64
> > 13:48:46.892825 IP 192.168.3.2 > 192.168.3.3: ICMP host 10.12.182.13
> > unreachable - admin prohibited, length 92
> 
> This is the reject iptables rule firing, so those other rules are not
> matching.
> You need to look at 'iptables -L -v -n -x' to see if their packet/byte
> counts
> are increasing or not. If not, start using things like 'ip route get
> $dest' to
> figure out what interfaces the kernel is using for output, which will
> help you
> fix those rules to be correct.

--> Thanks, I will try it.

> 
> -Brian
> 
> > ----- Original Message -----
> >> On 07/23/2013 11:41 PM, David Kang wrote:
> >>>
> >>>  A Redhat manual suggests the following rule to enable forwarding
> >>>  packets
> >>> among VMs and external network.
> >>> https://access.redhat.com/site/documentation/en-US/Red_Hat_OpenStack/2/pdf/Release_Notes/Red_Hat_OpenStack-2-Release_Notes-en-US.pdf
> >>>
> >>> iptables -t filter -I FORWARD -i qr-+ -o qg-+ -j ACCEPT
> >>> iptables -t filter -I FORWARD -i qg-+ -o qr-+ -j ACCEPT
> >>> iptables -t filter -I FORWARD -i qr-+ -o qr-+ -j ACCEPT
> >>>
> >>>  But it doesn't work for me.
> >>
> >> Can you elaborate on what "it doesn't work" means?
> >>
> >> Do any of those rules show increased packet/byte counts, indicating
> >> they've been
> >> matched?
> >>
> >> Is IP forwarding enabled?
> >>
> >> Is there a mis-configuration in your bridge config? Use 'brctl
> >> show'
> >> to see
> >> where all the tap and other devices are attached.
> >>
> >> Deleting that one FORWARD rule causing all the trouble is going to
> >> be
> >> a much
> >> quicker solution.
> >>
> >> -Brian
> >>
> >>> ----- Original Message -----
> >>>> On 07/23/2013 12:22 PM, David Kang wrote:
> >>>>>
> >>>>>  Hi,
> >>>>>
> >>>>>   We are running OpenStack Folsom on CentOS 6.4.
> >>>>> Quantum-linuxbridge-agent is used.
> >>>>> By default, the Quantum node has the following entries in its
> >>>>> /etc/sysconfig/iptables file.
> >>>>>
> >>>>> -A INPUT -j REJECT --reject-with icmp-host-prohibited
> >>>>> -A FORWARD -j REJECT --reject-with icmp-host-prohibited
> >>>>>
> >>>>>  With those two lines, VM cannot get IP address from the DHCP
> >>>>>  server
> >>>>>  running on the Quantum node.
> >>>>> More specifically, the first line prevents a VM from getting IP
> >>>>> address from DHCP server.
> >>>>> The second line prevents a VM from talking to other VMs and
> >>>>> external
> >>>>> worlds.
> >>>>> Is there a better way to make the Quantum network work well
> >>>>> than just commenting them out?
> >>>>
> >>>> Since Quantum isn't adding them, and you want the system to act
> >>>> as
> >>>> a
> >>>> DHCP server
> >>>> and gateway, I think you have two choices:
> >>>>
> >>>> 1. Delete them
> >>>> 2. Craft rules to sit above them (using -I) to allow certain
> >>>> packets
> >>>>
> >>>> #2 gets tricky as you'll need to account for DHCP, metadata, etc.
> >>>> in
> >>>> the INPUT
> >>>> chain, and in the FORWARD chain you could maybe start by allowing
> >>>> all
> >>>> traffic
> >>>> from your bridge. You would need to do some more work there.
> >>>>
> >>>> I believe any DHCP iptables rules will be on the compute hosts,
> >>>> and
> >>>> will be put
> >>>> in place for anti-spoofing. Since this is the network node you
> >>>> won't
> >>>> see them here.
> >>>>
> >>>> -Brian
> >>>
> >

-- 
----------------------
Dr. Dong-In "David" Kang
Computer Scientist
USC/ISI