openstack team mailing list archive
-
openstack team
-
Mailing list archive
-
Message #24479
Re: Security Group of Quantum ovs plugin (Folsom) is not working
Hi, Aaron,
Sorry for my unclear explanation.
I can ping or ssh into the VM with default security group even there are no
rules setting...
Here is my security group information,
[root@controller ~]# nova secgroup-list
+---------+-------------+
| Name | Description |
+---------+-------------+
| default | default |
+---------+-------------+
[root@controller ~]# nova secgroup-list-rules default
[root@controller ~]#
After I created a VM with default security group, I checked the iptables at
compute node:
[root@compute1 ~]# iptables -L -v -n
Chain INPUT (policy ACCEPT 26495 packets, 22M bytes)
pkts bytes target prot opt in out source
destination
289 120K nova-compute-INPUT all -- * * 0.0.0.0/0
0.0.0.0/0
0 0 ACCEPT udp -- virbr0 * 0.0.0.0/0
0.0.0.0/0 udp dpt:53
0 0 ACCEPT tcp -- virbr0 * 0.0.0.0/0
0.0.0.0/0 tcp dpt:53
0 0 ACCEPT udp -- virbr0 * 0.0.0.0/0
0.0.0.0/0 udp dpt:67
0 0 ACCEPT tcp -- virbr0 * 0.0.0.0/0
0.0.0.0/0 tcp dpt:67
1036 64284 ACCEPT tcp -- * * 0.0.0.0/0
0.0.0.0/0 tcp dpt:5900
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source
destination
0 0 nova-filter-top all -- * * 0.0.0.0/0
0.0.0.0/0
0 0 nova-compute-FORWARD all -- * * 0.0.0.0/0
0.0.0.0/0
0 0 ACCEPT all -- * virbr0 0.0.0.0/0
192.168.122.0/24 state RELATED,ESTABLISHED
0 0 ACCEPT all -- virbr0 * 192.168.122.0/24
0.0.0.0/0
0 0 ACCEPT all -- virbr0 virbr0 0.0.0.0/0
0.0.0.0/0
0 0 REJECT all -- * virbr0 0.0.0.0/0
0.0.0.0/0 reject-with icmp-port-unreachable
0 0 REJECT all -- virbr0 * 0.0.0.0/0
0.0.0.0/0 reject-with icmp-port-unreachable
Chain OUTPUT (policy ACCEPT 30821 packets, 14M bytes)
pkts bytes target prot opt in out source
destination
30218 14M nova-filter-top all -- * * 0.0.0.0/0
0.0.0.0/0
261 80864 nova-compute-OUTPUT all -- * * 0.0.0.0/0
0.0.0.0/0
Chain nova-compute-FORWARD (1 references)
pkts bytes target prot opt in out source
destination
Chain nova-compute-INPUT (1 references)
pkts bytes target prot opt in out source
destination
Chain nova-compute-OUTPUT (1 references)
pkts bytes target prot opt in out source
destination
Chain nova-compute-inst-783 (1 references)
pkts bytes target prot opt in out source
destination
0 0 DROP all -- * * 0.0.0.0/0
0.0.0.0/0 state INVALID
0 0 ACCEPT all -- * * 0.0.0.0/0
0.0.0.0/0 state RELATED,ESTABLISHED
0 0 nova-compute-provider all -- * * 0.0.0.0/0
0.0.0.0/0
0 0 ACCEPT udp -- * * 30.0.0.2
0.0.0.0/0 udp spt:67 dpt:68
0 0 ACCEPT all -- * * 30.0.0.0/24
0.0.0.0/0
0 0 nova-compute-sg-fallback all -- * * 0.0.0.0/0
0.0.0.0/0
Chain nova-compute-local (1 references)
pkts bytes target prot opt in out source
destination
0 0 nova-compute-inst-783 all -- * * 0.0.0.0/0
30.0.0.5
Chain nova-compute-provider (1 references)
pkts bytes target prot opt in out source
destination
Chain nova-compute-sg-fallback (1 references)
pkts bytes target prot opt in out source
destination
0 0 DROP all -- * * 0.0.0.0/0
0.0.0.0/0
Chain nova-filter-top (2 references)
pkts bytes target prot opt in out source
destination
261 80864 nova-compute-local all -- * * 0.0.0.0/0
0.0.0.0/0
If I add rules to security group default:
[root@controller ~]# nova secgroup-list-rules default
+-------------+-----------+---------+-----------+--------------+
| IP Protocol | From Port | To Port | IP Range | Source Group |
+-------------+-----------+---------+-----------+--------------+
| icmp | -1 | -1 | 0.0.0.0/0 | |
| tcp | 22 | 22 | 0.0.0.0/0 | |
+-------------+-----------+---------+-----------+--------------+
the Chain nova-compute-inst-783 will be :
Chain nova-compute-inst-783 (1 references)
pkts bytes target prot opt in out source
destination
0 0 DROP all -- * * 0.0.0.0/0
0.0.0.0/0 state INVALID
0 0 ACCEPT all -- * * 0.0.0.0/0
0.0.0.0/0 state RELATED,ESTABLISHED
0 0 nova-compute-provider all -- * * 0.0.0.0/0
0.0.0.0/0
0 0 ACCEPT udp -- * * 30.0.0.2
0.0.0.0/0 udp spt:67 dpt:68
0 0 ACCEPT all -- * * 30.0.0.0/24
0.0.0.0/0
* 0 0* ACCEPT tcp -- * * 0.0.0.0/0
0.0.0.0/0 tcp dpt:22
*0 0* ACCEPT icmp -- * * 0.0.0.0/0
0.0.0.0/0
0 0 nova-compute-sg-fallback all -- * * 0.0.0.0/0
0.0.0.0/0
The iptables chain rule can reflect the security group rules correctly but
there are no packets go through this iptables chain rule.
Thanks,
Chandler
2013/6/18 Aaron Rosen <arosen@xxxxxxxxxx>
> Hi,
>
> I think it would also be helpful if you attached the output of:
>
> nova secgroup-list
> then: nova secgroup-list-rules for each group so we could see what rules
> you have set in nova.
>
> Aaron
>
>
> On Mon, Jun 17, 2013 at 6:22 PM, Chandler Li <lichandler116@xxxxxxxxx>wrote:
>
>> Hi Aaron,
>>
>> Thanks for your reply!
>>
>> Yes, I have set /etc/nova/nova.conf as follows, but it seems not working.
>>
>> libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver
>> firewall_driver=nova.virt.libvirt.firewall.IptablesFirewallDriver
>> libvirt_use_virtio_for_bridges=True
>>
>> I can't figure out why network packets didn't follow the rules of
>> iptables created by nova.
>>
>> There are no traffic in FORWARD chain rule and nova-compute-local chain
>> rule as I posted before.
>>
>> Thanks again!
>>
>> Chandler
>>
>>
>>
>> 2013/6/18 Aaron Rosen <arosen@xxxxxxxxxx>
>>
>>> Do you have:
>>>
>>> firewall_driver=nova.virt.firewall.IptablesFirewallDriver
>>>
>>> in your nova.conf? In folsom, quantum leveraged nova security groups
>>> implementation directly so you need that. (looks like you have that set
>>> though by your output).
>>>
>>> Aaron
>>>
>>>
>>>
>>> On Sun, Jun 16, 2013 at 7:38 PM, Chandler Li <lichandler116@xxxxxxxxx>wrote:
>>>
>>>> Hi,
>>>> I checked the compute node's iptables rules and found out the
>>>> nova-compute-inst-xxx have no traffic flow.
>>>> The traffic flow stopped at nova-filter-top chain rule, so security
>>>> group is not working.
>>>> Any idea how to resolve this problem?
>>>>
>>>> Thanks,
>>>> Chandler
>>>>
>>>> [root@compute1 ~]# iptables -L -v -n
>>>> Chain INPUT (policy ACCEPT 714 packets, 335K bytes)
>>>> pkts bytes target prot opt in out source
>>>> destination
>>>> 369 117K nova-compute-INPUT all -- * * 0.0.0.0/0
>>>> 0.0.0.0/0
>>>> 0 0 ACCEPT udp -- virbr0 * 0.0.0.0/0
>>>> 0.0.0.0/0 udp dpt:53
>>>> 0 0 ACCEPT tcp -- virbr0 * 0.0.0.0/0
>>>> 0.0.0.0/0 tcp dpt:53
>>>> 0 0 ACCEPT udp -- virbr0 * 0.0.0.0/0
>>>> 0.0.0.0/0 udp dpt:67
>>>> 0 0 ACCEPT tcp -- virbr0 * 0.0.0.0/0
>>>> 0.0.0.0/0 tcp dpt:67
>>>> 0 0 ACCEPT tcp -- * * 0.0.0.0/0
>>>> 0.0.0.0/0 tcp dpt:5900
>>>>
>>>> Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
>>>> pkts bytes target prot opt in out source
>>>> destination
>>>> 0 0 nova-filter-top all -- * * 0.0.0.0/0
>>>> 0.0.0.0/0
>>>> 0 0 nova-compute-FORWARD all -- * * 0.0.0.0/0
>>>> 0.0.0.0/0
>>>> 0 0 ACCEPT all -- * virbr0 0.0.0.0/0
>>>> 192.168.122.0/24 state RELATED,ESTABLISHED
>>>> 0 0 ACCEPT all -- virbr0 * 192.168.122.0/24
>>>> 0.0.0.0/0
>>>> 0 0 ACCEPT all -- virbr0 virbr0 0.0.0.0/0
>>>> 0.0.0.0/0
>>>> 0 0 REJECT all -- * virbr0 0.0.0.0/0
>>>> 0.0.0.0/0 reject-with icmp-port-unreachable
>>>> 0 0 REJECT all -- virbr0 * 0.0.0.0/0
>>>> 0.0.0.0/0 reject-with icmp-port-unreachable
>>>>
>>>> Chain OUTPUT (policy ACCEPT 779 packets, 378K bytes)
>>>> pkts bytes target prot opt in out source
>>>> destination
>>>> 437 233K nova-filter-top all -- * * 0.0.0.0/0
>>>> 0.0.0.0/0
>>>> 396 216K nova-compute-OUTPUT all -- * * 0.0.0.0/0
>>>> 0.0.0.0/0
>>>>
>>>> Chain nova-compute-FORWARD (1 references)
>>>> pkts bytes target prot opt in out source
>>>> destination
>>>>
>>>> Chain nova-compute-INPUT (1 references)
>>>> pkts bytes target prot opt in out source
>>>> destination
>>>>
>>>> Chain nova-compute-OUTPUT (1 references)
>>>> pkts bytes target prot opt in out source
>>>> destination
>>>>
>>>> Chain nova-compute-inst-767 (1 references)
>>>> pkts bytes target prot opt in out source
>>>> destination
>>>> 0 0 DROP all -- * * 0.0.0.0/0
>>>> 0.0.0.0/0 state INVALID
>>>> 0 0 ACCEPT all -- * * 0.0.0.0/0
>>>> 0.0.0.0/0 state RELATED,ESTABLISHED
>>>> 0 0 nova-compute-provider all -- * * 0.0.0.0/0
>>>> 0.0.0.0/0
>>>> 0 0 ACCEPT udp -- * * 30.0.0.2
>>>> 0.0.0.0/0 udp spt:67 dpt:68
>>>> 0 0 ACCEPT all -- * * 30.0.0.0/24
>>>> 0.0.0.0/0
>>>> 0 0 ACCEPT tcp -- * * 0.0.0.0/0
>>>> 0.0.0.0/0 tcp dpt:22
>>>> 0 0 ACCEPT icmp -- * * 0.0.0.0/0
>>>> 0.0.0.0/0
>>>> 0 0 nova-compute-sg-fallback all -- * * 0.0.0.0/0
>>>> 0.0.0.0/0
>>>>
>>>> Chain nova-compute-local (1 references)
>>>> pkts bytes target prot opt in out source
>>>> destination
>>>> 0 0 nova-compute-inst-767 all -- * * 0.0.0.0/0
>>>> 30.0.0.5
>>>>
>>>> Chain nova-compute-provider (1 references)
>>>> pkts bytes target prot opt in out source
>>>> destination
>>>>
>>>> Chain nova-compute-sg-fallback (1 references)
>>>> pkts bytes target prot opt in out source
>>>> destination
>>>> 0 0 DROP all -- * * 0.0.0.0/0
>>>> 0.0.0.0/0
>>>>
>>>> Chain nova-filter-top (2 references)
>>>> pkts bytes target prot opt in out source
>>>> destination
>>>> 396 216K nova-compute-local all -- * * 0.0.0.0/0
>>>> 0.0.0.0/0
>>>>
>>>>
>>>>
>>>> 2013/6/14 Chandler Li <lichandler116@xxxxxxxxx>
>>>>
>>>>> Hello,
>>>>>
>>>>> I'm trying to use security group of Quantum ovs plugin(Folsom) in
>>>>> CentOS 6.3 (2012.2.3-1.el6@epel).
>>>>>
>>>>> Everything looks good, except security group,
>>>>>
>>>>> and there are no error message in /var/log/nova/compute.log file.
>>>>>
>>>>> After I created VM, I can see the bridges and interfaces have been
>>>>> created normally.
>>>>>
>>>>> [root@compute1 ~]# brctl show
>>>>> bridge name bridge id STP enabled interfaces
>>>>> br-int 0000.3eca2e714b4d no
>>>>> qvo756ead5d-32
>>>>> br-tun 0000.824651aab541 no
>>>>> qbr756ead5d-32 0000.ca57ea41484c no
>>>>> qvb756ead5d-32
>>>>> vnet0
>>>>>
>>>>> The chain rules in filter table of iptables can reflect security group
>>>>> rules correctly too.
>>>>>
>>>>> Chain nova-compute-inst-749 (1 references)
>>>>> num target prot opt source destination
>>>>> 1 DROP all -- 0.0.0.0/0 0.0.0.0/0
>>>>> state INVALID
>>>>> 2 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
>>>>> state RELATED,ESTABLISHED
>>>>> 3 nova-compute-provider all -- 0.0.0.0/0
>>>>> 0.0.0.0/0
>>>>> 4 ACCEPT udp -- 10.0.0.2 0.0.0.0/0
>>>>> udp spt:67 dpt:68
>>>>> 5 ACCEPT all -- 10.0.0.0/24 0.0.0.0/0
>>>>> 6 nova-compute-sg-fallback all -- 0.0.0.0/0
>>>>> 0.0.0.0/0
>>>>>
>>>>> Obviously, the packets do not follow these rules correctly.
>>>>>
>>>>> Please advise me how to resolve this problem.
>>>>>
>>>>> Thanks a lot,
>>>>> Chandler
>>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Mailing list: https://launchpad.net/~openstack
>>>> Post to : openstack@xxxxxxxxxxxxxxxxxxx
>>>> Unsubscribe : https://launchpad.net/~openstack
>>>> More help : https://help.launchpad.net/ListHelp
>>>>
>>>>
>>>
>>
>
Follow ups
References
