openstack team mailing list archive
-
openstack team
-
Mailing list archive
-
Message #24481
Re: Security Group of Quantum ovs plugin (Folsom) is not working
Hi Chandler,
whats your libvirt_vif_driver set in nova-compute.conf?
On Tue, Jun 18, 2013 at 1:08 PM, Chandler Li <lichandler116@xxxxxxxxx>wrote:
> Hi, Aaron,
>
> Sorry for my unclear explanation.
>
> I can ping or ssh into the VM with default security group even there are
> no rules setting...
>
> Here is my security group information,
>
> [root@controller ~]# nova secgroup-list
> +---------+-------------+
> | Name | Description |
> +---------+-------------+
> | default | default |
> +---------+-------------+
> [root@controller ~]# nova secgroup-list-rules default
>
> [root@controller ~]#
>
>
> After I created a VM with default security group, I checked the iptables
> at compute node:
>
> [root@compute1 ~]# iptables -L -v -n
> Chain INPUT (policy ACCEPT 26495 packets, 22M bytes)
> pkts bytes target prot opt in out source
> destination
> 289 120K nova-compute-INPUT all -- * * 0.0.0.0/0
> 0.0.0.0/0
> 0 0 ACCEPT udp -- virbr0 * 0.0.0.0/0
> 0.0.0.0/0 udp dpt:53
> 0 0 ACCEPT tcp -- virbr0 * 0.0.0.0/0
> 0.0.0.0/0 tcp dpt:53
> 0 0 ACCEPT udp -- virbr0 * 0.0.0.0/0
> 0.0.0.0/0 udp dpt:67
> 0 0 ACCEPT tcp -- virbr0 * 0.0.0.0/0
> 0.0.0.0/0 tcp dpt:67
> 1036 64284 ACCEPT tcp -- * * 0.0.0.0/0
> 0.0.0.0/0 tcp dpt:5900
>
> Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
> pkts bytes target prot opt in out source
> destination
> 0 0 nova-filter-top all -- * * 0.0.0.0/0
> 0.0.0.0/0
> 0 0 nova-compute-FORWARD all -- * * 0.0.0.0/0
> 0.0.0.0/0
> 0 0 ACCEPT all -- * virbr0 0.0.0.0/0
> 192.168.122.0/24 state RELATED,ESTABLISHED
> 0 0 ACCEPT all -- virbr0 * 192.168.122.0/24
> 0.0.0.0/0
> 0 0 ACCEPT all -- virbr0 virbr0 0.0.0.0/0
> 0.0.0.0/0
> 0 0 REJECT all -- * virbr0 0.0.0.0/0
> 0.0.0.0/0 reject-with icmp-port-unreachable
> 0 0 REJECT all -- virbr0 * 0.0.0.0/0
> 0.0.0.0/0 reject-with icmp-port-unreachable
>
> Chain OUTPUT (policy ACCEPT 30821 packets, 14M bytes)
> pkts bytes target prot opt in out source
> destination
> 30218 14M nova-filter-top all -- * * 0.0.0.0/0
> 0.0.0.0/0
> 261 80864 nova-compute-OUTPUT all -- * * 0.0.0.0/0
> 0.0.0.0/0
>
> Chain nova-compute-FORWARD (1 references)
> pkts bytes target prot opt in out source
> destination
>
> Chain nova-compute-INPUT (1 references)
> pkts bytes target prot opt in out source
> destination
>
> Chain nova-compute-OUTPUT (1 references)
> pkts bytes target prot opt in out source
> destination
>
> Chain nova-compute-inst-783 (1 references)
> pkts bytes target prot opt in out source
> destination
> 0 0 DROP all -- * * 0.0.0.0/0
> 0.0.0.0/0 state INVALID
> 0 0 ACCEPT all -- * * 0.0.0.0/0
> 0.0.0.0/0 state RELATED,ESTABLISHED
> 0 0 nova-compute-provider all -- * * 0.0.0.0/0
> 0.0.0.0/0
> 0 0 ACCEPT udp -- * * 30.0.0.2
> 0.0.0.0/0 udp spt:67 dpt:68
> 0 0 ACCEPT all -- * * 30.0.0.0/24
> 0.0.0.0/0
> 0 0 nova-compute-sg-fallback all -- * * 0.0.0.0/0
> 0.0.0.0/0
>
> Chain nova-compute-local (1 references)
> pkts bytes target prot opt in out source
> destination
> 0 0 nova-compute-inst-783 all -- * * 0.0.0.0/0
> 30.0.0.5
>
> Chain nova-compute-provider (1 references)
> pkts bytes target prot opt in out source
> destination
>
> Chain nova-compute-sg-fallback (1 references)
> pkts bytes target prot opt in out source
> destination
> 0 0 DROP all -- * * 0.0.0.0/0
> 0.0.0.0/0
>
> Chain nova-filter-top (2 references)
> pkts bytes target prot opt in out source
> destination
> 261 80864 nova-compute-local all -- * * 0.0.0.0/0
> 0.0.0.0/0
>
>
> If I add rules to security group default:
>
> [root@controller ~]# nova secgroup-list-rules default
> +-------------+-----------+---------+-----------+--------------+
> | IP Protocol | From Port | To Port | IP Range | Source Group |
> +-------------+-----------+---------+-----------+--------------+
> | icmp | -1 | -1 | 0.0.0.0/0 | |
> | tcp | 22 | 22 | 0.0.0.0/0 | |
> +-------------+-----------+---------+-----------+--------------+
>
>
> the Chain nova-compute-inst-783 will be :
>
> Chain nova-compute-inst-783 (1 references)
> pkts bytes target prot opt in out source
> destination
> 0 0 DROP all -- * * 0.0.0.0/0
> 0.0.0.0/0 state INVALID
> 0 0 ACCEPT all -- * * 0.0.0.0/0
> 0.0.0.0/0 state RELATED,ESTABLISHED
> 0 0 nova-compute-provider all -- * * 0.0.0.0/0
> 0.0.0.0/0
> 0 0 ACCEPT udp -- * * 30.0.0.2
> 0.0.0.0/0 udp spt:67 dpt:68
> 0 0 ACCEPT all -- * * 30.0.0.0/24
> 0.0.0.0/0
> * 0 0* ACCEPT tcp -- * * 0.0.0.0/0
> 0.0.0.0/0 tcp dpt:22
> *0 0* ACCEPT icmp -- * * 0.0.0.0/0
> 0.0.0.0/0
> 0 0 nova-compute-sg-fallback all -- * * 0.0.0.0/0
> 0.0.0.0/0
>
>
> The iptables chain rule can reflect the security group rules correctly but
> there are no packets go through this iptables chain rule.
>
> Thanks,
> Chandler
>
>
>
> 2013/6/18 Aaron Rosen <arosen@xxxxxxxxxx>
>
>> Hi,
>>
>> I think it would also be helpful if you attached the output of:
>>
>> nova secgroup-list
>> then: nova secgroup-list-rules for each group so we could see what rules
>> you have set in nova.
>>
>> Aaron
>>
>>
>> On Mon, Jun 17, 2013 at 6:22 PM, Chandler Li <lichandler116@xxxxxxxxx>wrote:
>>
>>> Hi Aaron,
>>>
>>> Thanks for your reply!
>>>
>>> Yes, I have set /etc/nova/nova.conf as follows, but it seems not working.
>>>
>>> libvirt_vif_driver=nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver
>>> firewall_driver=nova.virt.libvirt.firewall.IptablesFirewallDriver
>>> libvirt_use_virtio_for_bridges=True
>>>
>>> I can't figure out why network packets didn't follow the rules of
>>> iptables created by nova.
>>>
>>> There are no traffic in FORWARD chain rule and nova-compute-local chain
>>> rule as I posted before.
>>>
>>> Thanks again!
>>>
>>> Chandler
>>>
>>>
>>>
>>> 2013/6/18 Aaron Rosen <arosen@xxxxxxxxxx>
>>>
>>>> Do you have:
>>>>
>>>> firewall_driver=nova.virt.firewall.IptablesFirewallDriver
>>>>
>>>> in your nova.conf? In folsom, quantum leveraged nova security groups
>>>> implementation directly so you need that. (looks like you have that set
>>>> though by your output).
>>>>
>>>> Aaron
>>>>
>>>>
>>>>
>>>> On Sun, Jun 16, 2013 at 7:38 PM, Chandler Li <lichandler116@xxxxxxxxx>wrote:
>>>>
>>>>> Hi,
>>>>> I checked the compute node's iptables rules and found out the
>>>>> nova-compute-inst-xxx have no traffic flow.
>>>>> The traffic flow stopped at nova-filter-top chain rule, so security
>>>>> group is not working.
>>>>> Any idea how to resolve this problem?
>>>>>
>>>>> Thanks,
>>>>> Chandler
>>>>>
>>>>> [root@compute1 ~]# iptables -L -v -n
>>>>> Chain INPUT (policy ACCEPT 714 packets, 335K bytes)
>>>>> pkts bytes target prot opt in out source
>>>>> destination
>>>>> 369 117K nova-compute-INPUT all -- * * 0.0.0.0/0
>>>>> 0.0.0.0/0
>>>>> 0 0 ACCEPT udp -- virbr0 * 0.0.0.0/0
>>>>> 0.0.0.0/0 udp dpt:53
>>>>> 0 0 ACCEPT tcp -- virbr0 * 0.0.0.0/0
>>>>> 0.0.0.0/0 tcp dpt:53
>>>>> 0 0 ACCEPT udp -- virbr0 * 0.0.0.0/0
>>>>> 0.0.0.0/0 udp dpt:67
>>>>> 0 0 ACCEPT tcp -- virbr0 * 0.0.0.0/0
>>>>> 0.0.0.0/0 tcp dpt:67
>>>>> 0 0 ACCEPT tcp -- * * 0.0.0.0/0
>>>>> 0.0.0.0/0 tcp dpt:5900
>>>>>
>>>>> Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
>>>>> pkts bytes target prot opt in out source
>>>>> destination
>>>>> 0 0 nova-filter-top all -- * * 0.0.0.0/0
>>>>> 0.0.0.0/0
>>>>> 0 0 nova-compute-FORWARD all -- * * 0.0.0.0/0
>>>>> 0.0.0.0/0
>>>>> 0 0 ACCEPT all -- * virbr0 0.0.0.0/0
>>>>> 192.168.122.0/24 state RELATED,ESTABLISHED
>>>>> 0 0 ACCEPT all -- virbr0 * 192.168.122.0/24
>>>>> 0.0.0.0/0
>>>>> 0 0 ACCEPT all -- virbr0 virbr0 0.0.0.0/0
>>>>> 0.0.0.0/0
>>>>> 0 0 REJECT all -- * virbr0 0.0.0.0/0
>>>>> 0.0.0.0/0 reject-with icmp-port-unreachable
>>>>> 0 0 REJECT all -- virbr0 * 0.0.0.0/0
>>>>> 0.0.0.0/0 reject-with icmp-port-unreachable
>>>>>
>>>>> Chain OUTPUT (policy ACCEPT 779 packets, 378K bytes)
>>>>> pkts bytes target prot opt in out source
>>>>> destination
>>>>> 437 233K nova-filter-top all -- * * 0.0.0.0/0
>>>>> 0.0.0.0/0
>>>>> 396 216K nova-compute-OUTPUT all -- * * 0.0.0.0/0
>>>>> 0.0.0.0/0
>>>>>
>>>>> Chain nova-compute-FORWARD (1 references)
>>>>> pkts bytes target prot opt in out source
>>>>> destination
>>>>>
>>>>> Chain nova-compute-INPUT (1 references)
>>>>> pkts bytes target prot opt in out source
>>>>> destination
>>>>>
>>>>> Chain nova-compute-OUTPUT (1 references)
>>>>> pkts bytes target prot opt in out source
>>>>> destination
>>>>>
>>>>> Chain nova-compute-inst-767 (1 references)
>>>>> pkts bytes target prot opt in out source
>>>>> destination
>>>>> 0 0 DROP all -- * * 0.0.0.0/0
>>>>> 0.0.0.0/0 state INVALID
>>>>> 0 0 ACCEPT all -- * * 0.0.0.0/0
>>>>> 0.0.0.0/0 state RELATED,ESTABLISHED
>>>>> 0 0 nova-compute-provider all -- * * 0.0.0.0/0
>>>>> 0.0.0.0/0
>>>>> 0 0 ACCEPT udp -- * * 30.0.0.2
>>>>> 0.0.0.0/0 udp spt:67 dpt:68
>>>>> 0 0 ACCEPT all -- * * 30.0.0.0/24
>>>>> 0.0.0.0/0
>>>>> 0 0 ACCEPT tcp -- * * 0.0.0.0/0
>>>>> 0.0.0.0/0 tcp dpt:22
>>>>> 0 0 ACCEPT icmp -- * * 0.0.0.0/0
>>>>> 0.0.0.0/0
>>>>> 0 0 nova-compute-sg-fallback all -- * *
>>>>> 0.0.0.0/0 0.0.0.0/0
>>>>>
>>>>> Chain nova-compute-local (1 references)
>>>>> pkts bytes target prot opt in out source
>>>>> destination
>>>>> 0 0 nova-compute-inst-767 all -- * * 0.0.0.0/0 30.0.0.5
>>>>>
>>>>> Chain nova-compute-provider (1 references)
>>>>> pkts bytes target prot opt in out source
>>>>> destination
>>>>>
>>>>> Chain nova-compute-sg-fallback (1 references)
>>>>> pkts bytes target prot opt in out source
>>>>> destination
>>>>> 0 0 DROP all -- * * 0.0.0.0/0
>>>>> 0.0.0.0/0
>>>>>
>>>>> Chain nova-filter-top (2 references)
>>>>> pkts bytes target prot opt in out source
>>>>> destination
>>>>> 396 216K nova-compute-local all -- * * 0.0.0.0/0
>>>>> 0.0.0.0/0
>>>>>
>>>>>
>>>>>
>>>>> 2013/6/14 Chandler Li <lichandler116@xxxxxxxxx>
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I'm trying to use security group of Quantum ovs plugin(Folsom) in
>>>>>> CentOS 6.3 (2012.2.3-1.el6@epel).
>>>>>>
>>>>>> Everything looks good, except security group,
>>>>>>
>>>>>> and there are no error message in /var/log/nova/compute.log file.
>>>>>>
>>>>>> After I created VM, I can see the bridges and interfaces have been
>>>>>> created normally.
>>>>>>
>>>>>> [root@compute1 ~]# brctl show
>>>>>> bridge name bridge id STP enabled
>>>>>> interfaces
>>>>>> br-int 0000.3eca2e714b4d no
>>>>>> qvo756ead5d-32
>>>>>> br-tun 0000.824651aab541 no
>>>>>> qbr756ead5d-32 0000.ca57ea41484c no
>>>>>> qvb756ead5d-32
>>>>>> vnet0
>>>>>>
>>>>>> The chain rules in filter table of iptables can reflect security
>>>>>> group rules correctly too.
>>>>>>
>>>>>> Chain nova-compute-inst-749 (1 references)
>>>>>> num target prot opt source destination
>>>>>> 1 DROP all -- 0.0.0.0/0 0.0.0.0/0
>>>>>> state INVALID
>>>>>> 2 ACCEPT all -- 0.0.0.0/0 0.0.0.0/0
>>>>>> state RELATED,ESTABLISHED
>>>>>> 3 nova-compute-provider all -- 0.0.0.0/0
>>>>>> 0.0.0.0/0
>>>>>> 4 ACCEPT udp -- 10.0.0.2 0.0.0.0/0
>>>>>> udp spt:67 dpt:68
>>>>>> 5 ACCEPT all -- 10.0.0.0/24 0.0.0.0/0
>>>>>> 6 nova-compute-sg-fallback all -- 0.0.0.0/0
>>>>>> 0.0.0.0/0
>>>>>>
>>>>>> Obviously, the packets do not follow these rules correctly.
>>>>>>
>>>>>> Please advise me how to resolve this problem.
>>>>>>
>>>>>> Thanks a lot,
>>>>>> Chandler
>>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Mailing list: https://launchpad.net/~openstack
>>>>> Post to : openstack@xxxxxxxxxxxxxxxxxxx
>>>>> Unsubscribe : https://launchpad.net/~openstack
>>>>> More help : https://help.launchpad.net/ListHelp
>>>>>
>>>>>
>>>>
>>>
>>
>
> _______________________________________________
> Mailing list: https://launchpad.net/~openstack
> Post to : openstack@xxxxxxxxxxxxxxxxxxx
> Unsubscribe : https://launchpad.net/~openstack
> More help : https://help.launchpad.net/ListHelp
>
>
--
Regds,
Ashok ,
Delivery Consultant,
HP.
Follow ups
References
