About me 北京上海无锡湖州 openstack 私有云解决方案和实施 openstack 公有云解决方案和实施 openstack / vmware 培训

Size: px

Start display at page:

Download "About me 北京上海无锡湖州 openstack 私有云解决方案和实施 openstack 公有云解决方案和实施 openstack / vmware 培训"

坦位蔡
5 years ago
Views:

1 Neutron DVR 技术分析上海 meetup 九州云信息科技有限公司

2 About me 北京上海无锡湖州 openstack 私有云解决方案和实施 openstack 公有云解决方案和实施 openstack / vmware 培训

3 Content problem fixed by DVR, distributed virtual router key trick to implement DVR related features or skills L2pop,arp proxy,flow tables and namespace sample traffic flow analysis

4 Content problem fixed by DVR, distributed virtual router key trick to implement DVR related features or skills L2pop,arp proxy,flow tables and namespace sample traffic flow analysis

5 problems to solve 流量 l3 router 东西走向, 即使是同一个计算节点的不同子网之间南北走向造成的问题单点故障拥堵 subnet1 subnet2

6 problems to solve 流量 l3 router 1:N 1 : 1 东西走向, 同计算节点的虚拟机互通, 跨计算节点也是两者直接互通 1:1 subnet1 subnet2 南北走向, 1 : 1 和第三个结点没关系, 1 : N 还是需要走向第三节点 l3 router l3 router

7 Content problem fixed by DVR, distributed virtual router key trick to implement DVR related features or skills L2pop,arp proxy,flow tables and namespace sample traffic flow analysis

8 关键技术网关 mac 地址伪装把 macb 转成 dvr mac1 maca qr1 qr2 macb br-int 4 5 vm1 9 把 dvr mac2 转成 maca 把 maca 转成 dvr mac2 macb qr2 qr1 maca br-int vm2 6 把 dvr mac1 转成 macb DVR MAC1 DVR MAC2 第一步发出的 arp request 是不能跨越计算节点的, 网关是不能对任何地址做跨节点 arp request 的, 所以需要在 qr 设备所在的 namespace 插入所有虚拟机的 arp entry

9 limit DVR requirements You must use the ML2 plug-in for Open VSwitch (OVS) to enable DVR. Be sure that your firewall or security groups allows UDP traffic over the VLAN, GRE, or VXLAN port to pass between the compute hosts. DVR limitations 1.Distributed virtual router configurations work with the Open vswitch Modular Layer 2 driver only for Juno. 2.In order to enable true north-south bandwidth between hypervisors (compute nodes), you must use public IP addresses for every compute node and enable floating IPs. 3.For now, based on the current neutron design and architecture, DHCP cannot become distributed across compute nodes. 4.one subnet can be connected to DVR router only once, which is confirmed by BP 5.one network cannot have more than one subnets connecting to DVR routers 6.only tunnel is fine now, vlan mode is in progress +spec/neutron-ovs-dvr-vlan 7.since the DVR cannot do arp request across compute nodes, the 'address pair' cannot work

10 Content problem fixed by DVR, distributed virtual router key trick to implement DVR related features or skills L2pop,arp proxy,flow tables and namespace sample traffic flow analysis

11 l2population 相关知识 ML2 的一个 mechanism driver before l2population, n*(n-1) tunnels broadcast 到没关系的节点上去

12 l2 population 管理 tunnel 管理 tunnel 的建立, 动态建立 tunnel 减少广播域

13 ARP responder 本地实现 VM arp proxy VM.4 br-int VM.5 br-int arp req

14 vm br-int 1 patch in port open flow tables table 20 table 10 LEARN_FROM_TUN table 1 DVR_PROCESS table 2 UCAST_TO_TUN dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 table 9 DVR_NOT_LEARN dl_src= 对方节点的 DVR mac in_port=2 table 4 VXLAN_TUN_TO_LV table 0 PATCH_LV_TO_TUN arp,dl_dst=ff:ff:ff:ff:ff:ff in_port=1 dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 table 22 FLOOD_TO_TUN table 21 ARP_RESPONDER ovs bridge 2 tunnel port

15 vm 10 patch tun port ovs bridge br-int table 0 LOCAL_SWITC HING in_port=10 mac=dvr MAC else normal table 1 DVR_TO_SRC_MAC vm1 fa:16:3e:7e:e5:41 vm2 fa:16:3e:7e:e5:42 qr-xx fa:16:3e:a1:f7:6a cookie=0x0, duration= s, table=1, n_packets=0, n_bytes=0, idle_age=27707, priority=4,dl_vlan=1,dl_dst=fa:16:3e:7e:e5:41 actions=strip_vlan,mod_dl_src:fa:16:3e:a1:f7:6a,output:5

16 需要公网 IP 虚拟机网络 tagged br-ex NS:snat-<rid> qg-<12 of snat intr portid per rt> sg-<12 of snat portid per subnet>.2 NS:fip-<netid> fg-<12 of gw portid per node > fpr-<12 of rid> rfp-<12of rid> veth br-ex VM br-int.1.6 NS:fip-<netid> rfp-<12of rid> qr-<gw portid> fg-<12 of gw portid per node > fpr-<12of rid> veth 1 NS:qrouter-<rid> VM.4 br-int.1 qr-<12 of gw portid> NS:qrouter-<rid> 多少个计算节点, 就要有多少个公网 IP 耗掉, 每个 router 还要有一个公网 IP 耗掉

17 Content problem fixed by DVR, distributed virtual router key trick to implement DVR related features or skills L2pop,arp proxy,flow tables and namespace sample traffic flow analysis

18 同网络同节点虚拟机 vm.5 ping vm.6 当 sudo ovs-appctl fdb/show br-int 没有目标机的条目时, br-int table0 的 normal action 会进行广播, 但是进入 tunnel 的 ARP 包会没有音信 (arp responder 只对外部的虚拟机有条目 ), 目标机 vm 回应后, 这时 fdb entry 就有了后面的 vm.5 到 vm.6 的流量不会在隧道上过改进 : 可以在 br-int 桥上过滤掉针对本地端口 arp ping 包去 tunnel 桥 VM VM br-int.6 1 vxlan-ac105b01 gongysh@fedora20:/mnt/data/gongysh$ sudo ovs-appctl fdb/show br-int port VLAN MAC Age 6 1 fa:16:3e:7c:73: fa:16:3e:6f:53:07 1

19 同网络不同节点 VM.4 br-int 1 VM.6 br-int arp 2 vm traffic 3 vm.4 ping vm.6 当 sudo ovs-appctl fdb/show br-int 没有目标机的条目时, br-int table0 的 normal action 会进行广播, 进入 tunnel 的 ARP request 会被 arp proxy 处理, 然后 vm.4 直接和目标机对话

20 不同网络同节点虚拟机由于 qrouter-<rid> 中有各个虚机的 arp 条目, 所以不需要到上做 arp proxy 1 VM 4 qr d VM br-int.1.4 qr-2312a96e-82 2 NS:qrouter-<rid>

21 不同网络不同节点虚拟机由于 qrouter-<rid> 中有各个虚机的 arp 条目, 所以不需要到上做 arp proxy 1 qr d qr d VM br-int.1 VM br-int.1.4 qr-2312a96e-82.6 qr-2312a96e-82 NS:qrouter-<rid> NS:qrouter-<rid> 4

22 permanent arp entry $ sudo ip netns exec qrouter-9d19202e-8f82-4b53-ac8c-3cf6e998b0fe arp Address HWtype HWaddress Flags Mask Iface ether fa:16:3e:6f:53:07 CM qr-2312a96e ether fa:16:3e:7d:08:30 CM qr-2312a96e (incomplete) qr d ether fa:16:3e:e9:7b:65 CM qr-2312a96e ether fa:16:3e:bf:60:f4 CM qr-2312a96e ether fa:16:3e:2b:f2:ae CM qr d ether fa:16:3e:07:03:37 CM qr-2312a96e ether fa:16:3e:f8:9b:c2 CM qr d ether fa:16:3e:d1:49:06 CM qr-2312a96e ether fa:16:3e:38:18:bd CM qr d ether fa:16:3e:9f:f5:ae CM qr-2312a96e ether fa:16:3e:33:0e:e6 CM qr d ether fa:16:3e:7e:e5:41 CM qr-2312a96e ether ba:5d:a1:c5:29:e1 C rfp-9d19202e ether fa:16:3e:7c:73:95 CM qr-2312a96e-82 how much entries can arp table contain? can we do without the static arp population and use the arp responder on?

23 5 br-ex 4 NS:fip-<netid> fg-<12 of gw portid per node > 1:1 FIP fpr-<12 of rid> rfp-<12of rid> VM br-int.1 veth qr-<12 of gw portid> 1 NS:qrouter-<rid> 2 $sudo ip netns exec qrouter-xxx ip rule list : from lookup : from /24 lookup $ sudo ip netns exec qrouter-xxx ip r l table 16 default via dev rfp-9d19202e-8 $ sudo ip netns exec fip-xxx ip r default via dev fg-e203c0ca /24 dev fg-e203c0ca-77 proto kernel scope link src via dev fpr-9d19202e /31 dev fpr-9d19202e-8 proto kernel scope link src

24 4 3 1:N snat br-ex NS:snat-<rid> qg-<12 of snat intr portid per rt> veth $sudo ip netns exec qrouter-xxx ip rule list : from lookup : from /24 lookup sg-<12 of snat portid per subnet> rfp-<12of rid> $ sudo ip netns exec qrouter-xxx ip r l table default via dev qr-2312a96e-82 M.5 br-int 1.1 qr-<12 of gw portid> NS:qrouter-<rid> $ sudo ip netns exec snat-xxx iptables-save... *nat... -A neutron-l3-agent-postrouting! -i qg-c62a504f-4d! -o qg-c62a504f-4d -m conntrack! --ctstate DNAT -j ACCEPT -A neutron-l3-agent-snat -o qg-c62a504f-4d -j SNAT --to-source

25 Thanks

幻灯片 1

幻灯片 1 SDN 和分布式网络虚拟化设计 QQ : 67278439 新浪微博 :@ 盛科张卫峰 Topic SDN 对云计算网络的价值 SDN 网络虚拟化方案一览从 OVN 看分布式网络虚拟化设计基于硬件 SDN 交换机的网络虚拟化 SDN 是一种思想 SDN 不是一种具体的