untitled

Size: px

Start display at page:

Download "untitled"

丑宦
5 years ago
Views:

2 SDN 2

3 Core Elements of a Data Center Applications Databases Database Management System (DBMS) and the physical and logical storage of data Servers/Operating Systems Networks Storage Arrays 3

4 An Example Client Local Area Network Storage Area Network Storage Array Application User Interface Server Database Consider an order processing system consisting of: Application for order entry. Database Management System (DBMS) to store customer and product information. Server/Operating System on which the Application and Database programs are run. Networks that provide Connectivity between Clients and the Application/Database Server Connectivity between the Server and the Storage system. Storage Array

5 An Example..Closer Look Client Local Area Network Server Storage Area Network Storage Array Application User Interface A customer order is entered via the Application User Interface on a client.

6 An Example..Closer Look Client Local Area Network Server Storage Area Network Storage Array A customer order is entered via the Application User Interface on a client The client accesses the Server over a Local Area Network.

7 An Example..Closer Look Client Local Area Network Server Storage Area Network Storage Array O/S and DBMS A DBMS uses the operating system on the server to read and write this data to the physical location on a disk.

8 An Example..Closer Look Client Local Area Network Server Storage Area Network Storage Array A DBMS uses the operating system on the server to read and write this data to the physical location on disk. A Network provides the communication link between the server and the storage array, and transports the read/write commands and data between the server and the storage array.

9 An Example..Closer Look Client Local Area Network Server Storage Area Network Storage Array Database A DBMS uses the operating system on the server to read and write this data to the physical location on disk. A Network provides the communication link between the client and the server, and transports the read/write commands and data between the server and the storage array. A storage array receives the read/write commands and data from the server and performs the necessary operations to store the data on the physical disks.

11 Google Data Center Google Dalles

12 at Lenoir, North Carolina treetview/

13 Gallery

14 IBM Data Center

15 Equinix Data Center

19 Elements of a Data Center The Site Command Center Cable Management Network Infrastructure Terminal Servers Environmental Controls Power

20 Standards 1

21 Standard 2

Their must be a pathway for equipment to be moved in & out of the

23 Structural Aspects a raised floor ceiling Basement data center locations near water are not a good idea. Their must be a pathway for equipment to be moved in & out of the data center. Make sure the floor where the data center is to be located is rated for the estimated load.

24 UPS

25 HVAC (Heating, Ventilation, and Air Conditioning)

26 Hot Aisle and Cold Aisle Layout

27 Networking Core Layer Switch/Router: Layer 2 Switch: Cisco Core Layer Router DLink 48ports Gbps Layer-2 Switch

28 Security Physical Access Levels of Access Monitoring

29 Past to Future a single computer occupied the space of an entire Data Center. That same space can be occupied by thousands of servers today.

31 (Tower) (Server Rack) : (Blade)

32 Rack Unit (RU) Unit=44.45mm (height) Width=19 inch or 23 inch 1U, 2U, 4U (half rack) 1U 4U

33 Server Rack and Cluster

34 Data Center Topology Core Internet Layer-3 router Data Center Aggregation Layer-2/3 switch Access Top of Rack Switch Layer-2 switch Servers

35 Top of Rack (ToR) Switch

43 Data Center Costs James Hamilton published basic 2008 breakdown Servers: 45% CPU, memory, disk Infrastructure: 25% UPS, cooling, power distribution Power draw: 15% Electrical utility costs Network: 15% Switches, links, transit

44 Power Cost Approximate distribution of peak power usage by hardware subsystem in one of Google s datacenters (circa 2007).

45 Data Center Challenges Traffic load balance Support for VM migration Achieving bisection bandwidth Power savings / Cooling Network management (provisioning) Security (dealing with multiple tenants)

46 Oversubscription

49 What is Virtualization? Run multiple OSes and user applications on the same hardware e.g., run both Windows and Linux on the same laptop How is it different from dual boot? Both OSes run simultaneously OSes are completely isolated from each other

53 Reduce costs by consolidating services onto the fewest number of physical machines

54 Non Virtualized Data Centers Too many servers for too little work High costs and infrastructure needs Maintenance Networking Floor space Cooling Power Disaster Recovery

55 Dynamic Data Center Virtualization helps us break the one service per server model Consolidate many services into a fewer number of machines when workload is low, reducing costs Conversely, as demand for a particular service increases, we can shift more virtual machines to run that service We can build a data center with fewer total resources, since resources are used as needed instead of being dedicated to single services

56 VM Workload Multiplexing Separate VM sizing VM multiplexing s 3 s 2 s 1 We expect s 3 < s 1 + s 2. Benefit of multiplexing! Multiplex VMs workload on same physical server

57 So, it is just like Java VM, right?

59 Two Types of Hypervisors (or VMM) Hypervisor is a software layer that allows several VMs to run on a physical machine The physical OS and hardware are called the Host VM OS and applications are called the Guest Type 1 (bare metal) Type 2 (hosted) Process Process Process Process VM1 VM2 Guest Guest VM1 VM2 Process Hypervisor Host Hypervisor Hardware OS Hardware Host VMware ESX, Microsoft Hyper V, Xen VMware Workstation, Microsoft Virtual PC, Sun VirtualBox, QEMU, KVM

60 Bare metal or Hosted? Bare metal Has complete control over hardware Doesn t have to fight an OS Hosted Avoid code duplication: need not code a process scheduler, memory management system the OS already does that Can run native processes alongside VMs Familiar environment how much CPU and memory does a VM take? Use top! How big is the virtual disk? ls l Easy management stop a VM? Sure, just kill it! A combination Mostly hosted, but some parts are inside the OS kernel for performance reasons, e.g., KVM

61 VM on Multi core CPUs Each core can be configured for multiple VMs A Quad core CPU could be configured as a 32 node multi computer Limiting factor is often memory. Each guest OS has its own requirements (512 MB?) 61

62 Installing a Virtual machine Base OS is Windows 7 First install Sun VirtualBox as hypervisor Then, Guest OS will be Ubuntu

63 Installing Sun/Oracle VirtualBox 63

64 Installing VirtualBox 64

65 Installing VirtualBox 65

66 Installing VirtualBox 66

67 Installing Ubuntu VM 67

68 Installing Ubuntu VM 68

69 Installing Ubuntu VM 69

70 Installing Ubuntu VM 70

71 71

72 Installing Ubuntu VM 72

73 73

74 Installing Ubuntu VM 74

75 Data Center Topology Core Internet Layer-3 router Data Center Aggregation Layer-2/3 switch Access Top of Rack Switch Layer-2 switch Servers

78 Data Center Network Traffic Analysis 78

79 Motivation Flow based traffic monitoring Volume of processed data is reduced Popular flow statistics tools Cisco NetFlow sflow

80 What is sflow? Accurate sampling is simple enough to be performed in hardware, at wire speed. Good accuracy under heavy loads Detailed complete packet header and switching/routing information for L2 L7 traffic flows. Scalable capable of monitoring networks at 10Gbps, 100Gbps and beyond. Thousands of devices can be monitored by a single sflow Collector. Low Cost sflow Agent is very simple to implement and adds negligible cost to a switch or router. Timely an up to the minute view, for real time controls. QoS and DoS attack.

81 sflow Architecture

sflow in Operation Switch/Router sflow Datagram forwarding tables interface counters Switching ASIC sflow agent 1 in N sampling packet header src/dst i/f sampling parms

82 sflow in Operation Switch/Router sflow Datagram forwarding tables interface counters Switching ASIC sflow agent 1 in N sampling packet header src/dst i/f sampling parms forwarding user ID URL i/f counters eg 128B rate pool src 802.1p/Q dst 802.1p/Q next hop src/dst mask AS path communities localpref src/dst Radius TACACS sflow Collector & Analyzer

83 Statistical Model for Packet Sampling Estimating Traffic per Protocol Total number of frames = N Total number of samples = n Number of samples in class = c Number of frames in the class estimated by: N c = c n N Relative Sampling Error 100% % Error 75% 50% %error c 25% 0% Number of Samples in Class

84 Even so, Data Volume is Huge! Flow data in our campus network ( /16 prefix ) # of Routers 1 Day 1 Month 1 Year GB 13 GB 156 GB 5 6 GB 65 GB 780 GB GB 130 GB 1.5 TB GB 2.6 TB 30 TB Short term period of flow data Massive flow data from anomaly traffic data of Internet worm and DDoS Cluster file system and cloud computing platform Google s programming model, MapReduce, big table Open source system, Hadoop 84

85 An Experiment: Travel Booking Website 2 BNT switches

86 Traditional Flow based Traffic Monitoring Run on a high performance central server Routers Flow Data Storage High Performance Server 86

87 Occupancy of Different Network Service Types Heavy Hitters Workload

88 Offline Spike Analysis

89 Google System Health

90 Count Min Sketch (1/3) Consider a vector a (dimension n), which is presented in an implicit, incremental fashion. Current state at time t: [ ] at ( ) = a1 ( t),..., ai( t),..., an( t) a(0) = 0, i = 1,..., n Updates to individual entries of the vector are presented as a stream of pairs. The t th update is ( i, c ) t t Question: how can we approximate the true value of a data stream, e.g. application of heavy hitters

91 Count Min Sketch (2/3) Data structure: Accuracy parameters Pr a$ i () t ai () t + ε a() t δ, i = 1,..., n 1 represented by a two dimensional array counts with width depth d = lnδ 10 d hash functions: w= e/ ε 2000 h1... hd : {1... n} {1...w} chosen uniformly at random from a pairwise independent family Key Space cost: - an array: (wd*4) Bytes - d hash functions, each of which can be stored using 2 words when using the pairwise functions.

92 Count Min Sketch (3/3) Stage 1: Count CM t+ 1 j, hj( a ) i CM t j, hj( ai) + ci( t) where: j = 1,2,..., d i = 1,2,..., n Stage 2: Min (or Point Query at time t) return an approximation at $ ( ) = a$ $ $ 1( t),..., ai( t),..., an( t) min a$ () t min CM j, h ( a () t ) = i t j i j= 1,2,..., d where: i = 1, 2,..., n Why minimum? hashing collision overestimation!

93 CM with Conservative Update CM sketch always overestimates the true value! Enhanced algorithm: Count Min Conservative Update Update a counter, if and only if, the current value of the counter is smaller than the estimated value added by the new count. new value of the counter CM ( $ ) t+ 1, ( ) max, ( ), i j hj a i CM t j hj a i a ( t) + ci( t) where: $ i a () t = min CM t j, hj( ai) j= 1,2..., d at ( ) = a1 ( t),..., ai( t),..., an( t) [ ] estimated value current value of the counter

94 Time Complexity Analysis: CM&CU CM: O(nd) CU: O(2nd) n n>m w w d 94

95 CU a CU sketch a CU 1 a CM 0<a<1 CU, CM O((1+a)nd) 95

96 CU Zipfian z z sketch z sketch 96

97 CU sketch sketch z sketch CU w P(w) CU d P(d) CU 97

98 CU P(d) CU CU w d CU sketch K sketch n K sketch sketch K P(d) CU CM P(w) CU CU 98

99 CPU

100 L T L<T T L sketch sketch P(d) CU sketch L (1) sketch γ (2) 100

101 Example 1 I wanna know currently which interfaces are highly overloaded across the entire DC List out the top K router/switch interfaces with workload ranking Key: Hash_if(routerIP, if_id) (routerip>>16 + routerip + if_id) % tablesize; Sketch size 4KB: width 256, depth 4 Unit: KB Index Router_IP If_ID Esti_allocts Real_allocts Real_sendocts Real_recvocts Abs_Error Error_rate % % % % % % % % % %

102 Example 2 Currently which set of flows carries most traffic across the entire DC List out the top K source destination IP pairs with its workload ranking Key: Hash_ipp(src_IP, dest_ip) a = (src_ip * dest_ip), (a>>32+a) % tablesize; Sketch size 65KB: width: 1024, depth: 16 Unit: Byte Index Src_IP Dest_IP Esti_allocts Real_allocts Real_sendocts Real_recvocts Abs_Error Error_rate % % % % % % % % % %

103 Example 3 which set of flows cause the interface congestion at a particular switch List out the top K source destination IP pairs "per interface" with its workload ranking Key: Hash_ifipp(routerIP, if_id, src_ip, dest_ip) (Hash_ipp(src_ip,dst_ip) + Hash_if(router_ip,rif_id)) % tablesize; Sketch size 131KB: width: 2048, depth: 16 Unit: KB Index Router_IP If_ID Src_IP Dst_IP Esti_alloct Real_alloct Real_sendoct Real_recvoct Abs_Error Error_rate % % % % % % % % % %

104 Implementations (source/destination IP/MAC addresses, source/destination ports, packet length, port of the switch/router, connection type) 104

105 Result ARE: average relative error sketch P(d) CU CM CM CU P(w) CU CU 105

106 Result 106

107 Software Defined Networking (SDN) What is SDN? Background An OS for networks What is OpenFlow? How it helps SDN The current status & the future of SDN Conclusions 10 7

108 Limitations of Current Networks Switches 108

We have lost our way Routing, management, mobility management, access

5400 RFCs Barrier to entry Specialized Packet Forwarding Hardware 500M

the infrastructure OSPF, BGP, multicast, differentiated services, Traffic

109 We have lost our way Routing, management, mobility management, access control, VPNs, App App App Operating System Million of lines of source code 5400 RFCs Barrier to entry Specialized Packet Forwarding Hardware 500M gates 10Gbytes RAM Bloated Power Hungry Many complex functions baked into the infrastructure OSPF, BGP, multicast, differentiated services, Traffic Engineering, NAT, firewalls, MPLS, redundant layers, An industry with a mainframe mentality

110 Reality App App App Operating System Specialized Packet Forwarding Hardware App App Operating System App Specialized Packet Forwarding Hardware Lack of competition means glacial innovation Closed architecture means blurry, closed interfaces Vertically integrated, complex, closed, proprietary Not suitable for experimental ideas Not good for network owners & users Not good for researchers

111 Glacial process of innovation made worse by captive standards process Idea Standardize Deployment Wait 10 years Driven by vendors Consumers largely locked out Lowest common denominator features Glacial innovation

112 Old Ways to Configure a Network App App App Operating System Specialized Packet Forwarding Hardware App App App Operating System App App App Operating System Specialized Packet Forwarding Hardware App App App Operating System Specialized Packet Forwarding Hardware App App App Specialized Packet Forwarding Hardware Operating System Specialized Packet Forwarding Hardware 112

113 No control plane abstraction for the whole network! It s like old times when there was no OS 11 3 Wilkes with the EDSAC, 1949

114 Idea: An OS for Networks!!! App App App Closed Operating System Specialized Packet Forwarding Hardware App App App Operating System App App App Operating System Specialized Packet Forwarding Hardware App App App Operating System Specialized Packet Forwarding Hardware App App App Specialized Packet Forwarding Hardware Operating System Specialized Packet Forwarding Hardware

115 Idea: An OS for Networks Control Programs Network Operating System App App App Operating System Specialized Packet Forwarding Hardware App App App Operating System App App App Operating System Specialized Packet Forwarding Hardware App App App Operating System Specialized Packet Forwarding Hardware App App App Specialized Packet Forwarding Hardware Operating System Specialized Packet Forwarding Hardware 115

116 Idea: An OS for Networks Control Programs Network Operating System Simple Packet Forwarding Hardware Simple Packet Forwarding Hardware Simple Packet Forwarding Hardware Simple Packet Forwarding Hardware Simple Packet Forwarding Hardware 116

117 Idea: An OS for Networks NOX: Towards an Operating System for Networks Software Defined Networking (SDN) Control Programs Global Network View Network Operating System Control via forwarding interface Protocols Protocols 117

118 Trend App App App App App App Windows Windows Windows (OS) (OS) (OS) Linux Linux Linux Mac Mac Mac OS OS OS NOX Controller Controller 1 1 (Network OS) Controller Controller Network 22 OS Virtualization layer x86 (Computer) Virtualization or Slicing OpenFlow Computer Industry Network Industry

119

120 Outline What is SDN? Limitations of current networks The idea of Network OS What is OpenFlow? How it helps SDN The current status & the future of SDN Conclusions 12 0

121 OpenFlow OpenFlow: Enabling Innovation in Campus Networks Like hardware drivers interface between switches and Network OS 121

122 OpenFlow Control Path (Software) Data Path (Hardware) 122

123 OpenFlow OpenFlow Controller OpenFlow Protocol (SSL/TCP) Control Path OpenFlow Data Path (Hardware) 123

124 OpenFlow Basics Control Program A Control Program B Network OS OpenFlow Protocol SSL TCP IBM 10 gigabit ethernet OpenFlow switch G8264, which has GbE SFP+ ports and 4 40 GbE QSFP+ ports

Dst TCP sport TCP dport Action * * * 5.6.7.

125 OpenFlow Switching Software Layer Hardware Layer MAC src MAC dst OpenFlow Client IP Src OpenFlow Table IP Dst TCP sport TCP dport Action * * * * * port 1 Controller PC port 1 port 2 port 3 port

126 Flow Table Control Program A Control Program B Network OS Packet Forwarding If header = p, send to port 4 If header = q, overwrite header with r, add header s, and send to ports 5,6 If header =?, send to me Packet Forwarding Flow Table(s) Packet Forwarding

127 Flow Table Flow 1. Rule (exact & wildcard) Action Statistics Flow 2. Rule (exact & wildcard) Action Statistics Flow 3. Rule (exact & wildcard) Action Statistics Flow N. Rule (exact & wildcard) Default Action Statistics

128 Flow Entry Match fields Match against packets Action Modify the action set or pipeline processing Stats Update the matching packets Match Fields Action Stats In Port Src MAC Dst MAC Eth Type Vlan Id IP Tos IP Proto IP Src IP Dst TCP Src Port TCP Dst Port Layer 2 Layer 3 Layer 4 1. Forward packet to port(s) 2. Encapsulate and forward to controller 3. Drop packet 4. Send to normal processing pipeline 1. Packet 2. Byte counters

129 Examples Switching Switch Port MAC src MAC dst Eth type VLAN ID IP Src IP Dst IP Prot TCP sport TCP dport Action * * 00:1f:.. * * * * * * * port6 Flow Switching Switch Port MAC src MAC dst Eth type VLAN ID IP Src IP Dst IP Prot TCP sport TCP dport Action port3 00: :1f vlan port6 Firewall Switch Port MAC src MAC dst Eth type VLAN ID IP Src IP Dst IP Prot TCP sport TCP dport Action * * * * * * * * * 22 drop

130 Examples Routing Switch Port MAC src MAC dst Eth type VLAN ID IP Src IP Dst IP Prot TCP sport TCP dport Action * * * * * * * * * port6 VLAN Switching Switch Port * MAC src MAC dst Eth type VLAN ID IP Src IP Dst IP Prot TCP sport * 00:1f.. * vlan1 * * * * * TCP dport Action port6, port7, port9

131 OpenFlow Usage OpenFlow Alice s Switch Rule Alice s code Controller PC Decision? OpenFlow Protocol Alice s OpenFlow Rule Switch Alice s OpenFlow Rule Switch 13 1

132 OpenFlow Usage» Alice s code: Simple learning switch Per Flow switching Network access control/firewall Static VLANs Her own new routing protocol: unicast, multicast, multipath Home network manager Packet processor (in controller) IPvAlice Alice s code Controller PC 13 2

133 OpenFlow Standard way to control flow tables in commercial switches and routers Just need to update firmware Essential to the implementation of SDN 133

Controller Controller OpenFlow Switch OpenFlow

134 Centralized/Distributed Control Onix: A Distributed Control Platform for Large scale Production Networks, UNIX OSDI Centralized Control Distributed Control Controller Controller OpenFlow Switch OpenFlow Switch Controller OpenFlow Switch OpenFlow Switch OpenFlow Switch OpenFlow Switch

135 Virtualizing OpenFlow Network operators Delegate control of subsets of network hardware and/or traffic to other network operators or users Multiple controllers can talk to the same set of switches Imagine a hypervisor for network equipments Allow experiments to be run on the network in isolation of each other and production traffic

136 Switch Based Virtualization Research VLAN 2 Research VLAN 1 Production VLANs Flow Table Flow Table Normal L2/L3 Processing Controller Controller 136

137 FlowVisor A network hypervisor developed by Stanford A software proxy between the forwarding and control planes of network devices

138 FlowVisor based Virtualization Topology discovery is per slice Aaron s Controller Heidi s Controller Craig s Controller OpenFlow Protocol OpenFlow Switch OpenFlow FlowVisor & Policy Control OpenFlow Protocol OpenFlow Switch OpenFlow Switch 138

139 Outline What is SDN? Limitations of current networks The idea of Network OS What is OpenFlow? How it helps SDN The current status & the future of SDN Conclusions 13 9

140 OpenFlow Building Blocks oftrace oflops openseer Monitoring/ debugging tools Stanford Provided ENVI (GUI) LAVI n Casting Expedient Applications NOX Beacon Trema Maestro ONIX Controller FlowVisor Console FlowVisor Slicing Software Commercial Switches HP, NEC, Pronto, Juniper.. and many more Stanford Provided Software NetFPGA Ref. Switch OpenWRT PCEngine WiFi AP Broadcom Ref. Switch Open vswitch OpenFlow Switches 140

141 Current status of SDN Hardware support Juniper MX series NEC IP8800 WiMax (NEC) HP Procurve 5400 Netgear 7324 PC Engines Pronto 3240/3290 Ciena Coredirector More coming soon

142 Current status of SDN Industry support Google built hardware and software based on the OpenFlow protocol VMware purchased Nicira for $1.26 billion in 2012 IBM, HP, NEC, Cisco and Juniper also are offering SDNs that may incorporate OpenFlow, but also have other elements that are specific to that vendor and their gear.

143 Future Focuses of SDN New policies for security Programmable WLANs The placement of controllers (amount; location; centralized/distributed) Debugger for SDN 14 3

144 Conclusions What is SDN? A system layered abstraction Programmable, flexible, and extensible What is OpenFlow? Interface between switches and controllers Enabling SDN Future SDN Enabling innovation 14 4

145 Wireless Data Center Networking 60 GHz spectrum 7 GHz (57 64 GHz) waveband data rate over 1 Gb/s 10 meters Line of Sight (LoS)

Microsoft PowerPoint - Aqua-Sim.pptx

Microsoft PowerPoint - Aqua-Sim.pptx Peng Xie, Zhong Zhou, Zheng Peng, Hai Yan, Tiansi Hu, Jun-Hong Cui, Zhijie Shi, Yunsi Fei, Shengli Zhou Underwater Sensor Network Lab 1 Outline Motivations System Overview Aqua-Sim Components Experimental