Go构建日请求千亿微服务最佳实践的副本

Size: px

Start display at page:

Download "Go构建日请求千亿微服务最佳实践的副本"

絺色汲
4 years ago
Views:

1 Go 构建请求千亿级微服务实践项超

4 万 3000 亿

7 Goroutine & Channel

8 Goroutine Channel Goroutine func gen() chan int { out := make(chan int) go func(){ for i:=0; i<100; i++ { out <- i func seq(input chan int) { for num := range input { fmt.println(num) } } } }() return out func main(){ seq(gen()) } }

9 Goroutine Goroutine Channel Goroutine Goroutine func main(){ in := gen() go seq(in) go sep(in) go seq(in) }

10 Goroutine Goroutine Channel Goroutine Goroutine func main(){ out := make(chan int) func spout(out chan int) { for i:=0; i<100; i++ { out<-i } go spout(out) go spout(out) go spout(out) seq(out) } }

11 Problem: To print in ascending order all primes less than Use an array of processes, SIEVE, in which each process inputs a prime from its predecessor and prints it. The process then inputs an ascending stream of numbers from its predecessor and passes them on to its successor, suppressing any that are multiples of the original prime. Communicating Sequential Processes

13 func main() { origin, wait := make(chan int), make(chan struct{}) Processor(origin, wait) for num := 2; num < 10000; num++ { origin <- num } close(origin) <-wait }

14 func Processor(seq chan int, wait chan struct{}) { go func() { prime, ok := <-seq if!ok { close(wait) return } fmt.println(prime) out := make(chan int) Processor(out, wait) for num := range seq { if num%prime!= 0 { out <- num } } close(out) }() }

15 Wait Cancel

16 Server A Server B Requests GW Server C Server D Server E

17 Wait Processor Processor Wait Processor

18 Wait wg := sync.waitgroup{} wg.add(3) go func() { defer wg.done() // do... go func() { defer wg.done() // do... }() }() go func() { defer wg.done() // do... }() wg.wait()

19 Cancel Processor Cancel Processor Processor

20 Cancel type SIG struct{} exit := make(chan SIG) func Processor(exit chan SIG) { for { select { go Processor(exit) go Processor(exit) go Processor(exit) case <-exit: return default: // do... // Cancel after 1s time.sleep(time.second) close(exit) } } }

21 User Request User Request User Request User Request User Request User Request User Request User Request Dependency A Dependency B Dependency C Dependency D Dependency E Dependency F Dependency G Dependency H Dependency I Dependency J Dependency K Dependency L Dependency M Dependency N Dependency O Dependency P

22 Reset ReadTimeout Start Connect Write Read End Reset WriteTimeout 写超时读超时

23 Real Process Done Event Request Concurrent Ctrl Yes Wait Response No Timeout Clock Timeout Event

24 Go1.7 context

25 import ( ) "context" func Handler(r *Request) { timeout := r.value("timeout") ctx, cancel := context.withtimeout(context.background(), timeout) defer cancel() done := make(chan struct{}, 1) go func() { Do(ctx,...) done <- struct{} }() select{ case <-done: // nice... case <-ctx.done(): // timeout... } }

27 并发超时控制

28 Go Go Go Go

29 Go Go Go Go

34 GC

35 SetHistory GetHistoryByRange

36 SetHistory GetData Decompress Deserialize Serialize Compress SetData GetHistoryByRange Loop GetData Decompress Deserialize

37 GC Deserialize

38 Pipeline type Reader interface { } Read(p []byte) (n int, err error) type Writer interface { } Write(p []byte) (n int, err error)

39 Data Decompress Data Deserialize Data Data Decompress Buffer Deserialize Data Pipeline

40 Data Decompress Buffer Deserialize Data Pool Data Decompress Deserialize Data

41 Loop GetData Decompress Deserialize Go GetData Decompress Deserialize Go GetData Decompress Deserialize Go GetData Decompress Deserialize

46 Thread local Panic,Recover

49 工程性 Golang Python

50 import ( "go/parser" "go/token" ) fset := token.newfileset() // positions are relative to fset f, err := parser.parsefile(fset, example.go, nil, parser.allerrors)

52 Thanks!

Microsoft Word - 201110.doc

Microsoft Word - 201110.doc 2011 年 10 月信徒交通月刊目錄一本期目錄編輯室 1 二牧者的話教會轉化 -- 得到更新皮袋衣立凡 2 三講章精華清潔的心思 -- 除去論斷講員衣立凡 / 賴美如整理 4 清潔的心思 -- 除去情慾講員葉志偉 / 林慶如整理 9 四精選文章等候