Go 构建 请求千亿级微服务实践 项超
100+ 700 万 3000 亿
Goroutine & Channel
Goroutine Channel Goroutine func gen() chan int { out := make(chan int) go func(){ for i:=0; i<100; i++ { out <- i func seq(input chan int) { for num := range input { fmt.println(num) } } } }() return out func main(){ seq(gen()) } }
Goroutine Goroutine Channel Goroutine Goroutine func main(){ in := gen() go seq(in) go sep(in) go seq(in) }
Goroutine Goroutine Channel Goroutine Goroutine func main(){ out := make(chan int) func spout(out chan int) { for i:=0; i<100; i++ { out<-i } go spout(out) go spout(out) go spout(out) seq(out) } }
Problem: To print in ascending order all primes less than 10000. Use an array of processes, SIEVE, in which each process inputs a prime from its predecessor and prints it. The process then inputs an ascending stream of numbers from its predecessor and passes them on to its successor, suppressing any that are multiples of the original prime. Communicating Sequential Processes
10 9 8 7 6 5 4 3 2 2 9 7 5 3 3 7 5 5 7 7
func main() { origin, wait := make(chan int), make(chan struct{}) Processor(origin, wait) for num := 2; num < 10000; num++ { origin <- num } close(origin) <-wait }
func Processor(seq chan int, wait chan struct{}) { go func() { prime, ok := <-seq if!ok { close(wait) return } fmt.println(prime) out := make(chan int) Processor(out, wait) for num := range seq { if num%prime!= 0 { out <- num } } close(out) }() }
Wait Cancel
Server A Server B Requests GW Server C Server D Server E
Wait Processor Processor Wait Processor
Wait wg := sync.waitgroup{} wg.add(3) go func() { defer wg.done() // do... go func() { defer wg.done() // do... }() }() go func() { defer wg.done() // do... }() wg.wait()
Cancel Processor Cancel Processor Processor
Cancel type SIG struct{} exit := make(chan SIG) func Processor(exit chan SIG) { for { select { go Processor(exit) go Processor(exit) go Processor(exit) case <-exit: return default: // do... // Cancel after 1s time.sleep(time.second) close(exit) } } }
User Request User Request User Request User Request User Request User Request User Request User Request Dependency A Dependency B Dependency C Dependency D Dependency E Dependency F Dependency G Dependency H Dependency I Dependency J Dependency K Dependency L Dependency M Dependency N Dependency O Dependency P
Reset ReadTimeout Start Connect Write Read End Reset WriteTimeout 写超时读超时
Real Process Done Event Request Concurrent Ctrl Yes Wait Response No Timeout Clock Timeout Event
Go1.7 context
import ( ) "context" func Handler(r *Request) { timeout := r.value("timeout") ctx, cancel := context.withtimeout(context.background(), timeout) defer cancel() done := make(chan struct{}, 1) go func() { Do(ctx,...) done <- struct{} }() select{ case <-done: // nice... case <-ctx.done(): // timeout... } }
并发超时控制
Go 1 2 3 4 Go 1 2 3 4 Go 1 2 3 4 Go 1 2 3 4
1 2 3 4 1 2 3 4 1 2 3 4 1 2 3 4 Go Go Go Go
GC
SetHistory GetHistoryByRange
SetHistory GetData Decompress Deserialize Serialize Compress SetData GetHistoryByRange Loop GetData Decompress Deserialize
GC Deserialize
Pipeline type Reader interface { } Read(p []byte) (n int, err error) type Writer interface { } Write(p []byte) (n int, err error)
Data Decompress Data Deserialize Data Data Decompress Buffer Deserialize Data Pipeline
Data Decompress Buffer Deserialize Data Pool Data Decompress Deserialize Data
Loop GetData Decompress Deserialize Go GetData Decompress Deserialize Go GetData Decompress Deserialize Go GetData Decompress Deserialize
Thread local Panic,Recover
工程性 Golang Python
import ( "go/parser" "go/token" ) fset := token.newfileset() // positions are relative to fset f, err := parser.parsefile(fset, example.go, nil, parser.allerrors)
Thanks!