QCon北京2018-Saga-姜宁.key

Size: px

Start display at page:

Download "QCon北京2018-Saga-姜宁.key"

药华
5 years ago
Views:

1 Saga 分布式事务解决方案与实践演讲者 / 姜宁

2 关于我华为开源能力力中心 ServiceComb 项目负责人 Apache Member, IPMC, 多个 Apache 项目 RedHat, IONA, Travelsky

3 议题微服务事务一致性问题? 业界 Saga 的解决方案 ServiceComb Saga 的演进后续的开发计划

4 微服务架构微服务架构将一个应用分成多个相互独立的服务好处是各个服务能够持续独立的开发和部署难题是服务的数据需要采用什什么样的方式来进行行存储?

5 多个微服务使用同一数据库 A B C commit rollback

6 一个典型的微服务架构应用 Browser HTML Front UI Customer Service Customer Database REST REST EdgeService Order Service Mobile Device Inventory Service Order Database Inventory Database

7 两阶段提交 2PC Transaction Manager RM 1 RM 2 提供强一致保障 prepare() OK prepare() OK 准备阶段完成资源操作如果准备过程中出现问题, 可以回滚提交阶段不不允许出错 commit() OK commit() OK 资源层面提供保障业务侵入性低协议成本高, 并且存在全局锁的问题

8 ACID 与 BASE ACID ( 刚性事务 ) BASE ( 柔性事务 ) 原子性 (Atomicity) 基本可用 (Basically Available) 一致性 (Consistency) 柔性状态 (Soft state) 隔离性 ( Isolation) 最终一致性 (Eventually Consistent) 持久性 (Durability)

9 TCC Try: 尝试执行行业务 try confirm cancel 完成所有业务检查, 预留留必须的业务资源 Confirm: 确认执行行业务业务服务真正执行行业务, 不不做业务检查 Cancel: 取消执行行业务释放 Try 阶段预留留的业务资源

10 通过领域建模来解决

11 微服务事务一致性建议内刚微服务内 : 聚合通过数据库事务保证强一致外柔微服务间 : 最终一致

12 Saga 简介 1987 年年 Hector & Kenneth 发表论文 Sagas Saga = Long Live Transaction (LLT) LLT = T1 + T2 + T Tn 每个本地事务 Tx 有对应的补偿 Cx T1 T2 T3... Tn C1 C2 C3... Cn T1 T2 T3... Tn 正常情况 T1 T2 T3 C3 C2 C1 异常情况

13 业界 Saga 的研究应用情况 Caitie McCaffrey Distributed Sagas Chris Richardson Microservice saga pattern

14 Saga Started Flight Started Flight Ended Hotel Started Hotel Ended Car Started Car Ended Payment Started Payment Ended Saga Ended request Saga StartSaga EndSaga

15 Saga Started request Flight Started Flight Ended StartSaga Hotel Started Hotel Aborted Flight Compensated Saga Saga Ended Transaction Rollback EndSaga

16 对服务的要求幂等 T = T T T time saga car rental T T'

17 对服务的要求幂等 T = T T T 可交换补偿 T C = T C T time saga car rental time saga car rental T T T' T' C 保留留所有事务数据!

18 ACID 与 Saga ACID Saga 只提供 ACD 保证原子性 (Atomicity) 原子性 ( 通过 Saga 协调器器实现 ) 一致性 (Consistency) 一致性 ( 本地事务 + Saga log) 隔离性 ( Isolation) 隔离性 (Saga 不不保证 ) 持久性 (Durability) 持久性 (Saga log)

19 缺乏隔离性带来的问题两个 Saga 事务同时操作一个资源会出现数据语义不不一致的的情况两个 Saga 事务同时操作一个订单, 彼此操作会覆盖对方 ( 更更新丢失 ) 两个 Saga 事务同时访问扣款账号, 无法看到退款 ( 脏读取问题 ) 在一个 Saga 事务内, 数据被其他事务修改前后的读取值不不一致 ( 模糊读取问题 )

20 如何应对隔离性问题隔离的本质是控制并发, 防止并发事务操作相同资源而引起结果错乱在应用层面加入逻辑锁的逻辑 Session 层面隔离来保证串串行行化操作业务层面采用预先冻结资金金的方式隔离此部分资金金业务操作过程中通过及时读取当前状态的方式获取更更新

21 Saga 的实现方式集中式的实现方式 (Orchestration-based coordination) 集中式协调器器负责服务调用以及事务协调分布式的实现方式 (Choreography-based coordination) 通过事件驱动的方式来进行行事务协调

Order Service create() OrderService Order state total 集中式 Saga 实现 //transaction invocation action from("direct:creditreservation") Customer Service.saga().propagation(SagaPropagation.SUPPORTS).

22 Order Service create() OrderService Order state total 集中式 Saga 实现 //transaction invocation action from("direct:creditreservation") Customer Service.saga().propagation(SagaPropagation.SUPPORTS).option("CreditId", body()) // mark the current body as needed in the compensating action.compensation("direct:creditrefund").bean(creditservice, create() "reservecredit") reservecredit().log("credit ${header.amount} reserved. Custom Id used creditlimit is $ {body}"); CreateOrder Saga creditreserved() CustomerService // called only if the saga is cancelled from("direct:creditrefund").transform(header("creditid")) // retrieve the CreditId option from headers.bean(creditservice, "refundcredit").log("credit for Custom Id ${body} refunded"); creditreservate

23 分布式 Saga 实现 Order created Create Order Order Service Credit Reserved Customer Service create() OR reservecredit() approve()/reject() state total Order Credit Limit Exceeded Customer creditlimit creditreservations

24 Service Center Java Chassis 让云原生应用开发更更简单代码 : 网站 : 华为云 : Saga

25 ServiceComb Saga 演进 Saga Log Saga Started Service A Caller params { T1:[a,b], C1:[c,d],... } Saga Coordinator config { T1: name, path, C1:name, path,... } Saga ID: Saga x Saga T1 Started T1 Ended T2 Started Transaction Viewer Omega Service B Omega Alpha Alpha Alpha Dynamic Config Aggregator Service Registry 集中式的 Saga 协调器器分布式 Saga 协调器器

26 集中式 Saga 协调器器 Saga Log Saga Started Caller params { T1:[a,b], C1:[c,d],... } Saga Coordinator config { T1: name, path, C1:name, path,... } Saga ID: Saga x Saga T1 Started T1 Ended T2 Started Transaction Viewer Dynamic Config Aggregator Service Registry

27 集中式的 Saga 基于图形 Caller requests { T1:[a,b], C1:[c,d],... } Saga Execution Component FromJsonFormat GraphBuilder request graph Saga TaskRunner TaskConsumer SagaTask events Saga Log Saga Started T1 Started T1 Ended T2 Started SagaRequest Transaction Compensation EventRepo Transport ToJsonFormat Service Registry 1 2 3

28 集中式 Saga 基于 Actor 模型 Saga Execution Component Saga Actors Saga Log Saga Started Caller requests { T1:[a,b], C1:[c,d],... } FromJsonFormat ActorBuilder T1 Started T1 Ended T2 Started SagaTask events SagaRequest Transaction Compensation Transport EventRepo ToJsonFormat Service Registry 1 2 3

29 集中式 Saga 实现过程中问题好处集中式的服务调用易易于监控和协调坏处通过 JSon 描述 Saga 事务执行行灵活性不不高业务描述与代码相分离, 需要依赖 UI 工具的帮助如何解决自动获取 Saga 事务定义的问题?

30 自动构造事务调用信息 A {GT} {GT,LT a} B {GT, LT b} {GT, LT b} {GT,LT a} E Service Consumer Global Tx Id Parent Tx Id Local Tx Id Request Global Tx Id Local Tx Id Service Provider Global Tx Id Parent Tx Id Local Tx Id C D

31 Saga 的狼群架构 Alpha 是狼群首领, 负责协调事务执行行情况 Omega 是狼群成员, 负责收集事务, 向狼群首领上报情况, 并执行行相关指令 Service A Omega Service B Alpha Alpha Alpha Omega

32 Omega 内部实现 Request (Tx id) Request Interceptor Pre Processor Build TxContext Transaction Begin DB Transaction Alpha Post Processor Transaction End / Abort

33 Omega 与 Alpha 之间交互 Request (Tx id) Request Interceptor Saga Started Pre Processor A Started DB Transaction Alpha A Started B Started Post Processor

34 User Service A Omega Service B Omega Alpha request intercept pre-process inject tx id record saga started event pre-process record tx started event transaction post-process record tx ended event intercept inject tx id intercept retrieve tx id pre-process record tx started event transaction post-process record tx ended event successful response post-process record saga ended event

35 User Service A Omega Service B Omega Scanner Alpha request intercept pre-process inject tx id record saga started event record tx started event transaction post-process record tx ended event intercept inject tx id intercept retrieve tx id pre-process record tx started event transaction failed response throw ex record tx aborted event compensate record tx compensated event record saga ended event

36 未来的开发计划 Alpha 高可用多租户架构基于消息队列列的服务提供 TCC 的协调控制服务通过 Omege 提供幂等操作功能可视化的事务拓拓扑, 定位异常最多服务 Omega 进一步解决多线程间共享调用链问题

37 小结微服务事务一致性问题? 业界 Saga 的解决方案 ServiceComb Saga 的演进后续的开发计划

让云原生应用开发更更简单代码 : https://github.com/apache?

38 让云原生应用开发更更简单代码 : 网站 : 华为云 :

LC3-分布式事务-姜宁

LC3-分布式事务-姜宁 Saga 分布式事务解决方案与实践演讲者 / 姜宁关于我华为开源能力力中心 ServiceComb 项目负责人 Apache Member, IPMC, 多个 Apache 项目 RedHat, IONA, Travelsky 议题微服务事务一致性问题? 业界 Saga 的解决方案 ServiceComb Saga 的演进后续的开发计划微服务架构微服务架构将一个应用分成多个相互独