中国科技网

Size: px

Start display at page:

Download "中国科技网"

现镜习
9 years ago
Views:

1 基于深度学习的医学影像大数据分析 Zhao Di Computer Network Information Center Chinese Academy of Sciences 2016 年图形处理器技术大会北京国际饭店会议中心,2016 年 9 月 13 日

2 关于我

3 关于我 : 博士毕业论文细胞间的热传导 ; 采用微分方程模型, 全文上千公式 ;

4 关于我 : 哥伦比亚大学工作

5 关于我 : 哥伦比亚大学生物医学信息中心生物医学信息分析 ; 采用贝叶斯模型 ;

6 关于我 :OSU 脑科学中心

7 关于我 :OSU 脑科学中心 MRI 成像与图像分析的训练, 西门子公司 ; 采用贝叶斯模型 ; 导师 :Mark A. Pitt 教授,Jay Myung 教授,Zhong-Lin Lu 教授, 认知科学

8 基于深度学习的医学图像分析医学影像技术提供了新的契机描述人体组织解剖结构信息 PET SPECT 描述人体组织代谢信息

9 基于卷积神经网络的医学图像分析基于深度神经网络的医学图像分析 Hinton 于 2006 年提出, 与 GPU 计算结合, 使得大规模的数据学习和多层网络训练成为可能深度学习中的主要网络 : (1) 自编码机 Auto Encoder (2) 稀疏编码 Sparse Coding (3) 限制性玻尔兹曼机 RBM (4) 深度置信网络 Deep Belief Network (5) 深度卷积神经网络

10 深度学习疾病的分类脑 : 脑血管病, 神经退行性疾病 ( 阿尔茨海默病, 帕金森病, 癫痫 ), 精神疾病 ( 抑郁症, 精神分裂症 ), 脑瘤胸 : 心脏疾病, 肺结节 / 肺癌, 乳腺结节 / 乳腺癌颈 : 颈动脉检测, 甲状腺癌眼 : 糖尿病眼病皮肤 : 皮肤癌腹部 : 胃癌男性骨盆 : 前列腺癌女性骨盆 : 子宫颈癌耳鼻背四肢臀腰

阿尔茨海默病 (Alzheimer's disease) A B C 阿尔茨海默病 (Alzheimer s disease) 阿尔茨海默病 (Alzheimer s disease) 是一是一种起病隐匿的进行性发展的神经种起病隐匿的进行性发展的神经系统退行系统退行性疾病阿尔茨海默病导致性疾病阿尔茨海默病导致脑神经细胞死脑神经细胞死亡, 脑组织缺失如左亡, 脑组织缺失

11 阿尔茨海默病 (Alzheimer's disease) A B C 阿尔茨海默病 (Alzheimer s disease) 阿尔茨海默病 (Alzheimer s disease) 是一是一种起病隐匿的进行性发展的神经种起病隐匿的进行性发展的神经系统退行系统退行性疾病阿尔茨海默病导致性疾病阿尔茨海默病导致脑神经细胞死脑神经细胞死亡, 脑组织缺失如左亡, 脑组织缺失如左图所示, 阿尔茨海图所示, 阿尔茨海默病将导致严重默病将导致严重的脑萎缩 ; 的脑萎缩临床上以记忆障碍失语失用失认视空间技能损害执行功能障碍以及人格和行为改变等全面性痴呆表现为特征, 病因迄今未明临床上以记忆障碍失语失用失认视空间技能损害执行功能障碍以及人格和行为改变等全面性痴呆表现为特征, 病因迄今未明 ; 阿尔茨海默病尚未找到有效的治疗手段, 临床上早期发现, 早期干预, 对于减轻病人脑部损害有非常重要的意义

Classification of Alzheimer s Disease Structural MRI Data by Deep Learning Convolutional Neural Networks In this paper, we used convolutional neural network to classify Alzheimer s brain from normal

The importance of classifying this kind of medical data is to potentially develop a predict model or system in order to recognize the type disease from normal subjects or to estimate the stage of the

12 Classification of Alzheimer s Disease Structural MRI Data by Deep Learning Convolutional Neural Networks In this paper, we used convolutional neural network to classify Alzheimer s brain from normal healthy brain. The importance of classifying this kind of medical data is to potentially develop a predict model or system in order to recognize the type disease from normal subjects or to estimate the stage of the disease. Using Convolutional Neural Network (CNN) and the famous architecture LeNet-5, we successfully classified structural MRI data of Alzheimer s subjects from normal controls where the accuracy of test data on trained data reached 98.84%. Saman Sarraf, Ghassem Tofighi, Classification of Alzheimer's Disease Structural MRI Data by Deep Learning Convolutional Neural Networks,

Classification of Alzheimer s Disease Using fmri Data and Deep Learning Convolutional Neural Networks In this paper, we used convolutional neural network to classify Alzheimer s brain from normal

13 Classification of Alzheimer s Disease Using fmri Data and Deep Learning Convolutional Neural Networks In this paper, we used convolutional neural network to classify Alzheimer s brain from normal healthy brain. of clinical data such as Alzheimer s disease has been always challenging and most problematic part has been always selecting the most discriminative features. Using Convolutional Neural Network (CNN) and the famous architecture LeNet- 5, we successfully classified functional MRI data of Alzheimer s subjects from normal controls where the accuracy of test data on trained data reached 96.85%. Saman Sarraf, Ghassem Tofighi, Classification of Alzheimer's Disease using fmri Data and Deep Learning Convolutional Neural Networks, arxiv: [cs.cv].18: /9/12

DeepAD: Alzheimer s Disease Classification via Deep Convolutional Neural Networks using MRI and fmri This work outlines state-of-the-art deep learning-based pipelines employed to distinguish

Using these pipelines, which were executed on a GPU-based high performance computing platform, the data were strictly and carefully preprocessed.

14 DeepAD: Alzheimer s Disease Classification via Deep Convolutional Neural Networks using MRI and fmri This work outlines state-of-the-art deep learning-based pipelines employed to distinguish Alzheimer's magnetic resonance imaging (MRI) and functional MRI data from normal healthy control data for the same age group. Using these pipelines, which were executed on a GPU-based high performance computing platform, the data were strictly and carefully preprocessed. Next, scale and shift invariant low- to high-level features were obtained from a high volume of training images using convolutional neural network (CNN) architecture. Saman Sarraf, Ghassem Tofighi, DeepAD: Alzheimer s Disease Classification via Deep Convolutional Neural Networks using MRI and fmri,

15 深度学习脑疾病图像分析数据源 :ADNI; 合作单位 : 北京天坛医院等

16 深度学习脑疾病图像分析

17 深度学习脑疾病图像分析本项目获得了 2016 年北京市自然科学基金重点项目的资助

18 深度学习脑疾病图像分析

19 深度学习具有专业知识的深度学习更高的准确率更小的训练样本与首都医科大学附属宣武医院合作研究

20 1. AD 诊断通道 1 fmri ROIs 通道 N 多通道深度学习模型 1 MRI ROIs 通道 1 通道 N 多通道深度学习模型 2 通道 1 权重 w1 权重 w2 权重 w3 权重 w(n-1) 权重 w(n) 临床诊断决策结果 PET ROIs 通道 N 加权贝叶斯网络多通道深度学习模型 3 CSF 其它临床诊断深度学习模型 N

21 1. AD 诊断 -ADNI AD 生物标记物 : 脑内 Aβ 沉积, 脑脊液的 tau,pet, 结构性 MRI

22 1. AD 诊断 - 临床

23 1. AD 诊断 - 预处理分割各个脑区

24 1. AD 诊断 - 预处理杏仁核海马内嗅皮层结构标记 ( 线性测量, 体积 ) 线性指标中, 颞角宽度敏感度最高双侧颞角水平正好代表海马的中部, 在此区域包含了大量的联系纤维 ( 内嗅区海马海马旁回等与大脑额颞叶的纤维联系 ), 早期的病理改变常局限在此区域线性测量的重复性和特异性较低

25 1. AD 诊断 - 预处理面积测量测量前后连合间层面的额叶颞叶侧脑室体部断面颞角和外侧裂平均横断面面积萎缩率最高的是颞角, 然后依次为侧脑室体部外侧裂颞叶, 最后为额叶面积测量忽略了皮层厚度对于萎缩程度的影响体积测量轻度 AD 海马萎缩约为 15%- 22%, 严重者可达 40% 内嗅皮层旁嗅皮层及颞极皮层在 AD 均显示萎缩, 以内嗅皮层萎缩最明显海马及内嗅皮层萎缩可以早期诊断 AD, 体积测定诊断 AD 的准确度约 85%-94%

26 深度学习疾病的分类脑 : 脑血管病, 神经退行性疾病 ( 阿尔茨海默病, 帕金森病, 癫痫 ), 精神疾病 ( 抑郁症, 精神分裂症 ), 脑瘤胸 : 心脏疾病, 肺结节 / 肺癌, 乳腺结节 / 乳腺癌颈 : 颈动脉检测, 甲状腺癌眼 : 糖尿病眼病皮肤 : 皮肤癌腹部 : 胃癌男性骨盆 : 前列腺癌女性骨盆 : 子宫颈癌耳鼻背四肢臀腰

27 Deep Learning-Parkinsons Diseases

PD Disease State Assessment in Naturalistic Environments using Deep Learning In this work we propose an assessment system that abides practical usability constraints and applies deep learning to

28 PD Disease State Assessment in Naturalistic Environments using Deep Learning In this work we propose an assessment system that abides practical usability constraints and applies deep learning to differentiate disease state in data collected in naturalistic settings. Based on a large data-set collected from 34 people with PD we illustrate that deep learning outperforms other approaches in generalisation performance, despite the unreliable labelling characteristic for this problem setting, and how such systems could improve current clinical practice. Nils Y. Hammerla, James M. Fisher, Peter Andras, Lynn Rochester, Richard Walker, Thomas Plotz, PD disease state assessment in naturalistic environments using deep learning, AAAI'15 Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence,

29 深度学习疾病的分类脑 : 脑血管病, 神经退行性疾病 ( 阿尔茨海默病, 帕金森病, 癫痫 ), 精神疾病 ( 抑郁症, 精神分裂症 ), 脑瘤胸 : 心脏疾病, 肺结节 / 肺癌, 乳腺结节 / 乳腺癌颈 : 颈动脉检测, 甲状腺癌眼 : 糖尿病眼病皮肤 : 皮肤癌腹部 : 胃癌男性骨盆 : 前列腺癌女性骨盆 : 子宫颈癌耳鼻背四肢臀腰

30 深度学习肺癌筛查医学诊断现状 : 仅通过单次 CT 图像不能判断碘灌注增强仅通过医学图像不能判断病理分析医学诊断期望 : 通过医学图像判断肿瘤的良恶性并给出量化的特征, 如某一评分指标或是尺寸等

31 深度学习肺癌筛查划分标准肿瘤肉眼形态肿瘤发生部位类型管内型, 管壁浸润型, 结节型, 块状型, 弥漫浸润型中央型, 周围型, 弥漫型组织病理鳞状上皮细胞癌, 腺癌, 细支气管肺泡癌, 小细胞未分化肺癌, 大细胞未分化肺癌类型特征鳞状上皮细胞癌息肉状腔内肿块和 ( 或 ) 支气管阻塞 ; 肺门的肿块 ( 累及淋巴结 ) 腺癌圆形或分叶状的孤立结节邻近肺浸润, 其边缘常不规则呈毛刺边缘

Deep learning is based on deep neural networks.

32 Using Deep Learning for Pulmonary Nodule Detection & Diagnosis This study uses a revolutionary image recognition method, deep learning, for the classification of potentially malignant pulmonary nodules. Deep learning is based on deep neural networks. Ross Gruetzemacher, Ashish Gupta, Using Deep Learning for Pulmonary Nodule Detection & Diagnosis, Twenty-second Americas Conference on Information Systems, San Diego, 2016.

Computer-aided classification of lung nodules on computed tomography images via deep learning technique In this study, we attempted to simplify the image analysis pipeline of conventional CAD with

33 Computer-aided classification of lung nodules on computed tomography images via deep learning technique In this study, we attempted to simplify the image analysis pipeline of conventional CAD with deep learning techniques. Specifically, we introduced models of a deep belief network and a convolutional neural network in the context of nodule classification in computed tomography images. Two baseline methods with feature computing steps were implemented for comparison. The experimental results suggest that deep learning methods could achieve better discriminative results and hold promise in the CAD application domain. Hua KL, Hsu CH, Hidayati SC, Cheng WH, Chen YJ, Computer-aided classification of lung nodules on computed tomography images via deep learning technique. Onco Targets Ther Aug 4;8:

Automated detection of pulmonary nodules in PET/CT images: Ensemble false-positive reduction using a convolutional neural network technique In the CT images, a massive region is first detected using

34 Automated detection of pulmonary nodules in PET/CT images: Ensemble false-positive reduction using a convolutional neural network technique In the CT images, a massive region is first detected using an active contour filter, which is a type of contrast enhancement filter that has a deformable kernel shape. Subsequently, high-uptake regions detected by the PET images are merged with the regions detected by the CT images. FP candidates are eliminated using an ensemble method; it consists of two feature extractions, one by shape/metabolic feature analysis and the other by a CNN, followed by a two-step classifier, one step being rule based and the other being based on support vector machines. Teramoto A, Fujita H, Yamamuro O, Tamaki T, Automated detection of pulmonary nodules in PET/CT images: Ensemble false-positive reduction using a convolutional neural network technique, Med Phys. 2016;43(6):2821

35 深度学习肺癌筛查设计的 CNNs 网络训练数据 ( 三个病人的三张 Slice, 取 1600 个 28*28 大小正样本,600 个 28*28 大小负样本 ) 正样本负样本

36 深度学习疾病的分类脑 : 脑血管病, 神经退行性疾病 ( 阿尔茨海默病, 帕金森病, 癫痫 ), 精神疾病 ( 抑郁症, 精神分裂症 ), 脑瘤胸 : 心脏疾病, 肺结节 / 肺癌, 乳腺结节 / 乳腺癌颈 : 颈动脉检测, 甲状腺癌眼 : 糖尿病眼病皮肤 : 皮肤癌腹部 : 胃癌男性骨盆 : 前列腺癌女性骨盆 : 子宫颈癌耳鼻背四肢臀腰

37 深度学习颈部超声图像识别 The Problem: 超声图像采用 CNN+CSF 等算法参加 Kaggle s data science competitions:ultrasound Nerve Segmentation CCF HPCChina 2016 已经投稿

38 1 存在问题

39 深度学习颈部超声图像识别

40 深度学习颈部超声图像识别 Loss / Accuracy 曲线

41 深度学习疾病的分类脑 : 脑血管病, 神经退行性疾病 ( 阿尔茨海默病, 帕金森病, 癫痫 ), 精神疾病 ( 抑郁症, 精神分裂症 ), 脑瘤胸 : 心脏疾病, 肺结节 / 肺癌, 乳腺结节 / 乳腺癌颈 : 颈动脉检测, 甲状腺癌眼 : 糖尿病眼病皮肤 : 皮肤癌腹部 : 胃癌男性骨盆 : 前列腺癌女性骨盆 : 子宫颈癌耳鼻背四肢臀腰

42 Mitosis Detection in Breast Cancer Histology Images with Deep Neural Networks We use deep max-pooling convolutional neural networks to detect mitosis in breast histology images. The networks are trained to classify each pixel in the images, using as context a patch centered on the pixel. Simple postprocessing is then applied to the network output. Our approach won the ICPR 2012 mitosis detection competition, outperforming other contestants by a significant margin. Cireşan DC, Giusti A, Gambardella LM, Schmidhuber J, Mitosis detection in breast cancer histology images with deep neural networks, Med Image Comput Comput Assist Interv. 2013;16(Pt 2):411-8.

43 深度学习疾病的分类脑 : 脑血管病, 神经退行性疾病 ( 阿尔茨海默病, 帕金森病, 癫痫 ), 精神疾病 ( 抑郁症, 精神分裂症 ), 脑瘤胸 : 心脏疾病, 肺结节 / 肺癌, 乳腺结节 / 乳腺癌颈 : 颈动脉检测, 甲状腺癌眼 : 糖尿病眼病皮肤 : 皮肤癌腹部 : 胃癌男性骨盆 : 前列腺癌女性骨盆 : 子宫颈癌耳鼻背四肢臀腰

44 Ensemble of Deep Convolutional Neural Networks for Learning to Detect Retinal Vessels in Fundus Images In this work we present a computational imaging framework using deep and ensemble learning for reliable detection of blood vessels in fundus color images. An ensemble of deep convolutional neural networks is trained to segment vessel and non-vessel areas of a color fundus image. During inference, the responses of the individual ConvNets of the ensemble are averaged to form the final segmentation. In experimental evaluation with the DRIVE database, we achieve the objective of vessel detection with maximum average accuracy of 94.7% and area under ROC curve of Debapriya Maji, Anirban Santara, Pabitra Mitra, Debdoot Sheet, Ensemble of Deep Convolutional Neural Networks for Learning to Detect Retinal Vessels in Fundus Images, arxiv: [cs.lg].

45 深度学习疾病的分类脑 : 脑血管病, 神经退行性疾病 ( 阿尔茨海默病, 帕金森病, 癫痫 ), 精神疾病 ( 抑郁症, 精神分裂症 ), 脑瘤胸 : 心脏疾病, 肺结节 / 肺癌, 乳腺结节 / 乳腺癌颈 : 颈动脉检测, 甲状腺癌眼 : 糖尿病眼病皮肤 : 皮肤癌腹部 : 胃癌男性骨盆 : 前列腺癌女性骨盆 : 子宫颈癌耳鼻背四肢臀腰

H&E-stained Whole Slide Deep Learning Predicts SPOP Mutation State in Prostate Cancer We constructed a statistical model that predicts whether or not SPOP is

Using a cohort of 177 prostate cancer patients where 20 had mutant SPOP, we trained multiple ensembles of residual networks, accurately distinguishing SPOP

46 H&E-stained Whole Slide Deep Learning Predicts SPOP Mutation State in Prostate Cancer We constructed a statistical model that predicts whether or not SPOP is mutated in prostate cancer, given only the digital whole slide after standard hematoxylin and eosin [H&E] staining. Using a cohort of 177 prostate cancer patients where 20 had mutant SPOP, we trained multiple ensembles of residual networks, accurately distinguishing SPOP mutant from SPOP wild type patients. Andrew J Schaumberg, Mark A Rubin, Thomas J Fuchs, H&E-stained Whole Slide Deep Learning Predicts SPOP Mutation State in Prostate Cancer,

47 Prostate cancer detection using photoacoustic imaging and deep learning This paper introduces a new database which consists of a large sample size of patients gathered using multispectral photoacoustic imaging. As an alternate to the standard two class labeling (malignant, normal), our voxel based ground truth diagnosis consists of three classes (malignant, benign, normal). We explore deep neural nets, experiment with three popular activation functions, and perform different sub feature group analysis. Our initial results serve as a benchmark on this database. Greedy based feature selection recognizes and eliminates noisy features. Ablation feature ranking at the feature and group level can simplify clinician effort and results are contrasted with medical literature. Our database is made freely available to the scientific community. Arjun Raj Rajanna, Raymond Ptucha, Saugata Sinha, Bhargava Chinni, Vikram Dogra, Navalgund A. Rao, Prostate cancer detection using photoacoustic imaging and deep learning, IS&T International Symposium on Electronic Imaging 2016.

48 深度学习脑模拟与对人脑视觉系统的深入理解拟用大规模神经网络建模和高性能计算机仿真方法来探明视觉认知的动态交互机制, 并在此基础上与深度学习理论结合, 研发出高效的类脑物体识别算法, 为新一代智能视觉应用提供最基本的信息处理加工技术仿真规模神经元达到 1 亿量级突触达到 1000 亿量级 ; 能模拟高级视皮层在 1 秒内完成的轮廓识别任务的精细神经活动过程 ; 与北京师范大学合作研究获得了北京市科委大脑高级视觉系统的仿真计算模型与应用验证的资助

49 深度学习人脑如何学习和记忆? 人的学习过程中, 大脑对外部环境进行感知, 注意机制对感兴趣的信息保持关注 ; 在工作记忆中, 新知识在旧知识的基础上通过检索被快速建立起来 ; 而后经过神经元的加工整理, 形成难被遗忘的长时记忆由此, 人不断地从生活经验中建立并整合知识, 从而学会处理日益复杂的任务在持续不断的学习过程中, 对以往知识检索利用, 使得人们只需要少量的训练就能快速地学会新的任务综上, 一个真正的智能系统应具备以下两方面的作用 : 在长时记忆系统中建立一个可检索的知识库, 在交互过程中持续不断的整合更新知识库

50 感谢厚爱与支持!

报告 1: 郑斌教授, 美国俄克拉荷马大学医学图像特征分析与癌症风险评估方法摘要 : 准确的评估癌症近期发病风险和预后或者治疗效果是发展和建立精准医学的一个重要前

报告 1: 郑斌教授, 美国俄克拉荷马大学医学图像特征分析与癌症风险评估方法摘要 : 准确的评估癌症近期发病风险和预后或者治疗效果是发展和建立精准医学的一个重要前东北大学中荷生物医学与信息工程学院 2016 年度生物医学与信息工程论坛会议时间 2016 年 6 月 8 日, 星期三,9:30 至 16:00 会议地址会议网址主办单位东北大学浑南校区沈阳市浑南区创新路 195 号生命科学大楼 B 座 619 报告厅 http://www.bmie.neu.edu.cn