ISSN 1000-9825, CODEN RUXUEW E-mail: jos@iscas.ac.cn Journal of Software,2017,28(11):3018 3029 [doi: 10.13328/j.cnki.jos.005332] http://www.jos.org.cn. Tel: +86-10-62562563 1,2, 1,2 1 (, 100876) 2 ( ( ), 100876) :, E-mail: panghao@bupt.edu.cn :,,.,,,.,.,.,. : ; ; ; ; : TP182 :,..,2017,28(11):3018 3029. http://www.jos. org.cn/1000-9825/5332.htm : Pang H, Wang C. Deep learning model for diabetic retinopathy detection. Ruan Jian Xue Bao/Journal of Software, 2017,28(11):3018 3029 (in Chinese). http://www.jos.org.cn/1000-9825/5332.htm Deep Learning Model for Diabetic Retinopathy Detection PANG Hao 1,2, WANG Cong 1,2 1 (School of Software, Beijing University of Posts and Telecommunications, Beijing 100876, China) 2 (Key Laboratory of Trustworthy Distributed Computing and Service (Beijing University of Posts and Telecommunications), Ministry of Education, Beijing 100876, China) Abstract: In recent years, deep learning in the computer vision has made great progress, showing good application prospects in medical image reading. In this paper, a model with construction of two-level deep convolution neural network is designed to achieve feature extraction, feature blend, and classification of the fundus photo. By comparing with doctor s diagnosis, it is shown that the output of the model is highly consistent with the doctor's diagnosis. In addition, an improved method of fine-grained image classification using weak supervised learning is proposed. Finally, future research direction is discussed. Key words: computer vision; convolutional neural network; deep learning; weak supervised learning; diabetic retinopathy (diabetic retinopathy, DR),.,,. 10,,,., DR,. : (2016YFF0201003) Foundation item: National Key Research and Development Program of China (2016YFF0201003). : 2017-01-03; : 2017-04-11, 2017-06-16, 2017-08-23; : 2017-09-06
: 3019,,., DR,., DR., DR..,,. 1,, (convolutional neural network, CNN). ImageNet (ImageNet Large Scale Visual Recognition Challenge, ILSVRC) CNN. 2012 AlexNet [1] 2014 VGGNet [2] GoogLeNet [3] 2015 Residual Network [4],CNN ;,., 1 000,CNN.,CNN, [5] [6] [7].,. Deep Vessel [8].DRIU [9]. DR, [10] DR.,, CNN DR : 1 CNN, 2 CNN. CNN, DR. 2 1 CNN 1 CNN 3 :., VGGNet,, ;, DR, CNN ; GoogLeNet Residual Network,,. 1. Fig.1 Level 1 CNN framework of the proposed DR detection 1 DR CNN 1 2.1, DR,, DR.,,,,, 512 512
3020 Journal of Software Vol.28, No.11, Novermber 2017, 8G.,,,.. CIFAR-10 CNN, 32 32., 32 32. 1. Table 1 1 Layout of the input part / / 32/3 3 2 512 512 3/256 256 32 32/3 3 1 256 256 32/256 256 32 64/3 3 1 256 256 32/256 256 64 64/3 3 2 256 256 64/128 128 64 64/3 3 1 128 128 64/128 128 64 128/3 3 1 128 128 64/128 128 128 3 3 2 128 128 128/64 64 128 192/3 3 1 64 64 128/64 64 192 192/3 3 1 64 64 192/64 64 192 3 3 2 64 64 192/32 32 192 : (2016 ) [11], (ETDRS) 2. Table 2 Classification of DR in the early treatment of DR study 2 1 2 3 4 5, DR.,,,,.,. DR,, DR,.,,., DR DR DR DR.,,.,. 8 8 2048, 3. 2.2 Table 3 3 Layout of the output part, 8 8, 1, 5, 1 1 Dropout, 0.5, 1 024, 2 Softmax, 1 2014,ImageNet GoogLeNet.,.2015,ImageNet Residual Network.
: 3021, 100., Inception v3 ResNet v2,,. Inception ResNet,, 4. ; ;, ;..Zagoruyko [12] : ResNet,,.,.Lee Wide-Residual-Inception [13], Residual Block, Inception 3 3,.Wu [14] :ResNet,,.. [13] ResNet, Inception ResNet, Inception, Inception Residual,., 32 32 16 16 8 8, 3. 1 32 32 192, 32 32 288. 3 4 Inception, Residual, 1. 2. Fig.2 Standard inception module 2 Inception 1,. Residual Idnetity, Block Residual., 3 inception, 32 32 16 16, [15]. 2 16 16, 16 16 768, 16 16 768. 4 4 Inception, 3,, Inception Residual, 2. 4. 2,, Identity, Residual.,,.,.,,.., 3 inception, 16 16 8 8, [15].
3022 Journal of Software Vol.28, No.11, Novermber 2017 Fig.3 Inception module used in Stage 2 3 2 Inception Fig.4 Multi-Branch design in Stage 2 4 2 3 8 8, 8 8 1280, 8 8 2048. 2 6 Inception, Inception Residual, 3. 5.,,.,. 3 Identity,,, Residual. 1,,, Residual,.,., 1 CNN InceptionV3, Residual,,,., 1 CNN. 1 CNN, DR.
: 3023 Fig.5 Two inception modules in Stage 3 5 3 inception 3 2 CNN, DR,. CNN,.,. 1 CNN,. CNN,., 1 CNN DR,,, DR., Kaggle 2. 8 8 2048,,, 1 1 2048., 2 048.,,, CNN., 2 048,., 2 048 4 096.,,,,,. 8 192., 32 32 8, CNN. 6. Fig.6 Steps of feature extraction 6
3024 Journal of Software Vol.28, No.11, Novermber 2017, AlexNet CNN.,, 2 CNN. 4. 4 : Table 4 Design of level 2 CNN 4 :Intel i5-6600k,32g,nvida GTX 1080; 2 CNN 32/3 3 1 3 3 2 64/3 3 1 3 3 2 128/3 3 1 3 3 2 500 Dropout 0.5 500 2 1 :Ubuntu 16.04,CUDA8.0,cudnn5.1,Theano0.8,Lasagne,Keras1.x., DR Kaggle DR (Kaggle diabetic retinopathy detection competition). Kappa,. 35 124, 5. Table 5 Details of data 5 (%) 0 Normal 25 808 73.48 1 Mild NPDR 2 443 6.96 2 Moderate NPDR 5 292 15.07 3 Severe NPDR 873 2.48 4 PDR 708 2.01,,90%,10%.,, Gary King [16]. Nesterov momentum [17], [18]. 1 CNN,, 0.838 361, Kappa 0.754 13. 7. ( ),, Kappa., 0.783 153, Kappa 0.765 98. 8. ( ), Kappa,. 7 8,,., CNN,.
: 3025 Fig.7 Training curves of level 1 CNN (classification) on the Kaggle dataset 7 1 CNN Kaggle Fig.8 Training curves of level 1 CNN (regression) on the Kaggle dataset 8 1 CNN Kaggle, Softmax., 1,,,. 7,, Kappa.,Kappa.,, DR.,,, DR. Kappa. 8,,, Kappa., Kappa,, 2 CNN. 1 CNN,, 2 CNN,., 1 CNN,, 9. : 0.812,Kappa 0.833; : 0.816,Kappa 0.838.., 3, 150 ;, 3, 150. 2 CNN 3, 3,
3026 Journal of Software Vol.28, No.11, Novermber 2017 6, 3., 3 Kappa 0.842, 3 Kappa 0.85, 6 Kappa 0.848.,. Fig.9 Comparison of training curves for the level 2 CNN training using two different feature sources 9 2 CNN,Softmax 5,., 1, 1.,. Kappa. 7. 3 514, Kappa 0.85 6. Table 6 Confution matrix 6 DR / 329, 0.093 6. DR DR 2 408 153 16 0 0 109 114 32 0 0 68 95 325 30 0 0 3 40 50 0 0 2 13 36 20, CNN,., DR Table 7 Conclusion of computer-aided diagnosis 7 * * DR (0~4) (0~1) 0.700 1 0.044 7 DR 0.471 4 0.776 3 DR 7, DR, 0.7, 2 CNN. 0.5,1.5,2.5 3.5, DR. 0.044 7 1 CNN,,,. DR,, DR 0.5,, DR.
: 3027 5 CNN,,., ;,..,. CNN. cnn param k num k size k num 2 _ = [ _ l ( _ l) + _ l], l,cnn_param,k_num l l,k_size l l. DR (fine-grained image classification), DR,.,.,.,,. DR,,., DR,.. CNN,,,.,.,,. 10,,... Fig.10 Weak supervisory characterization 10,,., ( ),.,,.,,,,,., 512 512,.,,.
3028 Journal of Software Vol.28, No.11, Novermber 2017.,. 6 CNN, DR, CNN, 2 CNN DR., Kaggle, CNN, CNN., CNN, 0.85 Kappa. Kaggle.,,.,. Inception Xception [19] ResNet ResNeXt [20],.,ResNeXt 2016 ImageNet. DR,,, Kappa, Kappa., [21], 5,..,,,,.,,. Kaggle 5,,.,.2016,Google JAMA, DR [22].2017, Nature, [23].,. References: [1] Krizhevsky A, Sutskever I, Hinton GE. ImageNet classification with deep convolutional neural networks. Communications of the ACM, 2017,60(8):84 90. [doi: 10.1145/3065386] [2] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition. In: Proc. of the Int l Conf. on Learning Representations. 2015. [3] Szegedy C, Liu W, Jia YQ, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A. Going deeper with convolutions. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2015. [doi: 10.1109/CVPR.2015.7298594] [4] He KM, Zhang XY, Ren SQ, Sun J. Deep residual learning for image recognition. arxiv: 1512.03385 [cs.cv], 2015. [5] Girshick R, Donahue J, Darrell T, Malik J. Region-Based convolutional networks for accurate object detection and segmentation. IEEE T-PAMI, 2016,38(1):142 158. [doi: 10.1109/TPAMI.2015.2437384] [6] Hariharan B, Arbelaez P, Girshick R, Malik J. Hypercolumns for object segmentation and fine-grained localization. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2015. [doi: 10.1109/CVPR.2015.7298642] [7] Xie SN, Tu ZW. Holistically-Nested edge detection. In: Proc. of the Int l Conf. on Computer Vision. 2015. [doi: 10.1109/ICCV. 2015.164] [8] Fu HZ, Xu YW, Lin S, Wong DWK, Liu J. Deep vessel: Retinal vessel segmentation via deep learning and conditional random field. In: Proc. of the Medical Image Computing and Computer Assisted Intervention. 2016. [doi: 10.1007/978-3-319-46723-8_16] [9] Maninis KK, Pont-Tuset J, Arbelaez P, Van Gool L. Deep retinal image understanding. In: Proc. of the Medical Image Computing and Computer Assisted Intervention. 2016. [doi: 10.1007/978-3-319-46723-8_17] [10] Haloi M. Improved microaneurysm detection using deep neural networks. arxiv: 1505.04424 [cs.cv], 2015.
: 3029 [11] American Academy of Ophthalmology Retina/Vitreous Panel. Preferred practice pattern guidelines. In: Proc. of the Diabetic Retinopathy. San Francisco: American Academy of Ophthalmology, 2016. http://www.aao.org/ppp [12] Zagoruyko S, Komodakis N. Wide residual networks. arxiv:1605.07146 [cs.cv], 2016. [13] Lee YW, Kim HE, Park ES, Cui XN, Kim HK. Wide-Residual-Inception networks for real-time object detection. In: Proc. of the IEEE Intelligent Vehicles Symp. 2017. [doi: 10.1109/IVS.2017.7995808] [14] Wu ZF, Shen CH, van den Hengel A. Wider or deeper: Revisiting the ResNet model for visual recognition. arxiv: 1611.10080 [cs.cv], 2016. [15] Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z. Rethinking the inception architecture for computer vision. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2015. [doi: 10.1109/CVPR.2016.308] [16] King G, Zeng LC. Logistic regression in rare events data. Political Analysis, 2001,9:137 163. [doi: 10.1093/oxfordjournals.pan.a00 4868] [17] Sutskever I, Martens J, Dahl G, Hinton G. On the importance of initialization and momentum in deep learning. In: Proc. of the 30th Int l Conf. on Machine Learning. 2013. 1139 1147. [18] He KM, Zhang XY, Ren SQ, Sun J. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proc. of the Int l Conf. on Computer Vision. 2015. [doi: 10.1109/ICCV.2015.123] [19] Chollet F. Xception: Deep learning with depthwise separable convolutions. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2017. [20] Xie SN, Girshick R, Dollár P, Tu ZW, He KM. Aggregated residual transformations for deep neural networks. In: Proc. of the IEEE Conf. on Computer Vision and Pattern Recognition. 2017. [21] Vaughn D, Justice D. On the direct maximization of quadratic weighted Kappa. arxiv: 1509.07107 [cs.lg], 2015. [22] Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S, Widner K, Madams T, Cuadros J, Kim R, Raman R, Nelson PC, Mega JL, Webster DR. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA Online, 2016. https://jamanetwork.com/journals/jama/article-abstract/2588763 [23] Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, Thrun S. Dermatologist-Level classification of skin cancer with deep neural networks. Nature, 2017,542:115 118. [doi: 10.1038/nature21056] (1979 ),,,, (1958 ),,,,,,, CCF,.,,,.