2 209 Softmax Python TensorFlow drop- out 2 Fig.2 Remotesensingimageclassificationflowchart 1 1 ;2 ;3 ; ( ) 3 (3) 1 (4) Fig.1 Remotesensing

Size: px

Start display at page:

Download "2 209 Softmax Python TensorFlow drop- out 2 Fig.2 Remotesensingimageclassificationflowchart 1 1 ;2 ;3 ; ( ) 3 (3) 1 (4) Fig.1 Remotesensing"

乘凌
5 years ago
Views:

1 GeologicalScienceandTechnologyInformation Vol.37 No.2 Mar doi /.cnki.dzkq [J] (2) ( ; ; 3. ( ) ) Python TensorFlow Dropout Dropout ; ;dropout ; P627 A (2018) Lü [11] SVM DBN Shim [12] DBN EMG DBN EMG [13] [1] Li [2-4] [14] (CNN) [5] [6] SVM [15] 96% [7] AlphaGo BN [16] [8] (LSTM) [9] [10] (GRU) ( ) (1981 (1981 ) liwei@ccccltdwrh.com ) @qq.com

2 2 209 Softmax Python TensorFlow drop- out 2 Fig.2 Remotesensingimageclassificationflowchart 1 1 ;2 ;3 ; ( ) 3 (3) 1 (4) Fig.1 Remotesensingimageclassificationmethod 2 2 (1) 1 (deepbeliefnetworks DBN) 2 (RBM) 3 RBM (2)

3 sigmac i +W iv () i ( ) RBM - lgp ( v) = Ev [ p ( h i =1 v) ]-v (6) c i - lgp ( v) = Ev [ p ( v h) ]-v () i b RBM Gibbs N Gibbs RBM 3 v 0 v 1 S=(S 1 S 2 S 3 S N ) h 0 h 1 h 2 S i-p(s i S i-1) S i-1 N-1 S RBM S Gibbs 3 RBM Fig.3 RBM networkconstruction RBM E(vh) CD 2 RBM 1 ; ;2 p(v h)= p(v h) Persistent CD (3) p(h v)= p(h i v) p(vh) i k RBM (DBN) RBM p(h i =1 v)=sigma(c i +W iv) (4) RBM p(v =1 h)=sigma(b +W h) DBN RBM 0 1 DBN RBM F(v)=-bv- lg(1+e (c i +W i v) ) (5) i BP - lgp (v) = Ev [ p ( h i =1 v ) v ] W i -v () i sigma ( W iv () i ) ; (9) h ( n+ ) v ( n+ ) 1 ~sigmac+wv ( () n ) 1 ~sigmab+wh ( () n ) (7) (7) sigma(c+wv) 1 0 sigma(b+wh) 1 E(vh)=-bv-ch-hWv (1) 0 W ;b c p(vh) Hinton [17] F(v)=-bv- i lg eh i (c i +W iv) hi (2) CD PersistentCD DBN ( ) RBM sigmoid

4 DBN Fig.4 DBNnetworkconstruction DBN DBN v Dropout Gibbs h Gibbs Dropout cost Dropout RBM 6 Gibbs mini-batch Dropout 5 2 DBN DBN BP BP DBN BP BP 2.3 Dropout Hinton Dropout [18] Dropout 6 Dropout Fig.6 Dropout TensorFlow RBM DBN DBN 5 DBN (MLP) Fig.5 DBNnetworkconstructionlearningdiagram RBM MLP RBM

5 TensorFlow DBN 7 7 TensorFlow DBN Fig.7 TheuseofTensorFlowtoachievetheDBN model 784; 1; l r= (RBM) NWPU-RESISC45 (MLP) 45 (DBN) TensorFlow TensorGraph TIF [19] 4 ( )28 28=784 ( ) [ ] (weight) ( 400 (bias) Cost ) (400) epoch mini-batch learningrate0.3 epochs100 learningrate0.3 epochs RBM DBN DBN MLP 10 [ ] 8 7 DBN Cost 0.1 l r= (l r=0.01 =100) Fig.9 Fine-tuningstageclassificationaccuracychanges 8 Cost Fig.8 ChangesinCostvaluesinpre-learning 9

6 2 213 DBN DBN Cost 10 Cost Fig.10 ChangesinCostvaluesinPre-learning 13 DBN ( Dropout) Fig.13 DBNclassificationaccuracy(stochasticDropout) DBN Dropout ( 12 75% ) 11 Cost Fig.11 ChangeofCostvalueinthefine-tuningstage 14 DBN ( Dropout) 11 (ep- Fig.14 DBNclassificationaccuracy(Dropout) och) Cost 12 DBN SVM 0.76 SVM DBN Python ( ) scikit-learn(sklearn) 13 (epoch) 1 (1 ) DBN DBN DBN 20 DBN 12 Fig.12 Fine-tuningstageclassificationaccuracychanges 1 Table1 Classificationaccuracy SVM KDTree 63.5 SGD DBN 73.9 DBN(Dropout) 75.3 /%

7 DBN DBN Dropout Dropout ingresearch201011(6) [1] BruzzoneLChiMMarconciniM.AnoveltransductiveSVM forsemisupervisedclassificationofremote-sensingimages[j]. GeoscienceandRemoteSensingIEEE Transactionson (11) [2] LiXWang G.Optimalbandselectionforhyperspectraldata withimproveddiferentialevolution[j].journalofambientin- teligence & HumanizedComputing20156(5) [3] LiXWangL.Onthestudyoffusiontechniquesforbadgeo- logicalremotesensingimage[j].journalofambientinteli- gence & HumanizedComputing20156(1) [4] LiXZhangH.IdentificationofRemoteSensingImageofAd- delberg tivedivergence[m].[s.l.]mitpress2002. [5]. [18]YuDHintonGMorgan Netal.Introductiontothespecial [J] (6) sectionondeeplearningforspeechandlanguageprocessing [J] (3) [7] ArelIRoseD CKarnowskiT P.Deep machinelearning-anew frontierinartificialinteligenceresearch[researchfrontier][j]. ComputationalInteligenceMagazineIEEE20105(4) [8] BengioYCourvileAVincentP.RepresentationlearningA reviewandnewperspectives[j].paternanalysisand Machine InteligenceIEEETransactionson201335(8) [9] Oquab MBotouLLaptevIetal.Isobectlocalizationfor free -weakly-supervisedlearning with convolutionalneural networks[c] Anon.ProceedingsoftheIEEE Conferenceon ComputerVisionandPaternRecognition sificationusingremotelysensedsardatathroughdeepbelief networks[j].journalofsensors201520(15)1-10. perience[j] (5) [13]. [J] (1) [14]. [J]. 2017(1) [15]. [J] (1) [16]. [J]. 2017(1) versegeologicalbodybasedonclassification[m].bio-inspired Computing -Theoriesand Applications.SpringerBerlin Hei- [10]VincentPLarocheleHLaoieIetal.Stackeddenoisingau- toencoderslearningusefulrepresentationsinadeepnetwork withalocaldenoisingcriterion[j].journalofmachinelearn- [11]LüQDouYNiuXetal.Urbanlanduseandlandcoverclas- [12]Shim H MLee S.Multi-channel electromyography patern classificationusingdeepbeliefnetworksforenhanceduserex- [17]HintonG.Trainingproductsofexplertsbymininizingcontras- ing201220(1)4-6. tionbenchmarkandstateoftheart[j].proceedingsofthe IEEE (10) ModelofDeepBeliefNetworkBasedonRemoteSensingClassification LiWei 1 WuLiang 2 ChenGuanyu 3 (1.NorthwestRegionalHeadquartersChinaCommunicationsConstructionCo.Xi an710065china; 2.SchoolofInformationandEngineeringWuchangUniversityofTechnologyWuhan430223China; 3.SchoolofComputerScienceChinaUniversityofGeosciences(Wuhan)Wuhan430074China) [6]. [J].IEEETransactionsonAudioSpeech & LanguageProcess- [19]ChengGHanJLuX.Remotesensingimagesceneclassifica- AbstractHighresolutionremotesensingimagescandescribethegeometricfeaturesspatialfeaturesand texturefeaturesofobectsmoreaccurately.howtogetmoreusefulinformationfromtheremotesensing imageandthentherecognitionandclassificationoftheimagefromtheinformationhasatractedmorein- terestsinthefieldofhighresolutionremotesensingimageresearch.deeplearningalearningalgorithm basedonthedepthnetworkstructurecanbeterfittheintrinsicstructureofthesamplecomparedwith thetraditionalshalowclassifier.inthisstudythedeepbeliefnetworkisrealizedwithpythonandtensor FlowandanimprovedDropoutstrategyisproposed.Thestrategywhichonlyselectspartofthelocalarea datatocleartheweightnotonlypreservesthelocalinformationoftheimageitselfbutalsoenhancesthe generalizationabilityofthemodel.moreoverthediferentialevolutionalgorithmisadoptedtooptimize theweightsandbiasvaluesoftheneuralnetwork.theexperimentalresultsshowthattheimproveddrop- outstrategyhasasignificantefectonpreventingtheoverfitingproblemandthediferentialevolutional- gorithmisbeterthanthetraditionalshalowclassifierforparametertuningofthedeepbeliefnetwork. Keywordsremotesensingimage;deepbeliefnetwork;dropoutstrategy;diferentialevolution;classification

GeologicalScienceandTechnologyInformation Vol.36 No.4 Jul doi: /j.cnki.dzkq ,,. [J].,2017,36(4): , 2 2, (1.

36 4 2017 7 GeologicalScienceandTechnologyInformation Vol.36 No.4 Jul. 2017 doi:10.19509/j.cnki.dzkq.2017.0432. [J]. 201736(4):244-249. 1 2 2 (1. 430074;2. 430206) : Kappa 92% : ; ; :P627 :A :1000-7849(2017)04-0244-06