國立中山大學學位論文典藏.PDF

Size: px

Start display at page:

Download "國立中山大學學位論文典藏.PDF"

婷俭凤
6 years ago
Views:

1 Air Visibility Forecasting via Artificial Neural Networks and Feature Selection Techniques

2 2003 7

3 Air Visibility Forecasting via Artificial Neural Networks and Feature Selection Techniques Sequential Floating Search Method RBF 7 RBF 92 i

4 i... ii... v... vii Trend ARMA Model Sequential Backward Selection ii

5 2.3.2 Sequential Forward Selection Sequential Floating Search Method 29 RBF Early Stop SFSM iii

6 5.2.2 SFSM SFS SFSM RBF MLP iv

7 2.1 (0~2 km) (2~8 km) (>8 km) ~88 PM ~88 PM ~88 SO ~88 NO ~88 O ~88 CO ARMA(4,4) model ARMA(4,4) model ARMA(4,4) model ARMA(4,4) model ARMA(5,3) model Sequential Backward Selection Sequential Forward Selection Sequential Floating Search Method Sequential Floating Search Method v

8 Early stop RBF x. 55 x SFSM SFSM SFS SBS RBF RBF ~ vi

9 ~ ~ ~ / SFS SFSM SFS RBF SFSM RBF ~92 SFSM ~92 SFSM ~92 SFSM ~92 SFSM RBF ~92 SFSM RBF ~92 SFSM RBF ~ ~ ~ vii

10 ~92 MLP ~ ~ ~ ~ ~ viii

11 1.1 (Neural Network, NN) (Pattern Recognition) (Function Approximation) (mapping) RBF (Radial Basis Function Neural Network) 1

12 RBF (PSI) % % 4.05 % (PM 10 ) (extinction) (O 3 ) (1999~2000) % 2~ % 0~2 km 2~8 km 8 2

13 (Multiple Regression Model) RBF RBF

14 RBF 4

15 (Feature Selection Method) 2.1 (1999) 6~12 3~5 5

16 (PM 10 ) 19,145 (30%) (16%) 11% 7~9% 2. (SO x ) ( 95%) (1~2%) (37%) ( 23% 13%) 3. (NO x ) 52,902 ( 71%) ( 11% 12%) (24%) ( 22% 12%) 4. (NMHC) 60,146 ( 31 %) ( 6

17 21%) ( 14% 13%) 5. (CO) 372,425 ( 58%) ( 31%) 5% ( ) ~ ~ ~12 8.3~ %~80 % 7

18 2.1 ( ) (m/s) (%) (mm) (hr) MJ/m 2 (mm) (bar) N N N WNW WNW SSE SSE SSE W WNW N N N ( ) 8

19 75.7% % % MJ/m MJ/m MJ/m m/s 1.9~2.5 m/s m/s m/s ~

20 0~ ~5.99 6~ Pasquill D E F ( 57.2% ~ 60.5%) 10. (12 14 ) 800 ~ 1100 (4 6 ) ~2 km( 2.1) 2~8 km( 2.2) 8 km ( 2.3)

21 88/89 ( ) (0~2 km) 11

22 2.2 (2~8 km) 2.3 (>8 km) 12

23 2.2 t t Trend (Trend) (day)

24 PM 10 (12~2 ) (6~8 ) PM 10 ( g/m 3 ) ~88 PM (Trend)

26 (PM 10 ) (SO 2 ) (NO 2 ) (O 3 ) (CO)

27 PM 10 ( g/m 3 ) ~88 PM SO2 (ppb) ~88 SO 2 17

28 NO 2 (ppb) ~88 NO O 3 (ppb) ~88 O 3 18

29 CO (ppm) ~88 CO ARMA Model Analysis 1968 Box Jenkins Autoregressive and Moving Average (ARMA) Model Analysis (Autoregressive Model) (Moving Average Model) ARMA(p,q) p q 19

30 ARMA(p,q) 88~92 (2-1) Xt = α1xt 1+ α2xt αpx t p + β Z + β Z + + β Z (2-1) 1 t 1 2 t 2 q t q X t t Z t white noise t α β p q p q α β ARMA(p,q) model MATLAB Statistics Toolbox ARMA(p,q) model ARMA(p,q) p q ARMA(4,4) ARMA model ARMA(4,4) model 20

31 ARMA(4,4) model ARMA(4,4) model 21

32 ARMA(4,4) model ARMA(4,4) model 22

33 2.17 ARMA(4,4) model ARMA(5,3) model ARMA model 88~90 ARMA(4,4) ARMA(p,q) model ARMA(5,3) model 23

34 2.3 (Feature Selection Method) n 2 n

35 2.3.1 Sequential Backward Selection Sequential Backward Selection(SBS) top down SBS 2.19 SBS 4 2 Criterion Function(CF) { x, x, x, x } Criterion Function Value(CF ) MLP [ x, x, x, x ] T CF CF CF C( x 1, x 2, x 3) C( x 1, x 2, x 4 ) C( x 1, x 3, x 4 ) C( x 2, x 3, x 4 ) CF 3 { x1, x2, x3} { x, x, x } C( x 1, x 2 ) C( x 1, x 3 ) C( x 2, x 3 ) CF { x1, x2} SFS SFS n m 1 + [( n+ 1) n m( m+ 1)]/2 2 n Sequential Forward Selection Sequential Forward Selection(SFS) bottom up SBS 25

36 n { F1 ~ F n } * * C({ F1 ~ F n } X ) X { F ~ F n } { F ~ F X { ~ * 1 n } F } 1 Fn 1 1 n=n-1 n-1 m m { ~ } F1 F m 2.19 Sequential Backward Selection 26

37 SFS 2.20 SFS 4 3 { x, x, x, x } CF x CF C( x 1, x 2 ) C( x 1, x 3 ) C( x 1, x 4 ) CF { x1, x3} CF { x, x, x } SFS n m SFS nm m( m 1)/ 2 SBS SFS Sequential Backward Selection Sequential Forward Selection SFS SFS SBS SBS SFS Nesting Effect SBS SFS Nesting Effect SBS SFS Nesting Effect 1 27

38 n X ~ 1 X n CF k=1 F 1 k=k+1 CF { F ~ F + F } 1 k 1 k k m m { F ~ F m } Sequential Forward Selection 28

39 2.3.3 Sequential Floating Search Method Sequential Floating Search Method (Pudil et.,1994) (SFSM) SBS SFS 2.21 SFSM SFS SBS k-1 CF k-1 k-1 SBS SBS SFS k SFS SFSM SFSM Inclusion Test Exclusion 2.22 n m Xk = { x1, x2,, x k } k Y n k n-k Inclusion x 1 = arg max C({ X, y}) x + k+ y Y n k k Y n k k 1 X k CF k+1 X = { X + x } k+ 1 k k+ 1 Test 1. xr = arg max x X C( X 1 k+ 1 { xr}) X k+ 1 SBS r k+ xr 2. r=k+1 k=k+1 SFS SBS k X k SFS 3. r k + 1 C ( Xk+ 1 { xr}) C( Xk) k=k+1 k k 29

40 Start Forward Selection Stop Criterion No Backward Selection Yes Stop No Test Yes 2.21 Sequential Floating Search Method Sequential Forward Selection Sequential Backward Selection Stop Criterion Test Backward Selection k k CF Backward Selection k k Backward Selection Forward Selection k+1 Forward Selection Sequential Floating Search Method 30

41 X k X SFS X k + 1 k SFS 4. k=2 X = X 1 { x } C( X ) = C( X { x }) k k+ r k-1 k=2 SBS 5. Exclusion ' 1. X = X x k k k+ 1 { r} ' 2. x = arg max ' C( X { y}) SBS s y X k ' k 3. C X ' ' ' ( k { xs}) C( Xk 1) Xk = X k C( Xk) = C( Xk) X k SBS ' 4. Xk = X x k=k-1 ' 1 k { s} 5. k=2 X = X C X = C X k ' ' k ( k) ( k) 6. k k k+ 1 r SFSM Nesting Effect SFSM 31

42 Floating Search Method xk+ 1 = argmax y Y n k C({ X, y}) k X = { X + x } k+ 1 k k+ 1 x = arg max C ( X { }) + x r r x X k r k+ 1 1 k =k+1 Yes Yes r=k+1 No C( X { x }) C( X k+ 1 r k) X k = Xk+ 1 { xr} CX ( ) = CX ( { x}) k k+ 1 r Yes No k = 2 No ' Xk = Xk+ 1 { x r } x s = y ' argmax C( Xk { }) y X ' k ' C( Xk { xs}) C( Xk 1 No ' ' X = k 1 X k { xs} k=k-1 ) Yes X = X ' k k ' ( k) = C( Xk) C X No k = 2 Yes 2.22 Sequential Floating Search Method 32

43 RBF RBF (Input Layer) (Hidden Layer) (Output Layer) (Activation Function) Chen and Billings(1992) (Orthogonal Least Squares Algorithm, OLS) 3.1 RBF (Perceptron) (Full Connected) 3.1 RBF (3-1) φ j 2 x c j x j = exp 2 ρ (3-1) ( c, ρ ) x c j RBF ρ (width) (Euclidean Norm) RBF (3-2) f R n R m f J ( ) yˆ = φ ( x c, ρ) θ i = 1, m x = (3-2) i i j j ji, j= 1 J RBF m ŷ i i θ ji j i 33

44 3.1 (3-2) RBF RBF f : x y RBF 3.2 Chen and Billings(1992) (OLS) RBF RBF (3-3) y ( t) i J = j= 1 φ ( t) θ + e ( t) 1 i m (3-3) j ji i 34

45 φ j j t θ ji y i e i (3-4) Y = F T + E (3-4) (3-5) (Objective Function) E N m 1 2 c y ki y ki (3-5) 2 (,θ ) = ( ˆ ) k= 1 i= 1 N m y ki ŷ ki k i OLS (Error Reduction Ratio, ERR) RBF (Least Squares,LS) (3-3) ji F [ ] F = WB (3-6) T W = w i j w = 0 B 1 w J 1 (3-7) i w j 35

46 1 β12 β1j 0 B = (3-7) β J 1J 0 1 W B Gram-Schmidt (Björck 1967) (3-8) (3-9) w 1 = F 1 (3-8) ß w = w F T ( w w ) T ik i k i i k k 1 = k F βikw i i= 1 1 i < k k = 2,..., J (3-5) (3-10) (3-11) Y WG + E (3-9) = (3-10) BT = G (3-11) OLS γˆ γˆ 11 1m ˆ T 1 T G = ( W W ) W Y = (3-12) γˆ J1 γˆ Jm T w j yi γ ˆji =, 1 j J, 1 i m (3-13) T ( w w ) j j OLS Ĝ Tˆ BTˆ = Gˆ (3-14) (3-14) Tˆ (3-5) E W Y (Covariance) trace T Y Y trace N = 1 N J m j= 1 i= 1 2 T 1 T γ ji w j w j + trace E E (3-15) N (3-16) 36

47 1 N m i=1 2 T γ ji w j w j (3-16) W j explained trace W j (3-17) [ err] j = m γ 2 ji w i = 1 trace T j w T ( Y Y ) j (3-17) OLS 1. j =1 2. k =1 (3-6) d 3. c j = x k k j RBF 4. (3-5) E 5. k = k x k RBF 7. j = j dmax ρ = (3-18) j + 1 d max max 37

48 OLS (1) (2) RBF (1) (2) (3) overfitting OLS 3.3 RBF activation function 3.2 OLS OLS (err) C (MSE) (3-19) j 38

49 i= N MSE y i yˆ N i (3-19) N yi desired output ŷ i output RBF RBF (3-18) RBF RBF (classifier) (Nearest Neighbor, nn) 3.4 (prototype) 39

50 3.4 ( ) (3-20) new new old ρ = min center center (3-20) new new ρ cneter old cneter nn RB (3-18) 40

51 3.4 Early Stop Early Stop OLS 3.5 RBF (Noise) (Overfitting) RBF RBF (Cross-Validation) RBF (training set) (test set) (validation set) RBF RBF (Early Stop) 3.5 Early stop Early stop 41

52 Early stop Early stop k+1 k k+2 k k+2 early stop k 3.6 RBF 42

53 MSE 3.6 RBF 43

54 RBF ( 87 ) PM 10 O 3 NO /01/01~86/12/31 87/11/25~92/4/ Index Index PM 10 O 3 NO 2 44

55 PM 10 O 3 NO 2 4 ( )

56 4.1.2 (Scaling) 4-1 L L max L min 0 ~ 1 (Rescaling) 4-2 D D max D min L L max L min 4-1 S = L L L L max min min (4-1) D D Dmax D min Rs = ( Lmax Lmin) + Lmin min S (scaling data) Rs (rescaling data) L min L max D max D min (4-2) % mm mm 150 mm 0~150 46

57 ( ) (10th) (%) (MJ/m2) (hpa) (hr) (m/s) ( ) PM 10 O 3 NO ( ) ( ) (mm) (ug/m3) (ppb) (ppb) (km) 4.2 RBF 13 Input X Designed output D X D (1) (2) 2 km 8km Training data Validation data Early stop Testing data RBF X y 0.1 i i 0.1 < y y i i 47

58 RBF RBF 1. RBF SFSM RBF 80% Training data 20% Validation data Training data 10 CF SFSM RBF Early stop Training data SFSM 2. RBF RBF SFSM RBF SFSM 80% Training data 10% Validation data 10% Testing data 100 Testing data RBF SFSM Testing data 80 % Training data 20 % Validation data

59 RBF Testing data RBF (Learning Rate) (Momentum) 4.3 (3~5 ) (6~8 ) (9~11 ) (12~2 ) ( 90.91%) 10~4 (10~4 ) (5~9 ( 6 ) ~92 ( ) 0~2 km ~8 km >8 km

60 4.4 83~92 ( %) 0~2 km ~8 km >8 km RBF ~ /1~11/ (0~2 km) 83 0 (>8 km) ARMA(p,q) model analysis 91 3 (1)83/1/1~92/4/24( 87/1/1~87/11/23)(2)88/1/1~92/4/24(3)91/1/1~92/4/ ~92 ( ) ~2 km ~8 km >8 km

61 4.6 / 83~92 88~92 91~92 0~2 km 374(12.37%) 282(18.24%) 61(13.56%) 2~8 km 1820(60.21%) 889(57.50%) 265(58.89%) >8 km 829(27.42%) 375(24.26%) 124(27.56%) 4.5 Sequential Floating Search Method(SFSM) RBF (1) 83/1/1~92/4/24( 87/1/1~87/11/23) (2) 88/1/1~92/4/24 (3) 91/1/1~92/4/24 7 (1) (2) (10~4 ) (3) (5~9 ) (4) (3~5 ) (5) (6~8 ) (6) (9~11 ) (7) (12~2 ) (1) (2) RBF 51

62 (3) 92 Multilayer Perceptron (MLP) SFSM

63 83~92 88~92 91~92 7 SFSM RBF 92 MLP

64 MLP ~92/4/24 Sequential Floating Search Method ( 5.2 ) RBF Learning rate=0.1 Momentum=0.0 80% Training data 10% Validation data 10% Testing data Training Validation Testing % % % 5.2 Training Validation Testing % % % 54

65 5.2 SFSM 7 SFSM SFSM 13 SFSM SFSM Graph Class 1 Class Feature Feature x x

66 { x, x,, x } x } 1 80% } 85% { { x, x { x, x, x } 90% { x7, x8,x 9, x10} 95% { x, x 2 3 } { x { x } 2 } 3 100% (Nearest Neighbor, nn) SFSM 5.2 index SFSM 2 3 { x } 1 { x, x } { x, x, x, x } { x, x, x } { x, x SFS } { x, x, x } % % % % % % % % % % % % % % SFSM 56

67 SBS SFSM SFSM SFS SFS SFSM SFS SFSM SFS 5.3 SFS O 3 PM 10 NO 2 O 3 PM SFSM PM 10 O 3 RBF 80% Training data 10% Validation data 10% Testing data 100 Testing data SFSM SFS 6 SFS 7 SFSM SFS Nesting Effect 57

68 5.5 SFS RBF RBF (SFS) Input Training Validation Testing SFS Dimension % % % % % % 5.6 SFSM RBF RBF (SFSM) Input Training Validation Testing SFSM Dimension % % % SFSM SFSM SFSM SFSM CF RBF O

69 SFS SBS CF 80% Training data 20% Validation data RBF RBF CF SFS SBS SFSM 5.3 SFSM RBF 59

70 5.7 83~92 SFSM 83~92 O % NO % O 3 O % % O % % O 3 PM 10 O 3 NO % ~92 SFSM 88~ % O 3 NO 2 PM 10 O % PM % PM % PM 10 O 3 NO % % O 3 NO % PM 10 O 3 60

71 5.9 91~92 SFSM 91~92 PM 10 O 3 NO % Index Index PM 10 O 3 NO

72 5.3 RBF 5.2 RBF 4.2 RBF ~92 88~92 91~92 RBF ~92 88~92 83~82 88~ ~92 SFSM 88~92 SFSM 80% Training data 10% Validation data 10% Testing data RBF RBF 5.4 RBF 62

73 ~92 SFSM RBF Training Validation Testing % % % 10~ % % % % % % 3~ % % % 6~ % % 63.25% 9~ % 78.25% % 12~ % % % ~92 SFSM RBF Training Validation Testing % % % 10~ % % % % % 63.4% 3~ % % % 6~ % % % 9~ % % % 12~ % % 75.8% 63

74 ~92 SFSM RBF Training Validation Testing % % 71.25% { } { { } 83~92 88~ ~ % % % ~ % % % 5.4 MLP ~92 88~ ~92 64

75 12.37% 27.42% % 8~19 83~ ~92 ( %) 0~2 km ~8 km >8 km Training : Class 1 / 0 /13 Class 2 / 302 /469 Class 3 / 332 /476 Validation : Class 1 / 0 /2 Class 2 / 42 /63 Class 3 / 32 /55 Testing : Class 1 / 0 /2 Class 2 / 36 /58 Class 3 / 45 / ~92 65

76 MLP ~92 MLP Training Validation Testing % % % 6~ % % % /1~4/24 Testing data 80 % Training data 20 % Validation data 100 RBF Testing data 92 1/1~4/ RBF

77 ~91 92 Training Validation Testing % % % 10~ % % % 3~ % 70.6% % 12~ % % % ~91 92 Training Validation Testing % % % 10~ % % % 3~ % % % 12~ % % % ~92 and % % % ~92 and % % % 67

78 RBF RBF RBF RBF RBF Testing data Training data 100 Validation data ~92 SFSM ~ Training Validation Testing % 68.52% 68.7% 68

79 6.1 (1) (2)RBF (3)92 MLP 1. Sequential Floating Search Method Sequential Forward Selection Sequential Backward Selection Nesting Effect 2. SFSM n ( N) stop criterion k (>n) n 3. input 4. RBF { } { 69

80 } 5. 83~92 88~ (1) (2) (3) 7. MLP SFSM RBF 10 CF 70

81 Validation data RBF MLP Averaging Committee AdaBoost committee SFSM CF 71

82 G.E.P. Box, G.M. Jenkins, and G.C. Reinsel., 1994, Time Series Analysis, Forecasting and Control, 3 edition, Ed. Englewood Cliffs: Prentice Hall. G.Zhang, B.E. Patuwo, and M.Y.Hu, Forecasting with Artificial Neural Networks: The State of The Art, Int. J. Forecast., vol. 14, pp.35-62, J. Barry Gomm, and Ding Li Yu, Selecting Radial Basis Function Network Centers with Recursive Orthogonal Least Squares Training, IEEE Transactions on Neural Networks, vol.11, pp , J.T. Connor and R.D. Martin., 1994, Recurrent Neural Networks and Robust Time Series Prediction., IEEE Transactions of Neural Networks, 2(5): , P. A. Devijver and J. Kittler., 1982, Pattern Recognition: A Statistical Approach., Prentice-Hall. P. Pudil, J. Novovičová, and J. Kittler., Floating search methods in feature selection., Pattern Recognition Letters, 15(11): , T. M. Cover and P. E. Hart, Nearest neighbor pattern classification, IEEE Trans. Inform. Theory, vol. IT-13, pp.21-27,

83 ,,,, 1998, RBF,,, 2001,,,, 2002,,,,,,,,

SVM OA 1 SVM MLP Tab 1 1 Drug feature data quantization table

SVM OA 1 SVM MLP Tab 1 1 Drug feature data quantization table 38 2 2010 4 Journal of Fuzhou University Natural Science Vol 38 No 2 Apr 2010 1000-2243 2010 02-0213 - 06 MLP SVM 1 1 2 1 350108 2 350108 MIP SVM OA MLP - SVM TP391 72 A Research of dialectical classification