5 : 603., , (Just-in-time learning, JITL) 13 18,,,. JITL,, / /,. KL, LSSVR SVR,, 10 12, 15.,, LSSVR (Recursive LSSVR, R-LSSVR) 11, 15 KL. : 1) J

Size: px

Start display at page:

Download "5 : 603., , (Just-in-time learning, JITL) 13 18,,,. JITL,, / /,. KL, LSSVR SVR,, 10 12, 15.,, LSSVR (Recursive LSSVR, R-LSSVR) 11, 15 KL. : 1) J"

出渍郁
5 years ago
Views:

1 39 5 Vol. 39, No ACTA AUTOMATICA SINICA May, (Kernel learning, KL)., (Just-in-time kernel learning, JITL),,.,,,,.,,. DOI,,,,,,.., 2013, 39(5): /SP.J Online Identification of Time-varying Processes Using Just-in-time Recursive Kernel Learning Approach LIU Yi 1 JIN Fu-Jiang 2 GAO Zeng-Liang 1 Abstract An online identification method using just-in-time recursive kernel learning (KL) is proposed to trace the timevarying characteristics of nonlinear chemical processes. For each query sample, a just-in-time kernel learning (JITL) model is established using the similar set constructed by a presented cumulative similarity factor. Different from traditional justin-time learning approaches discarding their models at each time, an efficient modeling strategy is proposed to reduce the computational load by utilizing the similarity between two neighborhood models. Consequently, a new just-in-time kernel learning model can be quickly constructed using the recursive updating algorithm, by introducing new samples and deleting different samples. The superiority of the proposed online identification method is demonstrated by a continuous stirred tank reactor process with time-varying parameters, showing better prediction performance compared with conventional recursive kernel learning approaches. Key words Process identification, just-in-time learning (JITL), kernel learning (KL), least squares support vector regression (LSSVR), recursive identification Citation Liu Yi, Jin Fu-Jiang, Gao Zeng-Liang. Online identification of time-varying processes using just-in-time recursive kernel learning approach. Acta Automatica Sinica, 2013, 39(5): ,., Manuscript received May 15, 2012; accepted December 19, 2012 ( , , ) Supported by National Natural Science Foundation of China ( , , ) Recommended by Guest Editor WANG Hong Institute of Process Equipment and Control Engineering, Zhejiang University of Technology, Hangzhou School of Information Science and Engineering, Huaqiao University, Xiamen ;.,., (Support vector regression, SVR) (Least squares SVR, LSSVR) (Kernel learning, KL), 4 12.,., : 1) /, ; 2) /, ; 3),,

2 5 : 603., , (Just-in-time learning, JITL) 13 18,,,. JITL,, / /,. KL, LSSVR SVR,, 10 12, 15.,, LSSVR (Recursive LSSVR, R-LSSVR) 11, 15 KL. : 1) JITL,,. 2) JITL,,. 1 LSSVR 1.1 LSSVR f: X Y S = {(x 1, y 1 ),, (x k, y k )} X Y, x, y. KL φ( ) X H, φ(x i ): X H, 12, y k = f(y k 1,, y k ny, u k 1,, u k nu )+e k (1), u k y k k, n u n y, f( ). e k. x k = y k 1,, y k ny, u k 1,, u k nu T, KL 12,15 y k = f(w k, b k, x k ) + e k = w T k φ(x k ) + b k + e k (2), w k b k k, LSSVR f 5 : min J (w k ) = 1 2 w k 2 + γ 2 e k 2 s.t. y i w T k φ(x i ) b k e i = 0, i = 1,, k (3), e k = e 1, e 2,, e k T ; w k 2. 2 γ > 0,., 5 K k + I k 1 γ k a k y = k (4) 1 T k 0 0, a k = a k,1,, a k,k T k, y k = y 1,, y k T, I k R k k, 1 k = 1 1 T R k 1 ; K k, Mercer 4 5, K k (i, j) = φ (x i ), φ (x j ) i, j = 1,, k. H k = K k + I k, :, P γ k = H 1 k b k a k = P k y k 1 k 1 T k P ky k 1 T k y k 1 k b k = 1T k P ky k 1 T k y k 1 k (5) k+1 : f(w k, b k, x k+1 ) = a T k k k+1 + b k (6), w k = k i=1 a k,iφ (x i ), k k+1 (i) = φ (x i ), φ (x k+1 ), i = 1,, k. 1.2,. k R-LSSVR N k ( ). (4), 11, 15 : 1 Nk K Nk + I N k γ 1 T N k 0 a Nk b k = y Nk 0 (7), (4), R-LSSVR N k. H Nk = K Nk + I Nk, P Nk = H 1 N k, a Nk = P Nk y Nk 1 Nk 1 T N P Nk y k Nk 1 T N y k Nk 1 Nk b Nk = 1T N k P Nk y Nk 1 T N k y Nk 1 Nk (8) x k+1, y k+1 R-LSSVR, x Nk+1, y Nk+1,. a Nk b Nk H Nk P Nk., P Nk+1 19 P Nk+1 = P Nk r Nk+1 r T N k+1 z Nk+1 (9)

3 T;, r Nk+1 = V T N k+1 P Nk, 1 V Nk+1 = ( ) ( ) T; K x1, x Nk+1,, K xnk, ) x znk+1 Nk+1 = 1 /(v Nk+1 V TNk+1 P Nk V Nk+1 v Nk+1 = K ( ) x Nk+1, x Nk+1 + 1/γ.,, P Nk+1 P Nk,, O(N 3) O(N 2). k k, ;, 11, 15., H Nk ( H N ), P N, 15., O(N 3) k O(N 2 ). R-LSSVR k,,. R- LSSVR, R-LSSVR,, 11, R-LSSVR JITL /, SVR, LSSVR,, ;. JITL,, x q,,,, JITL x q, (Lazy learning) JITL,. JITL :,, x q,,. JITL SVR LSSVR,, 13 18,., x q X x i s qi : s qi = ρ x i x q + (1 ρ) cos(θ qi ) (10), ρ, ρ = 1( ρ = 0) ( ) , s qi l S sim l min, l max l ,, l,., l x q, l., s l : s l = l s qi i=1 l max, l min l l max (11) s qi i=1, x q, l., l ( s l = 0.9), ( 0.75 s l 0.95) l,. s l x q, l,. 2.2 JITL,, JITL ,, Ge,, 18.,,.,. JITL,,,,. 1, ,,.,, 12, 15. JITL,

4 5 : 605. Fig. 1 1 Flowchart of just-in-time recursive learning method for online identification 1, k k + 1,., S 1, S 2, S 3, ; k D k, ; k + 1 D k+1,.,,.,, ( ),., k + 1 k,,,. k + 1 N k, k m, O(N 3 k ), O(mN 2 k ), m N k,. (10),, ρ s l S sim., ρ s l, (Fast leave-one-out cross-validation, FLOO-CV) 15. S sim l, FLOO-CV 15 : l E FLOO CV l = e FLOO CV i (12) i=1 FLOO-CV O(l 4 ) O(l 3 ), LSSVR,. t, O(tl 4 ) O(tl 3 ) 15., KL : 1. : k,, S sim ρ, s l, LSSVR, FLOO- CV LSSVR,,. 2. : k + 1, ρ, s l, S sim.,,, E FLOO CV l ;,, ρ, s l S sim S sim, E FLOO CV l ;, E FLOO CV l ρ, s l,,,.,. ρ, s l,,, ρ, s l.,,.,, 13.,, s l l,, 0.7,, 0.75 s l 0.95 l.,, : 1) :,, FLOO-CV,,. 2) JITL : JITL, 13 18,,,. JITL,,,, JITL. 3 :

5 , 20,., SVR, LSSVR, 12, 20,. C a, T, q c, q c C a, C a,k+1, x k = C a,k, C a,k 1, q c,k, q c,k 1 T12, 20, 1%. 2 k p, k p , LSSVR : , 20., k p,,,. Fig. 2 2 Main input and output variables and the time-varying parameter of the process JITL, LSSVR, JR-LSSVR (Just-in-time recursive LSSVR, JR- LSSVR); R-LSSVR, 11, 15 ; LSSVR 3 5. (Root mean squares error, RMSE), (Relative RMSE, RE) (Maximal absolute error, MAE), 3 12., ( d) 12,., FLOO-CV LSSVR, γ, d R- LSSVR JR-LSSVR. N R-LSSVR, N,, N 10. JR-LSSVR, ρ, s l FLOO-CV, ρ 0.8, 0.85, 0.9, s l 0.95 l.,, JR-LSSVR R-LSSVR ,, JR-LSSVR R-LSSVR LSSVR. R-LSSVR, N, N, N, N., N,, N ;,, N. 3 4 JR-LSSVR, R-LSSVR (N = 40, ) LSSVR 3., LSSVR 75,,, ( ),. 3 4 R-LSSVR JR-LSSVR, , JR-LSSVR, R-LSSVR,. 5, JR-LSSVR FLOO-CV ρ l, S sim., ( ),, X x i x q, s l, S sim x q. X, S sim. JITL

6 5 : 607 l min, l max l 13 15,18, l. s l l,., 11 12,15,,.. 5,,, JR-LSSVR. Fig. 3 Fig Comparison of prediction results with three 4 identification methods 3 Comparison of absolute prediction errors with three identification methods, 5,., 69 77, 68 76, ,,, 10 %,, 5 Fig. 5 ρ l Online selection of the similarity parameter ρ and the number of similar samples l for each query sample, JR-LSSVR, 3 % 5 %, JR-LSSVR R-LSSVR, JR-LSSVR. ( ), ; FLOO-CV,.,. R-LSSVR N = 30 40, N,., JR-LSSVR R-LSSVR,.,,, Gao 21. 4,. JITL : JITL

7 Table 1 1 JR-LSSVR, R-LSSVR LSSVR Comparison of identification results with JR-LSSVR, R-LSSVR, and LSSVR methods 1 % 1 % 1 % 3 % 5 % 3 % 5 % RMSE RE (%) MAE RMSE RE (%) JR-LSSVR R-LSSVR ( ) R-LSSVR (N = 10) R-LSSVR (N = 20) R-LSSVR (N = 30) R-LSSVR (N = 40) R-LSSVR (N = 50) R-LSSVR (N = 60) R-LSSVR (N = 70) LSSVR ;,. JITL,... References 1 Ljung L, Hjalmarsson H, Ohlsson H. Four encounters with system identification. European Journal of Control, 2011, 17(5): Himmelblau D M. Accounts of experiences in the application of artificial neural networks in chemical engineering. Industrial and Engineering Chemistry Research, 2008, 47(16): Wang L X. Adaptive Fuzzy Systems and Control: Design and Stability Analysis. New Jersey: Prentice-Hall, Scholkopf B, Smola A J. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. Cambridge, MA: MIT Press, Suykens J A K, van Gestel T, de Brabanter J, De Moor B, Vandewalle J. Least Squares Support Vector Machines. Singapore: World Scientific, Rojo-Alvarez J L, Martinez-Ramon M, de Prado-Cumplido M, Artes-Rodriguez A, Figueiras-Vidal A R. Support vector method for robust ARMA system identification. IEEE Transactions on Signal Processing, 2004, 52(1): Toivonen H T, Totterman S, Akesson B. Identification of state-dependent parameter models with support vector regression. International Journal of Control, 2007, 80(9): Totterman S, Toivonen H T. Support vector method for identification of Wiener models. Journal of Process Control, 2009, 19(7): Li C H, Zhu X J, Cao G Y, Sui S, Hu M R. Identification of the Hammerstein model of a PEMFC stack based on least squares support vector machines. Journal of Power Sources, 2008, 175(1): Wang H, Pi D Y, Sun Y X. Online SVM regression algorithm-based adaptive inverse control. Neurocomputing, 2007, 70(3): Tang H S, Xue S T, Chen R, Sato T. Online weighted LS- SVM for hysteretic structural system identification. Engineering Structures, 2006, 28(12): Liu Y, Wang H Q, Yu J, Li P. Selective recursive kernel learning for online identification of nonlinear systems with NARX form. Journal of Process Control, 2010, 20(2): Cheng C, Chiu M S. A new data-based methodology for nonlinear process modeling. Chemical Engineering Science, 2004, 59(13): Pan T H, Li S Y, Cai W J. Lazy learning-based online identification and adaptive PID control: a case study for CSTR process. Industrial and Engineering Chemistry Research, 2007, 46(2): Liu Yi. Research on Kernel Learning Adaptive Modeling and Control for Industrial Batch Processes Ph. D. dissertation, Zhejiang University, China, 2009 (..,, 2009) 16 Fujiwara K, Kano M, Hasebe S, Takinami A. Soft-sensor development using correlation-based just-in-time modeling. AIChE Journal, 2009, 55(7): Liu Y Q, Huang D P, Li Y. Development of interval soft sensors using enhanced just-in-time learning and inductive confidence predictor. Industrial and Engineering Chemistry Research, 2012, 51(8): Ge Z Q, Song Z H. A comparative study of just-in-timelearning based methods for online soft sensor modeling. Chemometrics and Intelligent Laboratory Systems, 2010, 104(2): Golub G H, van Loan C F. Matrix Computations. Baltimore: The John Hopkins University Press, 1996.

5 : 609 20 Nikravesh M, Farell A E, Stanford T G. Control of nonisothermal CSTR with time varying parameters via dynamic neural network control (DNNC).

IEEE Transactions on Industrial Informatics, 2008, 4(4): 271 279. 2009.,.. E-mail: yliuzju@zjut.edu.cn (LIU Yi Associate professor at Zhejiang University of Technology. He received his Ph. D.

Corresponding author of this paper.). 2002.,. E-mail: jinfujiang@163.com (JIN Fu-Jiang Professor at Huaqiao University. He received his Ph. D. degree form Zhejiang University in 2002.

8 5 : Nikravesh M, Farell A E, Stanford T G. Control of nonisothermal CSTR with time varying parameters via dynamic neural network control (DNNC). Chemical Engineering Journal, 2000, 76(1): Gao Z W, Dai X W, Breikin T, Wang H. Novel parameter identification by using a high-gain observer with application to a gas turbine engine. IEEE Transactions on Industrial Informatics, 2008, 4(4): ,.. yliuzju@zjut.edu.cn (LIU Yi Associate professor at Zhejiang University of Technology. He received his Ph. D. degree from Zhejiang University in His research interest covers process systems engineering, including statistical learning methods with applications to industrial process modeling and control. Corresponding author of this paper.) ,. jinfujiang@163.com (JIN Fu-Jiang Professor at Huaqiao University. He received his Ph. D. degree form Zhejiang University in His research interest covers industrial process modeling, control and optimization.) zlgao@zjut.edu.cn (GAO Zeng-Liang Professor at Zhejiang University of Technology. He received his Ph. D. degree from East China University of Science and Technology in His research interest covers process equipment and control engineering in industrial processes.)

* CUSUM EWMA PCA TS79 A DOI /j. issn X Incipient Fault Detection in Papermaking Wa

* CUSUM EWMA PCA TS79 A DOI /j. issn X Incipient Fault Detection in Papermaking Wa 2 *. 20037 2. 50640 CUSUM EWMA PCA TS79 A DOI 0. 980 /j. issn. 0254-508X. 207. 08. 004 Incipient Fault Detection in Papermaking Wastewater Treatment Processes WANG Ling-song MA Pu-fan YE Feng-ying XIONG