[15] (Max-margin domain transforms, MMDT),,, SVM (Support vector machine),., MMDT,, QP (Quadratic programming),, (Extreme learning machi

Size: px

Start display at page:

Download "[15] (Max-margin domain transforms, MMDT),,, SVM (Support vector machine),., MMDT,, QP (Quadratic programming),, (Extreme learning machi"

累乐
5 years ago
Views:

1 44 2 Vol. 44, No ACTA AUTOMATICA SINICA February, , (Extreme learning machine, ELM), ELM,.,,.,,.,,. DOI,,,,,,,,.., 2018, 44(2): /j.aas.2018.c Domain Adaption Algorithm with ELM Parameter Transfer XU Su-Hui 1 MU Xiao-Dong 1 CHAI Dong 2 LUO Chang 3 Abstract In allusion to transfer learning problem with a small number of labeled samples, a domain adaption method through transferring extreme learning machine (ELM) parameters is proposed in this paper. The core idea is projecting the target ELM parameters on to the source and making the parameters maximally aligned with the source. In addition, considering the transformation may cause negative transfer, a regular term is added to the objective function. Unlike the existing domain adaption method, the parameters of classifier and the transformation matrix can be calculated simultaneously, and the objective function can be easily solved. Experiments demonstrate the proposed method has potential advantages in terms of accuracy and efficiency compared to the state-of-the-art approaches. Key words feature Domain adaption, transfer learning, extreme learning machine, regularization, middle-level feature, deep Citation Xu Su-Hui, Mu Xiao-Dong, Chai Dong, Luo Chang. Domain adaption algorithm with ELM parameter transfer. Acta Automatica Sinica, 2018, 44(2): ,, [1].,,, [2 3]. [4 5] [6 11] [12].,, Manuscript received December 11, 2016; accepted March 30, 2017 Recommended by Associate Editor WANG Zhan-Shan Department of Information Engineering, Rocket Force University of Engineering, Xi an Beijing Aeronautical Technology Research Institute, Beijing Air and Missile Defense College, Air Force Engineering University, Xi an , [1].,., Kulis [6, 13]. [13],, [6] (Asymmetric regulized cross-domain tranformation, ARC-t), [13]., ARC-t,. [7] (Heterogeneous features augmented, HFA),, ARC-t,. [14] (Geodesic flow kernel, GFK),,, GFK

2 [15] (Max-margin domain transforms, MMDT),,, SVM (Support vector machine),., MMDT,, QP (Quadratic programming),, (Extreme learning machine, ELM) [16],, ;,,. ELM, [17 18]., (Domain adaption with parameter transfer, DAPT), ELM,. DAPT ELM,,, ( 1 ),.. ω i b i, ELM,,. ELM y = L β i g(ω T i x + b i ) (1) i=1 g(ω1 T x 1 + b 1 )... g(ωlx T 1 + b L ) H =....., g(ω1 T x m + b 1 )... g(ωlx T m + b L ) ELM : min Hβ T 2 + C β 2 (2) β 2 C. : β = (H T H + CI) H T T (3), A A Moore-Penrose., MMDT QP, ELM (3), MMDT. 1 Table 1 The application comparison between DAPT and previous methods ARC-t GFK HFA MMDT DAPT!!!!!!!!!! ELM, 1 : m, L, n. x 1, x 2,, x p, t 1, t 2,, t p. g(ω T i x j +b i ), ω m L, ω i i, b i i. β, L n, β i i 1 Fig. 1 ELM network 1.2 ELM DAPT MMDT, DAPT ELM. ELM β, DAPT W,. DAPT : min J = L s(β)+l t (β, W )+R(β)+Q(β, W ) (4) β,w L s (β) = 1 2 H sβ T s 2 (5)

3 2 : 313 L t (β, W ) = 1 2 H tw β T t 2 (6) R(β) = C 1 2 β 2 (7) Q(β, W ) = C 2 2 W β β 2 (8) L s (β), L t (β, W ),, W β, R(β),., Q(β, W ),. C 1, C 2. : W β. : 1. W = I; 2. W, β:, J 1 = J, β : β = arg min J 1 (9) J 1 β =H T s (H s β T s ) + W T H T t (H t W β T t )+ C 1 β + C 2 (W I) T (W I)β 0 : β = (H T s H s + W T H T t H t W + C 1 I + C 2 (10) (W I) T (W I)) (H T s T s + W T H T t T t ) (11) 3. β, W : W = arg min J 2 = ( 1 arg min 2 H tw β T t 2 + C ) 2 W β β 2 2 (12) W : J 2 W = H T t (H T t W β T t )β T +C 2 (W β β)β T (13) 0, : W = (H T t H t + C 2 I) (H T t H t β T + C 2 ββ T )(ββ T ) (14) 4. 2 β. : 2 y = H t (x)w β (15),. : ARCt [6] GFK [14] HFA [7] MMDT [15] ,, office-caltech256 4, : amazon webcam dslr caltech, 10, backpacks keyboards., 12. webcam dslr caltech, 8, amazon, 20 ; 3 ;., 12, 20, 20,. ARC-t GFK HFA MMDT.,,, SVM ELM, 4, SVMs, SVMt., ELMs ELMt SVMs SVMt. DAPT, DAPT, DAPT n,. BOVW (Bag of visual words),,, BOVW,. BOVW :, SURF (Speed-up robust features),,, 800. Caffe, Alexnet

4 314 44, fc ,., ARC-t GFK HFA MMDT, DAPT 3, L C 1 C 2, : C 1 = , C 2 = 2 000, L = 200, L = : 2 : 1) SVMs SVMt HFA MMDT, 4 SVM, HFA MMDT SVMs SVMt;, ELM, DAPT ELMs ELMt, :. 2) SVMs SVMt ELMs ELMt, ELM SVM, ELMt MMDT,, ELM MMDT, : ELM, SVM. 3) 9, DAPT 11,, webcam dslr, GFK w d, d w. 4) c a, c w, 4,,, DAPT ; 5), 5, ARC-t HFA, ARC-t, HFA,. DAPT, ELM. 6) DAPT n DAPT, c a, c w, ELMt,. DAPT.,. 3 : 1) 4 096, ), : ; ELM SVM;. 3) 9, DAPT. 4), , DAPT. 2.2 bing-caltech DAPT, bing-caltech., 256. bing 300, caltech 80, [19]. bing 2 BOVW (%) Table 2 Accuracy for all the methods when using BOVW feature (%) SVMs SVMt ELMs ELMt ARC-t GFK HFA MMDT DAPT n DAPT a w a d a c w a w d w c d a d w d c c a c w c d acc (%) time (s)

5 2 : (%) Table 3 Accuracy for all the methods when using deep feature (%) SVMs SVMt ELMs ELMt ARC-t GFK HFA MMDT DAPT n DAPT a w a d a c w a w d w c d a d w d c c a c w c d acc (%) time (s) , Caltech., : 1) n t = 10, n s = 5, 10, 20, 50, 100, ) n s = 50, n t = 5, 10, 15, 20, 25, 30.,, ARC-t HFA, : 1) DAPT n s n t,, : 2) DAPT n DAPT,, SVMt ELMt, : 3) DAPT,,. 2.3 DAPT L C 1, C 2, office-caltech256,., C 1, C 2, L. C 1 = 2 000, C 2 = 2 000, L 17, ,, 12, 5 L 3. 3, 2 bing-caltech Fig. 2 The accuracy on bing-caltech dataset

6 ,,.,,. References Fig. 3 3 L The accuracy curves varying with L 1 Zhuang Fu-Zhen, Luo Ping, He Qing, Shi Zhong-Zhi. Survey on transfer learning research. Journal of Software, 2015, 26(1): (,,,.., 2015, 26(1): 26 39) 2 Pan S J, Yang Q. A survey on transfer learning. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10): Weiss K, Khoshgoftaar T M, Wang D D. A survey of transfer learning. Journal of Big Data, 2016, 3(1): Article No. 9 4 Wang C, Mahadevan S. Heterogeneous domain adaptation using manifold alignment. In: Proceedings of the 22nd International Joint Conference on Artificial Intelligence. Barcelona, Spain: AAAI, 2011, 2: C 1 C 2 5 Domeniconi G, Moro G, Pagliarani A, Pasolini R. Markov chain based method for in-domain and cross-domain sentiment classification. In: Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K). Lisbon, Portugal: IEEE, Fig. 4 The accuracy curves varying with C 1 and C 2 L 200,,,,.,, 200. L = 200, C 1, C 2 1, 10, 100, 1 000, 2 000, BOVW , 4,., C , C 1 ; C 2 ; C , C ,. 3,. ELM,,,, 6 Kulis B, Saenko K, Darrell T. What you saw is not what you get: domain adaptation using asymmetric kernel transforms. In: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Providence, RI, USA: IEEE, Duan L X, Xu D, Tsang I W. Learning with Augmented Features for Heterogeneous Domain Adaptation. In: Proceedings of the 29th International Conference on Machine Learning. Edinburgh, UK: ARXIV, Zhu Y, Chen Y, Lu Z, Pan S J, Xue G R, Yu Y, Yang Q. Heterogeneous Transfer Learning for Image Classification. In: Proceedings of the 25th AAAI Conference on Artificial Intelligence. San Francisco, USA: AAAI, Oquab M, Bottou L, Laptev I, Sivic J. Learning and transferring mid-level image representations using convolutional neural networks. In: Proceedings of the 2004 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Columbus, OH, USA: IEEE, van Opbroek A, Ikram M A, Vernooij M W, de Bruijne M. Transfer learning improves supervised image segmentation across imaging protocols. IEEE Transactions on Medical Imaging, 2015, 34(5):

2期许夙晖等: 基于极限学习机参数迁移的域适应算法 11 Shu Xing, Yu Hui-Min, Zheng Wei-Wei, Xie Yi, Hu Hao-Ji, Tang Hui-Ming.

基于边际 Fisher 准则和迁移学习的小样本集分类器设计算法. 自动化学报, 2016, 42(9): 1313 1321) 12 Nam J, Kim S. Heterogeneous defect prediction. In: Proceedings of the 10th joint Meeting on Foundations of Software Engineering.

In: Proceedings of the 11th European Conference on Computer Vision: Part IV. Heraklion, Greece: Springer, 2010. 213 226 14 Gong B Q, Shi Y, Sha F, Grauman K.

2066 2073 15 Hoffman J, Rodner E, Donahue J, Darrell T, Saenko K. Efficient learning of domain-invariant image representations.

7 2期许夙晖等: 基于极限学习机参数迁移的域适应算法 11 Shu Xing, Yu Hui-Min, Zheng Wei-Wei, Xie Yi, Hu Hao-Ji, Tang Hui-Ming. Classifier-designing algorithm on a small dataset based on margin Fisher criterion and transfer learning. Acta Automatica Sinica, 2016, 42(9): (舒醒, 于慧敏, 郑伟伟, 谢奕, 胡浩基, 唐慧明. 基于边际 Fisher 准则和迁移学习的小样本集分类器设计算法. 自动化学报, 2016, 42(9): ) 12 Nam J, Kim S. Heterogeneous defect prediction. In: Proceedings of the 10th joint Meeting on Foundations of Software Engineering. Bergamo, Italy: ACM, Saenko K, Kulis B, Fritz M, Darrell T. Adapting visual category models to new domains. In: Proceedings of the 11th European Conference on Computer Vision: Part IV. Heraklion, Greece: Springer, Gong B Q, Shi Y, Sha F, Grauman K. Geodesic flow kernel for unsupervised domain adaptation. In: Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Providence, RI, USA: IEEE, Hoffman J, Rodner E, Donahue J, Darrell T, Saenko K. Efficient learning of domain-invariant image representations. In: Proceedings of the 2013 International Conference on Learning Representations. Arizona, USA, Huang G B, Zhu Q Y, Siew C K. Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of the 2004 IEEE International Joint Conference on Neural Networks. Budapest, Hungary: IEEE, 2004, 2: Horata P, Chiewchanwattana S, Sunat K. Robust extreme learning machine. Neurocomputing, 2013, 102: Huang G, Song S J, Gupta J N D, Wu C. Semi-supervised and unsupervised extreme learning machines. IEEE Transactions on Cybernetics, 2014, 44(12): Bergamo A, Torresani L. Exploiting weakly-labeled Web images to improve object classification: a domain adaptation approach. In: Proceedings of the 23rd International Conference on Neural Information Processing Systems. Vancouver, British Columbia, Canada: Curran Associates Inc., 许夙晖火箭军工程大学信息工程系博士研究生. 主要研究方向为遥感图像处理技术. 本文通信作者. xu suhui@163.com (XU Su-Hui Ph. D. candidate in the Department of Information Engineering, Rocket Force University of Engineering. Her main research interest is remote sensing image processing. Corresponding author of this paper.) 慕晓冬火箭军工程大学信息工程系教授. 主要研究方向为智能信息处理与计算机仿真. muxiaodong402@163.com (MU Xiao-Dong Professor in the Department of Information Engineering, Rocket Force University of Engineering. His research interest covers intelligent information processing and computer simulation.) 柴栋北京航空工程技术研究中心工程师, 博士. 主要研究方向为智能信息处理. chaibaodong@126.com (CHAI Dong Ph. D, Engineer at Beijing Aeronautical Technology Research Institute. His main research interest covers intelligent information processing.) 罗畅空军工程大学防空反导学院博士研究生. 主要研究方向为智能信息处理. luochang1988@126.com (LUO Chang Ph. D. candidate at the Air and Missile Defense College, Air Force Engineering University. His main research interest is intelligent information processing.)

第 2 期王向东等 : 一种运动轨迹引导下的举重视频关键姿态提取方法 257 竞技体育比赛越来越激烈, 为了提高体育训练的效率, 有必要在体育训练中引入科学定量的方法许多

第 2 期王向东等 : 一种运动轨迹引导下的举重视频关键姿态提取方法 257 竞技体育比赛越来越激烈, 为了提高体育训练的效率, 有必要在体育训练中引入科学定量的方法许多 2014 年 4 月图学学报 April 2014 第 35 卷第 2 期 JOURNAL OF GRAPHICS Vol.35 No.2 一种运动轨迹引导下的举重视频关键姿态提取方法王向东 1, 张静文 2, 毋立芳 2, 徐文泉 (1. 国家体育总局体育科学研究所, 北京 100061;2. 北京