R α Cortes Vapnik 90 R emp α 1 - η 2 - VC 3 support vector machine SVM 1 h R α R emp ( ln 2l + 1 h ) - ln η 4 α + 1 槡 l α f x α α Λ 2 SVM α Λ

Size: px

Start display at page:

Download "R α Cortes Vapnik 90 R emp α 1 - η 2 - VC 3 support vector machine SVM 1 h R α R emp ( ln 2l + 1 h ) - ln η 4 α + 1 槡 l α f x α α Λ 2 SVM α Λ"

件储
5 years ago
Views:

1 ELECTRI C MACHINES AND CONTROL Vol. 17 No. 3 Mar : 为提高支持向量机的泛化能力, 根据统计学习理论中学习机推广能力的界和 VC 维理论, 提出了一种同时以特征空间中样本数据分布半径最小化和类间间隔最大化为优化目标的支持向量机模型通过合理设计其目标函数, 将该支持向量机的建模问题转化为二次规划问题, 从而可以采用与传统 SVM 相似的算法快速实现用 UCI 数据库中的部分数据进行了仿真, 实验证明与传统的 SVM 相比, 在分类准确度不降低, 且有所提高的基础上, 使其支持向量的数目得到减少 ; 在支持向量数目相近的情况下, 预测精度得到提高体现出更强的泛化能力 : 支持向量机 ; 泛化性能 ; 数据分布 ; 分类 TF 181 A X Support vector machine with minimum-data distribution and maximum-margin in feature space CHEN Jun 1 PENG Xiao-qi 1 2 SONG Yan-po 1 LIU Zheng 1 TANG Xiu-ming 3 1. School of Information Science and Engineering Central South University Changsha China 2. Department of Information Science and Engineering Hunan First Normal University Changsha China 3. Institute of Information and Electrical Engineering Hunan University of Science and Technology Xiangtan China Abstract In order to improve the generalization capability of support vector machine a new support vector machine with minimum-data distribution and maximum-margin in feature space was presented based on the bounds for the generalization error in machine-learning and the VC dimension theory. The support vector machine SVM optimization problem here was treated as a quadratic programming problem after reformulating a reasonable objective function so the problem was solved by applying algorithms similar to that of the traditional SVM. Simulation using UCI datasets shows that compared to the traditional SVM the improved SVM classifier reduces the support vectors number with the same or even higher prediction accuracy on the other hand it improves the prediction accuracy with similar support vectors number. The simulation results show that the proposed SVM have more generalization ability. Key words support vector machine generalization data distribution classification : 国家自然科学基金重点项目 ( ) ; 国家自然科学基金青年项目 ( ) ; 湖南省自然科学基金青年项目 ( 11JJ4057) ; 湖南省教育厅资助项目 ( 10C0696) : 陈君 ( 1977 ), 男, 博士研究生, 研究方向为智能决策, 工业过程优化决策与控制 ; : 陈君彭小奇 ( 1962 ), 男, 博士, 教授, 博士生导师, 研究方向为系统建模智能决策等 ; 宋彦坡 ( 1979 ), 男, 博士, 副教授, 研究方向为热工过程优化决策支持等 ; 刘征 ( 1979 ), 女, 博士研究生, 研究方向为工业过程优化决策与控制 ; 唐秀明 ( 1977 ), 女, 博士研究生, 研究方向为电力系统负荷建模

2 R α Cortes Vapnik 90 R emp α 1 - η 2 - VC 3 support vector machine SVM 1 h R α R emp ( ln 2l + 1 h ) - ln η 4 α + 1 槡 l α f x α α Λ 2 SVM α Λ < η < η f x α ε ε > 0 h f x α VC l SVM 8-11 Φ l h( ln ( ) 2l + 1 h ) - ln η 4 2 h 8-9 槡 l SVM 10 R α R emp α + Φ( l h ) 3 SVM 3 Φ Φ 11 VC h SVM 12 VC h l VC SVM SVM VC 15 SVM SVM R α Φ 2 - SVM 3 support vector machine with minimum - data distribution and maximum x 1 y 1 x l y l X R F - margin MDMM - SVM Δ l f w Ff w sgn x w w 1 w f w y i f w x i SVM Δ l F VC h R 2 h min( [ 2 Δ ] n ) n Vapnik 4 R2 Δ < n R 2

3 3 107 Δ F VC SVM r 2 + ξ r VC 4 5 R Δ VC h 5 Lagrange 2 L w b c r ξ r ξ w 1 2 w 2 + C w ξ w i 2. 1 l x 1 y 1 x l y l X X R d y i i x F x i X R d x i F - ξ r i - β r iξ r i 6 VC X 珔 α α w β β r Lagrange x i l i 1 6 w b c r ξ r i ξ w i KTT F c r F w x i - b 0 L w b c r ξ r ξ w w - c w y i α i x i 0 r L w b c r ξ r ξ w b y i α i 0 1 min w b r c ξ w ξ r 2 w 2 + C w ξ w i + r 2 + C r ξ r i L w b c r ξ r ξ w 2r 1 - r ( β i ) 0 x i - c 2 r 2 + ξ r i ξ r i 0 2 l 5 L w b c r ξ r ξ w s. t. c 2 β i x i - c 0 y i w x i + b 1 - ξ w i L w b c r ξ r ξ w ξ w i 0 2 l - β ξ r i - β r i + C r 0 C r C w 0 L w b c r ξ r ξ w - α ξ w i - α w i + C w 0 ξ r ξ w SVM w y i α i x i 8 ( + C w w ) r 2 ξ i x i - c 2 r 2 + ξ r i ξ r i 0 2 l SVM 5 c β i x / i - α i y i w x i + b ξ w i - C r ξ r i c 2 - r 2 + α w i ξ w i + r 2 + β i x i - y i α i 0 9 β 10 β i 11

4 C r β i + β r i C w α i + α w i ~ 13 6 α β max α β l s. t. { β i k x i x i - α i l i j 1 i j 1 β i β j k x i x k + y i y j α i α j k x i x j y i α i 0 0 α i C w β i 2 l 1 0 β i C r 2 l l w * b * y j - β * i y i x i 15 y i x i x i 16 f x sgn * ( y i x i x + b ) 17 3 UCI http / /archive. ics. uci. edu /ml /datasets. html 3 breast - cancer % % Statlog % % MDMM - SVM SVM MDMM - SVM heart 1 Statlog % Table % 8% Attributes of the datasets in the UCI database heart breast - cancer Statlog Fig. 1 MDMM-SVM 传统 SVM 1 k x i y i exp - x i - y i ( 2 2 σ ) 18 breast - cancer 1 VC 2 SVM 预测准确率核参数滓 MDMM-SVM SVM Prediction accuracy of MDMM-SVM and traditional SVM with different kernel parameters 1 MDMM - SVM SVM 2 2 Table 2 MDMM-SVM SVM Generalization performance compared between MDMM-SVM and traditional maximum-margin SVM MDMM - SVM /% /% SVM heart % % 10. 5% breast - cancer 1% % SVM 4

5 3 109 MDMM - SVM CORTES C VAPNIK V. Support vector networks J. Machine Learning VAPNIK V. An overview of statistical learning theory J. IEEE Transactions on Neural Networks VAPNIK V. The nature of statistical learning theory M. New York Springer - Verlag RODRIGUEZ - LUJAN I SANTA CRUZ C HUERTA R. Hierarchical linear support vector machine J. Pattern Recognition NIU Xiaoxiao SUEN Ching Y. A novel hybrid CNN-SVM classifier for recognizing handwritten digits J. Pattern Recognition Control and Decision SHIVASWAMY P K. Large Relative Margin and Applications J D. New York Faculty of Arts and Sciences Columbia University WANG Decheng LIN Hui. Vector control method based on support 17. vector machine and its application J. Electric Machines and Control GRYLLIAS K C ANTONIADIS I A. Support vector machine approach based on physical model training for rolling element bearing fault detection in industrial environments J. Engineering Applications of Artificial Intelligence ANGUITA D BONI A RIDELLA S. Evaluating the generalization ability of support vector machines through the bootstrap J. Neural Processing Letters J ZHOU Weida ZHANG Li JIAO Licheng. An improved principle for measuring generalization performance J. Chinese Journal of Computers WANG Xizhao HE Qiang. Enhancing generalization capability of SVM classifiers with feature weight adjustment knowledge - based J. Intelligent Information and Engineering Systems XUE Hui CHEN Songcan YANG Qiang. Structural regularized support vector machine a framework for structural large margin classifier J. IEEE Transactions on Neural Networks J WEN Chuanjun ZHAN Yongzhao CHEN Changjun. Maximal margin minimal volume hypersphere support vector machine J. 14 SHIVASWAMY P K JEBARA T. Maximum relative margin and data - dependent regularization J. The Journal of Machine Learning Research FUNG G MANGASARIAN O L SHAVLIK J. Knowledge - based support vector machine classifiers J Advances in Neural Information Processing Systems ( 编辑 : 张诗阁 )

SVM OA 1 SVM MLP Tab 1 1 Drug feature data quantization table

SVM OA 1 SVM MLP Tab 1 1 Drug feature data quantization table 38 2 2010 4 Journal of Fuzhou University Natural Science Vol 38 No 2 Apr 2010 1000-2243 2010 02-0213 - 06 MLP SVM 1 1 2 1 350108 2 350108 MIP SVM OA MLP - SVM TP391 72 A Research of dialectical classification