100084 E-mail: ahz@mail.tsinghua.edu.cn (Look Up Table, LUT) Adaboost 300 Adaboost TP391 Real Time Facial Expression Classification WANG Yubo, AI Haizhou, WU Bo, HUANG Chang Computer Science and Technology Department, Tsinghua University State Key Laboratory of Intelligent Technology and Systems E-mail: ahz@mail.tsinghua.edu.cn Abstract: In this paper, the problem of facial expression classification is discussed, which is very 603320101979 1964 1979 1981
difficult because of its diversity and complexity. An Adaboost algorithm based on Look-Up-Table (LUT) weak classifier is presented to train facial expression classifier. The experimental results show that compared to Support Vector Machines (SVMs), our method has almost the same correct rate; and nearly 300 times faster in speed. Our method is almost real time, and has significant value in application. Key Words: Facial expression classification; Adaboost; Look-Up-Table 1 [1](anger) (disgust)(fear)(happiness)(neutral)(sadness)(surprise) 1 [2]
1 7 [1](anger)(disgust)(fear) (happiness)(neutral)(sadness)(surprise) [2] M.J. Lyons [3] JAFFE 92%C. Padgett [4] Ekman [5] 86%M. Pantic [6] 265 91%T. Otsuka [7] [8] [9][10]Viola Jones[11] Haar AdaBoost M.S. Bartlett [12] Gabor AdaSVM Cohn-Kanade [13] (Look-Up-Table, LUT) Adaboost [14] SVM
300 2 2 Viola [11] Haar Adaboost [15] 2 3 Adaboost 3.1 Haar Haar Viola [11] Haar
3Haar Haar a. b. 1 A A 2 A+B 3 A+C 4 A+B+C+D 123 4 D 1+4-2-3i i c. d. Haar Haar 3 Haar Haar I( xy, ) I ( uv, ) u v I ( uv, ) I( xydxdy, ) = x= 0 y= 0 P P Haar 3 3.2 Adaboost Adaboost [16] 50% T h( x) = sgn( ah( x) b) h i T b i= 1 i i Adaboost D t
m r = D () i yh( x ) m t t i t i i= 1 1 1 rt h t D t r t Adaboost Adaboost Adaboost [16] T 1 {: i H ( x ) y } Z i i t m t= 1 h t 50% Z t 1[16] k Y = {1,, k} X Y [ 1,1] 1 yi = l Yil (, ) = 1 yi l Adaboost Adaboost [16] ( x 1, y 1 ),,( xm, ym) x i X y i Y m D 1 (, i l) = 1/( mk), i=1,,m, l=1,,k t = 1,,TT : 1. D t ht( xi, l) rt = Dt(, ilyilh ) (, ) t( xi, l) 1 1+ r 2. ln t αt = 2 1 rt 3. i, l
D t+ 1 (, i l) = ( α x ) D (, i l)exp Y(, i l) h(, l) t t t i Z t Z t T H( x) = argmax αtht( x, l) l t= 1 Adaboost r t Hamming Hamming 50%[16] Adaboost T k 1 Z t= 1 t 3.3 LUT Adaboost Viola Jones[11] 4 h( ) = sgn [ f ( ) b] x x f Haar Haar b Haar Haar Haar Adaboost LUT 4
4 LUT Haar f [0, 1] n bin [( j 1)/ n, j / n) Haar j = ( j) ( j) j=1,,n w 1, w 2 f ( x) bin h( x) = P P ( j ), ( j P P ) Haar j 1 2 1 2 Haar bin j ( ) P = P x w f ( x) bin, i = 1, 2, j = 1,, n ( j) i i Haar j [ ) [ ) 1 1/, / j u j n j n Bn ( u) =, j = 1,, n 0 u j 1/ n, j/ n LUT n ( j) ( j) j ( 1 2 ) ( ) h ( x) = P P B f ( x ) LUT n LUT j= 1 k w 1,, wk k>2 LUT f ( x) bin ( j) h( x, l) = 2P 1, l = 1,, k LUT l [ ) j, l 1 u j 1/ n, j/ n y = l Bn ( u, y) =, j = 1,, n, l = 1,, k 0 otherwise LUT Haar j n k ( j) j, l ( ) ( ) h ( x, y) = 2P 1 B f ( x), y LUT l n LUT j= 1 l= 1 4 JAFFE(Japanese Female Facial Expression)[1] 10 7 213 5 Adaboost (1) (2) 1.1 (3) 1.1 (4) 5
24 5112 Adaboost k=7 Adaboost 400 Haar 5 Support Vector MachinesSVMs[17] 7 5112 SVM 1870 JAFFE [1] 99.1% Internet 206 385 CPU Pentium IV 2.53GHz 512MB 1 Adaboost SVM 300 SVM 6 1 RBF-kernel SVM LUT Adaboost
92.6% 91.4% 28.7 0.11 6 7 7 LUT Adaboost
RBF SVM Haar Adaboost Adaboost SVM 300 LUT Adaboost 5 LUT Adaboost LUT Adaboost Adaboost SVM SVM 300 LUT Adaboost 6 [1] Lyons M J. The Japanese Female Facial Expression (JAFFE) Database [DB], http://www.mis.atr.co.jp/~mlyons/jaffe.html, 1998. [2] Pantic M and Rothkrantz Leon J M. Automatic Analysis of Facial Expressions: The State of Art [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(12): 14241445.
[3] Lyons M J, Budynek J, and Akamatsu S. Automatic Classification of Single Facial Images [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1999, 21(12): 1357 1362. [4] Padgett C and Cottrell G W. Representing Face Images for Emotion Classification. In: Proceedings of Conference Advances in Neural Information Processing Systems [A], Denver, USA, 1996. 894900. [5] Ekman P and Friesen W V. Unmasking the Face [M]. New Jersey: Prentice Hall, 1975. [6] Pantic M and Rothkrantz L J M. Expert System for Automatic Analysis of Facial Expression [J]. Image and Vision Computing J., 2000, 18(11): 881905. [7] Otsuka T and Ohya J. Spotting Segments Displaying Facial Expressions from Image Sequences Using HMM [A]. In: IEEE Proceedings International Conference on Automatic Face and Gesture Recognition, Nara, Japan, 1998. 442447. [8] Liang Luhong, Ai Haizhou, Xiao Xipan, et al. Face Detection Based on Template Matching and Support Vector Machines [J]. Chinese Journal of Computers, 2002, 25(1): 2229. ( in Chinese) [J] 2002, 25(1): 2229 [9] Huang J, Shao X, and Wechsler H. Face pose discrimination using support vector machines [A]. In: IEEE Proceedings International Conference on Pattern Recognition, Brisbane, Australia, 1998. 154156.
[10] Guo Guodong, Li S Z, Chan Kapluk. Face recognition by support vector machines [A]. In: IEEE Proceesings International Conference on Automatic Face and Gesture Recognition, Grenoble, France, 2000. 196201. [11] Viola P and Jones M. Rapid Object Detection using a Boosted Cascade of Simple Features [A]. In: IEEE Proceedings International Conference on Computer Vision and Pattern Recognition, Hawaii, USA, 2001. 511518. [12] Bartlett M S, Littlewort G, Fasel I, et al. Real Time Face Detection and Facial Expression Recognition: Development and Applications to Human Computer Interaction [A]. In: IEEE Workshop on Computer Vision and Pattern Recognition for Human-Computer Interaction, Wisconsin, USA, 2003. [13] Kanade T, Cohn J F, and Tian Y L. A comprehensive database for facial expression analysis [A]. In IEEE Proceedings International Conference on Automatic Face and Gesture Recognition, Grenoble, France, 2000. 4653. [14] Wu Bo, Ai Haizhou, Huang Chang. LUT-Based Adaboost for Gender Classification [A]. In: International Conference on Audio- and Video-Based Biometric Person Authentication, Guildford, UK, 2003. 104110. [15] Ai Haizhou, Xiao Xipan, and Xu Guangyou. Face Detection and Retrieval [J]. Chinese Journal of Computers, 2003, 26(7): 874881. ( in Chinese) [J]2003, 26(7): 874881 [16] Schapire R E and Singer Y. Improved boosting algorithms using confidence-rated predictions [J]. Machine Learning, 1999, 37(3): 297336.
[17] Moghaddam B and Yang M H. Gender Classification with Support Vector Machines [J], IEEE Transactions on Pattern Analysis and Machine Intelligence, 2002, 24(5): 707711.