Chinese Journal of Biomedical Engineering Vol. 31 No. 4 August 2012 * R A A Survey on

Size: px

Start display at page:

Download "Chinese Journal of Biomedical Engineering Vol. 31 No. 4 August 2012 * R A A Survey on"

型左
4 years ago
Views:

1 Chinese Journal of Biomedical Engineering Vol. 31 No. 4 August 2012 * R A A Survey on EEG Based Emotion Recognition NIE Dan WANG Xiao-Wei DUAN Ruo-Nan LV Bao-Liang * 1 Center for Brain-like Computing and Machine Intelligence Department of Computer Science and Engineering Shanghai Jiao Tong University Shanghai China 2 MOE-Microsoft Key Lab for Intelligent Computing and Intelligent Systems Shanghai Jiao Tong University Shanghai China Abstract Emotion is a state that comprehensively represents human feeling thought and behavior thus takes an important role in inter-personal human communication. Emotion recognition aims to automatically discriminate different emotional states by using physiological and non-physiological signals acquired from people in order to develop friendly and natural human-machine interface. EEG-based emotion recognition is one of the most common used and efficient methods for emotion recognition. This paper introduced the state-of-the-art techniques the read-world applications and some critical problems of EEG-based emotion recognition from the aspects of EEG signals recording signal preprocessing feature extraction feature selection and classification methods. Key words emotion emotion recognition EEG feature extraction machine learning - doi / j. issn CB * bllu@ sjtu. edu. cn

596 31 6 1 1. 1 6 1 Papez James Fig. 1 The Papez circuit theory of the functional 1884 neuroanatomy of emotion 6 1 1885 Lange 2 James-Lange James- 1.

2 Papez James Fig. 1 The Papez circuit theory of the functional 1884 neuroanatomy of emotion Lange 2 James-Lange James Lange 1927 Cannon James 3 Cannon 7 James Bard Ekman Cannon-Bard Cannon-Bard Clynes Izard 1937 Papez Papez 1 6 Papez 1 2 Lange Papez Maclean Papez-Maclean

4 597 11 2 Paul 12 13 2 Lange Fig. 2 The Lange theory of emotions 3 14-3 15 1 Lang 3 9 3 10 1.

3 Paul Lange Fig. 2 The Lange theory of emotions Lang Fig Stimuli distribution in the 2-D emotional space formed by the arousal and valence 11

4 EEG Tab. 1 Comparison of three technologies for emotion recognition 4 4 autonomic nervous system 2. 3 central nervous system Picard 8 20 Bartels 21 fmri Amuse EEG Infomax 21 4

5 4 599 Fig. 4 4 a b Comparison of the EEG signals before artifact removal a and after artifact removal b 2. 4 N delta 1 ~ 4 Hz theta 4 ~ 8 Hz alpha 8 ~ 13 Hz beta 13 ~ 30 Hz gamma 36 ~ 44 Hz 5 Khalil 5 22 Zhang 23 event-related potentials Zouridakis ms Eimer 28 Aftanas theta alpha beta 120 ms 29 Kothe ~

6 Aftanas Scotti 36 Zhang 23 McDonald theta alpha Lin gamma Christos 4 P100 N100 P200 N200 P delta theta alpha Murugappan Shi Murugappan alpha K Petrantonakis higher order crossings 33 Khosrowabadi 4 ~ 13 Hz PCA ICA

4 601 5 Fig. 6 Relationship between the emotion a b classification accuracies of six subjects and number of feature dimensions the Fig.

7 Fig. 6 Relationship between the emotion a b classification accuracies of six subjects and number of feature dimensions the Fig. 5 Comparison of the features before and after using the LDS approach. a The features before using the LDS approach. b The features after using the LDS approach CSP Li CSP Firpi Lin F F F F 31 Zhang

8 K K K Frantzidis Murugappan C Khosrowabadi Ko 45 Wang K Estepp K Schmidt 43 4 K K 1

4 603 49 Sarlo 52 4 53 3D alpha 54-50 Petrantonakis - asymmetry Index cross-correlation 4-51 3.

9 Sarlo D alpha Petrantonakis - asymmetry Index cross-correlation Li gamma gamma alpha beta gamma Matthews 7 39 Fig Distribution of emotional features

10 emotions Cannon W. examination and an alternative theory J Psychology Neuroscience Lang PJ Bradley MM Cuthbert BN manua M Emotion and Attention M 13 Coan JA Allen JJB McKnight PE. 5 1 James W. What is an emotion J. Mind Lange CG. The emotions a psychophysiological study J. The The James-Lange theory of emotions a critical - differences J emotions J American Journal of 4 James W Papez MD. A proposed mechanism of emotion J. Arch Neural Psychiatry Paul D Maclean MD. Psychosomatic disease and the Visceral Brain -recent developments bearing on the Papez theory of emotion J. Psychosomatic Medicine Dalgleish T. The emotional brain J. Nature Reviews 7 Healey J. Wearable and automotive systems for the recognition of affect from physiology D. Massachusetts MIT International Affective Picture System IAPS Technical Manual And Affective Ratings M. Gainesville University of Florida Frantzidis CA Bratsas C et al. Toward emotion aware computing an integrated approach using multichannel neurophysiological recordings and affective visual stimuli J. IEEE Transactions on Information Technology in Biomedicine 10 Lang PL Bradley MM Cuthbert BN. International Affective Picture System Affective ratings of pictures and instruction. Gainesville NIMH Center for the Study of. J Ekman P Friesen WV. The facial action coding system FACS a technique for the measurement of facial action. San Francisco Consulting Psychologists Press A capability model of individual differences in frontal EEG asymmetry J. Biological Psychology Crawford HJ Clarke SW Melissa KT. Self - generated happy and sad emotions in low and highly hypnotizable persons during waking and hypnosis laterality and regional EEG activity. International Journal of Psychophysiology 15 Damasio AR Grabowski TJ Bechara A et al. Subcortical and cortical brain activity during the feeling of self-generated. Nature Neuroscience

11 J. 31 Lin Yuanpin Wang Chihong Jung Tzyyping et al. EEG-based Anderson K McOwan PW. A real - time automated system for the recognition of human facial expressions J. IEEE Transaction on System Man and Cybernetics Part B Cybernetics Ang J Dhillon R Krupski A et al. Prosodybased automatic detection of annoyance and frustration in human computer dialog C / / Proceedeings of International Conference on Spoken Language Processing. Colorado ISCA J Khosrowabadi R Wahab A Ang KK et al. Affective computation on EEG correlates of emotion from musical and vocal 20 Picard RW Vyzas E Healey J. Toward machine emotional stimuli C / / Proceedings of International Joint Conference on intelligence analysis of affective physiological state J. IEEE Neural Networks. Atlanta IEEE Transaction on Pattern Analysis and Machined Intelligence 35 Aftanas LI Lotova NV Koshkarova VI et al. Non - linear Bartels G Shi LC Lu BL. Automatic artifact removal from EEG - a mixed approach based on double blind source separation analysis of emotion EEG calculation of Kolmogorov entropy and the principal Lyapunov exponent J. Neuroscience Letters and support vector machine C / / Proceedings of International 36 Scotti S Mauri M Barbieri R et al. Automatic quantitative conference of the IEEE Engineering in Medicine and Biology evaluation of emotions in e-learning applications C / / Society. Buenos Aires IEEE Khalili1 Z Moradi MH. Emotion recognition system using brain and peripheral signals using correlation dimension to improve the results of EEG C / / International Joint Conference on Neural Networks. Atlanta IEEE Proceedings of International Conference of the IEEE Engineering in Medicine and Biology Society New York City IEEE McDonald NJ Soussou W. QUASAR s QStates cognitive gauge performance in the cognitive state Assessment competition Zhang Qing Lee Minho. A hierarchical positive and negative emotion understanding system based on integrated analysis of visual and brain signals J. Neurocomputing M emotion recognition in music listening J. IEEE Transaction on Biomedical Engineering Murugappan M Rizon M Nagarajan R et al. Time - frequency analysis of EEG signals for human emotion detection C / / Proceedings of International Conference on Biomedical Engineering. Kuala Lumpur Springer Petrantonakis PC Hadjileontiadis LJ. Emotion recognition from EEG using higher order crossings J. IEEE Transaction on Information Technology in Biomedicine C / / Proceedings of International Conference of the IEEE Engineering in Medicine and Biology Society Boston IEEE Shi LC Lu BL. Off - line and on-line vigilance estimation based on linear dynamical system and manifold learning C / / Proceedings of International Conference of the IEEE Engineering 25 Eimer M Holmes A. An ERP study on the time course of in Medicine and Biology Society Buenos Aires IEEE 2010 emotional face processing J. Cognitive Neuroscience and Neuropsychology Nie D Wang XW Shi LC et al. EEG-based emotion 26 Martin E Amanda H. Event - related brain potential correlates recognition during watching movies C / / Proceedings of IEEE of emotional face processing J. Neuropsychologia EMBS Conference on Neural Engineering. Cancun IEEE Li Mu Lu BaoLing. Emotion classification based on gammaband 27. M Zouridakis G Patidar U Padhye NS et al. Spectral power of EEG C / / Proceedings of International Conference of the IEEE Engineering in Medicine and Biology Society. Minneapolis brain activity associated with emotion a pilot MEG study IEEE C / / Proceedings of International Conference on Biomagnetism 41 Firpi HA Vogelstein RJ. Particle swarm optimization-based Advances in Biomagnetism. Dubrovnik Springer feature selection for cognitive state detection C / / Proceedings of International Conference of the IEEE Engineering in Medicine 29 Aftanas LI Lotova NV Koshkarov VI et al. Non-linear and Biology Society. Boston IEEE dynamic complexity of the human EEG during evoked emotions 42 Khosrowabadi R Quek HC Wahab A et al. EEG-based J. International Journal of Psychophysiology emotion recognition using self-organizing map for boundary 76. detection C / / Proceedings of International Conference on 30 Kothe CA Makeig S. Estimation of task workload from EEG Pattern Recognition. Istanbul IEEE data new and current tools and perspectives C / / Proceedings 43 AlZoubi O Calvo RA Stevens RH. Classification of EEG for of International Conference of the IEEE Engineering in medicine and Biology Socit. Boston IEEE affect recognition an adaptive approach J. AI 2009

12 Advances In Artificial Intelligence Lecture Notes in Computer Science Frantzidis CA Bratsas C Klados MA et al. On the classification of emotional biosignals evoked while viewing affective pictures an integrated data-mining-based approach for healthcare applications J. IEEE Transaction on Information Technology in Biomedicine Ko KE Yang HC Sim KB. Emotion recognition using EEG signals with relative power values and Bayesian network J. International Journal of Control Automation and Systems Wang Ziheng Hope RM Wang Zuoguan et al. An EEG workload classifier for multiple subjects C / / Proceedings of International Conference of the IEEE Engineering in Medicine and Biology Society. Boston IEEE Albornoz EM Milone DH Rufiner HL. Multiple feature extraction and hierarchical classifiers for emotions recognition C / / Proceedings of Development of Multimodal Interfaces Active Listening and Synchrony. Dublin Springer Estepp JR Klosterman SL Christensen JC. An assessment of non - stationarity in physiological cognitive state assessment using artificial neural networks C / / Proceedings of International Conference of the IEEE Engineering in Medicine and Biology Society. Boston Massachusetts Schmidt LA Trainor LJ. Frontal brain electrical activity EEG distinguishes valence and intensity of musical emotions J. Cognition & Emotion Sarlo M Buodo G Poli S et al. Changes in EEG alpha power to different disgust elicitors the specificity of mutilations J. Neuroscience Letters Petrantonakis PC Hadjileontiadis LJ. A novel emotion elicitation index using frontal brain asymmetry for enhanced EEG-based emotion recognition J. IEEE Transactions on Information Technology in Biomedicine Galán F Nuttin M Lew E et al. A brain-actuated wheelchair Asynchronous and non-invasive Brain-computer interfaces for continuous control of robots J. Clinical Neurophysiology Tao Liu Leslie Goldberg Shangkai Gao et al. An online brain-computer interface using non-flashing visual evoked potentials J. Journal of Neural Engineering Qibin Zhao Liqing Zhang Andrzej Cichocki. EEG-based asynchronous BCI control of a car in 3D virtual reality environments J. Chinese Science Bulletin EEG J Matthews R Turner PJ McDonald NJ et al. Real time workload classification from an ambulatory wireless EEG system using hybrid EEG electrodes C / / Proceedings of International Conference of the IEEE Egineering in Medicine and Biology Society. Vancouver IEEE Li Xiaowei Hu Bin Zhu Tingshao et al. Towards affective learning with an EEG feedback approach C / / Proceedings of the first ACM international workshop on Multimedia technologies for distance learning. Beijing ACM J

(pleasure) 情緒喚起 (arousal) 情緒控制 (dominance) 以及依照 ( 王士逢 2004) 末梢血流與臉部肌電圖測量與焦慮狀態評估系統設計的結論得知, 肌電訊號由於每個人的標

(pleasure) 情緒喚起 (arousal) 情緒控制 (dominance) 以及依照 ( 王士逢 2004) 末梢血流與臉部肌電圖測量與焦慮狀態評估系統設計的結論得知, 肌電訊號由於每個人的標以生理量測系統與 SAM 量表分析圖片誘發情緒的關聯性多樂碩一甲 MA1K0203 黃正杰 0911-177-140 goiopoeva@gmail.com 摘要本研究將以 BioCapture 來測量受測者對於觀看我們從 International Affective Picture System (IAPS)