國立台灣大學圖書資訊學系四十週年系慶學術研討會論文格式

Size: px

Start display at page:

Download "國立台灣大學圖書資訊學系四十週年系慶學術研討會論文格式"

持净甄孙
5 years ago
Views:

1 Study on Keyword Spottng usng Prosodc Attrbute Detecton for Conversatonal Speech Yu-Ju Huang Department of Computer Scence and Informaton Engneerng Natonal Cha-Y Unversty Yn-We Chung Department of Computer Scence and Informaton Engneerng Natonal Cha-Y Unversty Ju-Feng Yeh Department of Computer Scence and Informaton Engneerng Natonal Cha-Y Unversty (SVM) SVM Abstract It s one of most essental ssues to extract the keywords from conversatonal speech for understandng the utterances from speakers. Ths thess ams at keyword spottng from spontaneous speech for keyword detectng. We proposed prosodc features that are used for keyword detecton. The prosody words are segmented from speaker s utterance accordng to the pre-tranng decson tree. The supported vector machne s further used as the classfer to judge the prosody word s keyword or not. The prosody word boundary segmentaton algorthm based on decson tree s llustrated. Besdes the data drven feature, the knowledge obtaned from the corpus observaton s ntegrated n the decson tree. Fnally, the keyword 231

2 n the focus part are extracted usng prosody features by sported vector machne (SVM). Accordng to the expermental results, we can fnd the proposed method outperform the phone verfcaton approach especally n recall and accuracy. Ths shows the proposed approach s operatve for keyword detectng. Keywords: Keyword spottng, prosodc feature, prosody word, spoken language. (Keyword spottng) () (Spontaneous speech) (Dalogue system)(speakng style) (Grammar) (Real tme) Kawahara (Keyword extracton) (Verfcaton) (Key-phrase detecton)(key-phrase verfcaton)(sentence parsng) (sentence verfcaton) (Incremental understandng)[1] Charpter [2] (Spoken Language Understandng, SLU) (Knowledge based)[3] (Prosodc attrbute) (Herarchcal Prosodc Phrase Groupng, HPG) [4][5](Prosodc word) Al [1]Weland B-gram Beam-search Vterb [6]Btar HMM [7]Rabner

3 [8]Tatsuya Kawahara Chn-Hu Lee Key-Phrase Detecton Verfcaton [9] Rose[10] HMM (fller)zhang[11] Bah[12] HMM Bazz HMM [13] Lee C.H.[14] Km[15] [16][17] Hazhou L, Bn Ma, and Chn-Hu Lee [18] AuToB[19] POS HMM Conke [20] delta HMM Srdhar[21] HMM HMM Erteschk-shr [22] [23] [24] [25] [26] MFCC 233

4 [27] SVM HPG Fujsak Model [4][5][28][29][30] HPG 1 1: (Prosodc Attrbutes Extracton) (Ptch)(Intensty)(Duraton) (HPG) (Prosodc Word Boundary)(Boundary Decson Tree) 234

5 SVM (Keyword Detector) (Prosodc Word Detecton) (Keyword Detecton) (syllable)(prosodc word) (ntonaton phrase) (Herarchcal Prosodc Phrase Groupng, HPG)[4][5] (syllable, Syl) (prosodc word, PW)(prosodc phrase, PPh)(breath-group) (prosodc phrase group, PG) B1B2B3B4 B5 B5 B1 B2 2 B 2(Prosody word) 235

6 [4][5] 3 9 > Case 1 (Pause) Case 2 Case 3 Case 4 Case 5 Case 6 Case 7 Case 8 Case 9 (Ptch Reset) (Ptch Reset) 3HPG 9 (Ptch reset) 1 1HPG Case 1 > Case 2 Case 3 Case 4 (Ptch reset) 236

7 Case 5 Case 6 Case 7 Case 8 (Ptch reset) Case 9 (1) = (2) ( 1) (slope) Pt () t ( 1) P(t) t b e 2 e ( tt )( P( t) P), [, ] ( t t ) tb t b e e 2 tb ( 2) t 3 P 4n t 1 ( e b ) ( 3) P 2 e 1 P() t ( 4) n t b 237

8 upper bound lower bound upper bound lower bound upper bound lower bound case 4 ptch reset case 8 ptch reset SVM (predct) T 1, f T s semantc object 1, otherwse ( 5) SVM (1) n P j j { P 1, P2,... Pn} PW P Dur j j B E Syl _ N Sylj _ b j Sylj _ e j (2) bpause epause 11 (3)

9 (Keyword spottng)(speech act) (Semantc slot) DA par Erteschk-shr [23] (Topc)(Focus) (Pragmatcs) : DA pars 5: DA pars (True Postve, TP) (False Negatve, FN) 239

10 (True Negatve, TN) (False Postve, FP) 6 6 (accuracy)(precson) (recall) 678 TP TN accuracy TP FP TN FN ( 6) TP precson TP FP TP recall TP FN ( 7) ( 8) (1) SVM SVM 2 3-5% 58% 80% % 2: SVM accuracy precson recall (c=1g=8) 77.16% 57.83% 68.25% 240

11 (c=10g=16) (c=1g=8) (c=10g=16) (c=1g=8) (c=10g=16) (c=1g=8) (c=10g=16) 74.10% 52.90% 69.19% 77.42% 58.17% 69.19% 74.10% 52.94% 68.45% 75.83% 54.69% 80.0% 73.04% 51.25% 77.73% 74.90% 54.01% 70.14% 71.58% 49.5% 70.62% (2) SVM SVM 3 100% SVM TP 3: SVM accuracy precson recall (c=1g=8) (c=10g=16) (c=1g=8) (c=10g=16) (c=1g=8) (c=10g=16) (c=1g=8) (c=10g=16) 83.38% 70.95% 75.33% 81.40% 65.41% 78.03% 83.51% 70.83% 75.56% 81.35% 64.91% 77.48% 82.45% 66.33% 85.15% 80.61% 63.00% 84.00% 80.47% 65.02% 75.33% 76.65% 58.42% 75.22% 241

12 [14] HTK forced algnment HMM (fller) 4 15% 4 accuracy precson recall Reference 68% 70.22% 68.45% Label + SVM 77.42% 58.17% 80% Decson Tree + SVM 83.51% 70.95% 85.15% HPG SVM SVM 51%~58% 68%~80% 51%~59% (True Postve, TP) 76%~83% 58%~71% 75%~85% (NSC E MY3). [1] Al, J. Van der Spegel, P. Mueller, G. Haentjens,and J. Berman, An Acoustc-Phonetc Feature-Based System for Automatc Phoneme Recognton n Contnuous Speech, ISCAS [2] N. Chater, M. Pckerng, and D. Mlward. What s ncremental nterpretaton? Ednburgh Workng Papers n Cogntve Scence, 11:1 22,

13 [3] J. L, Y. Tsao and C.H. Lee, A Study on Knowledge Source Integraton for Canddate Rescorng n Automatc Speech Recognton, ICASSP, IEEE Internatonal Conference, vol 1, pp , [4],, 11(2): , [5],, 9.3: , [6] E. Weland, F. Gallwtz, and H. Nemann. Combnng stochastc and lngustc language models for recognton of spontaneous speech. In Proceedngs of the IEEE Conference on Acoustcs, Speech, and Sgnal Processng, vol.1, Atlanta, May, pp , [7] N. N. Btar and C. Y. Espy-Wlson, Knowledge-based Parameters for HMM Speech Recognton, ICASSP [8] L. R. Rabner, A tutoral on hdden markov models and selected applcaton n speech recognton, Proceedngs of the IEEE, vol.77, no. 2, Feb [9] T. Kawahara, C.H. Lee, and B.H. Juang, Flexble Speech Understandng Based on Combned Key-Phrase Detecton and Verfcaton, IEEE TRANSACTIONS ON SPEECH AND AUDIO PROCESSING, vol.6, NO. 6, pp , [10] R. C. Rose, D. B. Paul, A Hdden Markov Model Based Keyword Recognton System Acoustcs, Speech, and Sgnal Processng, ICASSP, vol.1, Page(s): , [11] P. Zhang, J. Han, J. Shao, Y. Yan, A New Keyword Spottng Approach for Spontaneous Mandarn Speech Sgnal Processng, 8th Internatonal Conference on vol.1, [12] H. Bah, N. Benat, A New Keyword Spottng Approach Multmeda Computng and Systems, ICMCS, Internatonal Conference, pp.77 80, [13] I. Bazz and J. Glass, Modelng out-of-vocabulary words for robust speech recognton, Proc. ICSLP, Bejng, [14] H. Jang, C.H. Lee, A new approach to utterance verfcaton based on neghborhood nformaton n model space, IEEE Trans. Speech Audo Process. 11(5), pp , [15] T.-Y. Km and H. Ko, Bayesan Fuson of Confdence Measures for Speech Recognton, IEEE SIGNAL PROCESSING LETTERS, vol.12, NO. 12, Dec [16] Y. BenAyed, D. Fohr, J. P. Haton, G. Chollet, Improvng the Performance of a Keyword Spottng System by Usng Support Vector Machnes, n IEEE Auto Speech Recognton and Understandng Workshop ASRU, St, Thomas, U.S. Vrgn slands, Dec [17] R. Rose, Confdence measures for the Swtchboard database, Proc. of Internatonal Conference on Acoustcs, Speech and Sgnal Processng, pp , [18] H. L, B. Ma, and C.H. Lee. A Vector Space Modelng Approach to Spoken Language Identfcaton, Audo, Speech, and Language Processng, IEEE Transactons on vol. 15, 243

14 NO. 1, JANUARY, pp , [19] AuToB. [20] A. Conke, G. Rccard, and R. Rose. Prosody recognton from speech utterances usng acoustc and lngustc based models of prosodc events. In Eurospeech, [21] V. R. Srdhar, S. Bangalore, and S. Narayanan. Explotng acoustc and syntactc features for prosody labelng n a maxmum entropy framework. IEEE Transactons on Audo, Speech & Language Processng, 16(4): , [22] N. Erteschk-shr, Informaton Structure: The Syntax-Dscourse Interface, [23],,, 89 [24],,, 89 [25],,, 93 [26],,, 95 [27],,, 96 [28] C.Y. Tseng, Dscourse Speech Tempo. JAIST Symposum on Modelng of Speech and Audovsual Mechansm. Ishkawa, Japan [29] C.Y. Tseng, and C.H. Chang, Pause or No Pause?Phrase Boundares Revsted. The 9th Natonal Conference on Man-Machne Speech Communcaton NCMMSC).,, [30] ,,

15 Num 01 P ( PW) Dur 02 P ( PW) n j1 n P Dur j Dur _ Max Dur Dur Dur P ( PW ) Max P 1 P2 Pn {,,..., } Dur _ Mn Dur Dur Dur P ( PW ) Mn P 1 P2 Pn {,,..., } 05 Dur( PW ) B E Pause( PW) 06 Syl( PW ) Syl _ N Dur( Syl ) 07 1 Dur( Syl ) 08 2 Dur( Syl ) Syl _ e Syl _ b 1 1 Syl _ e Syl _ b 2 2 Syl _ e Syl _ b 3 3 Dur( Syl ) 4 Syl4_ e Syl4_ b Pause( PW ) e pause b pause 12 pos( PW ) B E 13 N( Speech) N 245

SVM [6] PCA+SVM 79.75% 9 FERE FERE. PCA LDA Adaboost SVM 5 1 SVM Moghaddam [6] M (x,y ) x R N y x y {0,1} M f ( x) = y α k( x, x ) + b x k f(x) = 1 x

SVM [6] PCA+SVM 79.75% 9 FERE FERE. PCA LDA Adaboost SVM 5 1 SVM Moghaddam [6] M (x,y ) x R N y x y {0,1} M f ( x) = y α k( x, x ) + b x k f(x) = 1 x 100084 PCA Fsher FLD Adaboost SVM 9 FERE Adaboost 1. Gollomb [1] SEXNE 30 30 Cottrell [] BP Edelman [3] Alce [4] PCA PCA [5] Moghaddam [6] (SVM) RBF Fsher FLD FERE 3.4% Shakhnarovch [7] Adaboost 78% ±15