5 : LCC Dan Moldovan, HLTRI IBM Sanda Harabagiu Jennifer Chu2Carroll, John Prager Tat2Seng Chua 8 Jimmy Lin 7 ISI Eduard Hovy 7 CMU LTI Eric Ny

Transcription

1 ACTA ELECTRONICA SINICA Vol. 37 No. 5 May ,2, 1, 1, 1 (1., ; 2., ) :,,, : ; ; ; ; : TP391 : A : (2009) Advance s in Open2Domain Que stion Answering ZHANG Zhi2chang 1,2, ZHANG Yu 1, LIU Ting 1, LI Sheng 1 (11 IR Laboratory of Computer Science and Technology School, Harbin Institute of Technology. Harbin, Heilongjiang , China ; 21Mathematic and Information Science School, Northwest Normal University, Lanzhou, Gansu , China) Abstract : Question answering is one of the research hotspots in information retrieval and natural language understanding. This paper summarizes the up2to2date research advances in open2domain question answering, compares the different approaches in question analysis, document and passage retrieval,answer extraction according to different question types. The shortcomings of state2 of2art techniques are also discussed,and further research directions for question answering are analyzed and prospected at last. Key words : open domain question answering ; question analysis ; relevant document retrieval ; passage retrieval ; answer extraction 1,,,, : ;, (Question Answering,QA), :, ; [1]. FAQ [2]., 1999, TREC, ACL SIGIR COLING NAACL EACL HLT IJCNLP EMNLP, 1.,, TREC QA, 20 60, TREC, : : ; : : (No ) ; (No ) ; 863 (No. 2008AA01Z144)

2 5 : LCC Dan Moldovan, HLTRI IBM Sanda Harabagiu Jennifer Chu2Carroll, John Prager Tat2Seng Chua 8 Jimmy Lin 7 ISI Eduard Hovy 7 CMU LTI Eric Nyberg, Jamie Callan Charles L. A. Clarke 4 Eric Brill 4 Dietrich Klakow 3 Albany ILS Tomek Strzalkowski 3 MIT AI Boris Katz 2 Joyce Y. Chai 2 (1) (factoid) (list), ; (2) ( definition), (3), : : ( a) why, ; ( b) how2to, ; ( c) ( scenario),,,,, ( In2 teractive QA),, MITRE Hirschman [1]. : (passage),, 1 :, 9 9 4, [3]. LCC Moldovan [4], (1),? : ( a) [5,6]., ( b), 2. (2),, : ( a) Cymfony Srihari [13], ; ( b). LCC Harabagiu [14] ; (c) Sun [15], 2 /,, Li TREC 10 ; ; SNoW, NE, Roth [7 ] 6,50 85 % Zhang 86 % ; SVM Lee [8 ] 80 % SVM/ Moschitti, [9 ], % ;NTCIR, Suzuki SVM/ : QAC CRL2QA,, [10 ] HDAG % NE,Chunk 693 TREC Wei Li [11 ] % ;7 NTCIR25,, Lin Bigram CLQA web Lu [12 ] :54 % ; , :

3 (1). ad hoc, Moldovan Harabagiu [4,14],,, TRECQA Lemur Lucene (2) Hovy [16], 1000, 8 %,, Thompson [17] :,,, [18].,, (3), :., ( a) ( stemming). Monz [19], ( ), MIT Bilotti [20],,,, ( b) Monz [19],, ( c) : Harabagiu [14] Hovy [16] WordNet IBM Ittycheriah [21]. Moldovan [22] WordNet (lexical chain) Web Sun [15], Google 50, 2 : Monz [19] Q, Q Agichtein [23] FAQ, what is a, refers to, describe, 2,, Google Riezler, 100 FAQ, [24] ,. (1),, Hovy [25] MITRE Light [26] ; n, Alicante Vicedo [27] ;,, Clarke [28]. (2), [26]., Alicante [27]. MultiText [28] idf,, MIT Tellex [29], :, (3) TREC,,, Roberts [30],,, n,, 31214,,,

4 5 : 1061,,,,, [3]. [31] [32]., : (1) [13]., Light [26], TREC, %, (2) :,, : ( a),, InsightSoft2M Soubbotin [33], TREC 2001 ( b),,,,,, Hovy [34], (3) :,, : ( a) :,, :, Kaisser [35] ;, Shen [36]. ( b) Albany Wu [37],,,, ( c), :,, : 2, : Cui [38] : ; Cui, Shen [39] :, ; ;,, ISI Echihabi [40],, 2, (4) : ;,, Web, TREC, Web, Clarke [41] Web, Web ITC Magnini [42] (5) LCC Moldovan Harabagiu, TREC :, Web ;,, [43]. Moldovan [44] GOGEX.

5 GOGEX,,, (6), IBM [21],,,,,, [45] ,, IBM PIQUANT [46] ; FDUQA [47], 7, (1), What is the vagus nerve?, vagus nerve.,, Gaizauskas [18] WordNet,, Web, (2) [48,49]., [50], n [48] : (1) : ; (2) : ; (3) :, ; (4) : (1),, (,definitional linguistic contructs). : 2 ; ; (Appositives) ; (Copulas) ; ( Rela2 tive Clauses) ; ( Harabagiu [51] ), ( Cui [50] ).,,, :, Cui [50],,, (2), (centroid vector) (profile),, : ( a) [48] ; ( b). [52]., [53]. VSM, [52]. Cui [50] TREC, 3. 3 TREC TREC System Definitional Linguistic Constructs Statistical Ranking Surface Pattern Patterns on Parsing Trees Appositives and Copulas Relative Clauses Predicates and Verb Phrases Centroid Vector or Profile NUS [50 ] Mining External Definitions BBN [48 ] Columbia [54 ] LCC [51 ] MIT [49 ] IBM PIQUANT [46 ] Sheffield [55 ]

6 5 : 1063 (3) ( a), :, ; n2gram Cui [50],n2gram,. ( b), [50,55] , (1) Verberne why, [56]. (2),, LCC, [57] : ( a),, ( b),, ;, 31412, [3].,, John McCain, Senator McCain., MacKinnon Wikipedia, [58].,, BM25, Yin how2to, [59] (1) ( a) n,,, Zhang [60], ( b),, Madnani [61] ;,,,, LCC Lacatusu [62]. (2) why Verberne [56] why :, ,,, : (1) (relevancy) AT&T Yang [63],, (2), Sun [64] (3) (intent),, Chai [65], 31512,, Sun [64],,, De Boni [66],,, 31513,

7 , : (1), Quarteroni [67] AIML. (2),, Chai [68]. (3), LCC Hickl [69], /, CRF, 316,, (1) [70], [72],, [71],, (2) Tang [73], CRF, (3) ( a) [74] [75,76], Web [77], [78] SVM,, ( b), BBN Peng [79],,,,, NTCIR25 NTCIR26, % [80], TREC, : ( a),, ( b),,lcc, ( c) NTCIR,, ( d),, (1),,, (2),,,?, (3),?,,,,,, (4),,, 412,

8 5 : 1065 :, ;,,,,,,, 413 (1) ( a), : MUC, ;,, ( b) [81], Web, Web, Web ( c),,,,,, (2) ( a), :, ( b).,,,,,,,,,, ( c) TREC,, (3),, how2to why, /,, why [57].,, ( ), ( ), 5,,, TREC, 70 % [3].,.,, : (1), TREC,,,

9 ,,. (2),., (3), (4) Web, blog FAQ,.,,,, (5),,,, ;,, (6),,,,,,,,,,, : [ 1 ] Hirschman L, Gaizauskas R. Natural language question answer2 ing : the view from Here [J ]. Natural Language Engineering, 2001,7 (4) : [2],,. [J ].,2005,19 (3) :1213. Wu Youzheng, Zhao J un,et al. Research on question answering & evaluation : a survey[j ]. J ournal of Chinese Information Pro2 cessing,2005,19 (3) :1213. (in Chinese) [ 3 ] Dang H2T,et al. Overview of the TREC 2007 question answer2 ing track[ A ]. Proc of the 16th Text REtrieval Conference[ C ]. Gaithersburg : NIST Special Publication, [4 ] Moldovan D,et al. Performance issues and error analysis in an open2domain question answering system[j ]. ACM Trans on In2 formation Systems,2003,21 (2) : [ 5 ] Hovy E, et al. Toward semantics2based answer pinpointing [A ]. Proc of the HL T Conference ( HL T2001) [ C ]. San Diego,USA : Association for Computational Linguistics, [6 ] Brill E, et al. An analysis of the AskMSR question2answering system[ A ]. Proc of 2002 Conference on Empirical Methods in Natural Language Processing ( EMNLP) [ C ]. Philadelphia, USA : Association for Computational Linguistics, [7] Li X, Roth D. Learning question classifiers : the role of seman2 tic information [ J ]. Natural Language Engineering, 2004, 1 (1) : [ 8 ] Zhang D,Lee W2S. Question classication using support vector machines[ A ]. Proc of the 26th Annual International ACM SI2 GIR Conference on Research and Development in Informaion Retrieval[ C ]. Toronto, Canada : ACM, [ 9 ] Moschitti A, et al. Exploiting syntactic and shallow semantic kernels for question/ answer classification [ A ]. Proc of the 45th Annual Meeting of the Association of Computational Linguis2 tics [ C ]. Prague, Czech Republic : Association for Computa2 tional Linguistics, [ 10 ] Suzuki J, et al. Question classification using HDAG kernel [ A ]. ACL 2003 Workshop on Multilingual Summarization and Question Answering [ C ]. Morristown, USA : Association for Computational Linguistics, [ 11 ] Li W. Question classification using language modeling [ R ]. CIIR Technical Report,2002. [12 ] Lin S2J, Lu W2H. Learning question focus and semantically related features from web search results for Chinese question classification [ A ]. Proc of AIRS 2006,the 3rd Asia Informa2 tion Retrieval Symposium [ C ]. Singapore : Springer, [13 ] Srihari R,Li W. A question answering system supported by in2 formation extraction [ A ]. Proc of the Sixth Conference on Applied Natural Language Processing [ C ]. Seattle, Washing2 ton : Morgan Kaufmann Publishers Inc, [ 14 ] Harabagiu S,et al. Falcon : boosting knowledge for answer en2

10 5 : 1067 gines [ A ]. Proc of the Ninth Text REtrieval Conference [ C ]. Gaithersburg : NIST Special Publication, [15 ] Sun R2X,et al. Using syntactic and semantic relation analysis in question answering [ A ]. Proc of the 14th Text REtrieval Conference [ C ]. Gaithersburg : NIST Special, [16 ] Hovy E, et al. Question answering in Webclopedia [ A ]. Proc of the Ninth Text REtrieval Conference [ C ]. Gaithersburg, USA : NIST Special Publication, [17 ] Thompson K C,et al. The effect of document retrieval quality on factoid question answering performance [ A ]. Proc of the 27th Annual International ACM SIGIR Conference on Re2 search and Development in Information Retrieval [ C ]. Sheffield,U K: ACM, [ 18 ] Gaizauskas R,et al. The University of Sheffield s TREC 2003 Q&A experiments [ A ]. Proc of the 12th Text REtrieval Con2 ference ( TREC 2003) [ C ]. Gaithersburg : NIST Special Publi2 cation, [19 ] Monz C. Document retrieval in the context of question an2 swering [ A ]. Proc of the 25th European Conference on Infor2 mation Retrieval Research ( ECIR203) [ C ], Lecture Notes in Computer Science 2633,Springer, [20 ] Bilotti M W,et al. What works better for question answering : stemming or morphological query expansion [ A ]. Proc of the SIGIR Workshop on Information Retrieval for Question An2 swering (IR4QA) [ C ]. Sheffield,U K: ACM, [21 ] Ittycheriah A, et al. IBM s statistical question answering sys2 tem22trec210[ A ]. Proc of the 10th Text REtrieval Confer2 ence[ C ]. Gaithersburg : NIST Special Publication, [ 22 ] Moldovan D,Novischi A. Lexical chains for question answer2 ing[ A ]. Proc of the 19th International Conference on Compu2 tational Linguistics [ C ]. Taipei : Association for Computation2 al Linguistics, [ 23 ] Agichtein E,et al. Learning to find answers to questions on the Web[J ]. ACM Trans on Internet Technology ( TOIT),2003,4 (2) : [24 ] Riezler S, et al. Statistical machine translation for query ex2 pansion in answer retrieval [ A ]. Proc of the 45th Annual Meeting of the Association of Computational Linguistics [ C ]. Prague, Czech Republic : Association for Computational Lin2 guistics, [25 ] Hovy E, et al. The use of external knowledge in factoid QA [A ]. Proc of the 10th Text REtrieval Conference [ C ]. Gaithersburg : NIST Special Publication, [26 ] Light M, et al. Analysis for elucidating current question an2 swering technology [J ]. Natural Language Engineering,2001, 7 (4) : [27 ] Vicedo J L, Ferrgndez A. University of Alicante at TREC210 [A ]. Proc of the 10th Text REtrieval Conference [ C ]. Gaithersburg : NIST Special Publication, [28 ] Clarke C,et al. Question answering by passage selection [ A ]. Proc of the Ninth Text REtrieval Conference [ C ]. Gaithers2 burg : NIST Special Publication, [ 29 ] Tellex S,et al. Quantitative evaluation of passage retrieval al2 gorithms for question answering[ A ]. Proc of the Twenty2Sixth Annual International ACM SIGIR Conference on Research and Development in Information Retrieval [ C ]. Toronto, Canada : ACM, [ 30 ] Roberts I, Gaizauskas R. Evaluating passage retrieval ap2 proaches for question answering [ A ]. Proc of 26th European Conference on Information Retrieval ( ECIR 04) [ C ]. Sun2 derland,u K: Springer2Verlag, [ 31 ] Moldovan D, Clark C, et al. Lymba s PowerAnswer 4 in TREC 2007 [ A ]. Proc of the Sixteenth Text REtrieval Con2 ference [ C ]. Gaithersburg : NIST Special Publication, [32 ] Yang H, Chuan T2S. Web2based list question answering [ A ]. Proc of the 20th International Conference on Computational Linguistics [ C ]. Geneva,Switzerland : Association for Compu2 tational Linguistics, [ 33 ] Martin M. Soubbotin, Sergei M. Soubbotin. Patterns of poten2 tial answer expressions as clues to the right answers [ A ]. Proc of the 10th Text REtrieval Conference [ C ]. Gaithersburg : NIST Special Publication, [34 ] Ravichandran D, Hovy E. Learning surface text patterns for a question answering system[ A ]. Proc of the 40th Annual Meet2 ing of the Association for Computational Linguistics [ C ]. Pennsylvania : Association for Computational Linguistics, [ 35 ] Kaisser M, Becker T. Question answering by searching large corpora with linguistic methods [ A ]. Proc of the 13th Text Retrieval Conference [ C ]. Gaithersburg : NIST Special Publi2 cation, [36 ] Shen D, Kruijff G2J M,et al. Exploring syntactic relation pat2 terns for question answering [ A ]. Proc of Second International J oint Conference on Natural Language Processing [ C ]. Jeju, Korea : Springer, [ 37 ] Wu M,et al. ILQUA2An IE2Driven question answering system [ A ]. Proc of the 14th Text REtrieval Conference [ C ]. Gaithersburg : NIST Special Publication, [38 ] Cui H,et al. National university of Singapore at the TREC 13 question answering[ A ]. Proc of the 13th Text Retrieval Con2 ference [ C ]. Gaithersburg : NIST Special Publication, [39 ] Shen D, Klakow D. Exploring correlation of dependency rela2 tion paths for answer extraction [ A ]. Proc of the 21st Interna2 tional Conference on Computational Linguistics and 44th An2 nual Meeting of the ACL [ C ]. Sydney : Association for Com2

11 putational Linguistics, [40 ] Echihabi A, Marcu D. A noisy2channel approach to question answering [ A ]. Proc of the 41st Annual Meeting of the Asso2 ciation for Computational Linguistics [ C ]. Sapporo, Japan : Association for Computational Linguistics, [41 ] Clarke C L A, et al. Exploiting redundancy in question an2 swering[ A ]. Proc of the 24th Annual International ACM SI2 GIR Conference on Research and Development in Information Retrieval[ C ]. New Orleans : ACM, [42 ] Magnini B,et al. Is it the right answer? exploiting web redun2 dancy for answer validation [ A ]. Proc of the 40th Annual Meeting of the Association for Computational Linguistics [ C ]. Philadelphia : Association for Computational Linguistics, [ 43 ] Harabagiu S,et al. Employing two question answering systems in TREC22005[ A ]. Proc of the 14th Text REtrieval Confer2 ence[ C ]. Gaithersburg : NIST Special Publication : [ 44 ] Moldovan D,et al. Cogex : a logic prover for question answer2 ing[ A ]. Proc of HL T2NAACL 2003[ C ]. Edmonton : Associa2 tion for Computational Linguistics, [45 ],,. [J ].,2007,35 (11) : Li Peng, Wang Xiaolong,et al. A classification method for im2 balance data set based on hybrid strategy[j ]. Acta Electronica Sinica,2007,35 (11) : (in Chinese) [46 ] Chu2Carroll J, et al. IBM s PIQUANT in TREC 2004 [ A ]. Proc of the 13th Text RETrieval Conference [ C ]. Gaithersburg : NIST Special Publication, [47 ] Wu L2D,et al. FDUQA on TREC 2005 QA Track [ A ]. Proc of the 13th Text REtrieval Conference [ C ]. Gaithersburg : NIST Special Publication, [48 ] Xu J,et al. Evaluation of an extraction2based approach to an2 swering definitional questions [ A ]. Proc of the 27th Annual International ACM SIGIR Conference on Research and Devel2 opment in Information Retrieval [ C ]. Sheffield, U K: ACM, [49 ] Katz B, et al. Answering multiple questions on a topic from heterogeneous resources [ A ]. Proc of the 13th Text REtrieval Conference [ C ]. Gaithersburg : NIST Special Publication, [50 ] Cui H, et al. Soft pattern matching models for definitional question answering [J ]. ACM Trans on Information Systems ( TOIS),2007,25 (2) : Article 8. [ 51 ] Harabagiu S, et al. Answer mining by combining extraction techniques with abductive reasoning [ A ]. Proc of the 12th Text REtrieval Conference [ C ]. Gaithersburg : NIST Special Publication, [ 52 ] Chen Y, Zhou M,et al. Reranking answers for definitional QA using language modeling [ A ]. Proc of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the ACL [ C ]. Sydney : Association for Computa2 tional Linguistics, [53 ] Zhang Z2Z, et al. Answering definition questions using web knowledge bases [ A ]. Proc of Second International J oint Con2 ference [ C ]. Jeju Island, Korea, [54 ] Blair2Goldensohn S, et al. A hybrid approach for QA track definitional questions [ A ]. Proc of the 12th Text REtrieval Conference [ C ]. Gaithersburg : NIST Special Publication, [ 55 ] Gaizauskas R,et al. The University of Sheffield s TREC 2004 Q&A experiments [ A ]. Proc of the 13th Text REtrieval Con2 ference [ C ]. Gaithersburg : NIST Special Publication, [ 56 ] Verberne S,et al. Discourse2based answering of why2questions [J ]. Traitement Automatique Des Langues, Special Issue on Computational Approaches to Discourse and Document Pro2 cessing,2007, (47) : [57 ] Harabagiu S, Lacatusu F, et al. Answering complex questions with random walk models [ A ]. Proc of the 29th Annual Inter2 National ACM SIGIR Conference on Research and Develop2 ment in Information Retrieval [ C ],Seattle : ACM, [ 58 ] Mac Kinnon I, Vechtomova O. Complex interactive question answering enhanced with Wikipedia [ A ]. Proc of the 16th Text Retrieval Conference [ C ]. Gaithersburg : NIST Special Publication, [59 ] Yin L. A two2stage approach to retrieving answers for how2to questions [ A ]. Proc of the 11th conference of the European Chapter of the ACL, Student Research Workshop [ C ]. Tren2 to,italy : Association for Computational Linguistics, [60 ] Zhang C, et al. Michigan State University at the 2007 TREC ciqa evaluation [ A ]. Proc of the Sixteenth Text REtrieval Conference [ C ]. Gaithersburg : NIST Special Publications, [ 61 ] Madnani N, et al. TREC 2007 ciqa Task : University of Maryland[ A ]. Proc of the Sixteenth Text REtrieval Confer2 ence [ C ]. Gaithersburg : NIST Special Publications, [62 ] Lacatusu F, Hickl A, et al. Impact of question decomposition on the quality of answer summaries [ A ]. Proc of the Seventh Language Resources and Evaluation Conference [ C ]. Genoa, Italy,2006. [63 ] Yang F,et al. A data driven approach to relevancy recognition for contextual question answering[ A ]. Proc of the IQA Work2 shop at HL T2NAACL 2006 [ C ]. New York City, USA : As2 sociation for Computational Linguistics,

12 5 : 1069 [64 ] Sun M, Chai J. Discourse processing for context question an2 swering based on linguistic knowledge knowledge2based sys2 tems [J ]. Special Issues on Intelligent User Interfaces, 2007, 20 (6) : [65 ] Chai J, Zhang C,et al. Towards conversational QA : automatic identification of problematic situations and user intent [ A ]. Proc of the COLING/ ACL 2006 Main Conference Poster Ses2 sions [ C ]. Sydney : Association for Computational Linguis2 tics, [ 66 ] De Boni M,Manandhar S. Implementing clarification dialogue in open2domain question answering [ J ]. Natural Language Engineering,2005,11 (4) : [67 ] Quarteroni S, Manandhar S. A chatbot2based interactive ques2 tion answering System[ A ]. Proc of the 11th Workshop on the Semantics and Pragmatics of Dialogue[ C ]. Trento,Italy : Uni2 versity of Trento, [ 68 ] Chai J, Baldwin T, et al. Automated performance assessment in interactive QA [ A ]. Proc of the 29th Annual International ACM SIGIR Conference on Research and Development in In2 formation Retrieval[ C ]. Seattle : ACM, [69 ] Hickl A, Harabagiu S. Enhanced interactive question2answer2 ing with conditional random fields [ A ]. Proc of the IQA Workshop at HL T2NAACL [ C ]. New York City, USA : As2 sociation for Computational Linguistics, [70 ],,. [J ].,2006,20 (2) : Wen Xu, Zhang Yu, et al. Syntactic Structure Parsing Based Chinese Question Classification [J ]. J ournal of Chinese Infor2 mation Processing,2006,20 (2) : (in Chinese) [71 ],,. [J ].,2007,21 (1) : Sun Jingguang, Cai Dongfeng, et al. HowNet Based Chinese Question Automatic Classification [J ]. J ournal of Chinese In2 formation Processing,2007,21 (1) : (in Chinese) [72 ] [ D ]. :,2005. [73 ] Tang H, et al. Query term selection strategies for web2based chinese factoid question answering[ A ]. Proc of the 11th Con2 ference on Artificial Intelligence and Applications ( TAAI 2006) [ C ]. Kaohsiung, Taiwan : KUAS,2006. [74 ] [ D ]. :,2004. [75 ],,. [J ].,2007,21 (2) : Wu Youzheng, Zhao J un, et al. Unsupervised Answer Pattern Acquisition [ J ]. J ournal of Chinese Information Processing, 2007,21 (2) : (in Chinese) [ 76 ] Wu Y2Z, et al. Learning unsupervised svm classifier for an2 swer selection in web question answering [ A ]. Proc of the 2007 J oint Conference on Empirical Methods in Natural Lan2 guage[ C ]. Prague, Czech Republic : Association for Computa2 tional Linguistics, [77 ] [ D ]. :,2005. [78 ],,. [J ].,2008,36 (5) : Sun Ang, Jiang Minghu, et al. Chinese question answering based on syntax analysis and answer classification [ J ]. Acta Electronica Sinica,2008,36 (5) : (in Chinese) [79 ] Peng F,et al. Combining deep linguistics analysis and surface pattern learning : a hybrid approach to chinese definitional question answering[ A ]. Proc of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing [ C ]. Vancouver, Canada : Association for Computational Linguistics, [ 80 ] Sasaki Y,et al. Overview of the NTCIR26 cross2lingual ques2 tion answering (CLQA) task[ A ]. Proc of NTCIR26 Workshop Meeting[ C ]. Tokyo,Japan : NII, [81 ],,. QA [J ].,2005,16 (8) : You Lan, Zhou Yaqian. A maximum entropy model based confidence scoring algorithm for QA[J ]. J ournal of Software, 2005,16 (8) : (in Chinese) :, com,1972