GATE(General Architecturefor Text Engi- neering) 1 Java [1] GATE (2) GATE GATE UIMA (Unstructured Information Manage- mentarchitecture) 2 [2]

Size: px

Start display at page:

Download "GATE(General Architecturefor Text Engi- neering) 1 Java [1] GATE (2) GATE GATE UIMA (Unstructured Information Manage- mentarchitecture) 2 [2]"

柚牲邰
7 years ago
Views:

1 JOURNAL OFCHINESEINFORMATIONPROCESSING Vol.25No.6 Nov.2011 : (2011) ( ) : LTP(LanguageTechnologyPlatform) CoNLL : ; :TP391 :A LanguageTechnologyPlatform LIU TingCHE WanxiangLIZhenghua (ResearchCenterforSocialComputingandInformationRetrieval HarbinInstituteofTechnologyHarbinHeilongjiang150001) Abstract:ChineseInformationProcessingnotonlyneedsthesupportofbasicdataplatformbutalsoneedsbasic technologyplatform.thispaperintroducesour LanguageTechnologyPlatform whichhasbeendevelopedandim- provedduringthepasteightyears.thisplatformiscomposedofseveralchineseprocessingtechnologiesincluding wordprocessingsyntacticparsingandsemanticparsing.amongthesetechnologiesoursyntacticandsemantic parsingsystem winnedthefirstplaceintheconferenceoncomputationalnaturallanguagelearning (CoNLL) 2009sharedtask.Webegantofreelyshareittotheacadamiccirclesince2006.andLTPgottheFirstPrizeof WeiChangQianChineseInformationProcessingScientificandTechnologicalAwardissuedbyChineseInformation ProcessingSocietyofChinain2010.SofarMorethan400researchinstituteshavesharedourplatform.InJune 2011wereleasedthesourcecodesandhopedthattheusersmaybeterstudyonapplicationtechnologiesbasedon ourplatformandtheymaymodifythesystemtogetherwithus. Keywords:Chineseinformationprocessing;languagetechnologyplatform 1 Web ( ) (1) : : : ( ) : (1972 ) ; (1980 ) ; (1983 )

2 GATE(General Architecturefor Text Engi- neering) 1 Java [1] GATE (2) GATE GATE UIMA (Unstructured Information Manage- mentarchitecture) 2 [2] (3) : UIMA GATE UIMA NLTK(NaturalLanguage Toolkit ) 3 Python [3] NLTK 2003 LTP(LanguageTechnologyPlatform) NLTK LTP 6 LTP XML (WebService) LTP LTP LTP (Language 2 Technology 1 htp://gate.ac.uk/ 2 htp:// :GATE UIMA NLTK 3 htp://

3 6 : 55 PlatformLTP) 1 LTP ( XML ) ( ) ( (API) (WebServ- ) ice) 1 LTP LTP 6 : ( ( ) ( ) ) ) ( [7] LTP 2) (POSTagging) LTP [8] (Generative Model) 1) (WordSegmentation) (Maximum Entropy Markov ModelsMEMM) [9] ( (Discriminative Model) ) [4] NianwenXue [5] LTP [10] (ConditionalRan- dom FieldCRF) [6] [11] LTP CRF 1 htp://ir.hit.edu.cn/ltp/

4 tion) 3) (NENamedEntityRecogni- [16] NE : [17] [18] [12] 2007SemEvalTask11 [13] [14] LTP MEMM 7 5) (DependencyParser) [15] (OpenInformation Extrac- 4) (WordSenseDisambiguation) LTP tion) [19] [20] ( ) 10% 8000 CoNLL (Conferenceon Computational NaturalLanguage Learning)2009 ( 7 42% ) htp://ufal.mf.cuni.cz/conl2009-st/results/results.php

5 6 : 57 6) (SemanticRoleLabeling) [19] (ILPIntegerLinearProgramming) CoNLL ( PropBank ) (Arg0~5) ( ) [21] Kernel ConvolutionTreeKernel 1 LTP [22] LTP 1 LTP F97.4% 185KB/s % ( ) 85.48% 56.3KB /s F92.3% 14.4KB/s (LAS)73.91% KB/s 9000 (UAS)78.23% % 7.2KB/s 94.34% ChinesePropBank2.0 (22277) 2556 F77.2% 3 1.3KB/s CoN- XML LL LTP XML 3.2 XML 1 2 3

6 <xmlversion= 1.0 encoding= gb2312 > <ltml> <doc> <paraid= 0 > <sentid= 2 cont= 40 > <wordid= 0 cont= pos= nl ne= O wsd= Cb05 parent= 1 relate= ATT /> <wordid= 1 cont= pos= n ne= O wsd= Al02 parent= 2 relate= ATT /> <wordid= 2 cont= pos= n ne= O wsd= Al01 parent= 6 relate= SBV /> <wordid= 3 cont= 40 pos= m ne= B-Nm wsd= -1 parent= 5 relate= QUN /> <wordid= 4 cont= pos= m ne= I-Nm wsd= Dn05 parent= 3 relate= RAD /> <wordid= 5 cont= pos= n ne= E-Nm wsd= Aa01 parent= 6 relate= SBV /> <wordid= 6 cont= pos= v ne= O wsd= Hj20 parent= -1 relate= HED > <argid= 0 type= Arg0 beg= 0 end= 2 /> <argid= 1 type= Arg0 beg= 3 end= 5 /> <argid= 2 type= Arg1 beg= 7 end= 7 /> </word> </doc> </ltml> LTML(LanguageTechnology MarkupLanguage) 2 LTML 2 LTML LTML 3.3 (word) cont( ) pos( ) ne( ) wsd( ) parent ( )relate( ) arg( ) (sent) (para) LTP XML 2 LTML 2 LTP [23] 600 ( ) ( ) [24] 6 LTML ; ; ; LTML ; LTP LTP

7 6 : 59 3 LTP 4 LTP LTP ;4) LTP Web 3.5 (WebService) Service LTP LTP WebService LTP WebService :1) 4 LTP LTP ;2) LTP ;3) 20099LTP

8 LTP LTP LTP LTP 400 (UI- UC) (NICT) (1) LTP LTP LTP LTP 500 LTP LTP 5 [25-26] 6 2 LTP LTP (2) 90% 80% LTP LTP LTP 8 LTP 6 (SemanticDe- pendencyparsingsdp) SDP 1 [28] (3) 80% 60%~70% (4) 1 htp://ir.hit.edu.cn/demo/ltp/sharing Plan.htm 2 [27]

9 6 : 61 Web2.0[12] GuodongZhouJianSu.2002.Namedentityrecog- nitionusingan HMM-basedchunktagger[C]//Pro- ceedingsofthe40th AnnualMeetingoftheAssocia- tionofcomparativelinguistics(acl): [13] HaiLeongChieuHweeTouNg.2002.NamedEn- tityrecognition:a Maximum EntropyApproach U- singglobalinformation[c]//proceedingsofthe19th InternationalConferenceon ComputationalLinguis- 30 tics(coling): [14] BurrSetles.2004.BiomedicalNamedEntityRecog- nition Using Conditional Random Fields and Rich FeatureSets[C]//ProceedingsofCOLING2004the InternationalJoint Workshop on Natural Language ProcessinginBiomedicineanditsApplications(NLP- BA)GenevaSwitzerland. [15] RuijiFuBing QinTing Liu.Generating Chinese [1] Hamish Cunningham Diana Maynard Kalina NamedEntityDatafromaParalelCorpus[C]//Pro- Bontchevaetal.GATE:anArchitectureforDevelop- mentofrobustnlt Applications [C]//Proceedings ofacl2002: [2] DavidFerrucciAdam Laly.BuildinganExampleAp- plication withthe UnstructuredInformation Manage- mentarchitecture[j].ibm SystemsJournal (3): [3] Steven BirdEdward Loper. NLTK: The Natural LanguageToolkit[C]//ProceedingsoftheACLdem- onstrationsession2004: ceedingsofthe5thinternationaljointconferenceon NaturalLanguageProcessing (IJCNLP2011). [16] WanxiangCheTing Liu. Word Sense Disambigu- ation Corpora Acquisition via Confirmation Code [C]//IJCNLP [17] ZhimaoLuHaifeng WangJianminYaoetal.An Equivalent Pseudoword Solution to Chinese Word SenseDisambiguation[C]//ACL2006. [18] Yuhang Guo Wanxiang CheYuxuan Huetal. HIT-IR-WSD:A WSD System for English Lexical [4]. [J]. SampleTask [C]//SemEval (2): [5] NianwenXue.Chinese WordSegmentationasCharac- tertagging[j].internationaljournalofcomputational LinguisticsandChineseLanguageProcessing20038 (1): [6] JohnLafertyAndrew McCalumFernandoPereira. ConditionalRandom Fields:Probabilistic Modelsfor Segmentingand Labeling Sequence Data [C]//Pro- ceedingsoficml2001: [19] WanxiangCheZhenghuaLiYongqiang Lietal. MultilingualDependency-basedSyntacticandSeman- ticparsing [C]//ProceedingsofCoNLL 2009: [20] ZhenghuaLiWanxiang CheTing Liu.Improving Dependency Parsing Using Punctuation [C]//Pro- ceedingsoftheinternational Conference on Asian LanguageProcessingofIALP2010.HarbinChina. [21] WanxiangCheMinZhangAiTiAwetal.Using [7]. ahybridconvolutiontreekernelforsemanticrole [C]// Labeling[C]//ACM TransactionsonAsianLanguage [8] Chris ManningHinrichSchütze.FoundationsofSta- tisticalnaturallanguageprocessing[m].mitpress. CambridgeMA:May1999. [9] Andrew McCalumDayneFreitagFernandoPereira. Maximum Entropy Markov ModelsforInformation ExtractionandSegmentation[C]//ProceedingsofIC- InformationProcessing.20087(4). [22] Adam L.BergerStephen A.DelaPietraVincent Jetal.A Maximum Entropy Approachto Natural Language[J].CL199622(1): [23]. [M] [24] TingLiuJinshan MaShengLi.BuildingaDepend- ML [10] VladimirVapnik.TheNatureofStatisticalLearning Theory[M].Springer-Verlag1995. ency TreebankforImproving Chinese Parser [J]. JournalofChineseInformationProcessing (4): [11]. SVMTool [25] Zhenghua Li Min Zhang Wanxiang Cheetal. [J] (4): JointModelsforChinesePOS TaggingandDepend-

10 encyparsing[c]//proceedingsofthe2011 Confer- enceonempiricalmethodsinnaturallanguagepro- cessing.edinburghscotlanduk : [26] WanxiangCheTingLiu.Jointly Modeling WSDand SRL with MarkovLogic[C]//Proceedingsofthe23rd InternationalConferenceon ComputationalLinguis- tics (Coling2010).BeijingChina : [27] WanxiangCheTingLiuShengLi.A New Chinese NaturalLanguageUnderstanding ArchitectureBased on Multilayer Search Mechanism [C ]//Third SIGHAN Workshopon Chinese Language Process- ing [28]. [C]// 櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚櫚 ( 52) [18]. Conference on Computational Linguistics (Coling [C]// 2010): : [19]. [J] [D] [20]. [25]. [J]. (JournalofSoftware) (4): [21] Jihong LIRuibo WANGWeilin WANG Bo GU GuochenLI.AutomaticLabelingofSemanticRoleon ChineseFrameNetUsingConditionalRandom Fields [C]//2009 IEEE/WIC/ACM International Confer- enceon WebInteligenceandInteligentAgentTech- nology-workshops. [22]. [28]. [J] (3): [24]. 24(1): [J] (4): [26]. [J] (6): [27] Ru LiShuanghongLi. The Semantic Computing Model of Sentence Similarity Based on Chinese FrameNet[C]//Web Inteligence/IAT Workshops 2009: [J] (2): [23] RuLiHaijingLiuShuanghongLi.ChineseFrameI- dentificationusing T-CRF Model[C]//International

6 : (FrameNet) [2] [3] [4] [5] [6] (ChineseFrameNetCFN) % 1520 ; [7] 3151 ( ) ; FrameNet [8] atract(

6 : (FrameNet) [2] [3] [4] [5] [6] (ChineseFrameNetCFN) % 1520 ; [7] 3151 ( ) ; FrameNet [8] atract( 256 201111 JOURNAL OFCHINESEINFORMATIONPROCESSING Vol.25No.6 Nov.2011 :1003-0077(2011)06-0046-07 ( 030006) : (ChineseFrameNetCFN) Filmore FrameNet 25 F1-74.16%52.70%61.62%; : ; ; :TP391 :A ResearchonChineseFrameNetConstructionandApplicationtechnologies