CFA Alderson & Banerjee 2002 Confirmatory Factor Analysis CFA cross-sectional design longitudinal design AERA et al CFA

Size: px

Start display at page:

Download "CFA Alderson & Banerjee 2002 Confirmatory Factor Analysis CFA cross-sectional design longitudinal design AERA et al CFA"

镜阮
7 years ago
Views:

1 Foreign Language Testing and Teaching Jul 2016 H319 A * CFA 1 Ute Knoch 2 3 Abstract Understanding the factor structure of a language test is crucial to the establishment of its construct validity. Despite the growing number of factor structure studies in the field of language testing few have used multi-year data to investigate the factor structure and factorial invariance of language tests. Adopting longitudinal design and multisample Confirmatory Factor Analysis CFA as its analytic methodology this study investigated the factor structure and factorial invariance of a university-based English proficiency test. Results indicated that the higher-order factor model fit the test data well and was comparatively parsimonious. Multi-sample CFA analyses demonstrated that this model had basically the same configuration on the two samples. While lending important empirical support to the construct validity of this test this study also has implications for other testing agencies in their test validation endeavors. Key words factor structure factorial invariance Confirmatory Factor Analysis multi-sample analysis 1. test validity argument AERA et al Chapelle et al AERA et al Gu 2014 Sawaki et al In nami & Koizumi 2012 TEAP In nami et al EPE Sims & Kunnan 2016 * 13CYY032 20

2 CFA Alderson & Banerjee 2002 Confirmatory Factor Analysis CFA cross-sectional design longitudinal design AERA et al CFA Oller 1976 Principal Component Analysis PCA Oller Oller Oller PCA Bachman & Palmer 1982 CFA 116 Sawaki et al CFA 2720 Sawaki Llosa 2007 Sawaki Gu 2014 CFA 1000 In nami & Koizumi Bachman & Palmer 1981 Sang et al

3 CFA Fudan English Test FET 2. 2 FET FET FET Bachman & Palmer FET 1 FET 2 FET 3 FET FET 2013 FET FET FET FET FET CFA 2 Fan & Ji 2014 FET 3 Fan et al FET FET FET FET FET 2 CFA FET FET % %

4 CFA % % % % FET 4 L1 * L2 2 L3 L4 5 3 R1 - R3 2 R4 - R5 2 W1 W2 3 S1 S2 S3 CFA 3. 4 Bachman & Palmer Gu 2014 In nmai & Koizumi 2012 Llosa 2007 Sawaki et al a 1 FET FET 1 b 2 FET FET * CFA 23

5 c A 3 FET + + FET 3 A d B 4 FET + + FET 24

6 CFA B CFA EQS 6. 3 Bentler & Wu Byrne 2006 Sawaki et al Comparative Fit Index CFI Non-Normed Fit Index NNFI Goodness of Fit Index GFI Standardized Root Mean Square Residual SRMR Root Mean Square Error of Approximation RMSEA Byrne 2006 In nami & Koizumi 2011 Byrne 2006 CFI NNFI GFI SRMR RMSEA RMSEA 90% In nami & Koizumi 2011 nested models Byrne

7 S2 ± 2 Mardia Bentler & Wu 2005 EQS robust estimation method CFI = GFI = SRMR = RMSEA = S - B x 2 = df = 5 p < S - B x 2 = df = 5 p < CFI = GFI = SRMR = RMSEA = S - B x 2 = df = 2 p < Cheung & Rensvold 2002 CFI CFI < Sawaki et al CFI FET N = 3988 S - B x 2 df CFI NNFI GFI SRMR * * RMSEA 90% C. I

8 CFA * * * p < A B 3 S - B x 2 EQS Satorra-Bentler C. I. Confidence Interval RMSEA 90% N = 3169 S - B x 2 df CFI NNFI GFI SRMR * * * * RMSEA 90% C. I S - B x 2 = df = 2 p < CFI CFI = FET 4. 3 CFA FET CFA Byrne 2006 metric invariance factorial invariance CFA configural model A B C 3 27

9 CFA S - B x 2 df CFI NNFI GFI SRMR Model A * Model B * Model C * RMSEA 90% C. I p < Model A Model B Model C 3 CFI = GFI = SRMR = RMSEA = Byrne CFI = GFI = SRMR = RMSEA = S - B x 2 = df = 10 p < CFI RMSEA Sawaki et al CFI = GFI = SRMR = RMSEA = S - B x 2 = df = 3 p < CFI RMSEA 5. CFA FET A 2013 FET Exploratory Factor Analysis EFA + + EFA CFA EFA EFA 28

10 CFA CFA Byrne 2006 EFA CFA CFA In nami & Koizumi 2011 B FET CFA Llosa 2007 Sawaki 2007 Sawaki et al FET Bachman & Palmer FET Sawaki et al * Sims & Kunnan 2016 CFA AERA et al FET Sawaki et al In nami et al Sim & Kunnan 2016 CFA Byrne 2006 Chapelle et al * 29

11 AERA APA & NCME. AERA Standards for Educational and Psychological Testing Z. Washington D.C. 2 Alderson J C & Banerjee J. Language testing and assessment Part 2 J. Language Teaching Bachman L F & Palmer A S. The construct validation of the FSI Oral Interview 1 J. Language Learning Bachman L F & Palmer A S. The construct validation of some components of communicative proficiency J. TESOL Quarterly Bachman L F & Palmer A S. Language Assessment in Practice Designing and Developing Useful Language Tests M. Oxford Oxford University Press Bachman L F & Palmer A S. Language Assessment in Practice Developing Language Assessments and Justifying their Use in the Real World M. Oxford Oxford University Press Bentler P M & Wu E J. EQS 6. 1 for Windows M. Encino CA Multivariate Software Byrne B M. Structural Equation Modeling with EQS Basic Concepts Applications and Programming 2nd Ed. M. Mahwah New Jersey Psychology Press Chapelle C A Enright M K & Jamieson J M. Building a Validity Argument for the Test of English as a Foreign Language C. New York and London Routledge Taylor & Francis Group Cheung G W & Rensvold R B. Evaluating goodness of fit indexes for testing measurement invariance J. Structural Equation Modeling Fan J & Ji P. Test candidates attitudes and their test performance The case of the Fudan English Test J. University of Sydney Papers in TESOL Fan J Ji P & Song X. Washback of university-based English language tests on students learning A case study J. The Asian Journal of Applied Linguistics Gu L. At the interface between language testing and second language acquisition Language ability and context of learning J. Language Testing In nami Y & Koizumi R. Structural equation modeling in language testing and learning research A review J. Language Assessment Quarterly In nami Y & Koizumi R. Factor structure of the revise TOEFL test A multi-sample analysis J. Language Testing In nami Y Koizumi R & Nakamura K. Factor structure of the Test of English for Academic Purposes TEAP test in relation to the TOEFL ibt P test J. Language Testing in Asia Llosa L. Validating a standards-based classroom assessment of English proficiency A multitrait-multimethod approach J. Language Testing Oller J W. Evidence for a general language proficiency factor An expectancy grammar J. Die Neueren Sprachen Sang F Schmitz B Vollmer H J Baumert J & Roeder P. Models of second language competence A structural equation approach J. Language Testing Sawaki Y. Construct validation of analytic rating scale in speaking assessment Reporting a score profile and a composite J. Language Testing Sawaki Y Stricker L J & Oranje A H. Factor structure of the TOEFL Internet-based test J. Language Testing

12 lexical approach 1 Bachman L F & Palmer A S. Language Testing in Practice M. Oxford Oxford University Press Taylor W L. Cloze procedure A new tool for measuring readability J. Journalism Quarterly CET J M M 櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟櫟 Sims J M & Kunnan A J. Developing evidence for a validity argument for an English placement exam from multi-year test performance data J. Language Testing in Asia J J J J J TEM J J

从语用的角度看口语水平测试

从语用的角度看口语水平测试对外汉语教学 : 语用为纲口语水平测试的实践与讨论香港中文大学吴伟平本文在语用为纲的框架下讨论大规模汉语口语水平测试中的三个重要问题 : 题库建设随机组卷和考后评核纵观语言测试的发展, 我们可以看到其重点随着语言