Microsoft Word - 20肖祝祝张敏强jz13-012_new_.doc

Size: px

Start display at page:

Download "Microsoft Word - 20肖祝祝张敏强jz13-012_new_.doc"

握詹扶
7 years ago
Views:

1 2013, Vol. 21, No. 10, Advances in Psychological Science DOI: /SP.J * 题组测验中处理局部项目依赖性 (LID) 的模型发展 (/, ) 题组是常用的测验题目组织形式, 题组中的题目往往会违背局部独立性假设, 产生局部项目依赖 (Local Item dependence, LID), 探讨如何处理题组中 LID 以提高参数估计精度具有重要意义本文在评述题组测验中处理 LID 的标准 IRT 方法多级评分 IRT 方法题组随机效应方法和两因子方法的基础上, 以参数估计精度作为准则, 对这 4 种处理 LID 方法的效果和适用条件进行了归纳并作出相应的评析, 为使用者选择合适的方法处理 LID 提供了参考依据同时展望了今后的研究方向题组 ; IRT 模型 ; 局部项目依赖性 (LID) B841 1 引言, (Testlet) (Rosenbaum, 1988),,,, (Item cluster, Wainer & Kiely, 1987)(Item bundle, Rosenbaum, 1988) (Item set, Keller, Swaminathan, & Sireci, 2003) (item-based test), (testlets -composed test) (1), ; (2),,, : * (GFA111009) (J ) 2012 (12YJC190016) (GZIT2012- ZB0292) :, zhangmq1117@qq.com ; (3) (Keller et al., 2003; Zenisky, Hambleton, & Sireci, 2001),, HSK GRE SAT ITBS IRT, (Local Item Independence, LII), (Rosenbaum, 1988), (Local Item dependence, LID), LID,,,,,, (Li, 2009) Yen (1993) LID,,, 1893

2 ,,, LID, 20, LID,, IRT 4 LID IRT IRT, 2 处理局部项目依赖性 (LID) 的方法 2.1 忽略 LID 的标准 IRT 模型 IRT,,, IRT (LII), LID, LII, IRT, LID, IRT,,,, (, Sireci, Thissen, & Wainer, 1991; Yen, 1993; Wainer, 1995; Wainer & Thissen, 1996; Wainer & Wang, 2000; Wang & Wilson, 2005b;,,,,, 2009),, IRT, Bradlow, Wainer Wang (1999) IRT 4~6, ; Wainer, Bradlow Du (2000), ( 4~6 ),,,, ; Tuerlinckx De Boeck (2001),, LID, Keller (2003),,,,,, LID IRT 2.2 多级评分 IRT 模型 LID IRT, (Super-item), (, Sireci et al., 1991; Thissen, Steinberg, & Mooney, 1989; Wainer, 1995) IRT,,, IRT LID IRT, (graded response models, GRM) IRT,, ( Sireci et al., 1991; Thissen et al., 1989),,,, IRT LID (Wainer, 1995),,,,,,, (Yen, 1993), IRT,,, Keller 等 (2003) LID IRT IRT,, IRT,,

3 10 : (LID) 1895, IRT IRT LID IRT LID, IRT LID, LID LID, LID, 2.3 题组反应模型 (TRT 模型 ),, (Testlet response theory, TRT) Bradlow (1999) 2PL-IRT LID, 2PL-TRT γ id ( j), i, d(j) d j γ id ( j) i d j Wainer (2000) 3-PL IRT, 3PL-TRT Wang, Bradlow 和 Wainer (2002), (General Bayesian Testlet Model),, 3-PL IRT, GRM Wang Wilson (2005a) Rasch, Rasch (Rasch Testlet Model), Wang Wilson (2005b) Rasch, (Random-Effects Facet Model) LID, ( ),,,, Li, Bolt Fu (2006) 2PL-TRT, 3PL-TRT, exp[ aj( θi bj+ γid(j) )] Py ( ij = 1) = cj + (1 cj) 1 + exp[ aj ( θi bj+ γid(j) )], aj, bj, cj j γ id ( j) i d(j), γ id ( j) θ 2 σ γ id ( j ) d(j), d(j), γ id ( j) =0, TRT IRT, TRT, ; LII, LID, IRT TRT, TRT IRT (,, 2009), Bradlow (1999) 2PL-TRT SAT,,, 2PL-TRT LID 2PL-IRT ; Wainer (2000) 3PL-TRT ( GRE SAT ), 3PL-TRT, Wainer Wang (2000) 3PL-TRT (TOEFL),, 3PL-TRT ; (2009) 3PL-TRT, MCMC,, LID, 3PL-TRT 玥 (2012) 2-PL TRT,,, 2-PL TRT,, Rasch Wang Wilson (2005a) Rasch, Rasch

4 , (RMSE), Rasch, Rasch, Wang 和 Wilson (2005b),, LID IRT,, 6, 2PL-TRT 3PL-TRT,, Wang (2002), LID,,,,,,,,,,,,,,,,,, IRT,,,, LID,,, IRT IRT,, (, ),, (Zhang, Shen, & Cannady, 2010) 2.4 两因子项目反应模型 (Bi-factor item response model),,,, (Full-information item Bi-factor model, Gibbons et al., 2007; Gibbons & Hedeker, 1992),,, Reise, Morizot Hays (2007), ;, Simms, Grös, Watson O Hara (2008) ;, Gibbons (2008),, DeMars (2006),, LID n S, 1 ( ) S-1 ( ),,,,, (Gibbons & Hedeker, 1992), j a j1, S-1 ( ajk, k = 2,..., s ) 2 4 α11 α12 0 α21 α22 0 α = α31 0 α33 α41 0 α43

5 10 : (LID) 1897 Gibbons Hedeker (1992) Gibbons (2007),, ;,,, (Bradlow et al., 1999)(Gibbons & Hedeker, 1992) (Rijmen, 2010),, (MMLE) TRT (MCMC)( 1),,, LID,,, (DeMars, 2006),, LID,, ( ) LID, 3,,, LID 4 ( 2),,, TRT LID 3 结语和展望 3.1 模型选择,, IRT,, 20 LID, LID IRT LID, LID, 表 1 LID 题组反应模型和两因子模型的参数估计方法区分度参数和适用计分方式汇总表 (Logistic Model) (2PL-TRT); (3PL-TRT); (General Testlet response Model) MCMC MCMC MCMC TRT (Rasch Model) Rasch (Rasch Testlet Model); (Random- Effects Facet testlet Model) MMLE MMLE (2PNO Model) (2PNO testlet Model) MCMC (Bi-factor Model) (Factor analysis) IRT (Bi-factor IRT Model) MMLE () () ;,

6 表 2 处理 LID 的 4 种方法比较 IRT,,,,,,, IRT TRT, LID, ; IRT,, ; ;,,,,,,,, LID,, LID,, LID,,, LID Pitt, Kim Myung (2003),,,,,,,,, LID, LID,,,, LID,, LID, (1) (4-6 ), IRT ; (2) LID, IRT LID ; (3) LID, TRT,, TRT,,, TRT 3.2 研究展望, (Multidimensional Item Response Theory, MIRT), MIRT LID, 2 MIRT LID,,,,,,,,,,,,

7 10 : (LID) 1899 (Marginal maximum likelihood estimation, MMLE) (Markov Chain Monte Carlo, MCMC) MMLE, MCMC, LID,,,,,,,,,, LID,, 刘玥, 刘红云. (2012). 贝叶斯题组随机效应模型的必要性及影响因素. 心理学报, 44(2), 涂冬波, 蔡艳, 漆书青, 丁树良, 戴海琦. (2009). 项目反应理论新进展题组模型及其参数估计的实现. 心理科学, 32(6), Bradlow, E. T., Wainer, H., & Wang, X. H. (1999). A Bayesian random effects model for testlets. Psychometrika, 64(2), DeMars, C. E. (2006). Application of the bi-factor multidimensional item response theory model to testlet-based tests. Journal of Educational Measurement, 43(2), Gibbons, R. D., Bock, R. D., Hedeker, D., Weiss, D. J., Segawa, E., Bhaumik, D. K., Stover, A. (2007). Full-information item bifactor analysis of graded response data. Applied Psychological Measurement, 31, Gibbons, R. D., & Hedeker, D. R. (1992). Full-information item bi-factor analysis. Psychometrika, 57, Gibbons, R. D., Weiss, D. J., Kupfer, D. J., Frank, L., Fagiolini, A., Grochocinski, V. J., Immekus, J. C. (2008). Using computerized adaptive testing to reduce the burden of mental health assessment. Psychiatric Services, 59, Keller, L. A., Swaminathan, H., & Sireci, S. G. (2003). Evaluating scoring procedures for context-dependent item sets. Applied Measurement in Education, 16(3), Li, F. F. (2009). An information correction method for testlet-based test analysis: from the perspectives of item response theory and generalizability theory. Unpublished doctorial dissertation, University of Maryland, College Park. Li, Y. M., Bolt, D. M., & Fu, J. B. (2006). A comparison of alternative models for testlets. Applied Psychological Measurement, 30(1), Pitt, M. A., Kim, W., & Myung, I. J. (2003). Flexibility versus generalizability in model selection. Psychonomic Bulletin & Review, 10, Reise, S. P., Morizot, J., & Hays, R. D. (2007). The role of the bifactor model in resolving dimensionality issues in health outcomes measures. Quality of Life Research, 16, Rijmen, F. (2010). Formal relations and an empirical comparison among the Bi-factor, the testlet, and a second-order multidimensional IRT model. Journal of Educational Measurement, 47(3), Rosenbaum, P. R. (1988). Items bundles. Psychometrika, 53, Simms, L. J., Grös, D. F., Watson, D., & O Hara, M. W. (2008). Parsing the general and specific components of depression and anxiety with bifactor modeling. Depression and Anxiety, 25, E34 E46. Sireci, S. G., Thissen, D., & Wainer, H. (1991). On the reliability of testlet-based tests. Journal of Educational Measurement, 28, Thissen, D., Steinberg, L., & Mooney, J. A. (1989). Trace lines for testlets: A use of multiple-categorical-response models. Journal of Educational Measurement, 26, Tuerlinckx, F., & De Boeck, P. (2001). The effect of ignoring item interactions on the estimated discrimination parameters in item response theory. Psychological Methods, 6, Wainer, H. (1995). Precision and differential item functioning on a testlet-based test: The 1991 Law School Admissions Test as an example. Applied Measurement in Education, 8(2), Wainer, H., Bradlow, E. T., & Du, Z. R. (2000). Testlet response theory: An analog for the 3PL model useful in testlet-based adaptive testing. In Computerized adaptive testing: Theory and practice (pp ). Netherlands: Springer.

8 Wainer, H., & Kiely, G. L. (1987). Item clusters and computerized adaptive testing: A case for testlets. Journal of Educational Measurement, 24, Wainer, H., & Thissen, D. (1996). How is reliability related to the quality of test scores? What is the effect of local dependence on reliability? Educational Measurement: Issues and Practice, 15, Wainer, H., & Wang, X. H. (2000). Using a new statistical model for testlets to score TOEFL. Journal of Educational Measurement, 37(3), Wang, W. C., & Wilson, M. (2005a). The Rasch testlet model. Applied Psychological Measurement, 29(2), Wang, W. C., & Wilson, M. (2005b). Exploring local item dependence using a random-effects facet model. Applied Psychological Measurement, 29(4), Wang, X. H., Bradlow, E. T., & Wainer, H. (2002). A general Bayesian model for testlets: Theory and applications. Applied Psychological Measurement, 26(1), Yen, W. M. (1993). Scaling performance assessments: Strategies for managing local item dependence. Journal of Educational Measurement, 30(3), Zenisky, A. L., Hambleton, R. K., & Sireci, S. G. (2001). Effects of local item dependence on the validity of IRT item, test, and ability statistics. Washington, DC: Association of American Medical. Zhang, O., Shen, L., & Cannady, M. (2010). Polytomous IRT or Testlet Model: An Evaluation of Scoring Models in Small Testlet Size Situations. paper presented at annual meeting of the 15th International Objective Measurement Workshop April, 2010; Boulder, Colorado. Model Development in Processing Local Item Dependency (LID) in Testlet-composed Test Analysis XIAO Zhuzhu; ZHANG Minqiang; WANG Xia; XIONG Siping (Center for Studies of Psychological Application, School of Psychology, South China Normal University, Guangzhou , China) Abstract: Testlets-composed tests are commonly used in education and psychology fields, but items in the same testlet may violate the assumption of local item independence and lead to local item dependency (LID). Thus, in order to increase parameter estimation accuracy, probing into methods of handling LID is of significant importance. In this paper, the authors summarize and assess four methods in processing LID, including standard IRT model, polytomous IRT model, testlet response model and bi-factor model. Taking parameter estimation accuracy as the criterion, the authors further summarize the effects and suitable conditions of each method and provided corresponding comments. This paper will provide reference for method selection when processing LID. Future research directions are discussed as well. Key words: testlet; IRT model; local item dependency (LID)

Developing an Item Bank of Fraction Concepts Jyun-Ji Lin Ph. D. Candidate, Department of Psychology, National Chung Cheng University Yuh-Yin Wu Profes

Developing an Item Bank of Fraction Concepts Jyun-Ji Lin Ph. D. Candidate, Department of Psychology, National Chung Cheng University Yuh-Yin Wu Profes 分數概念題庫之建立 : 跨學習階段的校準與測量林俊吉國立正大心理系博士候選吳毓瑩國立台北教育大心理與諮商系教授呂玉琴國立台北教育大數暨訊教育系教授摘要本研究目的旨在整合目前現有的分數概念研究之紙筆測驗題目,