教育學報31-2期.indd

Size: px

Start display at page:

Download "教育學報31-2期.indd"

旺钱
7 years ago
Views:

1 從 TIMSS 2007 臺灣八年級學生數學科作答反應檢視古典測驗理論和試題反應理論特性和測驗分析結果 TIMSS 2007 * ** TIMSS 2007 TIMSS R Excel 2003 SPSS 12.0 ConQuest 2.0 PCM TIMSS DOI: / * aguri.su@gmail.com ** 67

2 新竹教育大學教育學報第三十一卷第二期 classical test theory, CTT Gullikson, 1987; Lord & Novick, 1968 true score model modern test theory Lord, 1980 item response theory, IRT 1991 Embretson & Reise, 2000 Trends in International Mathematics and Science Study, TIMSS International Association for the Evaluation of Educational Achievement, IEA TIMSS 2007 TIMSS 68

3 從 TIMSS 2007 臺灣八年級學生數學科作答反應檢視古典測驗理論和試題反應理論特性和測驗分析結果 TIMSS 2007 TIMSS 2007 TIMSS 2007 TIMSS

4 新竹教育大學教育學報第三十一卷第二期 1999 Embretson & Reise, Embretson Reise Item response theory for psychologists, by S. E. Embretson & S. Reise, Mahwah, NJ: Lawrence Erlbaum Associates. 70

5 從 TIMSS 2007 臺灣八年級學生數學科作答反應檢視古典測驗理論和試題反應理論特性和測驗分析結果 1 Rasch model Linear logistic test model, LLTM Embretson & Reise,

6 新竹教育大學教育學報第三十一卷第二期 American Educational Research Association American Psychological Association National Council on Measurement in Education [AERA, APA, NCME] Lord 1980 standard error, SE IRT b 72

7 從 TIMSS 2007 臺灣八年級學生數學科作答反應檢視古典測驗理論和試題反應理論特性和測驗分析結果 Embretson & Reise, 2000 Bond Fox 2007 Raschinfit outfit outfit infit 11 1±0.3 logit joint maximum likelihood, JML marginal maximum likelihood, MML conditional maximum likelihood, CML Embretson Reise a 73

8 新竹教育大學教育學報第三十一卷第二期 TIMSS TIMSS

9 從 TIMSS 2007 臺灣八年級學生數學科作答反應檢視古典測驗理論和試題反應理論特性和測驗分析結果 TIMSS 2007 TIMSS TIMSS 2007 TIMSS ,

10 新竹教育大學教育學報第三十一卷第二期 booklet M01 M02 M01 TIMSS M02 TIMSS TIMSS 2007 M01 M02 M01 M02 M01 M02 M01 M02 M01 M TIMSS 2007TIMSS 76

11 從 TIMSS 2007 臺灣八年級學生數學科作答反應檢視古典測驗理論和試題反應理論特性和測驗分析結果 TIMSS R CTT Excel 2003 R erm SPSS 12.0 Excel 2003 ConQuest 2.0 Wu, Adams, & Wilson, partial credit model, PCM Masters, 1982 θi x x = 0,,m i i m i +1 x = j P ix (θ)= exp[ x j =0 (θ - δ ij )] [ mi exp[ r j =0 (θ- δ ij )] r=0 1 0 j =0 (θ - δ ij ) 0 δ ij j = 1,, m i jstep difficulty Embretson & Reise, 2000 θ i j 77

12 新竹教育大學教育學報第三十一卷第二期 δ ij SPSS 12.0 R erm infit outfit erm 2006 Rasch Rasch erm ConQuest 2.0eRm [p' i (θ)] 2 I i θ= i=1,,n 2 P i (θ) Q i (θ) 2 I i (θ) i θ p' i (θ) θ P i (θ)p i (θ) θ i Q i (θ)=1-p i (θ) 1992b standard error, SE 2006θ θ 1992b 1 78

13 從 TIMSS 2007 臺灣八年級學生數學科作答反應檢視古典測驗理論和試題反應理論特性和測驗分析結果 Cronbach's alpha 0.90 Cronbach's alpha

14 新竹教育大學教育學報第三十一卷第二期 Cronbach s alpha Embretson Reise TIMSS 2007 Olson, Martin, & Mullis, 2008 TIMSS & PIRLS International Study Center Science and Mathematics Item Review Committee [SMIRC] TIMSS 2007 Crocker Algina

15 從 TIMSS 2007 臺灣八年級學生數學科作答反應檢視古典測驗理論和試題反應理論特性和測驗分析結果 3 TIMSS 2007 Crocker & Algina 1986 Crocker Algina 1986 TIMSS National Research Coordinators [NRC] NRC 5. SMIRC NRC NRC TIMSS NRC 4. NRC SMIRC 81

16 新竹教育大學教育學報第三十一卷第二期 TIMSS Crocker Algina 1986 Bartlett 2, p = % % 3 3 Reckase Reckase 1979 Hattie

17 從 TIMSS 2007 臺灣八年級學生數學科作答反應檢視古典測驗理論和試題反應理論特性和測驗分析結果 Cronbach's alpha Outfit MSQ Infit MSQ Outfit MSQ Infit MSQ M * 1.30 M * 0.89 M M M * 1.29 M * 1.31* M M * 0.81 M M042301A M * 1.35* M042301B 0.64* 0.79 M * 0.82 M042301C 0.53* 0.62* M M M * 0.70 M M M M022234A M M022234B M M M * 0.85 M * 1.16 M * 1.20 M * 1±0.3 4 Outfit MSQ Infit MSQ ~1.3 Bond & Fox, 2007 M M M042301C 4 6 4M ~1.3 6 M M042301C

18 新竹教育大學教育學報第三十一卷第二期 4 M M042301C 6 M

19 從 TIMSS 2007 臺灣八年級學生數學科作答反應檢視古典測驗理論和試題反應理論特性和測驗分析結果 1991 weak assumption strong assumptions IRT ~0.80 M M

20 新竹教育大學教育學報第三十一卷第二期 5 TIMSS M M M M042301B M M042301C M M M M M M M M M M M M M M M042301A M022232* M M022234A* M M022234B* M M042220* M * ~ 0.5 logit 0 ~ 2 logit Wu Adams TIMSS

21 從 TIMSS 2007 臺灣八年級學生數學科作答反應檢視古典測驗理論和試題反應理論特性和測驗分析結果 1 Generalized partial credit model, GPCM Muraki, M M022234A 9 M022234B 10 M

22 新竹教育大學教育學報第三十一卷第二期 2006 IRT IRT

23 從 TIMSS 2007 臺灣八年級學生數學科作答反應檢視古典測驗理論和試題反應理論特性和測驗分析結果 M Content domain number Cognitive domain applying Maximum points 1 key C % (A) 200 (B) 300 (C) 1200 (D) 1600 (E) M TIMSS html M Content domain number Cognitive domain applying Maximum points 1 key A 60 30% (A) 18 (B) 24 (C) 30 (D) M TIMSS html 89

24 新竹教育大學教育學報第三十一卷第二期 6 TIMSS 2007 M A B C* D X M A* B C D X X 2. * M M M A M D M A M D

25 從 TIMSS 2007 臺灣八年級學生數學科作答反應檢視古典測驗理論和試題反應理論特性和測驗分析結果 13 M M ConQuest M % AM D 91

26 新竹教育大學教育學報第三十一卷第二期 7 7 TIMSS NA NA 7 92

27 從 TIMSS 2007 臺灣八年級學生數學科作答反應檢視古典測驗理論和試題反應理論特性和測驗分析結果 8 8 Cronbach's alpha Cronbach's alpha M M M042301C M M M M022234A M022234B M M M

28 新竹教育大學教育學報第三十一卷第二期 Cronbach's alpha 0.90Cronbach's alpha 0.90 TIMSS M M M M M042301C M042301C -3 ~ 0.5 logit 1 M M022234A M022234B M M M

29 從 TIMSS 2007 臺灣八年級學生數學科作答反應檢視古典測驗理論和試題反應理論特性和測驗分析結果 2011 Embretson & Reise,

30 新竹教育大學教育學報第三十一卷第二期 M M M IRT % 1 3% 2 28% 0.64~0.9 IRT -1.62~ M Content domain number Cognitive domain applying Maximum points M TIMSS html 96

31 從 TIMSS 2007 臺灣八年級學生數學科作答反應檢視古典測驗理論和試題反應理論特性和測驗分析結果 number sense National Council of teachers of Mathematics, NCTM Gurganus 2004 Gurganus 97

32 新竹教育大學教育學報第三十一卷第二期 M M M TIMSS 98

33 從 TIMSS 2007 臺灣八年級學生數學科作答反應檢視古典測驗理論和試題反應理論特性和測驗分析結果 GPCM General Partial Credit Model Muraki, 1992 Olson et al., 2008 Testlet Response Theory, TRT NSC I

34 新竹教育大學教育學報第三十一卷第二期 (6) a 9(1) b 9(6) (3) rcpet.ntnu.edu.tw/irt%e5%9c%a8%e9%87%8f%e8%a1%a8%e7 %B7%A8%E8%A3%BD%E4%B8%8A%E7%9A%84%E6%87%89% E7%94%A8(%E4%B8%8B) doc 2011 American Educational Research Association, American Psychological Association, National Council on Measurement in Education [AERA, APA, NCME] (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association, American Psychological Association, National Council on Measurement in Education. Bond, T. G., & Fox, C. M. (2007). Applying the Rasch model: Fundamental measurement in the human sciences (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates. Crocker, L., & Algina, J. (1986). Introduction to classical and modern test theory. New York, NY: Holt, Rinehart and Winston. Embretson, S. E., & Reise, S. (2000). Item response theory for psychologists. Mahwah, NJ: Lawrence Erlbaum Associates. 100

35 從 TIMSS 2007 臺灣八年級學生數學科作答反應檢視古典測驗理論和試題反應理論特性和測驗分析結果 Gullikson, H. (1987). Theory of mental tests. Hillsdale, NJ: Lawrence Erlbaum Associates. Gurganus, S. (2004). Promote number sense. Intervention in School and Clinic, 40(1), Hattie, J. (1985). Methodology review: Assessing unidimensionality of tests and items. Applied Psychological Measurement, 9, Lord, F. M. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbawn Associates. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley. Masters, G. N. (1982). A Rasch model for partial credit scoring. Psychometrika, 47, Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, Olson, J. F., Martin, M. O., & Mullis, I. V. S. (Eds.). (2008). TIMSS 2007 Technical Report. International Study Center, Boston College, Chestnut Hill, MA: TIMSS & PIRLS. Reckase, M. D. (1979). Unifactor latent trait models applied to multifactor tests: Results and implications. Journal of Educational Statistics, 4, Wu, M., & Adams, R. (2007). Applying the Rasch model to psycho-social measurement: A practical approach. Melbourne, Australia: Educational Measurement Solutions. Wu, M. L., Adams, R. J., & Wilson, M. R. (2007). ACER ConQuest: Generalized item response modeling software (2nd ed.). Hawthorn, Australia: Australia Council for Educational Research. 101

36 新竹教育大學教育學報第三十一卷第二期 Inspecting the Characteristics of the Classical Test Theory and Item Response Theory by Using Test Analysis Results and the Responses of Taiwanese Eighth-Grade Students in the TIMSS 2007 Database Hsu- Lin Su* Po-Hsi Chen** Abstract The purpose of this study was to investigate the characteristics of classical test theory (CTT) and item response theory (IRT) by using the responses given by eighth-grade Taiwanese students in the TIMSS 2007 database to conduct test and item analysis from 2 distinct test perspectives, to provide benefit for test design and education settings. A total of 287 students were included in the research, and Booklet 1 was selected as the research tool. Software such as R , Excel 2003, SPSS 12.0, and ConQuest 2.0 were used during data analysis and curve drawing. The results showed that the test difficulty ranged from medium to easy, and the 2 distractors embedded respectively in the 2 items showed that the proportion of average students was more than that of below-average students, according to the aforementioned theories. Moreover, item characteristic curves of 4 multiplechoice items were not ordered. In general, the test quality was high, despite slight flaws meaning that some items could not differentiate average students. However, items were easy for students. The reliability, construct validity, item parameters, category analysis, and overall scores of the test takers in CTT corresponded to concepts of test information function, construct validity, and model fit assessing, item parameters, item and category characteristic curves, abilities of participants in the IRT. The relative strengths of the IRT lie in test information function, latent trait assumption, and parameter invariance and test equating. Regarding the test design and educational implications, we suggest modifying presentations of 2 distractors and making connection to number sense education owing to a specific difficult item. Keywords: classical test theory (CTT), item response theory (IRT), PCM, TIMSS DOI: / Section editor: Shwu-Ching Young Received: December 31, 2013; Modified: March 25, 2014; Accepted: April 10, 2014 * Hsu-Lin Su, Doctoral student, Department of Educational Psychology and Counseling, National Taiwan Normal University, aguri.su@gmail.com ** Po-Hsi Chen, Associate Professor, Department of Educational Psychology and Counseling, National Taiwan Normal University 102

11-3-Cover-1

11-3-Cover-1 104 11 3259-279 2012-2013 2012 2013 2012 2013 2012 2013 2012 2013 260 1 4000 1 261 Computerized adaptive testing, CAT item response theory, IRT 2004 Wang & Chen, 2004Wang, Chen, & Cheng, 2004 2011 2012