NO.34 第九卷第二期中華民國 102 年 07 月 Standard Setting and its External Validation Abstract Mingchuan Hsieh Jin-Chang Hsieh The purpose of this study is to illu

Size: px

Start display at page:

Download "NO.34 第九卷第二期中華民國 102 年 07 月 Standard Setting and its External Validation Abstract Mingchuan Hsieh Jin-Chang Hsieh The purpose of this study is to illu"

老苏
5 years ago
Views:

專題標準設定實施程序與外在效度驗證 *** Yes/No Angoff * 國家教育研究院測驗及評量研究中心助理研究員 (Assistant Research Fellow, Research Center for Testing and Assessment, National Academy for Educational

1 專題標準設定實施程序與外在效度驗證 *** Yes/No Angoff * 國家教育研究院測驗及評量研究中心助理研究員 (Assistant Research Fellow, Research Center for Testing and Assessment, National Academy for Educational Research) ** 國家教育研究院測驗及評量研究中心副研究員 (Associate Research Fellow, Research Center for Testing and Assessment, National Academy for Educational Research) 專題標準設定實施程序與外在效度驗證 149

2 NO.34 第九卷第二期中華民國 102 年 07 月 Standard Setting and its External Validation Abstract Mingchuan Hsieh Jin-Chang Hsieh The purpose of this study is to illustrate the standard setting procedure and the validation approaches. The researchers demonstrate the process based on the experiences from the Taiwan Assessment of Student Achievement(TASA). In this case, the standard setting panelists are well-presented from each area in Taiwan. The final cut scores are also validated based on several external validity evidences. The suggestions for setting up cut scores for the national tests were provided in the end of this study. Key Words: Validity, Standard Setting, TASA 150

3 專題 % % 專題標準設定實施程序與外在效度驗證 151

4 NO.34 第九卷第二期中華民國 102 年 07 月 2012 Angoff modified Angoff Borderline Group Method Borderline Group Median Rasch Objective Standard Setting Method Criterion Development Refining MasteryEnsuring Precision Setting Standard Stone, Angoff 152

5 專題 TASA 1 (1) (2) (3) 教師學者 1. 至少有五年以上的教學經驗且避免是行政兼課的教師 2. 於 ( 該年段及社會科 ) 至少有二年以上的教學經驗 3. 在社會科專業領域具有某程度傑出或可受公認的表現 1. 熟悉該年段社會科之評量架構及內容 2. 在其專業領域具有某程度傑出或可受公認的表現專題標準設定實施程序與外在效度驗證 153

6 NO.34 第九卷第二期中華民國 102 年 07 月 Yes/No Angoff Angoff William Angoff 1971 Angoff Angoff Minimally Competent Examinee Angoff Angoff Angoff Impara Plake 1997 Yes/No Angoff yes no Yes No Angoff Yes/No Angoff (1) (2) (3) (4) 154

7 專題 2 Borderline Group / / / 3. Performace Level Description PLD PLD PLD YES/No Angoff ( ) ( ) 2010 Yes/No Angoff 1. 基礎最低能力者可以答對此題 2. 精熟最低能力者可以答對此題 3. 進階最低能力者可以答對此題 4. 進階較高能力者才可以答對此題 V 專題標準設定實施程序與外在效度驗證 155

8 NO.34 第九卷第二期中華民國 102 年 07 月 Yes/No Angoff Item Response Theory, IRT BILOG-MG Zimowski, Muraki, Mislevy, & Bock, 1996) TASA External Evidence for Validity Kane, 1994, 2001 Yes/No Angoff 8260 Kane 1994,

9 專題 Yes/No Angoff Contrast Group MethodZieky & Livingston, 1977 Zieky Livingston 1977 PLD TASA 1986 M 1 - [S 1 (M 1 - M 2 )/(S 1 + S 2 )] M 1 M 2 S 1 S 2 Yes/No Angoff VS. TASA Yes/No Angoff TASA % 74.5% Yes/No Angoff Yes/No Angoff 專題標準設定實施程序與外在效度驗證 157

10 NO.34 第九卷第二期中華民國 102 年 07 月 Angoff N = 491 Angoff 基礎以下基礎精熟進階總計 = 33.8% Angoff N = 491 Angoff 基礎以下基礎精熟進階總計 = 74.5 % Angoff N = 491 Angoff 基礎以下基礎精熟進階總計 =47.7 % 158

11 159 專題標準設定實施程序與外在效度驗證專題 Yes/No Angoff

12 NO.34 第九卷第二期中華民國 102 年 07 月 Yes/No Angoff NAEP 1. Yes/No Angoff 160

13 專題 Yes/No Angoff PISA2015 專題標準設定實施程序與外在效度驗證 161

14 NO.34 第九卷第二期中華民國 102 年 07 月參考書目 edu.tw/plan.htm (4) (2) * blog.udn.com/baogon/ w w w.t i n g-w e n.c o m/n e w s_s h o w. asp?exmno=2062 Angoff, W. H. (1971). Scales, norms, and equivalent scores. In R. L. Thorndike (Ed.), Educational Measurement (pp ). Washington, D.C.: American Council on Education. Impara, J. C., & Plake, B.S. (1997). Standard setting: An alternative approach. Journal of Educational Measurement, 34, Kane, M. T. (1994). Validating the performance standards associated with passing scores. Review of Educational Research, 64(3), Kane, M. T. (2001). So much remains the same: Conception and status of validation in setting standards. In G. J. Cizek (Ed.). Standard setting: Concepts, methods, and perspectives (pp ). Mahwah, NJ: Erlbaum. Stone, G. E. (2004). Objective standard setting (or truth in advertising). In Smith, E. V. Jr, & Smith R. M.,(Eds). Introduction to Rasch measurement: Theory, models, and applications(pp ). Maple Grove, MN: JAM Press. Zieky, M. J., & Livingston, S. A. (1977). Manual for setting standards on the Basic Skills Assessment Tests. Princeton, NJ: Educational Testing Service. Zimowski, M.F., Muraki, E., Mislevy, R.J., & Bock, R.D. (1996). BILOG-MG [Computer Software]. Chicago: Scientific Software International. 162

101 102 4

101 11 / 02 09:00 ~ 17:00 1 1024 101 102 4 ... 7... 8... 87 敍... 152 序 e 86 453 102 1 23 85 4 19 102 1 23 102 4 壹開幕致詞一主持人 : 李繼玄考選部政務次長 88 90 101 60 19 101 5 7 貳第一場次專題研討 101 11