124 / 26 型的 Prevalence Adjusted Kappa 值都在 0.6 以上, 顯示再測信度的信度良好, 但自評健康再測信度的 Kappa 值則為 0.378, 兩次回答一致的現象偏低, 但若考慮程度差異的 Weighted Kappa 值則為 0.665, 顯示回答方向一致的穩

Size: px

Start display at page:

Download "124 / 26 型的 Prevalence Adjusted Kappa 值都在 0.6 以上, 顯示再測信度的信度良好, 但自評健康再測信度的 Kappa 值則為 0.378, 兩次回答一致的現象偏低, 但若考慮程度差異的 Weighted Kappa 值則為 0.665, 顯示回答方向一致的穩"

音陨仰
5 years ago
Views:

1 123 研究論文臺灣健康危害行為監測系統之信度分析許勝懋洪百薰洪永泰摘要目標 : 介紹測量工具的信度的概念與評估方法, 首先, 探討受訪者個人特質回答穩定性的關係? 其次, 探討訪談語言與答案穩定性的關係? 進而討論缺乏信度的工具對研究結果的影響, 並以臺灣健康危害行為監測系統為例進行分析且提出建議方法 :2008 年 TBRFSS 使用 893 個再測訪問個案, 進行再次接觸且選擇五個題目進行測量結果 : 在完訪的個案中, 在信度檢測上, 內部一致信度依照資料格式檢測, 利用三高題目等名目尺度進行 KR 20 信度值為利用自評健康與生活滿意度等順序尺度進行 Cronbach s α 信度值為與利用身高體重之等比尺度進行 Guttman 折半係數信度值為 0.714, 顯示內部一致性良好 ; 再測信度則以 test-retest 信度測量, 其中事實題

2 124 / 26 型的 Prevalence Adjusted Kappa 值都在 0.6 以上, 顯示再測信度的信度良好, 但自評健康再測信度的 Kappa 值則為 0.378, 兩次回答一致的現象偏低, 但若考慮程度差異的 Weighted Kappa 值則為 0.665, 顯示回答方向一致的穩定性從二元勝算對數模型的分析可知, 年齡使用語言語言與城鄉的交互作用與語言教育程度的交互作用都是影響民眾再測變化的主要原因結論 : 臺灣健康危害行為監測系統具有良好之信度, 本研究顯示再測信度改變與受訪者的年齡訪談使用語言訪談使用語言與受訪者居住城鄉之交互作用, 以及訪談使用語言與受訪者教育程度之交互作用之間具有顯著相關關鍵字 : 臺灣健康危害行為監測系統內部一致性信度再測信度二元勝算對數模型 The Taiwan Behavior Risk Factor Surveillance System: Reliability Analysis Shen-Mao Hsu, Baai-Shyun Hurng, Yung-Tai Hung ABSTRACT Objectives: This purpose of this study was to develop and test reliability of the Taiwan Behavior Risk Factor Surveillance System (TBRFSS) Questionnaire. Methods: In TBRFSS, 893 cases were selected for internal consistency reliability and test-retest reliability. Result: 7 core items were selected for internal consistency with examining by KR 20 (0.374), Cronbach s alpha (0.477) and the Guttman split-half reliability tests. (0.714). It shows good reliability of the scale. In addition, 5 core items of the TBRFSS were selected for reliability with examining by test-retest reliability tests. It shows Postdoctoral Fellow, Survey Research Center, National Taiwan University Director, Population and Health Research Center, Bureau of Health Promotion, Department of Health Professor, Department of Political Science, National Taiwan University

3 125 good reliability with kappa 0.824, and 0.693, indicating adequate criterion. The binary logit model was used to identify factors significantly associated with adjusted odds rations: age, mixed language (Hainanese and Mandarin Chinese) and the interactions (such as county with language and education with language). Conclusion: The reliability depends on the level of random error measurement tools. The TBRFSS was confirmed to be reliable and it can be applied in future studies to enrich the existing system for evaluation of behavior risk factor surveillance system. It was significantly associated with the factors of age of respondent, response language, interaction with townships of respondent and response language, interaction with education of respondent and response language. Keywords: TBRFSS, internal consistency reliability, test-etest reliability, binary logit model 壹前言 2003 Severe Acute Respiratory Syndrome, SARS Center for Disease Control and Prevention, CDC Behavior Risk Factor Surveillance System, BRFSS

4 126 / pilot study test-retest BRFSS Core Sections 1 state added modules 250 3, Andresen et al., 2003 Behavioral Risk Factor Surveillance System: Operational and User s Guide. Atlanta, US Department of Health and Human Services, Centers for Disease Control and Prevention, Dec 12, Available at ftp://ftp.cdc.gov/pub/data/brfss/userguide.pdf CDC NHIS BRFSS Nelson et al., 2003

5 127 Optional Modules reliabilityconsistency validity BRFSS CDC 17 CDC States-Added Questions 4 20 A B C D

6 128 / 26 貳文獻檢閱 errors Fowler and Mangione, 1990: nominalism rationalism interpretative understanding

7 129 situation cognitive process context response effects 5 Sudman and Bradburn, : item non-response 5 sophisticated 1997

8 130 / 26 measurement Stevens, Lord and Novick, 1968; Nunnally, 1978 observed score true score error score X=T+E X T E 6 scale nominal categorical ordinal interval ratio classification absolute zero Stevens, 1946

9 131 systematic random errors Crocker and Algina, 1986 constant bias ρ x = VAR(t)/VAR(x) Carmines and Zeller, 1979: Kelsey et al., : ㈠㈡㈢㈣㈤

10 132 / 26 trustworthinessconsistency stability dependabilityaccuracy of precision reproducibility correlation coefficient Jones, 1971; Stanley, 1971; Carmines and Zeller, 1979 internal consistency Kuder-Richard 20 Reliability Cronbach s Reliability Split-Half Reliability 1. 庫李信度 ( 以下簡稱 KR 20) KR 20 Kuder- Richard KR 20 = K 1 pq K 1 S T 2

11 133 K p qq=1 p S T 2 2. 克朗巴阿爾發信度係數 ( 以下簡稱 Cronbach s α) Likert Scale Cronbach s α Cronbach s α α k = K 1 S i 2 K 1 S T 2 K S 2 i S 2 T Cronbach s α 折半信度 Guttman 2000 r xx =2 1 S a 2 + S b 2 S x 2

12 134 / 26 S 2 a S 2 b S 2 x intra-reliability1989 Intra-class Correlation Coefficient, ICC ICC ICC ICC Kappa 2004 Robinson et al., 1991 Everitt, 1992 Kappa K = P 0 P c 1 P c P 0 = r i =1 n it / N observed agreement

13 135 P c = 1 r n i n i /N N i =1 chance agreement 1 Everitt 表 1 假設兩位觀察者一致性程度表 Observer N % Observer Everitt, total P 0 = ( )/100=0.66 P c = = K = P 0 P c = =0 1 P c Everitt P 0 1 Kappa 1 Kappa

14 136 / 26 K W = wp 0 wp c quadratic weight=1 i j 2 n wp c k 1 wp 0 wp c k i j Kappa Landis Koch Everitt, 1992 Kappa < Poor Slight Fair Moderate Substantial Almost Perfect 參研究方法

15 KR 20 Cronbach s α Guttman Kappa Kappa weighted Kappa prevalence adjusted Kappa Kappa binary scale ordinal scale

16 138 / 26 multi-nominal2000: Long, 1997; Long and Freese, 2006 Logit Probit link function η ij =log φ ij 1 φ ij = 1 φij 1+e xβ η ij =β 0i +β 1j X ij +e ij log Pr (Y 1 1) 1+e =log xβ =log 1 =log (e xβ )=Xβ 1 Pr (Y 1 ) e xβ e xβ 1+e xβ φ ij odds ratio Kappa prevalence adjusted Kappa

17 139 weighted Kappa Item Non-Response systematic biases Y missing at random 8 肆資料分析 8 list-wise deletion list-wise deletion inefficient and biased estimates multiple imputation Allison, 2000; King et al., 2001 selection bias model

18 140 / 26 KR 20 Cronbach s α Guttman 2 ㈠ KR , ㈡ Cronbach s α Guttman , ㈢ Cronbach s α Guttman ,860 1,041

19 141 表年臺灣健康危害行為監測系統調查問卷內部一致性信度檢測 KR 20 Chronbach s α CS3. Guttman CS5. CS CS CS22. P1. P Cronbach s α Guttman

20 142 / 26

21 143 表年臺灣健康危害行為監測系統調查民眾自覺健康再測 N % Kappa =0.378 N= weighted Kappa= quadratic Weighting Kappa quadratic weighting weighted Kappa Fleiss Kappa 0.75 Kappa 0.4 Fleiss J.L., 1981:

22 144 / 26 表年臺灣健康危害行為監測系統調查醫師確診罹患糖尿病 N % prevalence adjusted Kappa=0.805 N= prevalence adjusted Kappa

23 145 表 5 臺灣健康危害行為監測系統調查民眾吸菸行為 N % prevalence adjusted Kappa=0.810 N= prevalence adjusted Kappa 0.810

24 146 / 26 表年臺灣健康危害行為監測系統調查民眾工作狀況 N % Kappa =0.693 N= Kappa Cronbach s α 0.866

25 表年臺灣健康危害行為監測系統調查民眾 N % Kappa =0.866 N= Kappa

26 148 / log φ ij = β 0 j + β 1 j * + β 2 j * + β 3j *+β 1 φ 4j * ij + β 5 j* + β 6 j** +β 7j* * + β 8 j **+ β 9 j ** + β 10 j * *+ β 11 j * * centering 10 10

27 149 Bryk and Raudenbush, [Exp ( 0.989)=0.011]= grand mean centering incremental

28 150 / 26 表 8 影響民眾再測變化之二元勝算對數模型 β S.E. OR 95%C.I. 0.88** (2.83,2.01) 0.01* (0.10,0.98) = (0.94,0.54) (1.32,0.93) = (1.24,1.02) = (1.83,1.24) 0.50* (0.83,0.39) (1.04,0.98) * 0.27 $ (1.47,1.15) * (1.10,0.75) * (1.05,0.41) * (1.05,0.36) * (1.55,1.02) * 0.45 $ (1.82,1.31) Pseudo R = 0= 2. $ P<0.1 * P<0.05 ** P<0.01 *** P <

29 Exp ( 0.495)= Exp (0.268)=

30 152 / EXP (0.448)=1.565 伍結論與討論 KR 20 Cronbach s α Guttman Kappa 0.378fair Weighted Kappa r ㈠㈡ 2 7 2

31 153 ㈢ Schuman and Presser, 1981 information bias

32 154 / 26 9 表年臺灣健康危害行為監測系統調查訪問使用語言

33 155 陸研究限制 11 BRFSS Rubenstein, BRFSS Andresen et al., 2003

34 156 / 26 2 參考資料 Allison, Paul D Multiple Imputation for Missing Data: A Cautionary Tale, Sociological Methods and Research 28: Andresen, Elena M., T. K. Catlin, K. W. Wyrwich, and J. Jackson-Thompson 2003 Retest Reliability of Surveillance Questions on Health Related Quality of Life, Journal of Epidemiology Community Health 57: Bryk, Anthony S. and Stephen W. Raudenbush 1992 Hierarchical Linear Models in Social and Behavioral Research: Applications and Data Analysis Methods. Newbury Park, CA: Sage Publications. Carmines, G. Edward and Richard A. Zeller 1979 Reliability and Validity Assessment. London: Sage. Crocker, L. and J. Algina 1986 Introduction to Classical and Modern Test Theory. New York: Holt, Rinehart, and Winstton. Everitt, Brian S The Analysis of Contingency Tables, 2 nd ed. London: Chapman and Hall. Fleiss, J.L Statistical Methods for Rates and Proportions. New York: John Wiley and Sons Inc. Fowler, Floyd J. and Thomas W. Mangione 1990 Standardized Survey Interviewing: Minimizing Interviewer-Related Error. London: England Sage.

35 157 Jones, Lyle V The Nature of Measurement, pp in Robert L. Thorndike (ed.) Educational Measurement. Washington, DC: American Council on Education. Kelsey, Jennifer L., Alice S. Whittemore, Alfred S. Evans, and W. Douglas Thompson 1986 Methods in Observational Epidemiology. New York: Oxford University Press. King, Gary, James Honaker, Anne Joseph, and Kenneth Scheve 2001 Analyzing Incomplete Political Science Data: An Alternative Algorithm for Multiple Imputation, American Political Science Review 95: Long, Scott J Regression Models for Categorical and Limited Dependent Variables. Thousand Oaks: Sage Publications. Long, Scott J. and Jeremy Freese 2006 Regression Models for Categorical Dependent Variables Using Stata. College Station, TX: Stata Corp LP. Lord, Frederic M. and Melvin R. Novick 1968 Statistical Theories of Mental Test Scores. Reading, Mass.: Addison-Wesley. Nelson D. E., Powell-Griner E, Town M, and Kovar M. G A Comparison of National Estimates from the National Health Interview Survey and the Behavioral Risk Factor Surveillance System, American Journal of Public Health 93: Nunnally, Jum C Psychometric Theory. New York: McGraw Hill. Robinson, J. P., P. R. Shaver, and L. S. Wrightsman 1991 Measures of Personality and Social Psychological Attitudes. San Diego, CA: Academic Press, Inc. Rubenstein, Sonda Miller 1995 Surveying Public Opinion. Belmont: Wadsworth Publishing Company. Schuman, Howard and Stanley Presser 1981 Questions and Answers in Attitude Surveys: Experiments on Question Form, Wording, and Context. New York: Academic Press. Stanley, Julian C Reliability, pp in Robert L. Thorndike (ed.), Educational Measurement. Washington, DC: American Council on Education. Stevens, Stanley S On the Theory of Scales of Measurement, Science 103:

36 158 / 26 Sudman, Seymour and Norman M. Bradburn 1974 Response Effects in Surveys. Chicago: Aldine. Behavioral Risk Factor Surveillance System: Operational and User s Guide Version 3.0, U.S. Department of Health and Human Services 2005 Behavioral Risk Factor Surveillance System: Operational and User s Guide. Atlanta, U.S. Department of Health and Human Services, Centers for Disease Control and Prevention. Retrieved Dec 12, 2005, from ftp://ftp.cdc.gov/pub/data/ brfss/userguide.pdf : (4):

37 159 附錄 1 自變數的測量題目與內部一致性信度檢測處理方式一覽表 CS CS CS CS CS P1. P

國民健康狀況是分配醫療資源及訂定全國健康目標之主要依據，更是衡量一個國家發展概況的重要指標之一

國民健康狀況是分配醫療資源及訂定全國健康目標之主要依據，更是衡量一個國家發展概況的重要指標之一 89 5 1 212 312 412~19 520~65 1 30 CDC, 1998 1957 1994 89 1 2 3 1994 3,814 15,000 1995 1989 4,049 60 1997 2 1980 1998 1956 2000 2000 3 (computer-assisted telephone interviewing, CATI) (computer-assisted