Microsoft PowerPoint _Hypothesis Testing (S9)

Size: px

Start display at page:

Download "Microsoft PowerPoint _Hypothesis Testing (S9)"

蝙唐
7 years ago
Views:

1 生物統計學 9 Hypothesis Testing ( 假設檢定 ) 陳光琦助理教授 (Kuang-Chi Chen) chichen6@mail.tcu.edu.tw,, Office: H

2 I. Hypothesis Testing Hypothesis tests, like CIs,, are a part of inferential statistics, the science of drawing statistical conclusions from specific data. A hypothesis test,, called a test of significance,, attempts to answer the question, "Could the data have occurred purely by chance?" Hypothesis testing is done constantly in medicine, business, education, polling, government, and the hard and soft sciences. It provides an objective framework (probability model) to reach a decision concerning the population (parameter) by examining a sample from the population. To do a hypothesis test, set up two contradictory statements which are the hypotheses.. The first hypothesis is often the accepted belief or is assumed to be true. Then conduct the test to determine if the data supports or does not support the first hypothesis.

3 Basic steps to a hypothesis test Step 1: Formulate the two hypotheses. They are called the null hypothesis and the alternative hypothesis. H 0 is the first hypothesis and is called the null hypothesis. H A is the second hypothesis and is called the alternative hypothesis. It is contradictory to the null hypothesis. Step 2: Determine the random variable and the distribution for the test.. Knowing the random variable and the distribution for the test, calculate a statistic from the data (for example, the sample mean or the estimated proportion) that will assess the evidence for or against H 0. Step 3: Using the statistic, calculate the p-value.. The p-value is the probability that the statistic calculated from the data will happen purely by chance when the null hypothesis is true.. A smaller p-value indicates stronger evidence against H 0.

4 Basic steps to a hypothesis test (cont d) Step 4: Make a comparison of the p-value with a fixed or preconceived significance level, α. α acts as a cut-off point below which we agree that the statistic calculated from the data is statistically significant: Then make a decision: - If p-value < α,, then we reject the null hypothesis. - If p-value > α,, then we do not reject the null hypothesis. - If p-value = α,, then our test is inconclusive. We, most likely, would gather more data and run at least one more test. Step 5: Write an appropriate conclusion to the hypothesis test so that everyone can understand the result. - If we reject the null hypothesis, we write the alternate hypothesis in a sentence as the conclusion. - If we do not reject the null hypothesis, we simply write it in a sentence as the conclusion..

5 The Null and Alternate Hypotheses and P-value The null hypothesis, H 0, if written as a mathematical statement, has one of the following symbols in it: =,, The alternate hypothesis, H A or H 1, if written as a mathematical statement, has one of the following symbols:, >, < (Note: H 0 and H A are contradictory) Example: Serum cholesterol level for the 20-to-74-year-old males in the US Prior knowledge: the mean serum cholesterol level is 211 mg/100 ml Purpose: we wonder whether the mean serum cholesterol level for the subpopulation of men who are hypertensive smoker is 211, too. Collected data: select a random sample of 25 men from this subpopulation and their mean serum cholesterol level is 230.

6 Example (cont d) Question: Is this sample mean comparable with a hypothesized mean of 211? (Sol) Let μ 0 = 211, the population mean μ= the mean of the subpopulation of interest = sample mean w/ sample size n from the subpopulation Formulating the H 0 and H A H 0 :μ=μ 0 = 211 vs. H A :μ μ 0 (a two-sided hypothesis test) H 0 :μ μ 0 = 211 vs. H A :μ>μ 0 (a one-sided hypothesis test) (use the information of the selected sample as an evidence for or against the hypotheses) - If there is evidence that the sample is not from the population with meanμ 0, we reject H 0. - If there is not sufficient evidence to doubt the validity of H 0, we could not reject H 0. (However, we do not say that we accept H 0 because the test does not prove H 0.)

7 Example Example: H 0 :μ= 5 vs. H A :μ< 5 Left-tailed test. The "<" in H A tells us this fact. Example: H 0 :μ 20 vs. H A :μ> 20 Right-tailed test. The ">" in H A tells us this fact.

8 Example (cont d) Example: H 0 : p 0.3 vs. H A : p < 0.3 Left-tailed test. The "<" in H A tells us this fact. Example: H 0 : p = 0.5 vs. H A : p 0.5 Two-tailed test. The in H A tells us this fact. Notice that the p-value is divided equally in both tails.

9 Preconceived Level of Significance, α Fixed α is also known as a preconceived α. It is the probability to which we compare the p-value. A fixed α level of 0.05 or 0.01 is most commonly used. The significance level α is chosen before the test is actually carried out. Many hypothesis testers choose other values, such as an α of (more conservative) or 0.1 (less conservative).

10 Possible Results of Hypothesis Testing (HT) Suppose that there are two kinds of truth: H 0 is true or H A is true, and two kinds of decisions will be made reject H 0 or do not reject H 0, then the possible results would be (i) (correct) (ii) (incorrect) reject H 0 do not reject H 0 when H 0 is true ; when H 0 is true ; (Type I error) (iii) (correct) reject H 0 when H A is true ; (power) (iv) (incorrect) do not reject H 0 when H A is true. (Type II error) Test Truth H 0 is true H A is true Do not reject H 0 Correct Type II error, β Reject H 0 Type I error, α Correct, 1 β (power)

11 Type I Error, Type II Error and Power 1. Type I Error, α A Type I error occurs when we reject H 0 when H 0 is really true. Fixed or preconceived α (the same α in hypothesis testing) is the probability of rejecting H 0 when H 0 is really true. So, α = P(Type I error) = P(reject H 0 H 0 is true). Example: Suppose H0 is The coin is fair. The Type I error is We do not think the coin is fair when, in fact, it really is. So, α = P(Type I error) = the probability that we think the coin is not fair when, in fact, it really is.

12 Example Suppose H 0 is p 0.6, where p is the proportion of Americans who vote in presidential elections. In words, H 0 is At most, 60% of Americans vote in presidential elections. The Type I error is We think that the proportion of Americans who vote in presidential elections is more than 60% when, in fact, the proportion is at most 60%. (If we reject H 0, then we think that p > 0.60.) So, α = P(Type I error) = the probability that we think the proportion of Americans who vote in presidential elections is more than 60% when, in fact, the proportion is at most 60%.

13 The Type II Error, β A Type II error occurs when we do not reject H 0 when H 0 is really false. Use the Greek letter β as the probability of not rejecting H 0 when H 0 is really false. So, β = P(Type II error) = P(do not reject H 0 H 0 is not true). Example: Suppose H 0 is The coin is fair. The Type II error is We think the coin is fair when, in fact, it really is not fair. So, β = P(Type II error) = the probability that we think the coin is fair when, in fact, it really is not fair.

14 Example Example: Suppose H 0 is p 0.6. The Type II error is We think that the proportion of Americans who vote in presidential elections is at most 60% when, in fact, the proportion is more than 60%. So, β = P(Type II error) = the probability that we think the proportion of Americans who vote in presidential elections is at most 60% when, in fact, the proportion is more that 60%.

15 3. The Power of A Test is defined as 1 - β 1 - β is the probability of rejecting H 0 when, in fact, H 0 is really false. It is desirable to have a high power since we would always want to reject H 0 when H 0 is really false. # Besides (1 - β) being used in the Power of the Test, β is also used to determine the appropriate sample size for the hypothesis test. - α = P(Type I error) = P(reject H 0 H 0 is true) - β = P(Type II error) = P(do not reject H 0 H 0 is not true) - Power = 1 β = P(reject H 0 H 0 is not true) The General Aim of hypothesis testing: To use statistical tests that make α and β as small as possible. However, it is impossible since α as β (or α as β ).

16 The Relation between α and β A strategy to select a (powerful) test: For a fixed α, use the test that minimize β (maximize the power). So, α is referred to as the significance level of a test.

17 About the p-value Definition of p-value: given that H 0 is true, the probability of obtaining a test statistic as extreme as or more extreme than the observed test statistic value (more extreme farther away from H 0 ). The p-value is the probability that the statistic calculated from the data will happen purely by chance when the null hypothesis is true. A smaller p-value indicates stronger evidence against H 0 ; a rare event has occurred; the null hypothesis is not true. Hence, if the p-value is sufficiently small, we reject H 0. What is a sufficiently small probability? In most applications, 0.05 is chosen.

18 About the p-value (cont d) Thus, we reject H 0 when the chance that the sample could have come from a population with meanμ 0 is less than or equal to 5%. That is, we reject incorrectly 5% of the time; i.e., given many repeated tests of significance, 5 times out of 100 we will erroneously reject H 0 when it is true. To be more conservative, a probability of 0.01 is chosen; to be less conservative, a probability of 0.10 might be used. The probability that we choose whether 0.05, 0.01, or some other value is known as the significance level (α) of the hypothesis test, which is specified before the test is carried out.

19 II. One-sample Test H A :μ<μ 0 II. One-sample Test for the mean of a normal distribution with known variance: One-Sided H A :μ<μ 0 ( < null mean) e.g., Let X i = the birth-weight for the i th baby from the lower SES area and sample size n = 100. Assume that X 1,, X n ~ N(μ,σ 2 ), where σ 2 = 25 and the birth-weight of a baby in US is from N(μ 0,σ 02 ) with μ 0 = 120 and σ 02 = 25. (S0) The observed sample x 1,, x n from a population N(μ,σ 2 ) (S1) Hypotheses to be tested H 0 :μ=μ 0 (σ=σ 0 = 5) vs. H A :μ<μ 0 (σ=σ 0 = 5) with a significance level of α.

20 II. One-sample Test H A :μ<μ 0 (cont d) (S2) The test statistic: 2 ( X μ) μ σ = σ / n under H 0 :μ=μ 0, we have Z ~ N(0, 1) n( X ) ~ N(0, ) and Z ~ N(0,1) (i) If the observed value of Z is departed from N(0, 1), then reject H 0. (ii) If the observed value of Z is not departed from N(0, 1), then do not reject H 0. Compute the values of the test statistic: e.g., z obs = ( x μ )/( σ / n) = ( )/(5/10) =

21 II. One-sample Test H A :μ<μ 0 (cont d) (S3) Calculate the p-value: ( 方向與 H A 相同 ) X μ0 x μ0 p-value = P( X < x H0 ) = P( < ) σ / n σ / n x μ0 x μ0 = PZ ( < ) =Φ ( ) =Φ ( zobs ) = σ / n σ / n (S4) Make a comparison of the p-value with a predetermined α. -If p is less than or equal to α, we reject H 0. -If p is greater than α, we do not reject H 0. Since p-value = < 0.05, reject H 0 at 0.05-significance level. (i) If p-value is sufficiently small, then the observed sample is less likely from H 0. (so reject H 0 ) (ii) If p-value is not small enough, then there is not sufficient evidence to doubt the validity of H 0.

22 II. One-sample Test H A :μ<μ 0 (cont d) (S5) Make a conclusion: Since p-value = < 0.05, reject H 0 at 0.05-significance level. We conclude that the true birth-weight is significantly lower in this hospital than in general population. Note that: (i) We can have a very small p-value if the sample size is large. (ii) Statistical Significance Scientific or Clinical Significance. The test result could actually have little practical consequence. Some statistically non-significant results can be scientifically important, if the same trends were also apparent in a large sample.

23 III. & IV. One-sample Test H A :μ>μ 0 & μ μ 0 III. One-sample Test for the mean of a normal distribution with known variance: One-Sided H A :μ>μ 0 ( > null mean) (S1) Hypotheses to be tested H 0 :μ=μ 0 (σ=σ 0 = 5) vs. H A :μ>μ 0 (σ=σ 0 = 5) p P X x H P 0 0 -value = ( > 0 ) = ( < ) σ / n σ / VI. One-sample Test for the mean of a normal distribution with known variance: Two-Sided H A :μ μ 0 ( null mean) (S1) Hypotheses to be tested H 0 :μ=μ 0 (σ=σ 0 = 5) vs. H A :μ μ 0 (σ=σ 0 = 5) X μ x μ X μ x μ p = P X > x H = P > σ / n σ / n 0 0 -value ( 0 ) ( ) n

24 V. One-sample t Test (for unknown variance) (S0) Assume that underlying population ~ N(μ,σ 2 ),σ 2 unknown. (S1) Hypotheses to be tested H 0 :μ=μ 0 vs. H A :μ μ 0 (σ unknown use s) with a significance level of α. (S2) The test statistic: n ( X μ0 ), where 2 ( i ) 2 /( 1) i= 1 T = s = X X n s/ n under H 0 :μ=μ 0, we have T ~ t n 1 (S3) Calculate the p-value: tobs = ( x μ0 )/( s/ n) and p-value = 2 P( T > t obs ) (S4) Make a comparison of the p-value with a predetermined α. (S5) Make a conclusion:

25 Recall that: Diagnostic Tests Test Status Disease (D) No disease (D c ) Test positive (T + ) a b Test negative (T ) c d (i) Sensitivity (True Positive, TP) = P(T + D) = a / (a + c) (ii) Specificity (True Negative, TN)= P(T D c ) = d / (b + d) (iii) False Negative (FN) = P(T - D) = c / (a + c) = 1 sensitivity (Type II error) (iv) False Positive (FP) = P(T + D c ) = b / (b + d) = 1 specificity (Type I error)

26 Combine Diagnostic Testing with Hypothesis Testing Test Status Disease (H A ) No disease (H 0 ) Test positive (Reject H 0 ) Test negative (Not reject H 0 ) Sensitivity, TP Power, 1 β FN Type II error, β (i) Sensitivity (True Positive, TP) = P(T + D) = a/(a+c) (ii) Specificity (True Negative, TN)= P(T D c ) = d/(b+d) FP Type I error, α Specificity, TN (iii) False Negative (FN) = P(T - D) = c/(a+c) = 1 sensitivity (Type II error) (iv) False Positive (FP) = P(T + D c ) = b/(b+d) = 1 specificity (Type I error)

27 Summary 1. Preconceived Level of Significance, α The significance level α is chosen before the test is actually carried out. It is the probability to which we compare the p- value. A fixed α level of 0.05 or 0.01 is most commonly used. 2. Possible Results of Hypothesis Testing & Decision Table Summarize the Type I, Type II errors and the correct decisions in a table. Test Truth H 0 is true H A is true Do not reject H 0 Correct Type II error, β Reject H 0 Type I error, α Correct, 1 β (power)

28 Summary (cont d) 3. The Type I and Type II Errors and Power of a Test - α = P(Type I error) = P(reject H 0 H 0 is true) - β = P(Type II error) = P(do not reject H 0 H 1 is true) - Power = 1 β = P(reject H 0 H 1 is true) (1) Type I Error, α (FP) A Type I error occurs if we reject H 0 when H 0 is really true. Fixed or preconceived α (the same α in hypothesis testing) is the probability of rejecting H 0 when H 0 is really true. (2) Type II Error, β (FN) A Type II error occurs when we do not reject H 0 when H 0 is really false. Use β as the probability of not rejecting H 0 when H 0 is really false.

29 Summary (cont d) (3) The Power of a Test is defined to be (1 β) 1 β is the probability of rejecting H 0 when, in fact, H 0 is really false. It is desirable to have a high power since we would always want to reject H 0 when H 0 is really false. - A strategy to select a (powerful) test: For a fixed α, use the test that minimize β (maximize the power). So, α is referred to as the significance level of a test.

30 Summary (cont d) 4. The P-value Definition of p-value: given that H 0 is true, the probability of obtaining a test statistic as extreme as or more extreme than the observed value of test statistic ( farther away from H 0 ). The p-value is the probability that the statistic calculated from the data will happen purely by chance when the null hypothesis is true. A smaller p-value indicates stronger evidence against H 0 ; a rare event has occurred; the null hypothesis is not true. If the p-value is sufficiently small, we reject H 0. In most applications, α = 0.05 is chosen. Thus, we reject incorrectly 5% of the time; i.e., given many repeated tests of significance, 5 times out of 100 we will erroneously reject H 0 when it is true.

31 Summary (cont d) 4. The P-value To be more conservative, a probability of 0.01 is sometimes chosen; to be less conservative, a probability of 0.10 might be used. (i) If p-value is sufficiently small, then the observed sample is less likely from H 0. (so reject H0) (ii) If p-value is not small enough, then there is not sufficient evidence to doubt the validity of H 0. Note: (i) We always can have a very small p-value if the sample size is large. (ii) Statistical Significance Scientific or Clinical Significance. The test result could actually have little practical consequence.

32 假設檢定之中文說文解字假設檢定 (hypothesis testing): 根據收集的資料來對於母群體的參數做統計判斷 ; 即先對母體參數假設一個可能數值, 然後以客觀收集來的資料計算出來的統計值, 來判斷這個假設是否合理假說 (hypothesis) : 將對於母體未知參數的原先有關陳述 / 想法, 用數量或統計學用語加以表達假說分為兩種 : 虛無假說對立假說一般來說通常採用統計學家 R.A. Fisher 的方法, 總是先提出一個比較不感興趣的敘述作為虛無假說, 而對立假說則為感興趣的敘述 ( 兩者互斥 ) 因此, 虛無假說是我們想要直接驗證的對象, 然後再依據收集來的資料, 拿出證據來推翻虛無假說

33 假設檢定之中文說文解字 (cont d) 另外, 也習慣把等號不相關沒有患病檢查結果為陰性的, 視為虛無假說的部份 ; 而對立假說則為不相等大於小於有相關有患病檢查結果為陽性的例 1: 假設檢察官想要起訴某嫌犯甲, 嫌犯甲宣稱自己是無罪的, 此時虛無假說為無罪, 對立假說為有罪的例 2: 某醫院的檢驗科想知道歲的男性族群的舒張壓是否比一般男性族群高, 因此虛無假說為歲的男性族群的舒張壓等於一般男性族群, 而對立假說則為歲的男性族群的舒張壓大於一般男性族群例 3: 抽菸與肺癌的相關性研究, 虛無假說為不相關 ( 相關性為 0), 對立假說為有相關

34 假設檢定之中文說文解字 (cont d) 根據收集來的資料做決定時, 與未知的真實狀況比較, 可能會有四種情況 :H 0 為真且沒有拒絕 H 0, 此為正確的判斷 ;H 0 為真卻拒絕 H 0, 此為錯誤的判斷, 稱為型 I 誤差 (Type I error); H 1 為真卻沒有拒絕 H 0, 此為型 II 誤差 (Type II error);h 1 為真且拒絕 H 0, 此為檢定力 (power) 在統計學上, 犯型 I 錯誤的機率, 通常以 α 表示 ; 犯型 II 錯誤的機率, 以 β 表示, 其中,α 又稱為顯著水準 (significant level) α = P(Type I error) = P(reject H 0 H 0 is true) β = P(Type II error) = P(do not reject H 0 H 1 is true) Power = 1 β = P(reject H 0 H 1 is true)

35 假設檢定之中文說文解字 (cont d) 一般採用 0.05 或 0.01 作為顯著水準, 顯著水準設定越小, 意謂著越嚴格保守因此, 對於犯型 I 錯誤的嚴重性較高的檢定, 採取較嚴格的顯著水準 (α= 0.01, 0.001), 對於犯型 I 錯誤的不太嚴重的檢定, 則採用一般的顯著水準 (α= 0.05) 例 1: 假設檢察官想要起訴某嫌犯甲, 嫌犯甲宣稱自己是無罪的, 此時虛無假說為無罪, 對立假說為有罪的若判決有罪但實際上嫌犯甲是無罪的, 此為型 I 錯誤 ; 若判決有罪的刑責是死刑, 則犯型 I 錯誤很嚴重, 宜採取保守的態度,α= 0.001, 控制讓犯型 I 錯誤的機率很小若判決無罪但實際上嫌犯是有罪的, 此為型 II 錯誤 ; 若判決有罪的刑責為勞動服務 50 天, 則可採一般的顯著水準.

36 假設檢定之中文說文解字 (cont d) 因此採用 0.05 或 0.01 作為顯著水準, 即控制型 I 誤差在 0.05 或 0.01 以內每 100 次有罪的判決中, 會有 5 次或 1 次的誤判檢定力 (power of test), 是指正確的拒絕 H 0 的機率,i.e. 拒絕 H 0 當 H 1 為真的機率, 通常以 1-β 來表示 p- 值 (p-value) 為在假設 H 0 為真的情況下, 計算比樣本平均數值更極端的機率如果 p- 值很小, 則樣本不太可能來自 H 0 的母體, 所以拒絕 H 0 如果 p- 值不夠小, 則沒有充分證據證明樣本不是來自 H 0 的母體, 所以無法拒絕 H 0

37 假設檢定之中文說文解字 (cont d) 假說檢定係當假說建立好之後, 自感興趣的母群體隨機收集樣本數 n 的資料, 將此樣本平均數與假設平均數 μ 0 做比較, 檢定樣本平均數與假設平均數 μ 0 的差異, 是否大到不可能僅為機會所造成的若證據顯示, 此樣本不可能來平均數 μ 0 的群體, 則拒絕 H 0 ; 也就是在假設 H 0 為真的情況下拒絕 H 0 所犯的型 I 錯誤機率很小時, 此時資料不支持 H 0, 而更支持 H 1 若沒有充分證據懷疑 H 0 的正當性, 則不能拒絕 H 0, 此時會下結論 : 母體平均數可能為 μ 0 與 μ 0 沒有統計上的顯著差異 ; 然而, 因為此檢定並無證明 H 0, 故不會宣稱接受 H 0, 因為也可能母體平均數是其他數值而非 μ 0

38 作業九 1. Exercise (a), (b), (c) (p. 255) 2. Exercise (a), (b), (c), (d) (p. 255) Reference: Principles of Biostatistics, 2 nd ed., chap. 9, Pagano and Gauvreau, Duxbury, 2000

39 References 中文 1. 生物統計學, 楊惠齡林明德編著,5 th, 新文京開發出版 2. 非讀不可 - 統計學評論, 趙民德謝邦昌蘇志雄, 鼎茂圖書 3. 生物醫學統計概論, 戴政江淑瓊, 翰蘆圖書英文 Fundamentals of Biostatistics, 5 th ed., Rosner, Duxbury, 2000

Microsoft Word - TIP006SCH Uni-edit Writing Tip - Presentperfecttenseandpasttenseinyourintroduction readytopublish

Microsoft Word - TIP006SCH Uni-edit Writing Tip - Presentperfecttenseandpasttenseinyourintroduction readytopublish 我难度 : 高级对们现不在知仍道有听影过响多少那次么 : 研英究过文论去写文时作的表技引示巧言事 : 部情引分发言该生使在中用过去, 而现在完成时仅表示事情发生在过去, 并的哪现种在时完态成呢时? 和难过道去不时相关? 是所有