專題 : Test for Non-normality:An Intensive Simulation Practice 目的 : 統計學者提出的檢定統計式 (Test Statistic) 通常需要經過大量的模擬, 藉以驗證其分配的假設與 Type I Error 的維持, 並觀察檢定力的高低本

Size: px

Start display at page:

Download "專題 : Test for Non-normality:An Intensive Simulation Practice 目的 : 統計學者提出的檢定統計式 (Test Statistic) 通常需要經過大量的模擬, 藉以驗證其分配的假設與 Type I Error 的維持, 並觀察檢定力的高低本"

词妤支
4 years ago
Views:

1 專題 : Test for Non-normality:An Intensive Simulation Practice 目的 : 統計學者提出的檢定統計式 (Test Statistic) 通常需要經過大量的模擬, 藉以驗證其分配的假設與 Type I Error 的維持, 並觀察檢定力的高低本專題以常態檢定為題, 作為程式寫作的進階, 因此選擇幾個較複雜的檢定統計式, 配合 Monte Carlo Method, 練習程式寫作的技巧細心與耐心為什麼 : 統計學者寫作程式可比喻為生化學者在實驗室操作器械與藥品統計方法需經過驗證才能為實務界所用, 而電腦模擬是一種有效的驗證的方式, 以電腦與軟體為實驗設備, 程式寫作為架構與程序, 藉由反覆執行來滿足統計上所根據的機率假設學者必須在從學過程中練習實驗設備的駕馭與實驗程序的熟捻, 本專題提供執行這個過程的引導做什麼 : 本專題選擇四種檢定統計式來測試其常態檢定能力, 利用 Monte Carlo Method 重複隨意的抽取資料, 觀察其 Type I Error 的維持與檢定力執行本單元的先決條件是必須具備基本的程式基礎, 才能應付本單元的幾個特點 : 資料本身具備群組性 (Treatments), 也就是假設資料來自不同的群組, 各組平均數不同 Monte Carlo Method 重複的次數要夠多, 譬如次此時程式執行的效率非常重要, 更凸顯 MATLAB 以矩陣作為運算基礎的概念觀察項目多, 程式結果的儲存與表達非常關鍵我們希望觀察不同組數在不同樣本數下的表現在檢定力的測試上, 需要假設不同的 H A 來源, 譬如 F 分配,T 分配,... 1

2 背景 : 四個常態檢定統計式以下四個檢定式基本上分為兩種形式, 其一為統計式的分配確定, 其二為統計式的分配未知未知分配的統計式以實驗式分配 (Emperical Distribution) 的方式計算相關的 p 值, 或做為拒絕假設與否的依據這是個無母數統計的手段, 非常實用有趣 1 W Test Suppose (x 1 x 2,, x n ) be the sample of size n to be tested for non-normality and y 1 < y 2 < < y n is its ordered counterpart.the W statistic is defined by where a = (a 1, a 2,, a n ) T is W = ( n i=1 a iy i ) 2 n i=1 (y i ȳ) 2 (1) a = (m T V 1 V 1 m) 1/2 m T V 1 where V is the covariance matrix of the order statistics of a sample of n standard normal random variables with expectation vector m. The vector a is antisymmetric, that is, a n = a 1 and for odd n, a (n/2)+1 = 0. Also a T a = 1. The approxiamtion of the coefficients (a 1, a 2,, a n ) according to the sample size n is given by for n = 3 for 4 n 1000 (ã 1, ã 2, ã 3 ) = ( ) ã n = c n x x x x x 5 ã n 1 = c n x x x x x 5 ã i = m i φ 2

3 for i = 2,, n 1(n 5) or i = 3,, n 2(n > 5) where x = 1 n m i = Φ 1 ( (i 3 8 )/(n ) ), Φ is the normal cdf c i = m i mt m φ = ( m T m 2 m 2 n)/(1 2ã 2 n) if n 5 = ( m T m 2 m 2 n 2 m 2 n 1)/(1 2ã 2 n 2ã 2 n 1) if n > 5 To find the p-value for W, Z = (w µ)/σ is referred to the upper tail of N(0, 1), where for 4 n 11 w = ln (γ ln(1 W )) γ = n µ = n n n 3 σ = exp( n n n 3 ) for 12 n 2000 x = ln n w = ln(1 W ) µ = x x x 3 σ = exp( x x 2 ) For n = 3, the p-value is directly given by p = 6 π ( sin 1 W sin )

4 2 Global Test:GW and GC GC is a global test for the normality by combining the test for each treatment and is written by K GC = 2 ln p i (2) where p i is the p-value for the i-th treatment by the W test and K represents i=1 the number of treatments. The GC statistic follows an asymptotical chisquare distribution with 2K degrees of freedom. The other global test that uses the transformed Z instead of p-value is called GW statistic which is defined by GW = K i=1 Z i K (3) The upper tail Z test then follows. 3 MQ test: The Test with Emperical Distribution The Q test proposed by Zhang(1999) is defined as the log-ratio of two unbiased estimator of the population variance. The two unbiased estimators are calcualted by q 1 = n i=2 g i/(n 1) and q 2 = n 4 i=1 h i/(n 4) where g i = y i y 1 m i m 1, i = 2, 3,, n h i = y i y i+4 m i m i+4, i = 1, 2,, n 4 where y i and m i represents the order statistics and their expected values under normality,i.e. m i = Φ 1 ( (i 3 8 )/(n )). The test is then defined as Q 1 = log q 1 q 2 4

5 The second statistic is constructed by reversing and rearranging y i, i.e. the new sequence of sample is (y1, y2,, yn) = ( y n, y n 1,, y 1 ). The second statistic Q 2 is then defined in the same way as Q 1 by employing yi. It is noted that both statistics(q 1, Q 2 ) do not follow any standard distribution. The emperical distributions for both statistics can be generated based upon Monte Carlo samples, 1 i.e. the discrete versions of CDF of Q 1 and Q 2 statistics. Figure 1demonstrates an example of using Monte Carlo smaples. Based on the emperical CDF, the null hypothesis is accepted only when both Q 1 and Q 2 are non-significant at the level α/2. Ths test procedure is denoted as the MQ test. 圖 1: Monte Carlo samples for the emperical CDF of Q 1 以上統計式的表達以英文為主, 希望習者能漸適應期刊文章的方式 1 A MATLAB instruction cdfcalc can be used to calculate CDF from the Monte Carlo samples of the statistic. 5

6 怎麼做型一誤的維持在虛無假設下, 樣本來自常態, 本專題想瞭解不同的檢定統計式, 在不同的樣本數及群組 (treatments) 組合下, 對於型一誤的維持 1. 先假設資料來自同一個常態群組, 平均數 µ = 1, 變異數 σ 2 = 1 針對不同的樣本數 n = 3, 4,, 60, 各產生組樣本, 對每組樣本進行 W-test 與 MQ-test, 假設型一誤 α = 0.05, 紀錄拒絕虛無假設的比例, 畫一張圖展示這兩種方法的拒絕比例與樣本數的關係 2. 假設資料來自 K 個常態群組, 其平均數分別為 µ = 2K 1, K = 2, 3,, 6, 變異數則同為 1 每個群組的樣本數設定為 n = 3, 4,, 60/K, 也就是總樣本數以 60 為限執行上述的四種檢定, 針對不同組數 K, 各畫一張圖展示拒絕比例與總樣本數 Kn 的關係請注意,W-test 與 MQ-test 處理多組數的情況, 需先將各組資料減去該組的樣本平均數, 再將各組資料合在一起處理檢定力當樣本來自對立假設, 則拒絕虛無假設的比例稱為檢定力 (Power), 因對立假設的範圍太廣泛 ( 所有不是常態分配的資料 ), 只能限制對立假設來自非常態的母體, 譬如以下分配 : 1. Lognormal distribution α = 3, β = 1 2. Chi-square distribution with degree of freedom N=1 3. Chi-square distribution with degree of freedom N=4 4. T distribution with degree of freedom N=1 5. T distribution with degree of freedom N=4 6. Uniform distribution on (0,1) 6

7 7. Weibull distribution alpha = 2, β = 1 8. Weibull distribution alpha = 0.5, β = 1 9. Beta distribution alpha = 0.5, β = Beta distribution alpha = 5.5, β = F distribution alpha = 50, β = 3 同樣的, 依上述作法, 針對不同組數 K, 各畫一張圖展示拒絕比例與總樣本數 Kn 的關係參考文獻 [1] Royston, J.P.(1992), Approximating the Shapiro-Wilk W-test for nonnormality, Statistics and Computing, 2,

Vol. 15 No. 1 JOURNAL OF HARBIN UNIVERSITY OF SCIENCE AND TECHNOLOGY Feb O21 A

Vol. 15 No. 1 JOURNAL OF HARBIN UNIVERSITY OF SCIENCE AND TECHNOLOGY Feb O21 A 5 200 2 Vol 5 No JOURNAL OF HARBIN UNIVERSITY OF SCIENCE AND TECHNOLOGY Feb 200 2 2 50080 2 30024 O2 A 007-2683 200 0-0087- 05 A Goodness-of-fit Test Based on Empirical Likelihood and Application ZHOU

專題 : Test for Non-normality:An Intensive Simulation Practice 目的 : 統計學者提出的檢定統計式 (Test Statistic) 通常需要經過大量的模擬, 藉以驗證其分配的假設與 Type I Error 的維持, 並觀察檢定力的高低 本

專題 : Test for Non-normality:An Intensive Simulation Practice 目的 : 統計學者提出的檢定統計式 (Test Statistic) 通常需要經過大量的模擬, 藉以驗證其分配的假設與 Type I Error 的維持, 並觀察檢定力的高低本