Previous Next First Last Ba

Size: px

Start display at page:

Download "Previous Next First Last Ba"

鸳雷
5 years ago
Views:

1 Office: 1006 Phone:

2 Previous Next First Last Back Forward 1

3 1.1 (predictor variables)(a.k.a, (independent variables), (regressors)) (response variable)(a.k.a (dependent variable), (explained variable), (regressand) ),,,. Previous Next First Last Back Forward 1

4 Multiple Regression Analysis x 1, x 2,..., x p 1, y y = β 0 + β 1 x 1 + β 2 x β p 1 x p 1 + e β = (β 0, β 1,..., β p 1 ) ( ) x 1,..., x p 1, ( ),. e, e (0, σ 2 ), E(ex i ) = 0, i = 1,..., p 1., n, y i = β 0 + β 1 x i1 + β 2 x i2 + + β p 1 x i(p 1) + e i Previous Next First Last Back Forward 2

5 y 1 1 x 11 x 1(p 1) y 2 1 x 21 x 2(p 1) =..... β + e 1 e 2. y n 1 x n1 x n(p 1) e n Y n 1 = X n p β + ϵ y i, x i1,..., x i(p 1) y, x 1,..., x p 1., : Ee i = 0 V ar(e i ) = σ 2 ( ) Cov(e i, e j ) = 0, i j Previous Next First Last Back Forward 3

6 (Multiple linear regression) Y n 1 = X n p β + ϵ Eϵ = 0, V ar(ϵ) = σ 2 I n. : y ij = µ + τ i + e ij, j = 1,..., n i, i = 1, 2, 3, (dummy variable), x ij = 1, i = j; 0. y ij = µ + τ 1 x i1 + τ 2 x i2 + τ 3 x i3 + e ij Previous Next First Last Back Forward 4

7 , : (, ) (, ) (, ) Previous Next First Last Back Forward 5

8 β : ˆβ = arg min β R p Y Xβ 2 = arg min β R p x i0 1. n ( p 1 ) 2 y i β k x ik i=1 k=0 X (p < n), β ˆβ = (X X) 1 X Y ˆβ β. Ŷ = X ˆβ = X(X X) 1 X Y = HY, H (Hat). ˆϵ = Y Ŷ = (I H)Y Previous Next First Last Back Forward 6

9 ˆϵ X = 0, ˆϵ Ŷ = 0. ˆϵ Ŷ = 0, Y Y Y Y = (Ŷ + Y Ŷ ) (Ŷ + Y Ŷ ) = Ŷ Ŷ + ˆϵ ˆϵ X 1, X ˆϵ = 0 1 ˆϵ = 0 = 1 Y = 1 Ŷ. 1 Y 1 Ŷ Y Y 1 Y = Ŷ Ŷ 1 Ŷ + ˆϵ ˆϵ n n n (y i ȳ) 2 = (ŷ j ȳ) 2 + i=1 i=1 i=1 SST = SS reg + SS e = + = + ˆϵ 2 i Previous Next First Last Back Forward 7

10 , R 2 = 1 SS e SST = SS reg SST (coefficient of determination). R ( ). E ˆβ = β, cov( ˆβ) = σ 2 (X X) 1 cov( ˆβ, ˆϵ) = 0 c β c ˆβ.(Gauss-Markov ) Previous Next First Last Back Forward 8

11 1.2, m, Y 1,..., Y m, x 1,..., x p 1,, : Y 1 = β 01 + β 11 x β (p 1)1 x p 1 + e 1 Y 2 = β 02 + β 12 x β (p 1)2 x p 1 + e 2. Y m = β 0m + β 1m x β (p 1)m x p 1 + e m Y i. e = [e 1, e 2,..., e m ] Ee = 0, Cov(e) = Σ = (σ ij ) Previous Next First Last Back Forward 9

12 n, j x j1, x j2,..., x j(p 1), y j = [y j1, y j2,..., y jm ], j = 1,..., n., y 11 y 12 y 1m y 21 y 22 y 2m Y n m = = [y. (1), y (2),, y (m) ] y n1 y n2 y nm x 10 x 11 x 1(p 1) x 20 x 21 x 2(p 1) X n p =. x n0 x n1 x n(p 1) x i0 1, i = 1,..., n. Previous Next First Last Back Forward 10

13 β 01 β 02 β 0m β 11 β 12 β 1m B p m = = [β. (1), β (2),, β (m) ] β (p 1)1 β (p 1)2 β (p 1)m ϵ 11 ϵ 12 ϵ 1m ϵ 1 ϵ 21 ϵ 22 ϵ 2m ϵ 2 ϵ n m = = [ϵ. (1), ϵ (2),, ϵ (m) ] =. ϵ n1 ϵ n2 ϵ nm, (Multivariate linear model) : ϵ n Y n m = X n p B p m + ϵ n m = [Xβ (1),, Xβ (m) ] + [ϵ (1), ϵ (2),, ϵ (m) ] Previous Next First Last Back Forward 11

14 Eϵ (i) = 0, Cov(ϵ (i), ϵ (j) ) = σ ij I n, i, j = 1, 2,..., m. k ϵ k Σ,. i y (i), y (i) = Xβ (i) + ϵ (i) Eϵ (i) = 0, Cov(ϵ (i) ) = σ ii I n, i = 1,..., m. i, n i y (i) β (i) ˆβ (i) = (X X) 1 X y (i) Previous Next First Last Back Forward 12

15 , ˆB = [ ˆβ (1), ˆβ (2),..., ˆβ (m) ] = (X X) 1 X [y (1), y (2),, y (m) ] = (X X) 1 X Y 1. X, ˆB B.., vec(y ) = (X I m )vec(b ) + vec(ϵ ) Cov(ϵ ) = I n Σ.,, Q(vec(B )) = vec(y ) (X I m )vec(b ) 2 Previous Next First Last Back Forward 13

16 vec(b ) vec(b ) = [((X I m )) (X I m )] 1 (X I m ) vec(y ) = [(X X) 1 X I m ]vec(y ),. 2. ˆB B, 1. ˆB B. 2. Cov( ˆβ (i), ˆβ (k) ) = σ ik (X X) 1, i, k = 1,..., p. 3. Σ ˆΣ = 1 n p Y P Y. P = I n X(X X) 1 X = I n H. Previous Next First Last Back Forward 14

17 . (1). (2), Cov( ˆβ (i), ˆβ (k) ) = (X X) 1 X Cov(ϵ (i), ϵ (k) )X(X X) 1 = σik(x 2 X) 1. (3). P X = X P = 0, Y P Y = (Y XB) P (Y XB) (EY P Y) ij = (E[(Y XB) P (Y XB)]) ij = E[(y (i) Xβ (i) ) P (y (j) Xβ (j) )] = tr{p E[ϵ (j) ϵ (i)]} = tr{p (σ ji I n )} = (n p)σ ij.. Previous Next First Last Back Forward 15

18 1.2.1 B, Ŷ = X ˆB = X(X X) 1 X Y ˆϵ = Y Ŷ = [I X(X X) 1 X ]Y = P Y Y Y }{{} totalsscp ˆϵ (i) = Ŷ Ŷ }{{} RegSSCP + ˆϵ ˆϵ }{{} ErrorSSCP Eˆϵ (i) = 0, Eˆϵ = 0, Eˆϵ (i)ˆϵ (j) = (n p)σ ij Eˆϵ ˆϵ = (n p)σ Previous Next First Last Back Forward 16

19 ˆB ˆϵ ϵ N n m (0, I n Σ), Σ m m > 0 Y N n m (XB, I n Σ) B Σ. 3. Y N n m (XB, I n Σ), X, B R p m Σ > 0, B Σ ˆB = (X X) 1 X Y ˆΣ = 1 n Y P Y., l(b, Σ) = 1 2 nln 2πΣ 1 2 tr[(y XB)Σ 1 (Y XB) ] Previous Next First Last Back Forward 17

20 tr[(y XB)Σ 1 (Y XB) ] = tr[σ 1 (Y X ˆB) (Y X ˆB)] + tr[σ 1 ( ˆB B) X X( ˆB B)] tr[σ 1 (Y X ˆB) (Y X ˆB)] = tr[σ 1 Y P Y] = tr[σ 1 ˆΣ ] B = ˆB, max l(b, Σ) = max l( ˆB, Σ) = max { 1 Σ>0,B Σ>0 Σ>0 2 nlog Σ 1 2 ntr[σ 1 ˆΣ ]} Σ = ˆΣ.. B, ˆB = ˆB; Σ. 4., ϵ N n m (0, I n Σ), ˆB ˆΣ, Previous Next First Last Back Forward 18

21 (1) ˆB N p m (B, (X X) 1 Σ); (2) ˆB ˆΣ ; (3) (n p)ˆσ W m (n p, Σ).. (1) Y N n m (XB, I n Σ), ˆB = (X X) 1 X Y N p m (B, (X X) 1 Σ). (2) ( 9), vec( ˆB) = [I (X X) 1 X ]vec(y) vec(p Y) = [I P ]vec(y) ˆB P Y vec( ˆB) vec(p Y) [I (X X) 1 X ][I P ] = I (X X) 1 X P = 0. (3) Y N n m (X ˆB, I Σ), Rank(P ) = n p, Wishart ( Cochran ) (n 1)ˆΣ = Y P Y W m (n p, Σ). Previous Next First Last Back Forward 19

22 5 (Gauss-Markov ). Θ = XB = [θ (1),..., θ (m) ], ϕ = m j=1 c jθ (j), c 1,..., c m m n, Ŷ = X ˆB = [ŷ (1),..., ŷ (m) ], ˆϕ = m j=1 c jŷ (j), ˆϕ ϕ.. ˆϕ., Ŷ = HY = [Hy (1),..., Hy (m) ], H = X(X X) X. ˆϕ = ˆϕ Y. m c jhy (j) j=1 ˆϕ. ϕ = m j=1 d jy (j) ϕ, m m Eϕ = d jθ (j) = ϕ = c jθ (j) j=1 j=1 Previous Next First Last Back Forward 20

23 m (d j c j ) θ (j) = 0, θ (j) L(X) j=1 (d j c j ) θ (j) = 0, θ (j) L(X) H(d j c j ) = 0 Hd j = Hc j, j = 1,..., m cov(y (j), y (k) ) = σ jk I n V ar(ϕ ) V ar( ˆϕ) m m = d j(i n H)d k σ jk j=1 k=1 = tr[d (I n H)DΣ] 0 D = [d 1,..., d m ], D(I n H) = 0, d j = Hd j = Hc j, j = 1,..., m., ˆϕ = ˆϕ. Previous Next First Last Back Forward 21

24 1.2.2 y n 1 = X n p β p 1 + e n 1, y X L(X) ŷ = X ˆβ = Hy. y Hy = (I H)y = ê, H = X(X X) 1 X. Previous Next First Last Back Forward 22

25 , Y y (i) L(X), Ŷ = X ˆB, Y Ŷ = (I n P )Y = ˆϵ : Y = XB + ϵ, Rank(X) = p AB = C, A s p, C s m, Rank(A) = s < m ϵ n m (0, I n Σ) B, ˆB H B. ˆB = (X X) 1 X Y B. Previous Next First Last Back Forward 23

26 Lagrange.,,. AB = C B = A C + (I A A)z, z A,, A = (X X) 1 A (A(X X) 1 A ) 1. Y XA C = X(I A A)z + ϵ z ẑ = [(I A A) X X(I A A)] 1 (I A A) X (Y XA C) Previous Next First Last Back Forward 24

27 B ˆB H = A C + (I A A)[(I A A) X X(I A A)] 1 (I A A) X (Y XA C) = ˆB (X X) 1 A (A(X X) 1 A ) 1 (A ˆB C), Σ ˆΣ H =.. 1 n p + s (Y X ˆB H ) (Y X ˆB H ) Previous Next First Last Back Forward 25

28 1.2.4 x 0 = [1, x 01,..., x 0(p 1) ], x 0B Ŷ 0 = x 0 ˆB, ˆB B. Ŷ 0 x 0B, EŶ 0 = x 0E ˆB = x 0B. x 0 ˆB x 0B i k E[x 0( ˆβ (i) β (i) )( ˆβ (i) β (i) ) x 0 ] = σ ik x 0(X X) 1 x 0. Y 0 Ŷ 0 i k E(Y 0i x 0 ˆβ (i) )(Ŷ0k x 0 ˆβ (k) ) = E(ϵ 0i x 0( ˆβ (i) β (i) ))(ϵ 0k x 0( ˆβ (k) β (k) )) = E(ϵ 0i ϵ 0k + x 0E( ˆβ (i) β (i) )( ˆβ (k) β (k) ) x 0 = σ ik (1 + x 0(X X) 1 x 0 ) Previous Next First Last Back Forward 26

: p Previous Next First Last Back Forward 1

: p Previous Next First Last Back Forward 1 7-2: : 7.2......... 1 7.2.1....... 1 7.2.2......... 13 7.2.3................ 18 7.2.4 0-1 p.. 19 7.2.5.... 21 Previous Next First Last Back Forward 1 7.2 :, (0-1 ). 7.2.1, X N(µ, σ 2 ), < µ 0;