,

zwp@ustc.edu.cn Office: 1006 Phone: 63600565 http://staff.ustc.edu.cn/~zwp/ http://fisher.stat.ustc.edu.cn

1.1................. 2 1.2,........... 9 1.3................. 13 1.4.................... 16 1.5.................. 18 1.6................. 21 1.7.......... 25 Previous Next First Last Back Forward 1

1.1,., x;, X.. n : x = [x 1,..., x n ] : x = x 2 1 + + x2 n x y : cos(θ) = x y x y = <x,y> x y x y = x y y y y = <x,y> y = x cos(θ) y = y y y y(y y) 1 y x = P y x, θ x y. P y = y(y y) 1 y. Previous Next First Last Back Forward 2

x 1,..., x p (linearly dependent): c 1,..., c p c 1 x 1 + + c p x p = 0 c 1,..., c p, x 1,..., x p (linearly independent). x 1,..., x p : c 1,..., c p c 1 x 1 + +c p x p = 0, c 1 = = c p = 0. x 1,..., x p, : i, x i = b 1 x 1 + + b i 1 x i 1 + b i+1 x i+1 + + b p x p b 1,, b p. Previous Next First Last Back Forward 3

R n H, (1) x H, y H x + y H, (2) cx H c. Definition {x = [x 1,..., x n ] R n x 1 + + x n = 0} R n x R n, x R n x 1,..., x k, k L(x 1,..., x k ) = { c i x i (c 1,..., c k ) R n } i=1 L(x 1,..., x k ), x 1,..., x k. Previous Next First Last Back Forward 4

n p A = [x 1,..., x p ] L(A). Rank(A) = dim(l(a)). H G, a H, b G, a b = 0, H G, H G., H + G = R n, G H, G = H. A, L(A) L(I A). A, L(A) = L(AA ), L(A ) = L(A A). ( N (A) = {x Ax = 0} A. N (A) A. dim(n (A)) + rank(a) = q(a ) N (A A) = N (A), N (AA ) = N (A ). Previous Next First Last Back Forward 5

.,, np n p n p ( ). n = p. A, A = A, A ; A = A, A. A, AA = A A = I; A, A 2 = A;. A = (a ij ) p, A = π ϵ πa 1j1 a 2j2 a pjp, (j 1,..., j p ) (1, 2,, p). π p!, ϵ π = 1-1. A A. Definition Previous Next First Last Back Forward 6

A = p j=1 a ija ij, A ij a ij. A p, aa = a p A. A B p q q p, I p +AB = I q +BA A, A = ±1 A p, A = 0, A ; A = 0, A. A p, B, AB = BA = I p, B A, B = A 1. Definition (A ) 1 = (A 1 ), (AC) 1 = C 1 A 1, A 1 = A 1 ( ) ( ). Previous Next First Last Back Forward 7

A B p q, C D p q q p. T = A + CBD, T 1 = A 1 A 1 C(B 1 + DA 1 C) 1 DA 1 A p q, A r, A r + 1, A r, Rank(A) = r. Definition Rank(A) = Rank(A ) = Rank(A A) = Rank(AA ) Rank(A) = 0, A = 0. 0 Rank(A) min(p, q). Rank(AB) min(rank(a), Rank(B)). Previous Next First Last Back Forward 8

Rank(A + B) Rank(A) + Rank(B). A C, Rank(ABC) = Rank(B). 1.2, p A, x 0, x R p, x Ax > 0. A > 0; p A, x R p, x Ax 0. A 0. Definition : ( ), ( ). Previous Next First Last Back Forward 9

A > 0, A 1 > 0. A > 0, B > 0, A B > 0, B 1 A 1 > 0, A > B. A > 0, A ( A11 A 12 A = A 21 A 22 ) A 11, A 11 > 0, A 22 > 0, A 11 2 = A 11 A 12 A 1 22 A 21 > 0, A 22 1 = A 22 A 21 A 1 11 A 12 > 0. A 0, Γ Γ AΓ = diag{λ 1, λ 2,..., λ p } λ i 0, i = 1,..., p A., ( ). : Previous Next First Last Back Forward 10

A, tr(a) = Rank(A). A, I A. A B, A + B = I, AB = BA = 0. X n p, n p, Rank(X) = p, P X = X(X X) 1 X, Rank(P X ) = p. 1. n P, x R n x P x = inf x u, u L(P ), L(P ), x P x x.. Previous Next First Last Back Forward 11

P 1 P 2, P 1 + P 2 P 1 P 2 = P 2 P 1 = 0. P 1 P 2 P 1 P 2 = P 2 P 1. P 1 P 2 P 1 P 2 = P 2 P 1 = P 2. A λ 1 λ 2 λ p, sup x Ax x x = sup x Ax = λ 1 x 0 x =1 inf x Ax x 0 x x = inf x Ax = λ p x =1 Previous Next First Last Back Forward 12

1.3 A p, Γ Λ = diag{λ 1,..., λ p }(λ 1 λ 2 λ p ), A = Γ ΛΓ Γ = [l 1, l 2,, l p ], p A = λ i l i l i i=1 A (spectral decomposition). A > 0( 0), A 1/2 > 0( 0), A = A 1/2 A 1/2. A 0 p r, r p r B, A = BB. Previous Next First Last Back Forward 13

p C, A = C ( Ir 0 0 0 ) C. T, A = T T. (Cholesky ), A > 0,. ( ) A n p, Rank(A) = r, n n U p p V A = UDV n p ( D11 0 r (p r) D = 0 0 ), D 11 = diag{λ 1,..., λ r }, λ 2 1... λ 2 r > 0 A A (λ i A.) V A A, U AA. Previous Next First Last Back Forward 14

(QR ) A A = QR, Q, R. A n p, r, n r P p r Q A = P Q. ( ) A 1,..., A k p, A i A j = 0, i j, i, j = 1, 2,..., k, H, H A ih = Λ i, Λ i, i = 1,..., k. A B p, A > 0, p H, A = H H, B = H ΛH Λ = diag{λ 1,..., λ p } A 1 B. Previous Next First Last Back Forward 15

1.4 A n p, : ( A11 A 12 A = A 21 A 22 ) A 11 k l. A. 1 A, A 11, A 11 0, A = A 11 A 22 1, A 22 1 A 21 A 1 11 A 12. A 22 0, A = A 11 2 A 22, A 11 2 A 12 A 1 22 A 21. = A 22 = A 11 2 A, A 11 A 22, Previous Next First Last Back Forward 16

A 11 = 0, A 1 = A 22 = 0, A 1 = ( A 1 11 + A 1 11 A 12A 1 22 1 A 21A 1 11 A 1 11 A 12A 1 22 1 A 1 22 1 A 21A 1 11 A 1 22 1 ( A 1 11 2 A 1 11 2 A 12A 1 22 A 1 22 A 21A 1 11 2 A 1 22 + A 1 22 A 21A 1 11 2 A 12A 1 22 A 11 = 0, A 22 0, A 1 = ( A 1 11 2 A 1 11 A 12A 1 22 1 A 1 22 A 21A 1 11 2 A 1 22 1 ) ) ) Previous Next First Last Back Forward 17

1.5 A n p, p n X, AXA = A, X A, X = A. Definition A, ( ). A, A, A = A 1. Rank(A ) Rank(A), Rank(A) = Rank(AA ) = tr(aa ). A, A A(A A) A = A, A(A A) A A = A. A(A A) A, (A A) Previous Next First Last Back Forward 18

A n p, p n X, AXA = A, XAX = X, (AX) = AX, (XA) = XA Definition X A Moore-Penrose, X = A +. A,. (A ) + = (A + ). Rank(A) = n, A + = A (A A) 1 ; Rank(A) = p, A + = (A A) 1 A ; Rank(A) = n = p, A + = A 1. (A + ) + = A Previous Next First Last Back Forward 19

(A A) + = A + (A + ) A + = (A A) + A = A (AA ) +. A, A + = A. AA + A + A. A = P Q, Rank(A) = Rank(P ) = Rank(Q) = r, A + = (Q ) + P +. A = A, { A = H ΛH, H. Λ = diag{λ 1,..., λ p }, λ 1 λ +, λ 0 = A + = H diag{λ + 1 0, λ = 0,..., λ+ p }H. Previous Next First Last Back Forward 20

1.6,.. A = (a ij ) = (a 1,..., a p ) n p, a j j, (j = 1,..., p). vec(a) = a 1. A Definition, vec(a) A. a p c, d, A B, vec(c A + d B) = c vec(a) + d vec(b). tr(ab) = ij a ijb ji = (vec(a )) (vec(b)). Previous Next First Last Back Forward 21

A = ij a ije ij = ij a ije i e j = j a je j = i e ia (i), e i i, E ij = e i e j, a (i) A i. a j = Ae j, a (i) = A e i. a ij = e iae j, tr(e ija) = a ij A = (a ij ) B n p m q, A B Kronecker A B A B = (a ij B) = a 11 B a 1p B.. a n1 B a np B Definition a, (aa) B = A (ab) = a(a B) Previous Next First Last Back Forward 22

(A B) = A B, (A B)(C D) = (AC) (BD), (A B) 1 = A 1 B 1. A, B, tr(a B) = (tr(a))(tr(b)) x y, x y = x y = y x. A n n, {λ i, i = 1,..., n}, {x i, i = 1,..., n}. B m m, {µ i, i = 1,..., m}, {y i, i = 1,..., m}. A B {λ i µ j, i = 1,..., n; j = 1,..., m}, {x i y j, i = 1,..., n; j = 1,..., m}. A n, B m, A B = A m B n. Rank(A B) = Rank(A)Rank(B) = Rank(B A) Previous Next First Last Back Forward 23

: vec(axb) = (B A)vec(X) vec(ab) = (I A)vec(B) tr(abc) = vec(a ) (I B)vec(C) vec(aa ) = a a tr(a BCD ) = vec(a) (D B)vec(C) Previous Next First Last Back Forward 24

1.7,,. Y = (y ij (t)) p q, t, Y t = ( ) yij (t) t Definition Y t. X+Y t = X t + Y t XY t = X t Y + X Y t Previous Next First Last Back Forward 25

X Y t = X t ( ) X t = X t X x ij = E ij Y + X Y t AXB x ij = AE ij B X 1 t 1 X = X t X 1 y = f(x) p q X, y X ( ) Definition y y X = x ij Previous Next First Last Back Forward 26

( ) y X = y X X p, tr(x) X tr(axb) X = A B tr(ax) X = = I p { A, X X A + A diag(a 11,..., a pp ), X = X x Ax x. = (A + A )x, x ; tr(x AX) X = (A + A )X, X a x x det(x) X = a, a, x. = (X ) 1 det(x) X, tr(x 1 A) X = (X 1 AX 1 ) Previous Next First Last Back Forward 27

x n, m y = (y 1,..., y m ) = (f 1 (x),..., f m (x)), y x y x = ( ) yj x i Definition y = Ax, y / x = A. Y = AXB, (vec(y )) vec(x) = B A. Previous Next First Last Back Forward 28

X m n, F (X) p q, F (X) = (f ij (X)) = (f ij ), F (X) X [vec(f (X))] = vec(x) Definition F (X) X. X m n, X X = I mn F (X) p q, G(X) q r, X m n, F (X)G(X) X = F (X) X (G(X) I p) + G(X) X (I r F (X)) Previous Next First Last Back Forward 29

AXB X = B A X 1 X = [X 1 (X 1 ) ] 2 (Chain rule). ψ(x) X, X x ij t, ψ(x) t [ ( ) ] X ψ = tr t X 3 (Chain rule). F (G(X)), det(x(t)) t lndet(x) t F (G(X)) = G(X) X X = det(x)tr ( ) X = tr ( X ) 1 X t 1 X t F (G) G Previous Next First Last Back Forward 30

g(x 1,..., x n )dx 1 dx n = D T g(f 1 (y)) J(x y) dy D R n, T = {y y = f(x), x D}, f ; J(x y) = x y +, A + A. X R m n, Y = F (X) R m n, F (X), Y = F (X) J(Y X) = J(Y : X) = Y X + Definition (1) Y = AXB, A m, B n Previous Next First Last Back Forward 31

, J(Y X) = (det(a)) n (det(b)) m = A n + B m + X n, Y = X 1, J(Y X) = X 2n. (2) G n, X n, Y = GX, J(Y X) = n i=1 g 11, g 22,..., g nn G. G n, X n, Y = XG, J(Y X) = n i=1 g i ii g n i+1 ii Previous Next First Last Back Forward 32

X,, Y = XX, J(Y X) = 2 n n i=1 x n i+1 ii : X, Y = XX. (3) X = X, G ( ), Y = G XG, J(Y X) = G n+1 + X = X, P, Y = P XP, J(Y X) = P n+1 + Previous Next First Last Back Forward 33

: A B n p, L(A) L(B). P A = A(A A) A P B = B(B B) 1 B R n L(A) L(B), P A P B = P B P A = P A. Previous Next First Last Back Forward 34