untitled

Size: px

Start display at page:

Download "untitled"

燥璩
5 years ago
Views:

1 Statistics & Data Analysis 2 (Part I) Zhu University

3 2.1 Example49613,983,816 13,983,816/52=268,919

4 Example n p(n) 12% 41% 70% 97% % % 1 ( ) 1 ( ) 100% Example

5 Example

6 E (Random experiment E ) E (Experiment E ) EE 1, E ω generic element ω Ω sample space Ω

7 Example 2.1E 1 Example 2.2E 2 Example 2.3E 3 DNA Example 2.4E 4 Example 2.5E 5 5 HBsAg() -HBs()еHBeAg( )е-hbe( )-HBc() 3 Example 2.6E 6 50% A Random event A A Event A Ω A, B, C, A1, A2, Ω Ω Example 2.1E 1 3

8 Example 2.2E 2 (H)(T) 3H Example 2.3E 3 DNA 3 Example 2.5E 5 5 HBsAg() -HBs()еHBeAg( )е-hbe( )-HBc() 3 3 ΩA, B, C, A k (k=1, 2, ) Ω 1 Ω 2

9 3 Ω 4 Ω 5 Ω 6 Ω

10 7 Ω 1AB=BAAB=BA 2(AB)C=A(BC)(AB)C=A(BC) 3A(BC)=(AB)(AC)A(BC)=ABAC 4(De Morgan) A B AB AB AB 5A 6A A A 7AA 8A B AB A( AB)

11 Example 2.7(a)(b)Ai ={i} AiA={LR} 1 (a) L 2 3 R 4 3 (b) L 1 4 R 2 5 (Complete events) ΩnA i (i=1, 2,, n) 1A i A j ( i j, i, j = 1, 2,, n) 2A 1 A 2 A n = Ω na i (i=1, 2,, n)ω, partition A 2 A 1 A 3 An 1 An

2.1.2 Kolmogorov19031987 σ σ σ-field or σ-algebra

12 2.1.2 Kolmogorov σ σ σ-field or σ-algebra ΩΩF Ω Ω σ: F C A F A F ( n N, A F ) A F n n 1 σ σ σω n

13 KolmogorovProbability Axioms Ω F Ω σf Ω ΩPF Ω PF Ω Ω AF Ω P(A)[0, 1] P(Ω)=1. A 1, A 2,... P A1 A2 P A i (...) ( ) ΩEF Ω σ A F Ω P(A)R + R + P(A) P(A)[0, 1] P(Ω)=1 A 1, A 2,... P ( A ) P ( A ) i i 1 i 1 P(A)A {ΩF Ω P(A)} {ΩF Ω Φ} i

14 1 P()=0 2 A 1,A 2,... A n n n P ( A ) P ( A ) i i1 i1 i 3 A P( A) 1 P( A) 4 ABP(A-B)=P(A)-P(B) 5 A, BP(AB)=P(A)+P(B)-P(AB) John Venn ( )Richard Von Mises ( ) n A naa n A /naf n (A) 10 f n (A) 1 2f n (Ω)1 3AB= f n (A+B)= f n (A)+ f n (B)

15 f n (A) n A f n (A) n n H f n (H) De Morgan Buffon K. Pearson K. Pearson n A f n (A)

16 f n (A) n EnAf n (A) p[0, 1]ppA P(A) nf n (A)p P(A) 10 P(A) 1 2P(Ω)1 3AB= P(A+B)=P(A)+P(B) 3k

18 Laplace Laplace E Laplace 1ΩΩ{ω 1, ω 2,, ω n } 2 P(ω 1 )=P(ω 2 )= =P(ω n )=1/n LaplaceEΩn AmA P(A) m PA ( ) n

19 P(A) 10 P(A) 1 2P(Ω)1 3ABAB=ΦP(A+B)=P(A)+P(B) 3k Example 2.8Simpson s paradox A B C

20 1 s (s 1)i r i i=1, 2,, s r 1 +r 2 + +r s 2 s (s 1)ir i i=1, 2,, s r 1 r 2 r s 3 n m(m n) m n! An n( n1)( n2)...( nm1) ( n m)! m=n n m(m n)n m n m(m n) m nn ( 1)( n2)...( nm1) n! Rn m m( n m)!

21 4 nm (m n) C m n C m n nn ( 1)( n2)...( nm1) n! mm ( 1)...1 m!( nm)! n m C n C C C m m1 m n n n1 C C... C 2 C 0 1 n n n n m m k m k n1 n C 2 nc 1 n2 k 0 n Example

22 Example n 2n Example 2.11 Capture/recapture method n

23 1Ω 2 Ω A PA ( ) D( A) D( ) A D(Ω)Ω D(A)A Ω ΩAΩA A

24 P(A) 10 P(A) 1 2P(Ω)1 3ABAB=ΦP(A+B)=P(A)+P(B) 3k Buffon's needle: D LL<D p=2l/(πd)π 1901 Mario Lazzarini 3408π 355/113 6

25 Example 2.12 [0, T]t (t T) y T t 0 t T x ΩA

26 Example 2.13Bertrand paradox 1

27 2.1.3 Example 2.14Digitalis Digitalis DigitalisTBeller DigitalisTDigitalis D D Total T 25 (0.185) 14 (0.104) 39 (0.289) T 18 (0.133) 78 (0.578) 96 (0.711) Total 43 (0.318) 92 (0.682) 135 D D T T Beller, G. et al. (1971). Digitalis intoxication: A prospective clinic study with serum level correlations. New England J. Med., 284:

28 Conditional probability EΩA, BP(A)>0 P ( AB ) P ( B A) P ( A) AB Ω AB A AB B P(A B) PB ( A) 1 2P(Ω A)1, P( A)=0 3A1, A2,, P ( Bi A) P( Bi A) A

29 P( AA B) P( A B) P( A B) P( AA B) P ( B A) 1 P( B A) Multiplication law P( AB) P( A) P( B A) ( P( A) 0) P( AB) P( B) P( A B) ( P( B) 0) P(ABC)=P(A)P(B A)P(C AB) P(A 1 A 2 A n )=P(A 1 )P(A 2 A 1 ) P(A n A 1 A 2 A n-1 )

30 Example Example Example 2.17Poloya br c nn 1 n 2 =nn 1 Poloya Poloyann 2?

31 Total probability EΩA 1,A 2,,A n Ω P(A i )>0 (i=1, 2,, n)b n P ( B ) P ( Ai) P ( B Ai) i 1 (Complete events) ΩnA i (i=1, 2,, n) 1A i A j ( i j, i, j = 1, 2,, n) 2A 1 A 2 A n = Ω na i (i=1, 2,, n)ω, partition A 2 A 1 A 3 An 1 An

32 BΩn A 1, A 2,, A n P(B A i ) A 2 A 1 B A 3 An 1 An Example 2.18 UML U 1 UU 2 UGlassHall1954EnglandWales Transition probability matrix 10%U40%M50%L UML Transition probability matrix U2 M2 L2 U M L U: upper level M: middle level L: lower level Glass, D., & Hall, J. (1954). A study of intergeneration changes in status. In Social Mobility in Britain.

33 Bayesian Example 2.18 UU Transition probability matrix U2 M2 L2 U M L U: upper level M: middle level L: lower level Glass, D., & Hall, J. (1954). A study of intergeneration changes in status. In Social Mobility in Britain. Bayesian EΩA 1,A 2,,A n Ω P(A i )>0 (i=1, 2,, n)bp(b)>0 P ( A B ) i n j 1 P ( A ) P ( B A ) i P ( A ) P ( B A ) j i j A 2 B A 1 A 3 An 1 An

34 B A 1, A 2,, A n BBayes B P ( A B ) i n j 1 P ( A ) P ( B A ) i P ( A ) P ( B A ) j i j P(A i ) {A i } (prior probability) P(B) B (prior probability) P(A i B) (posterior probability) Bayes A 2 A 3 B An 1 A 1 An

35 Structure of Bayesian network. Labeled ovals represent nodes; arrows (arcs) represent conditional dependence relationships. Each node is a data structure that contains conditional probability tables to quantify probabilistic relationships between variables. Cacalcifications, FCfibrocystic change, FHxfamily history of breast cancer, HRThormone replacement therapy, LNlymph node, P/A/O present, absent, or obscured. Elizabeth et al., (2006) Bayesian Network to Predict Breast Cancer Risk of Mammographic Micro-calcifications and Reduce Number of Benign Biopsy Results: Initial Experience. Radiology, 240, Example 2.19 X 0123 T 0,T 1,T 2,T 3 PD ( T) i PD ( ) PT ( i D) PD ( ) PT ( D) PD ( ) PT ( D) i i i P(T i D+) P(T i D-) P(D+) P(D+) 0.92 Diamond, G. & Forrester, J. (1979). Analysis of probability as an aid in the clinical diagnosis of coronary-artery disease. New Eng. J. Med., 300:

36 Example 2.20 Lie detector Gastwirth1987P(+ L)=0.88 P( T)=0.86LT + P(T)=0.99P(L)=0.01 Gastwirth, J. (1987). The statistical precision of medical screening procedures. Statistical Science, 3: Example

37 Example A={}B={ }P(A)P(B)P(AB)P(B A) 1. ABP(AB)=P(A)P(B)AB A, BP(A)>0A, B P(B A)=P(B) P(B)>0A, BP(A B)=P(A)

38 1AB 2A B 3 A B A B 4 2. A, B, C P(AB)=P(A)P(B); P(BC)=P(B)P(C); P(AC)=P(A)P(C); P(ABC)=P(A)P(B)P(C) A, B, C

39 n na 1, A 2,, A n k (1<kn) ki 1, i 2,, i k (1i 1 <i 2 < < i k n) na 1, A 2,, A n 1A 1, A 2,, A n k(1<kn) 2k=2 3nn 4 i i... i i i... i P A A A P A P A P A 1 2 k 1 2 n k 2 C k n n 2 n 1 k Example 2.7(a)(b)Ai={i} p A={LR} 1 (a) L 2 3 R 4 3 (b) L 1 4 R 2 5

untitled

untitled Statistics & Data Analysis 2 (Part I) Zhu Huaiqiu i @Peking University 2.1 21 Example49613,983,816l 496 13 983 816 13983816/52=268919 268919 Example232111 23 n 10 20 30 50 100 200 300 350 366 p(n) 12%