() 11-1 ()Classification Analysis( ) m() p.d.f prior (decision) (loss function) Bayes Risk for any decision d( ) posterior risk posterior risk Posterior prob. j (uniform prior) where Mahalanobis Distance(M-distance)
() 11-2 ( ) (score) Quadratic Discriminant Score Liner Discriminant score= score (1) (2) posterior prob. M-distance quadratic discriminant score posterior prob. M-distance linear discriminant score (m = 2 )
() 11-3 (1)/* */ DATA IRIS; TITLE ''; INPUT SEPALLEN SEPALWID PETALLEN PETALWID SPEC_NO @@; IF SPEC_NO=1 then SPECIES='SETOSA'; IF SPEC_NO=2 then SPECIES='VERSICOLOR'; IF SPEC_NO=3 then SPECIES='VIRGINICA'; DROP SPEC_NO; /*LABEL SEPALLEN=SEPAL LENGTH IN MM. SEPALWID=SEPAL WIDTH IN MM. PETALLEN=PETAL LENGTH IN MM. PETALWID=PETAL WIDTH IN MM.;*/ cards; 50 33 14 02 1 64 28 56 22 3 65 28 46 15 2 67 31 56 24 3 63 28 51 15 3 46 34 14 03 1 69 31 51 23 3 62 22 45 15 2 59 32 48 18 2 46 36 10 02 1 61 30 46 14 2 60 27 51 16 2 65 30 52 20 3 56 25 39 11 2 65 30 55 18 3 58 27 51 19 3 68 32 59 23 3 51 33 17 05 1 57 28 45 13 2 62 34 54 23 3 77 38 67 22 3 63 33 47 16 2 67 33 57 25 3 76 30 66 21 3 49 25 45 17 3 55 35 13 02 1 67 30 52 23 3 70 32 47 14 2 64 32 45 15 2 61 28 40 13 2 48 31 16 02 1 59 30 51 18 3 55 24 38 11 2 63 25 50 19 3 64 32 53 23 3 52 34 14 02 1 49 36 14 01 1 54 30 45 15 2 79 38 64 20 3 44 32 13 02 1 67 33 57 21 3 50 35 16 06 1 58 26 40 12 2 44 30 13 02 1 77 28 67 20 3 63 27 49 18 3 47 32 16 02 1 55 26 44 12 2 50 23 33 10 2 72 32 60 18 3 48 30 14 03 1 51 38 16 02 1 61 30 49 18 3 48 34 19 02 1 50 30 16 02 1 50 32 12 02 1 61 26 56 14 3 64 28 56 21 3 43 30 11 01 1 58 40 12 02 1 51 38 19 04 1 67 31 44 14 2 62 28 48 18 3 49 30 14 02 1 51 35 14 02 1 56 30 45 15 2 58 27 41 10 2 50 34 16 04 1 46 32 14 02 1 60 29 45 15 2 57 26 35 10 2 57 44 15 04 1 50 36 14 02 1 77 30 61 23 3 63 34 56 24 3 58 27 51 19 3 57 29 42 13 2 72 30 58 16 3 54 34 15 04 1 52 41 15 01 1 71 30 59 21 3 64 31 55 18 3 60 30 48 18 3 63 29 56 18 3 49 24 33 10 2 56 27 42 13 2 57 30 42 12 2 55 42 14 02 1 49 31 15 02 1 77 26 69 23 3 60 22 50 15 3 54 39 17 04 1 66 29 46 13 2 52 27 39 14 2 60 34 45 16 2 50 34 15 02 1 44 29 14 02 1 50 20 35 10 2 55 24 37 10 2 58 27 39 12 2 47 32 13 02 1 46 31 15 02 1 69 32 57 23 3 62 29 43 13 2 74 28 61 19 3 59 30 42 15 2 51 34 15 02 1 50 35 13 03 1 56 28 49 20 3 60 22 40 10 2 73 29 63 18 3 67 25 58 18 3 49 31 15 01 1 67 31 47 15 2 63 23 44 13 2 54 37 15 02 1 56 30 41 13 2 63 25 49 15 2 61 28 47 12 2 64 29 43 13 2 51 25 30 11 2 57 28 41 13 2 65 30 58 22 3 69 31 54 21 3 54 39 13 04 1 51 35 14 03 1 72 36 61 25 3 65 32 51 20 3 61 29 47 14 2 56 29 36 13 2 69 31 49 15 2 64 27 53 19 3 68 30 55 21 3 55 25 40 13 2 48 34 16 02 1 48 30 14 01 1 45 23 13 03 1 57 25 50 20 3 57 38 17 03 1 51 38 15 03 1 55 23 40 13 2 66 30 44 14 2 68 28 48 14 2 54 34 17 02 1 51 37 15 04 1 52 35 15 02 1 58 28 51 24 3 67 30 50 17 2 63 33 60 25 3 53 37 15 02 1 proc discrim simple wcov wcorr pcov pcorr listerr pool=test; class species; run; 150 Observations 149 DF Total 4 Variables 147 DF Within Classes 3 Classes 2 DF Between Classes Class Level Information Prior SPECIES Frequency Weight Proportion Probability SETOSA 50 50.0000 0.333333 0.333333 VERSIC 50 50.0000 0.333333 0.333333 VIRGIN 50 50.0000 0.333333 0.333333
() 11-4 Within Class Covariance Matrices SPECIES = SETOSA DF = 49 SEPALLEN 12.42489796 9.92163265 1.63551020 1.03306122 SEPALWID 9.92163265 14.36897959 1.16979592 0.92979592 PETALLEN 1.63551020 1.16979592 3.01591837 0.60693878 PETALWID 1.03306122 0.92979592 0.60693878 1.11061224 SPECIES = VERSIC DF = 49 SEPALLEN 26.64326531 8.51836735 18.28979592 5.57795918 SEPALWID 8.51836735 9.84693878 8.26530612 4.12040816 PETALLEN 18.28979592 8.26530612 22.08163265 7.31020408 PETALWID 5.57795918 4.12040816 7.31020408 3.91061224 SPECIES = VIRGIN DF = 49 SEPALLEN 40.43428571 9.37632653 30.32897959 4.90938776 SEPALWID 9.37632653 10.40040816 7.13795918 4.76285714 PETALLEN 30.32897959 7.13795918 30.45877551 4.88244898 PETALWID 4.90938776 4.76285714 4.88244898 7.54326531 Pooled Within Class Covariance Matrix DF = 147 SEPALLEN 26.50081633 9.27210884 16.75142857 3.84013605 SEPALWID 9.27210884 11.53877551 5.52435374 3.27102041 PETALLEN 16.75142857 5.52435374 18.51877551 4.26653061 PETALWID 3.84013605 3.27102041 4.26653061 4.18816327 Within Class Correlation Coefficients / Prob > R SPECIES = SETOSA SEPALLEN 1.00000 0.74255 0.26718 0.27810 0.0 0.0001 0.0607 0.0505 SEPALWID 0.74255 1.00000 0.17770 0.23275 0.0001 0.0 0.2170 0.1038 PETALLEN 0.26718 0.17770 1.00000 0.33163
() 11-5 0.0607 0.2170 0.0 0.0186 PETALWID 0.27810 0.23275 0.33163 1.00000 0.0505 0.1038 0.0186 0.0 SPECIES = VERSIC SEPALLEN 1.00000 0.52591 0.75405 0.54646 0.0 0.0001 0.0001 0.0001 SEPALWID 0.52591 1.00000 0.56052 0.66400 0.0001 0.0 0.0001 0.0001 PETALLEN 0.75405 0.56052 1.00000 0.78667 0.0001 0.0001 0.0 0.0001 PETALWID 0.54646 0.66400 0.78667 1.00000 0.0001 0.0001 0.0001 0.0 SPECIES = VIRGIN SEPALLEN 1.00000 0.45723 0.86422 0.28111 0.0 0.0008 0.0001 0.0480 SEPALWID 0.45723 1.00000 0.40104 0.53773 0.0008 0.0 0.0039 0.0001 PETALLEN 0.86422 0.40104 1.00000 0.32211 0.0001 0.0039 0.0 0.0225 PETALWID 0.28111 0.53773 0.32211 1.00000 0.0480 0.0001 0.0225 0.0 Pooled Within Class Correlation Coefficients / Prob > R SEPALLEN 1.00000 0.53024 0.75616 0.36451 0.0 0.0001 0.0001 0.0001 SEPALWID 0.53024 1.00000 0.37792 0.47053 0.0001 0.0 0.0001 0.0001 PETALLEN 0.75616 0.37792 1.00000 0.48446 0.0001 0.0001 0.0 0.0001 PETALWID 0.36451 0.47053 0.48446 1.00000 0.0001 0.0001 0.0001 0.0 Simple Statistics Total Sample Variable N Sum Mean Variance Std Dev SEPALLEN 150 8765 58.43333 68.56935 8.28066 SEPALWID 150 4586 30.57333 18.99794 4.35866 PETALLEN 150 5637 37.58000 311.62779 17.65298
() 11-6 PETALWID 150 1799 11.99333 58.10063 7.62238 SPECIES = SETOSA Variable N Sum Mean Variance Std Dev SEPALLEN 50 2503 50.06000 12.42490 3.52490 SEPALWID 50 1714 34.28000 14.36898 3.79064 PETALLEN 50 731 14.62000 3.01592 1.73664 PETALWID 50 123 2.46000 1.11061 1.05386 SPECIES = VERSIC Variable N Sum Mean Variance Std Dev SEPALLEN 50 2968 59.36000 26.64327 5.16171 SEPALWID 50 1385 27.70000 9.84694 3.13798 PETALLEN 50 2130 42.60000 22.08163 4.69911 PETALWID 50 663 13.26000 3.91061 1.97753 SPECIES = VIRGIN Variable N Sum Mean Variance Std Dev SEPALLEN 50 3294 65.88000 40.43429 6.35880 SEPALWID 50 1487 29.74000 10.40041 3.22497 PETALLEN 50 2776 55.52000 30.45878 5.51895 PETALWID 50 1013 20.26000 7.54327 2.74650 Within Covariance Matrix Information Covariance Natural Log of the Determinant SPECIES Matrix Rank of the Covariance Matrix SETOSA 4 5.35332 VERSIC 4 7.54636 VIRGIN 4 9.49362 Pooled 4 8.46214 Test of Homogeneity of Within Covariance Matrices Notation: K = Number of Groups
() 11-7 P N = Number of Variables = Total Number of Observations Number of Groups N(i) = Number of Observations in the i'th Group 1 Within SS Matrix(i) V = Pooled SS Matrix N(i)/2 N/2 2 1 1 2P + 3P 1 RHO = 1.0 SUM _ N(i) N _ 6(P+1)(K 1) DF =.5(K 1)P(P+1) PN/2 N V Under null hypothesis: 2 RHO ln PN(i)/2 _ N(i) _ is distributed approximately as chi square(df) Test Chi Square Value = 140.943050 with 20 DF Prob > Chi Sq = 0.0001 Since the chi square value is significant at the the within covariance matrices will be used in the discriminant function. 0.1 level, Reference: Morrison, D.F. (1976) Multivariate Statistical Methods p252. Pairwise Generalized Squared Distances Between Groups 2 1 D (i j) = (X X )' COV (X X ) + ln COV i j j i j j Generalized Squared Distance to SPECIES From SPECIES SETOSA VERSIC VIRGIN SETOSA 5.35332 110.74017 178.26121 VERSIC 328.41535 7.54636 23.33238 VIRGIN 711.43826 25.41306 9.49362
() 11-8 Classification Results for Calibration Data: WORK.IRIS Resubstitution Results using Quadratic Discriminant Function Generalized Squared Distance Function: 2 _ 1 _ D (X) = (X X )' COV (X X ) + ln COV j j j j j Posterior Probability of Membership in each SPECIES: 2 2 Pr(j X) = exp(.5 D (X)) / SUM exp(.5 D (X)) j k k Posterior Probability of Membership in SPECIES: Obs From Classified SPECIES into SPECIES SETOSA VERSIC VIRGIN 5 VIRGIN VERSIC * 0.0000 0.6050 0.3950 9 VERSIC VIRGIN * 0.0000 0.3359 0.6641 12 VERSIC VIRGIN * 0.0000 0.1543 0.8457 * Misclassified observation Classification Summary for Calibration Data: WORK.IRIS Resubstitution Summary using Quadratic Discriminant Function Generalized Squared Distance Function: 2 _ 1 _ D (X) = (X X )' COV (X X ) + ln COV j j j j j Posterior Probability of Membership in each SPECIES: 2 2 Pr(j X) = exp(.5 D (X)) / SUM exp(.5 D (X)) j k k Number of Observations and Percent Classified into SPECIES: From SPECIES SETOSA VERSIC VIRGIN Total SETOSA 50 0 0 50 100.00 0.00 0.00 100.00 VERSIC 0 48 2 50 0.00 96.00 4.00 100.00 VIRGIN 0 1 49 50
() 11-9 0.00 2.00 98.00 100.00 Total 50 49 51 150 Percent 33.33 32.67 34.00 100.00 Priors 0.3333 0.3333 0.3333 Error Count Estimates for SPECIES: SETOSA VERSIC VIRGIN Total Rate 0.0000 0.0400 0.0200 0.0200 Priors 0.3333 0.3333 0.3333 (2) /**/ DATA CROPS; TITLE 'REMOTE SENSING DATA ON FIVE CROPS'; INPUT CROP $ 1-10 X1-X4 XVALUES $ 12-22; CARDS; CORN 16 27 31 33 CORN 15 23 30 30 CORN 16 27 27 26 CORN 18 20 25 23 CORN 15 15 31 32 CORN 15 32 32 15 CORN 12 15 16 73 SOYBEANS 20 23 23 25 SOYBEANS 24 24 25 32 SOYBEANS 21 25 23 24 SOYBEANS 27 45 24 12 SOYBEANS 12 13 15 42 SOYBEANS 22 32 31 43 COTTON 31 32 33 34 COTTON 29 24 26 28 COTTON 34 32 28 45 COTTON 26 25 23 24 COTTON 53 48 75 26 COTTON 34 35 25 78 SUGARBEETS 22 23 25 42 SUGARBEETS 25 25 24 26 SUGARBEETS 34 25 16 52 SUGARBEETS 54 23 21 54 SUGARBEETS 25 43 32 15 SUGARBEETS 26 54 2 54 CLOVER 12 45 32 54 CLOVER 24 58 25 34 CLOVER 87 54 61 21 CLOVER 51 31 31 16 CLOVER 96 48 54 62 CLOVER 31 31 11 11 CLOVER 56 13 13 71 CLOVER 32 13 27 32 CLOVER 36 26 54 32 CLOVER 53 8 6 54 CLOVER 32 32 62 16 ; PROC DISCRIM DATA=CROPS POOL=YES LIST OUT=CROPCAL; CLASS CROP; ID XVALUES; VAR X1-X4; TITLE2 'CLASSIFICATION OF CROP DATA'; DATA TEST; INPUT CROP $ 1-10 X1-X4 XVALUES $ 12-22; CARDS;. 16 27 31 33. 21 25 23 24
() 11-10. 29 24 26 28. 54 23 21 54. 32 32 62 16 ; PROC DISCRIM DATA=CROPCAL TESTDATA=TEST TESTLIST; CLASS CROP; TESTCLASS CROP; TESTID XVALUES; VAR X1-X4; TITLE2 'CLASSIFICATION OF TEST DATA'; RUN;
() 11-11
() 11-12
() 11-13