untitled

Size: px

Start display at page:

Download "untitled"

秦珍田
5 years ago
Views:

1 Statistics & Data Analysis 4 Zhu Huaiqiu University A picture says more than a thousand words!

2 Descriptive statistics, or Statistical description In a clear and understandable way What does the Descriptive Statistics do? How to present your arguments or viewpoints? Open-air preaching in James Street, Covent Garden, London. The preacher is using an unusual style, he has his main points written on laminated plastic cards and is sticking them on a board.

3 What does the Descriptive Statistics do? Describe the basic features of the data gathered from an experimental study in various ways; Form the basis of quantitative analysis of data together with simple graphics analysis; Provide simple summaries about the sample and the measures; Proceed to inferential statistics if there are enough data to draw a conclusion It is necessary to be familiar with primary methods of describing data in order to understand phenomena and make intelligent decisions. (1) Graphical ldisplays of the data in which graphs summarize the data or facilitate comparisons. (2) Tabular description in which tables of numbers summarize the data. (3) Summary statistics (single numbers) which summarize y ( g ) the data.

4 (1) Collect data (2) Classify data (3) Summarize data (4) Present data (5) Proceed dto inferential statistics ti ti ifthere are enough data to draw a conclusion (Summary Statistics) 1.

5 2. 3. order statistics X 1, X 2,, X n X x 1, x 2,, x n x (1) x (2) x x (n) X X 1, X 2,, X n X (1), X (2),, X (n)

6 X F(x) f(x) X X (1), X (2),, X (n) 1 ( X (1), X (2),, X (n) ) n! f ( x ) f ( x )... f ( x ), x x... x 1 2 n X (k), k=1, 2,, n n! 1 f ( x)[ F( x)] [1 F( x)] ( k 1)!( n k)! 3 ( X (k), X (l) ), k<l k n k n! k 1 f( xk) f( xl) [ F( xk)] ( k 1)!( l k 1)!( n l)! l k 1 n l [ F( xl) F( xk)] [1 F( xl)], xk xl n 4. range X 1, X 2,, X n X X (1), X (2),, X (n) R n =max{x 1, X 2,, X n } min{x 1, X 2,, X n } =X (n) X X (1) X 1, X 2,, X n

7 R n ER ( ) n DR ( ) n c n d 2 2 n R n σ 5. Median 50% X X 1,X 2,..., X n x 1,x 2,..., x n x (*) 1 x 2... x n n 2k+1 x k 1 n=2k ( x k x k 1)/

8 6. Mode The value that occurs the most frequently in a data set or a probability distribution X X 1, X 2,..., X n x 1, x 2,..., x n x M 1 n x M 2 3 Example p 4.1 Plant Growth (# of leaves/plant)

9 7. X X 1, X 2,..., X n 1 n X X i n i 1 1 S X X C v S X n 2 2 ( i ) n 1 i 1 X coefficient of variation Example X 25.8 s= s= X

10 8. Chebyshev s theorem x 1, x 2,..., x N μ σ > 0 k>1 [μ kσ, μ+kσ] k=2: 75.0% k=3: 88.9% k=4: 93.8% 1 1 k 2

11 exploratory analysis outlier

12 4.3.1 Histogram Histogram Histogram: histos anything set upright gramma drawing, record, writing graphical display of tabulated frequencies, shown as bars, it shows what proportion of cases fall into each of several categories. 1. occurrence histogram 1 2 / 5 1 6~15 2 5

13 K Moore Moore, K C n 5 C =1~3 Sturges Sturges, 1928 k k k=1, 2,..., K K lg n n 1, n 2,..., n K, n k / k 2. frequency f n n, k 1,2,..., K k K k fk 1 k 1 frequency q ydensity fk nk p k, k 1,2,..., K n K k 1 p k k 1 k k

14 Example 4.3 Consider data collected by the U.S. Census Bureau on time to travel to work (2000 census). ) The census found that there were 124 million people who work outside of their homes. This rounding is a common phenomenon when collecting data from people. Interval Width Quantity Quantity/width Data by absolute numbers Histogram of travel time, US 2000 census. Area under the curve equals the total number of cases. This diagram uses Q/width from the table. Data by yproportionp Interval Width Quantity (Q) Q/total/width Histogram of travel time, US 2000 census. Area under the curve equals 1. This diagram uses Q/total/width from the table

15 1 2 3

16 Bar Bar chart 3. Pareto Pareto chart V. Pareto A type of chart that contains both bars and a line graph, where individual values are represented in descending order by bars, and the cumulative total is represented by the line. Pareto

17 A Pareto chart showing the relative frequency of reasons for arriving late at work Stem-leaf plot x i {x i }

18 1. x x i Stem Leaf 2. Stem 3. Stem Leaf 4. Stem Leaf Example / psi

19 Cumulative Freq. Stem Leaf (21) /4 3/4

21 4.3.3 Boxplot Boxplot Boxplot Box-whisker Plot John Tukey 1/4 3/4 3/4 1 2 Q1 Q3 3 Q3 1.5IQR Q1 1.5IQR Q3 3IQR Q1 3IQR mild outliers (extreme outliers) 4 5

22 IQR: Interquartile Range Extreme outliers Mild outliers Mild outliers Extreme outliers Mild outliers Extreme outliers

23 John Wilder Tukey Let the data speak for themselves! J. W. Tukey Example 4.5

24 Example Bayonne Clambers 1983 The boxplots... show many properties p of thedatarather strikingly. gy Thereisageneral reduction in sulphur dioxide concentration through time due to the gradual conversion to low sulphur fuels in the region. The decline is most dramatic for the highest quantiles. Also, there are higher concentrations during the winter months due to the use of heating oil. In addition, the boxplots show that the distributions areskewedtowardhighvaluesandthatthespreadofthedistributions...islargerwhenthe general level of concentration is higher.

25 Example 4.7 The total number of tornadoes recorded by State of USA (including the District of Columbia) from Table 1. The total number of recorded tornadoes in 2000, arranged alphabetically by State of USA and including the District of Columbia. (Source: National Oceanic and Atmospheric Administration's National Climatic Data Center) State No. of No. of State Tornadoes Tornadoes Alabama 44 Montana 10 Alaska 0 Nebraska 60 Arizona 0 Nevada 2 Arkansas 37 New Hampshire 0 California 9 New Jersey 0 Colorado 60 New Mexico 5 Connecticut 1 New York 5 Delaware 0 North Carolina 23 District of Columbia 0 North Dakota 28 Florida 77 Ohio 25 Georgia 28 Oklahoma 44 Hawaii 0 Oregon 3 Idaho 13 Pennsylvania 5 Illinois 55 Rhode Island 1 Indiana 13 South Carolina 20 Iowa 45 South Dakota 18 Kansas 59 Tennessee 27 Kentucky 23 Texas 147 Louisiana 43 Utah 3 Maine 2 Vermont 0 Maryland 8 Virginia 11 Massachusetts 1 Washington 3 Michigan 4 West Virginia 4 Minnesota 32 Wisconsin 18 Mississippi 27 Wyoming 5 Missouri 28

26 Graphical Summaries: Dispersion Graphs (dot plots) A dispersion graph places individual data values along a number line, thereby representing the position of each data value in relation to all the other data values. Researchers use dispersion graphs to identify patterns in data such as concentrations, locations of data 'gaps', or atypical data (i.e. observations that do not fit the general character of the dt data.) We can see that most of the data is concentrated at the lower end of the graph, indicating that most States had 20 or fewer tornadoes, and only a few States had more than 50 tornadoes. Texas had 147 tornadoes during 2000 and this data value is positioned on the far right-hand side of the graph. There are also several gaps in the data where there are no values; these are shown by the green rectangles on the graph. Since the data are concentrated toward the lower end of the dispersion graph, we can say that the number of tornadoes for Texas (147) is atypical of this particular data set. Atypical data values are also referred to as outliers. Graphical Summaries: Histograms We see that the first class contains all the States that experienced between zero and nineteen tornadoes during Notice that each class has the same width along the x- axis. The decision to set the width of each class at nineteen is arbitrary. A different width could easily be used and would likely change the overall appearance of the histogram. As with dispersion graphs, histograms can show gaps where no data values exist (the class). In the Figure, there are three empty classes: 80-99, , and When histogram data cluster to one side or the other, the shape of the histogram is described as skewed. In the Figure, the tornado data are clustered on the lower or left-hand side, which is known as positive skew. Due to the single outlier, the data is said to tail to the positive side.

27 Numerical Summaries: Measures of Central Tendency Mean If we examine the mean in relation to all data values, we can see that the mean lies toward the lower end of the dispersion graph, which makes sense because this is where the majority of the data values are concentrated. Numerical Summaries: Measures of Central Tendency Median We can see the tornado data median value of 11 on the dispersion graph. Note that for this data set, the median is positioned closer to the lower end of the data values than the mean. This shows that the median is not influenced by outliers as was the mean, but by the number of data values. When a data set has outliers, reporting the median as the central tendency of the data often gives a better 'typical' data value than the mean. Rank State No. of Tornadoes Rank State No. of Tornadoes 1 Alaska 0 27 Idaho 13 2 Arizona 0 28 Indiana 13 3 District i t of Columbia 0 29 South Dakota 18 4 Delaware 0 30 Wisconsin 18 5 Hawaii 0 31 South Carolina 20 6 New Hampshire 0 32 Kentucky 23 7 New Jersey 0 33 North Carolina 23 8 Vermont 0 34 Ohio 25 9 Connecticut 1 35 Mississippi Massachusetts 1 36 Tennessee Rhode Island 1 37 Georgia Maine 2 38 Missouri Nevada 2 39 North Dakota Oregon 3 40 Minnesota Utah 3 41 Arkansas Washington 3 42 Louisiana i Michigan 4 43 Alabama West Virginia 4 44 Oklahoma New Mexico 5 45 Iowa New York 5 46 Illinois Pennsylvania 5 47 Kansas Wyoming 5 48 Colorado Maryland 8 49 Nebraska California 9 50 Florida Montana Texas Virginia 11

28 Numerical Summaries: Measures of Central Tendency Mode The mode is the data value that occurs the most frequently in a data set. Although not used as often as the mean and the median, by identifying the most commonly occurring data value the mode may suggest the central tendency of the data. For the tornado data, the mode is 0. There are eight States that did not experience any tornadoes in However, it would be misleading to suggest that the central tendency of this data set is 0, since it is obvious from the data values that the value of 0 is not 'central' to the range of values. Numerical Summaries: Measures of Dispersions Virginia Variance No. of Squared Deviation Idaho State Tornad Deviation Scores Indiana oes Scores South Alaska Dakota Arizona Wisconsin District of South Columbia Carolina Delaware Kentucky Hawaii North Carolina New Hampshire Ohio New Jersey Mississippi Vermont Tennessee Connecticut Georgia Massachusetts Missouri North Rhode Island Dakota Maine Nevada Oregon Utah Minnesota Arkansas Louisiana Alabama Washington Oklahoma Michigan Iowa West Virginia Illinois New Mexico Kansas New York Colorado Pennsylvania Nebraska Wyoming Florida Maryland Texas California Montana Sum=0.0 Sum=

29 Numerical Summaries: Measures of Dispersions Standard Deviation Using the standard deviation of 26.6 for the tornado data, we can create bounds around the mean that describe data positions that are 1, 2, or 3 standard deviations. For example, if we add one standard deviation to and subtract one standard deviation from the mean we arrive at 47.7 and -5.5, respectively. FromtheFigure, we can see that most of the data fall within 1 standard deviation of the mean, which suggests that the data are concentrated about the mean. Notice that as the number of standard deviations increases, fewer data values are found. In fact, only six data values are found beyond 1 standard deviations from the mean. It is interesting to note that one data value is beyond 3 standard deviations. When interpreting any standard deviation value it is important to keep in mind that the greater the value of the standard deviation, the more spread out or dispersed a data set is likely to be. Numerical Summaries: Measures of Dispersions Interquartile Range A quartile can be thought of as one of the classes created from the division of an ordered data set into four equally-sized groups. The 25th quartile has 25% of the data falling below it and the 75th quartile has 75% of the data falling below it. The interquartile range describes the middle one-half (or 50%) of an ordered data set, so represents the range between the data value of the 25th quartile and the data value of the 75th quartile. The Figure illustrates the bounds of the interquartile range for the tornado data.

30 Numerical Summaries: Measures of Dispersions Interquartile Range Using the box-and-whisker plot, you can see the position of the central tendency with respect to the interquartile range. In our case, the median is positioned toward the lower end of the data, which suggests that the data is positively skewed. You can also see the length of the interquartile range compared to the entire data set, and identify atypical data values and the degree to which those values are atypical. The numbers on top of the circle and asterisk indicate the rank of the value, and allow you to locate the specific data value

31 Scatter plot A 3D scatter plot allows for the visualization of multivariate data of up to four dimensions. The Scatter plot takes multiple scalar variables and uses them for different axes in phase space. The different variables are combined to form coordinates in the phase space and they are displayed using glyphs and colored using another scalar variable.

32 x f t x sin t x cos t x sin2 t x cos2 f X t 2...

33 Statistics for Data Analysis Descriptive Statistics Classification & Arrangement Summarizing Presentation (figure, graph, table, ) Data + Working Hypothesis Inferential Statistics Sampling Parameter Estimation Hypothesis Test Conclusions

States and capital package

States and capital package : 1 Students are required to know 50 states and capitals and their geological locations. This is an independent working packet to learn about 50 states and capital. Students are asked to fulfill 4 activities