Chapter 13 - Hypothesis Testing

• I. Descriptive Versus Inferential Analysis

  In some studies go beyond descriptive analysis to verify specific statements, or hypotheses, about the population of interest.

  inferential analysis. Data analysis aimed at testing specific hypotheses

  II. Overview of Hypothesis Testing

  A. Null and Alternative Hypotheses

  H0 Null hypotheses Complement each other Mutually exclusive Collectively exhaustive. Ha Alternative hypotheses

  Hypotheses always pertain to population parameters or characteristics rather than to sample characteristics. Ha will not be accepted if the sample evidence strongly supports H0. Ha will be accepted if the sample evidence is strong enough to reject H0. H0 should be stated more conservatively Failure to reject H0 should preserve the status quo.

  B. Type I and Type II Errors

  Without a census of the population certainty of the validity of any H0 is impossible. Accept or reject H0 based on evidence from the sample. Type I and Type II errors Evidence can lead to wrong inferences due to sampling error. Evidence might reject H0 when it is true Evidence might not reject H0 when it is false Type I and Type II errors cannot be avoided Lower the risk to an acceptable level. Limit on Type I Decision maker specifies this limit Basis for deciding when to reject H0.

  Decision Reject H0 Do not reject H0 Truth H0 Type I Error Right Decision Ha Right Decision Type II Error

  C. Significance Level

  The significance level of a statistical hypothesis test is a fixed probability of wrongly rejecting the null hypothesis H0, if it is in fact true. It is the probability of a Type I error and is set by the investigator in relation to the consequences of such an error. The significance level is usually denoted by Significance Level = P(type I error) = Confidence level = Usually, the significance level is chosen to be = 0.05 = 5%. The Z-Distribution

  Type II

  A type II error occurs when H0, is not rejected when it is in fact false. A type II error is also called an error of the second kind. A type II error is frequently due to sample sizes being too small. The probability of a type II error is symbolized by   P(type II error) = Power of the test , which is the probability of avoiding a Type II error

  Researcher specifies is difficult to constrain because its value depends on the  unknown value of the population parameter.

  An inverse relationship exists between and . lower the probability of a Type I error and the probability of a Type II error increases. Difficult to limit both and within prespecified levels. The situation will dictate the suitable significance level There is no universal significance level Many hypothesis tests use significance levels of .01, .05, or .10 Significance levels of .20 or .30 may also be acceptable.

  D. Decision Rule for Rejecting the Null Hypothesis

  decision rule Guideline specifying the sample evidence necessary to reject H0. The value incorporated in the decision rule depends on the significance level specified. test statistic Standard variable value computed from sample data compared with a critical value (obtained from an appropriate probability table) to determine whether or not to reject the H0. Every hypothesis test has a corresponding test statistic, which depends on the sampling distribution involved

  E. One-Tailed Versus Two-Tailed Tests

  One Tailed Test Two Tailed Test A statistical hypothesis test in which the values for which we can reject H0 are located All in one tail of the distribution. in both tails of the distribution The critical region the set of values less than the critical value of the test or and the set of values greater than the critical value of the test.   the significance level is split each tail of the distribution For significance level of The tail portion of the distribution curve each critical value has a probability of

  One Tailed Example Suppose we wanted to test a the claim that, on average, viewers watch 50 hours of television per week. We could set up the following hypotheses

  Either of these two alternative hypotheses would lead to a one-sided test.

  Two Tailed Example Suppose we wanted to test a the claim that, on average, viewers watch 50 hours of television per week. The hypotheses below

  Nothing specific can be said about the average number viewing hours If we could reject H0 then we would know that the average number of viewing hours is likely to be less than or greater than 50

  Whether a hypothesis test should be one-tailed or two-tailed depends on the nature of the problem. Use a one-tailed test when interest centers primarily on one side of the issue. Use a two-tailed test there is no a priori reason to focus on one side of the issue.

  F. Steps in Conducting a Hypothesis Test

  The sequence of tasks involved in a typical hypothesis test are as follows:

  Step 1 Set up H0 and Ha. Step 2 Identify the nature of the sampling distribution curve specify the appropriate test statistic. Step 3 Determine whether the hypothesis test is one-tailed or two-tailed Step 4 Accounting for the significance level, find the critical value (two critical values for a two-tailed test) for the test statistic from the appropriate statistical table Step 5 State the decision rule for rejecting H0 Step 6 Compute the value for the test statistic from the sample data Step 7 Using the decision rule specified in step 5, either reject H0 or reject Ha.

  III. Role of Hypothesis Testing in Data Analysis

  A. Number of variables to be analyzed.

  Univariate analysis just one variable is the focus of the analysis. Bivariate analysis two variables are to be analyzed simultaneously. Multivariate analysis two or more variables are to be analyzed simultaneously.

  B. Type of Data

  Nonparametric procedures Analyses and hypothesis tests for nonmetric data Applicable to nominal and ordinal data Require only minimal assumptions about Nature of the data Measurement level Shape of their distribution. Parametric procedures Analysis techniques for metric data Applicable to interval and ratio data. Requires data with At least interval-scale properties Distribution that resembles the normal probability distribution.

  IV. Specific Hypothesis Tests

  A. Cross-Tabulations: Chi-Square Contingency Test

  Two-way tabulation

  Useful preliminary step in understanding the association between variables. Table shows the number of responses in each category of one variable falling into the categories of a second variable. To be meaningful Data on each variable is coded into a fixed set of categories Number of categories should not be large. Appropriate for Categorical (nominal or ordinal-scaled) variables. Transformed interval or ratio-scaled variables

  Chi-square contingency test Determines if a statistically significant relationship between two categorical (nominal or ordinal) exists testing the following hypotheses: H0: There is no association between the two variables (the two variables are independent of each other). Ha: There is some association between the two variables (the two variables are not independent of each other)

  Chi-square test can be used between two nominal scaled variables two ordinal scaled variables one nominal and one ordinal-scaled variable. Visual inspection of a crosstabulation can suggest an association. Chi-square test formally checks the relationship.

  B. Conducting the Test

  Compare the actual cell frequencies with expected cell frequencies. Expected cell frequencies assume that the null hypothesis is true. How would the total sample have partitioned itself into the various cells if the two variables were truly unrelated in the population The expected cell frequency marginal frequencies ni = the total units in category i (row total) nj = the total units in category i (column total) Rationale based on probability theory. Probability that any respondent will be in rowi ni/n. Probability that same respondent will be in columnj nj/n H0: rows and columns are independent Joint probability of cellij (ni/n)*(nj/n) Expected number of respondents in cellij =n*(ni/n)*(nj/n) =ni*nj/n chi-square test statistic in a contingency test r = number of rows c = number of columns Oij = Actual number in cellij Eij= Expected number in cellij (r-1)*(c-1) = degrees of freedom A chi-square contingency test of independence between two variables is always a one- tailed test. Minimum expected cell frequency is required. no cell should have an expected frequency of less than 1 no more than a 20% of cells should have expected frequencies <5. many low expected frequencies will inflate the computed chi-square value may lead to the false rejection of H0 The test may not be meaningful when the ni or nj values are very small for certain categories. Combine (recode) adjacent variable categories to get larger marginal frequencies.

  C. Cross-Tabulation Using SPSS for National Insurance Company

  How a customer's education was associated with whether or not she or he would recommend National to a friend.

  Sample Size for crosstabs

  Two-way table is based on 259 respondents. 26 of 285 responding customers did not answer education. Total sample size available for cross-tabulating is constrained by the variable with the larger number of missing values  If one of the two variables has a many missing values, the two-way table may be misleading. Exercise caution in interpreting the table The lower the educational level of a customer, the more likely she or he will recommend National to a friend There is a negative, association between a customer's education level and recommending National to a friend

  Percentages & frequencies

  Report the responses in each cell as raw frequencies percentages. Percentages are easy to compare helpful in gaining insight into the relationships among variables Cell percentages that said "Yes" to recommending National 91.9% with "high school or less" 90.1% with "some college" 89.4% with "college graduate" 71.9% with "graduate school"

  Row & Column Percentages

  Which direction (row or column) to compute percentages for the cell frequencies Example shows percentages of rows Guideline: Compute percentages in the direction of the presumed causal (independent) variable. if one variable is assumed a cause of the other base the percentages the causal variable. "Possible cause." Cross-tabs in descriptive research only suggests, does not prove, causality Which makes more sense? Education level influences willingness to refer Willingness to refer influences education level Example was made To reflect row percentages Probable causal variable on the left  (rows) Effect variable on the top (columns) It may not always be clear which variable is the likely causal variable.

  D. Precautions in Interpreting Two-Way Tables

  Two way tables are not conclusive evidence of a causal relationship. only suggest the possibility of a causal relationship. Watch out for small cell sizes percentages without raw totals. Check All individual cell sizes Overall sample size. Other variables two-way tables use data on just two variables at a time. relationship between two variables may depend on other variables. Critical variables excluded from the table may be masking a relationship that really does exist. The larger the number of variables included in the cross-tabulated data, the smaller the risk of making erroneous inferences.

  E. Test for a Single Mean

  Hypotheses H0: m = 0 Ha: m ¹ 0

  The theoretically correct sampling distribution for sample means is the t-distribution The appropriate test statistic is the t-statistic Get critical test statistic value (tc) from t-table The value of tc depends on a Degrees of freedom : n - 1

  z-statistic is used when the sample size is 30 or more When the sample size is 30 or more, the t-distribution closely resembles the standard normal curve

  Hypothesis-testing procedures limit only the probability of committing a Type I error If you find that H0 then review sample size. Small the sample size means high b-probability lower power less likelhood that H0 will be rejected.

  F. Test for a Single Proportion

  Hypothesis H0: p = 0; Ha: p ¹ 0

  The theoretically correct sampling distribution for sample proportions is the binomial distribution. For large sample size, binomial approaches normal distribution Z-statistic can be used as the test statistic p is the sample proportion A rule of thumb for sample sizes (both should apply) np > 10 n(1 - p) >10 p is its value when H0 is true Standard Error of the sample proportion no need to approximate denominator of this expression is really the standard error Know the standard error of the proportion exactly from p and n.

  G. Test of Two Means

  Equivalent Hypotheses H0: m1 = m2 H0:m1 -m2 = 0; Ha: m1 ¹ m2 Ha:m1 -m2 ¹ 0.

  When n1 > 30 and n2 > 30 sampling distribution resembles the normal distribution

  Test is two-tailed - null hypothesis is a strict equality Two critical values of z, one for each tail Z-Test Assumes large sample size independent samples Z-test is not appropriate for checking the difference between two means obtained at different times from the same sample.

  If n1 < 30 or n2 < 30 then use a t-test Assuming two sample populations are normally distributed with respect to the variable two populations have equal variances d.f. = n1 + n2 - 2. s* is the pooled standard deviation, given by

  H. Test of Two Means When Samples Are Dependent

  Requires modified hypothesis-testing procedure. Two sets of data from the same sample of households. Two sets of data from the same sample of respondents. First compute difference scores for related pairs. Two-sample data is now a set of single-sample difference scores. The rest of the procedure is similar to the hypothesis-testing procedure for a single mean.

  Hypotheses H0: md £ 0 Ha: md > 0

  sample estimate of the mean of differences T- statistic with (n - 1) degrees of freedom s is the standard deviation of the differences

  I. Test of Two Proportions

  Rule of thumb for ensuring that the sample sizes are adequate Each of the following conditions should be met

  Hypotheses H0: p1 - p2 = 0 Ha: p1 - p2 ¹ 0.

  test statistic sp1 - p2 Population standard error for the difference between proportions. sp1 - p2 sample standard error is used to estimate sp1 - p2 P weighted proportion across both samples n1p1 + n2p2 P = --------------------------- n1 + n2 Q Compliment of P Q = 1 – P