3. Collect sample data
4. Evaluate Null hypothesis
.
C. Errors in Hypothesis Testing
1. Type I error - reject Ho when true
2. Type II error - fail to reject Ho when false
II. Evaluating Hypotheses
A. Alpha Level (a)
- minimize risk of type I error
1. determine what data are expected if Ho true
2. determine what data are unlikely if Ho true
3. use distribution of sample means separated into two parts
- Xbar or M expected (hi prob) if Ho true
- Xbar or M unlikely (low prob) if Ho true
4. The alpha level defines very unlikely (e.g., extreme 5% of distribution) scores to obtain by chance
- Xbar or M compatible with middle of distribution
- Xbar or M compatible with extremes of distribution
5. When Ho falls into tails, we reject Ho
- very unlikely sample if the treatment had no effect
B. z scores in Hypothesis Testing
z = obtained difference / difference due to chance
Recall that the estimated standard error is: s/sqrt(n)
When n = 1, the estimated standard error will be equal to the population standard deviation
#1
A training program to increase friendliness is tried on one individual randomly selected from the general public. Among the general public (who do not get this procedure) the mean on
the friendliness measure is 30 with a standard deviation of 4. The researchers want to test their hypothesis at the 5% significance level. After going through the training program, this individual's score on the friendliness measure is 40. Compute the appropriate statistic and use a one-tailed test. What should the researchers conclude?
#2
A team of educational psychologists is interested in the effects of instructions on timed school achievement tests. The researchers
predict that people will do better on a test if they are told to answer each question with the first response that comes to mind. To test this theory, the researchers give a standard school achievement test to 64 randomly selected fifth-grade children. They give the test in the usual way, except that they add to the instructions a statement that children are to answer each question with the first response that comes to mind.
When given in the usual way, the test is known to have a mean of 200, a standard deviation of 48, and an approximately normal distribution. The 64 students with the special instructions score 220. Test with alpha = .05, one tail.
C. More about Alpha levels (two-tailed tests)
a = 0.05, critical region +/- 1.96
a = 0.01, critical region +/- 2.58
a = 0.001, critical region +/- 3.30
D. Assumptions
1. random sampling
2. independent observations
3. homogeneity of variance, s not changed by treatment
4. normal sampling distribution (sample size, population distribution)
III. Directional (one-tailed) tests
A. Critical region in only one tail
a = 0.05, critical region 1.64
a = 0.01, critical region - look up
a = 0.005, critical region - look up
a = 0.001, critical region - look up
- reject Ho with smaller difference between M and m
- more "sensitive"
- increase the possibility of Type I error (false alarm)
B. Power
- the probability of detecting a treatment effect when one is indeed present.
- power is the opposite of Type II error (when a treatment effect really exists in the population).
-power = 1 � (type II error) or 1 � (beta)
-as type II error decreases, power increases
- by decreasing type I error (move from .05 to .01) we directly increase type II error (and thereby decrease power).
The Relationship between Power and Sample Size
C. Effect Size
Important limitation of the hypothesis testing procedure:
It makes a relative comparison: the size of the treatment effect relative to the difference expected by chance. If the standard error is very small, then the treatment effect can also be very small and still be bigger than chance.
Therefore, a significant effect does not necessarily mean a big effect.
Also, if the sample size is large enough, any treatment effect, no matter how small, can be enough for us to reject the null hypothesis.
Figure 8-11 (p. 262)
The appearance of a 15-point treatment effect in two different situations. In part (a), the standard deviation is σ = 100 and the 15-point effect is relatively small. In part (b), the standard deviation is σ = 15 and the 15-point effect is relatively large. Cohen�s
d uses the standard deviation to help measure effect size.
Calculating effect size:
Cohen's d = mean difference / standard deviation
Magnitude of d Evaluation
0 < d < 0.2 Small effect
0.2 < d < 0.8 Medium effect
d > 0.8 Large effect
The Relationship between Power and Effect Size
#3
A mood questionnaire has been standardized so that the scores form a normal distribution with m = 50
and s = 15. A psychologist would like to use this test to examine how the environment affects mood. A sample of n = 25 individuals is obtained, and the individuals are given the mood test in a darkly painted, dimly lit room with plain metal desks and no windows. The average score for the sample is M = 43.
a) Is there sufficient evidence to conclude that the environment has a significant effect on mood? Use a two-tailed test at the .05 level of significance.
b) Calculate the effect size.
c) Repeat the hypothesis test using the .01 level of significance.
d) Compare the results from parts a & c. How does the level of significance influence the outcome of a hypothesis test?
e) Does the level of significance influence (or change) the effect size?
#4
Suppose that a researcher normally uses an alpha level of .01 for hypothesis tests, but this time uses an alpha level of .05.
a) What does this change in alpha level do to the amount of power?
b) What does this change in alpha do to the risk of a Type I error?