Making Inferences and Justifying Conclusions: Comparing Treatments Using Randomized Experiments (CCSS.S-IC.5)
Common Core High School Statistics And Probability · Learn by Concept
Help Questions
Common Core High School Statistics And Probability › Making Inferences and Justifying Conclusions: Comparing Treatments Using Randomized Experiments (CCSS.S-IC.5)
In a randomized study comparing two tutoring methods, the observed mean test score for Method 1 is 4 points higher than Method 2. A randomization test for the null claim of no difference gives p-value = 0.012. Which conclusion is supported at $\alpha=0.05$?
Fail to reject the null; the difference is not statistically significant.
Reject the null; there is evidence of a difference between methods.
The probability the null hypothesis is true is 0.012.
The p-value proves Method 1 causes higher scores with certainty.
Explanation
Because p-value 0.012 < 0.05, reject the null; the difference is statistically significant.
In a randomized trial, plants were assigned to fertilizer A or B, and under the null of equal means a randomization test gave a p-value of 0.001 for fertilizer A's higher average height. Which conclusion is supported at $\alpha=0.05$?
Fail to reject the null; there is insufficient evidence of a difference in mean plant height.
There is a 0.1% chance that fertilizer A truly increases height.
The p-value tells which fertilizer is more cost-effective.
Reject the null; there is strong evidence fertilizer A produces taller plants on average than fertilizer B.
Explanation
Because p = 0.001 < 0.05, reject the null; the data give strong evidence that fertilizer A results in a higher mean height.
Students are randomly assigned to take a short nap or not before a memory quiz. The nap group's mean completion time is 15 minutes faster; a randomization test of no mean difference gives p-value = 0.003. Which conclusion is supported at $\alpha=0.05$?
Fail to reject the null; there is no evidence of a difference.
The p-value means there is a 0.3% chance the null is true.
The difference is practically important for everyone.
Reject the null; there is strong evidence the mean times differ between groups.
Explanation
Because p-value 0.003 < 0.05, reject the null; the difference is statistically significant.
In a randomized experiment, students were assigned to tutoring Method A or Method B, and a simulation assuming no difference gave a p-value of 0.02 for Method A's higher average score. Which conclusion is supported at $\alpha=0.05$?
Fail to reject the null; there is not enough evidence that Method A differs from Method B.
There is a 2% chance that the null hypothesis is true.
Reject the null; the data provide evidence at $\alpha=0.05$ that Method A leads to higher average scores than Method B.
The p-value tells us the size of the score difference between methods.
Explanation
Because p = 0.02 < 0.05, reject the null of no difference; the data support that Method A produces higher average scores.
In a randomized study, participants were assigned to a new sleep app or to standard advice, and under the claim of no effect the simulation produced a p-value of 0.18 for a higher proportion getting at least 8 hours with the app. What does the p-value suggest?
Fail to reject the null; there is not enough evidence that the app changes the proportion getting at least 8 hours of sleep.
Reject the null; there is strong evidence the app increases the proportion getting at least 8 hours of sleep.
There is an 18% chance the app has no effect.
The p-value means 82% of participants slept at least 8 hours.
Explanation
Because p = 0.18 > 0.05, fail to reject the null; the difference is not statistically significant at $\alpha=0.05$.
In a randomized A/B test of two online ads, Ad A's conversion rate exceeds Ad B's by 1 percentage point. A simulation test of no difference gives p-value = 0.18. What does the p-value suggest at $\alpha=0.05$?
Reject the null; Ad A is superior.
The null is true with 82% probability.
Fail to reject the null; the difference is not statistically significant.
Accept the alternative because the p-value is less than 1.
Explanation
Because p-value 0.18 > 0.05, fail to reject the null; the observed difference is not statistically significant.
In a randomized trial of two pain-relief gels, the observed difference in average pain scores is small. A simulation test for the null claim of no mean difference yields p-value = 0.31. What does the p-value suggest at $\alpha=0.05$?
Reject the null; there is a significant difference.
There is a 31% chance the alternative hypothesis is true.
The treatments are identical.
Fail to reject the null; not enough evidence of a difference.
Explanation
Because p-value 0.31 > 0.05, fail to reject the null; the observed difference is not statistically significant.
In a randomized medical study, patients were assigned to Treatment X or Treatment Y, and a randomization test under no difference gave a p-value of 0.049 for a lower mean time to pain relief with Treatment X. What does the p-value suggest?
Reject the null; there is evidence at $\alpha=0.05$ that Treatment X reduces mean time to pain relief compared with Treatment Y.
Fail to reject the null; there is not enough evidence of a difference in mean time to pain relief.
There is a 4.9% chance that Treatment X works.
The p-value tells how many minutes faster Treatment X is.
Explanation
Because p = 0.049 < 0.05, reject the null; the data indicate a statistically significant reduction in mean time to pain relief for Treatment X.
A randomized field experiment compares two fertilizers. The mean yield with Fertilizer A exceeds Fertilizer B by 2.1 bushels. A randomization test of no mean difference gives p-value = 0.048. Which conclusion is supported at $\alpha=0.05$?
Reject the null; there is evidence the mean yield differs between fertilizers.
Fail to reject the null; the difference is due to chance.
The p-value is the probability that the randomization was fair.
Conclude Fertilizer A is better for all farms.
Explanation
Because p-value 0.048 < 0.05, reject the null; the difference is statistically significant.
In an A/B test, customers were randomly shown Ad Version 1 or Version 2, and a permutation test under equal conversion rates gave a p-value of 0.07 for Version 1's higher conversion. Which conclusion is supported at $\alpha=0.05$?
Reject the null; there is evidence Version 1 has a higher conversion rate.
Fail to reject the null; there is not enough evidence at $\alpha=0.05$ that the conversion rates differ.
There is a 7% chance the null hypothesis is true.
The p-value means 93% of customers prefer Version 1.
Explanation
Because p = 0.07 > 0.05, fail to reject the null; the observed difference is not statistically significant at the 5% level.