Here’s question 7 of our exam:
7. You conduct an experiment in which some people get a special get-out-the-vote message and others do not. Then you follow up with a sample, after the election, to see if they voted. If you follow up with 500 people, how large an effect would you be able to detect so that, if the result had the expected outcome, the observed difference would be statistically significant?
And the solution to question 6:
6. You are applying hierarchical logistic regression on a survey of 1500 people to estimate support for a federal jobs program. The model is fit using, as a state-level predictor, the Republican presidential vote in the state. Which of the following two statements is basically true?
(a) Adding a predictor specifically for this model (for example, state-level unemployment) could improve the estimates of state-level opinion.
(b) It would not be appropriate to add a predictor such as state-level unemployment: by adding such a predictor to the model, you would essentially be assuming what you are trying to prove.
Briefly explain your answer in one to two sentences.
(a) is true, (b) is false. The problem is purely predictive, and adding a good predictor should help (on average; sure, you could find individual examples where it would make things worse, but there’s no reason to think it wouldn’t help in the generically-described example above). When the goal is prediction (rather than estimating regression coefficients which will be given a direct causal interpretation), there’s no problem with adding this sort of informative predictor.
Just about all the students got this one correct.