(This article was originally published at Statistical Modeling, Causal Inference, and Social Science, and syndicated at StatsBlogs.)
Kelvin Leshabari writes:
We are currently planning to publish some few manuscripts on the outcome of treatment of some selected cancers occuring in children. The current dataset was derived from the natural admission process of those children with cancer found at a selected tertiary cancer centre. To the best of our understanding, our data are based on convenient sampling as we collected what was available. We are also quite sure that those patients were far from being representative of all other children with cancer in our area. We do know that convenient sampling is a non-probability sampling method. We plan to run several analyses that may include fitting proportional hazard models.
At present, I have a conflict with other co-investigators since I do believe that our dataset has violated some assumptions (e.g. proportionality assumption, randomness, independence, etc etc) and hence it may be statistically inappropriate to apply some statistical techniques (e.g. Proportional Hazard models, Kaplan-Meier curves). However, they provided me with a handful of publications that analysed data (based on convenience sampling!) even with obvious signs of violation of key assumptions of those statistical techniques.
Questions:
1. Is it statistically valid to analyse data using methods that need randomness and independence assumptions in cases of non-probability sampling? (e.g. Is it logical to fit a regression model to a dataset that was derived from convenient sampling?)
2. Can sample size (when infinitely large!) be a rescue for application of proportional hazard models (including fitting Kaplan-Meier curves!) in cases of non-probability sampling?
3. Can I fit proportional hazard models in a dataset whose nature is derived from convenience sampling?
My reply: If you do regression modeling using a convenience sample, you’re implicitly assuming the probability of a person being included in your sample depends only on the variables included in your regression model. To put it another way, you’re assuming equal-probability sampling within the poststratification cells determined by the predictors in your regression. So you can typically say that, the more predictors you have, the more reasonable it is to fit a regression model to a convenience sample. If you have some sense of the biases in the convenience sample (which sorts of people are more or less like to be in the sample), you should try to include predictors in your model that will capture this. And if necessary add a term to your log-likelihood to model any selection processes, but it’s rare that researchers ever get to this step.
The post If Yogi Berra could see this one, he’d spin in his grave: Regression modeling using a convenience sample appeared first on Statistical Modeling, Causal Inference, and Social Science.
Please comment on the article here: Statistical Modeling, Causal Inference, and Social Science
