Evaluation of the propensity score methods for estimating marginal odds ratios in case of small sample size (BMC Medical Research Methodology)

Background: Propensity score (PS) methods are increasingly used, even when sample sizes are small ortreatments are seldom used. However, the relative performance of the two mainlyrecommended PS methods, namely PS-matching or inverse probability of treatmentweighting (IPTW), have not been studied in the context of small sample sizes. Methods: We conducted a series of Monte Carlo simulations to evaluate the influence of sample size,prevalence of treatment exposure, and strength of the association between the variables andthe outcome and/or the treatment exposure, on the performance of these two methods. Results: Decreasing the sample size from 1,000 to 40 subjects did not substantially alter the Type Ierror rate, and led to relative biases below 10 %. The IPTW method performed better than thePS-matching down to 60 subjects. When N was set at 40, the PS matching estimators wereeither similarly or even less biased than the IPTW estimators. Including variables unrelated tothe exposure but related to the outcome in the PS model decreased the bias and the varianceas compared to models omitting such variables. Excluding the true confounder from the PSmodel resulted, whatever the method used, in a significantly biased estimation of treatmenteffect. These results were illustrated in a real dataset. Conclusion: Even in case of small study samples or low prevalence of treatment, PS-matching and IPTWcan yield correct estimations of treatment effect unless the true confounders and the variablesrelated only to the outcome are not included in the PS model.