An evaluation of the quality of statistical design and analysis of published medical research: results from a systematic survey of general orthopaedic journals (BMC Medical Research Methodology)

Background: The application of statistics in reported research in trauma and orthopaedic surgery hasbecome ever more important and complex. Despite the extensive use of statistical analysis, itis still a subject which is often not conceptually well understood, resulting in clearmethodological flaws and inadequate reporting in many papers. Methods: A detailed statistical survey sampled 100 representative orthopaedic papers using a validatedquestionnaire that assessed the quality of the trial design and statistical analysis methods. Results: The survey found evidence of failings in study design, statistical methodology andpresentation of the results. Overall, in 17% (95% confidence interval; 10-26%) of the studiesinvestigated the conclusions were not clearly justified by the results, in 39% (30-49%) ofstudies a different analysis should have been undertaken and in 17% (10-26%) a differentanalysis could have made a difference to the overall conclusions. Conclusion: It is only by an improved dialogue between statistician, clinician, reviewer and journal editorthat the failings in design methodology and analysis highlighted by this survey can beaddressed.