SYSTEM Research note on: A simulation method to test for potential accuracy of a selection bias test for RCTs

CONTEXT: Selection bias interferes with the internal validity of clinical trials and leads to favoring one clinical outcome over another. Risk of selection bias is introduced when knowledge of certain patient characteristics, known to be conducive to the success of one particular intervention, is applied together with foreknowledge regarding the allocation of such patients in a specific sequence of interventions.

PROBLEM: Selection bias testing has been proposed and recommended on basis of the reverse propensity score (RPS). So far there are no actual clinical randomised control trials (RCTs) available from which the accuracy of such test may be studied. The challenge is therefore to find a suitable simulation method that will allow establishing the summary measures of potential diagnostic test accuracy: sensitivity, specificity, positive/negative likelihood ratios and diagnostic odds ratio (DOR).

SUGGESTED SOLUTION: In order to investigate the test’s diagnostic accuracy the number of true positive (TP); true negative (TN); false positive (FP) and false negative (FN) results need to be established from a number of sufficiently relevant RCT simulations. From these, summary measures of potential diagnostic accuracy are calculated. It has to be noted that the so obtained results can only provide indication of potential (not actual) test accuracy, thus remain hypothetical and require verification through the application of the bias test in real RCTs at a later stage. This article describes one suggested simulation method in detail.