Lesion load reproducibility and statistical sensitivity of clinical trials in multiple sclerosis

To the Editor: Filippi et al.1 discussed strategies intended to improve the precision of measurement of the lesion load in MRI scans of MS patients. One technique, the use of a common set of rules to identify and outline lesions, reduced the “proportional variation (PV)” between observers to between 30% and 50%. Based on this result, they suggested that this technique may be useful in clinical trials. They did not furnish any estimate of the effect of this improved precision on the sensitivity of a clinical trial and provided insufficient information to allow readers to make such an assessment. Filippi et al.1 referenced two articles that study other methods intended to improve the precision of lesion load measurements.2,3 We believe both these papers have the same oversight, but Grimaud et al.3 did provide sufficient information to make an assessment of the effect of their method on the sensitivity of clinical trials. By our calculations (below), the effect is much less than the impression given by the article. The statistical significance of the results derived from a clinical trial is usually expressed by a “ p value,” where p < 0.05 is usually considered statistically significant. The technique of calculating the p value depends on the type of data available, the assumptions that can safely be made, and the statistical technique to be employed in the data analysis. As an illustrative example, consider the question of whether the mean measured lesion loads of two treatment arms from a randomized, placebo-controlled clinical trial are significantly different. We can calculate a p value from the means and corresponding standard errors using Student’s t -test. Provided the patients can be viewed as a random sample from a designated population, the standard errors of the means are equal to …