More on Strong Inference

I want to revisit the subject of my previous editorial on the subject “Strong inference.” It’s an important concept. That editorial focused on a specific set of clinical studies on the treatment of early-stage Hodgkin’s disease but the failure to use strong inference is generic in the design of clinical trials because of the nature of the participants; that is, human subjects. Asking fundamental questions is not easy in clinical trials.
John Platt, who coined the term strong inference, said it consists of applying the following three steps: the development of alternative hypotheses; the development of crucial experiments devised to exclude one or more hypotheses, and the execution of careful studies to obtain easily interpretable result. This process results in the minimum number of steps to solve a problem.
The studies on Hodgkin’s disease were not designed to exclude the hypothesis that an adequately delivered, standard chemotherapy program, not a favorite untested alternative, could perform as well alone as when combined with radiotherapy. These studies spanned four decades without answering definitively the question of the respective places of radiotherapy and chemotherapy in early-stage disease. The same is true of the testing of local and systemic treatments for localized breast cancer. All the necessary data and tools to test the alternate hypotheses that radical mastectomy was either too much for small tumors or too little for large tumors, were in place by the 1960s. The major reason for failure was heretofore unappreciated micrometastases. But, definitive clinical trials were not completed until the 1980s because studies not designed to exclude a hypothesis are often repetative. "We measure, we define, we compute, we analyze, be we do not exclude", Platt said.
By training, clinicians cannot alter their methods rapidly and they tend to be men and women of one method. Disproving a therapeutic hypothesis might also result in the shift of the major part of the management of a disease from one specialty to another, which is generally not well received in medicine; therefore, there is a tendency for specialty competition to dominate the design of clinical experiments. Management shifts eventually happened in the examples cited above but they took too long.
We need to increase the use of strong inference in the design of all of our clinical studies. Hypotheses need to be clearly visible and the experiments designed to exclude them rather than support them. It will redirect us to a problem rather than a method orientation. But, this requires investigators to be willing, repeatedly, to put aside their last methods and adopt new ones. Investigators should also be willing to design studies that may exclude their specialty from the management of the disease. When a fact fails to fit a hypothesis we should retain the fact and discard the hypothesis.
As we enter the arena of molecularly targeted therapy, we will, in my view, see a shift form doing large studies looking for small differences to doing small studies looking for large differences. We may also need to introduce these new treatments at earlier stages where they will necessarily compete with established treatments. The design of such trials will be daunting but important to capture the clinical value of the many new advances we see printed in this journal in every issue. The use of strong inference will guide us well. It is applicable to all research, both in a laboratory and in the clinic and it is what really distinguishes good from bad research regardless of the size of the particle under study. Try it, you’ll like it.