I want to revisit the subject of my previous editorial on the subject “Strong inference.” It’s an important concept. That editorial focused on a specific set of clinical studies on the treatment of early-stage Hodgkin’s disease but the failure to use strong inference is generic in the design of clinical trials because of the nature of the participants; that is, human subjects. Asking fundamental questions is not easy in clinical trials.
John Platt, who coined the term strong inference, said it consists of applying the following three steps: the development of alternative hypotheses; the development of crucial experiments devised to exclude one or more hypotheses, and the execution of careful studies to obtain easily interpretable result. This process results in the minimum number of steps to solve a problem.
The studies on Hodgkin’s disease were not designed to exclude the hypothesis that an adequately delivered, standard chemotherapy program, not a favorite untested alternative, could perform as well alone as when combined with radiotherapy. These studies spanned four decades without answering definitively the question of the respective places of radiotherapy and chemotherapy in early-stage disease. The same is true of the testing of local and systemic treatments for localized breast cancer. All the necessary data and tools to test the alternate hypotheses that radical mastectomy was either too much for small tumors or too little for large tumors, were in place by the 1960s. The major reason for failure was heretofore unappreciated micrometastases. But, definitive clinical trials were not completed until the 1980s because studies not designed to exclude a hypothesis are often repetative. "We measure, we define, we compute, we analyze, be we do not exclude", Platt said.
By training, clinicians cannot alter their methods rapidly and they tend to be men and women of one method. Disproving a therapeutic hypothesis might also result in the shift of the major part of the management of a disease from one specialty to another, which is generally not well received in medicine; therefore, there is a tendency for specialty competition to dominate the design of clinical experiments. Management shifts eventually happened in the examples cited above but they took too long.
We need to increase the use of strong inference in the design of all of our clinical studies. Hypotheses need to be clearly visible and the experiments designed to exclude them rather than support them. It will redirect us to a problem rather than a method orientation. But, this requires investigators to be willing, repeatedly, to put aside their last methods and adopt new ones. Investigators should also be willing to design studies that may exclude their specialty from the management of the disease. When a fact fails to fit a hypothesis we should retain the fact and discard the hypothesis.
As we enter the arena of molecularly targeted therapy, we will, in my view, see a shift form doing large studies looking for small differences to doing small studies looking for large differences. We may also need to introduce these new treatments at earlier stages where they will necessarily compete with established treatments. The design of such trials will be daunting but important to capture the clinical value of the many new advances we see printed in this journal in every issue. The use of strong inference will guide us well. It is applicable to all research, both in a laboratory and in the clinic and it is what really distinguishes good from bad research regardless of the size of the particle under study. Try it, you’ll like it.
Tuesday, February 26, 2008
Strong inference and Inductive Reasoning
In November, Ferme et al., (1)published an article on the treatment of early- stage Hodgkin’s disease and said the following; “Our study showed that a combination of chemotherapy and radiotherapy should now be the standard of care for all patients with localized stage, supradiaphragmatic Hodgkin’s disease” and, added somewhat amazingly, that “the remaining question now under investigation is whether early-stage Hodgkin’s disease can be cured by chemotherapy alone”.
I have to confess I was a bit stunned by this backward statement . And decided to editorialize about it in Nature CLinical Practice Oncology, where I serve as Editor in Chief. This , the initial editorial, will appear in the April issue. The one that will follow will appear in the May issue.
In 1970, the cure of advanced Hodgkin’s disease with combination chemotherapy was reported,(2 ) an observation that has been amply confirmed since then. The complete remissions in that study have remained durable over four decades. In the 1960s, Howard Skipper promulgated “the inverse rule”, which stated there is an invariable inverse relationship between the body burden of cancer cells and curability by chemotherapy in all experimental systems studied. In 1991, we published a randomized trial comparing MOPP chemotherapy to standard radiotherapy in poor prognosis early-stage Hodgkin’s disease.(3,4 ) Even after 25 years’ follow-up chemotherapy won hands down, as it has in all studies since, when compared with radiotherapy alone.(5 )The collective response and survival data from the chemotherapy arm of our 1991 study and others like it that use standard and adequate chemotherapy suggest the inverse rule is operative in human systems. No one has shown a survival advantage of adding radiotherapy to chemotherapy in early-stage disease compared with standard and adequate chemotherapy alone. Despite this, almost all large clinical trials have not tested a standard chemotherapy program alone compared with combined modality regimens. This is important because we know that radiotherapy is associated with very substantial late carcinogenic effects especially to the breast.
I used the words “adequate chemotherapy” repeatedly because one has to question the quality of the administered chemotherapy in the Ferme study although no data are supplied in the paper to judge this. It’s as if as long as the acronym is familiar just saying it was used is sufficient to assure the quality of administration. Administered in 92 different institutions it was only capable of achieving a complete remission in 64% of patients with early-stage disease, a substantially lower complete remission rate than everyone is achieving in patients with advanced disease who have a much larger tumor burden. When inadequate chemotherapy is used, radiotherapy always improves the outcome.
In 1964, John Platt coined the use of the term “strong inference” to describe a particular approach to research using an inductive reasoning process.(6) The process involves devising alternative hypotheses and designing experiment that will exclude a hypothesis. “How many of us,” he said, “focus on experiments to exclude a hypothesis? We measure, we define, we compute, and we analyze but we do not exclude." If one examines rapidly moving fields you will find that inductive reasoning is the backbone of the design of experiments. In the laboratory a hypothesis can be excluded in a week. Clinical investigators, because of the nature of their experimental subjects, have a special burden but also a special responsibility not to waste resources on experiments that take 5 years or more to complete that are not designed to exclude a hypothesis.
Strong inference and the inductive reasoning process have not been apparent in these large clinical studies of Hodgkin’s disease. The alternate hypothesis that standard and adequate chemotherapy, alone in early stage Hodgkin’s disease would be equivalent or better than combined modality therapy has not really been tested let alone excluded. We already know we can cure it. The reason for this is addressed in another posting to follow.
References
1.Fermé et al. (2007) Chemotherapy plus involved-field radiation in early-stage Hodgkin's disease. N Engl J Med 357: 1916–1927
2. DeVita VT Jr et al. (1970) Combination chemotherapy in the treatment of advanced Hodgkin's disease. Ann Intern Med 73: 881–895
3. DeVita VT Jr et al. (1980) Curability of advanced Hodgkin's disease with chemotherapy. Long-term follow-up of MOPP-treated patients at the National Cancer Institute. Ann Intern Med 92: 587–595
4. Longo DL et al. (1991) Radiation therapy vs. combination chemotherapy in the treatment of early stage Hodgkin's disease: Seven-year results of a prospective randomized trial. J Clin Oncol 9: 906–917
5. Longo DL et al. (2006) A prospective trial of radiation alone vs combination chemotherapy alone for early-stage Hodgkin’s disease: implications of 25-year follow-up to current combined modality therapy [abstract # 98]. Pro Am Soc Hem 108(11)
6. Platt JR (1964) Strong inference: certain systematic methods of scientific thinking may produce much more rapid progress than others Science 146: 347–353
I have to confess I was a bit stunned by this backward statement . And decided to editorialize about it in Nature CLinical Practice Oncology, where I serve as Editor in Chief. This , the initial editorial, will appear in the April issue. The one that will follow will appear in the May issue.
In 1970, the cure of advanced Hodgkin’s disease with combination chemotherapy was reported,(2 ) an observation that has been amply confirmed since then. The complete remissions in that study have remained durable over four decades. In the 1960s, Howard Skipper promulgated “the inverse rule”, which stated there is an invariable inverse relationship between the body burden of cancer cells and curability by chemotherapy in all experimental systems studied. In 1991, we published a randomized trial comparing MOPP chemotherapy to standard radiotherapy in poor prognosis early-stage Hodgkin’s disease.(3,4 ) Even after 25 years’ follow-up chemotherapy won hands down, as it has in all studies since, when compared with radiotherapy alone.(5 )The collective response and survival data from the chemotherapy arm of our 1991 study and others like it that use standard and adequate chemotherapy suggest the inverse rule is operative in human systems. No one has shown a survival advantage of adding radiotherapy to chemotherapy in early-stage disease compared with standard and adequate chemotherapy alone. Despite this, almost all large clinical trials have not tested a standard chemotherapy program alone compared with combined modality regimens. This is important because we know that radiotherapy is associated with very substantial late carcinogenic effects especially to the breast.
I used the words “adequate chemotherapy” repeatedly because one has to question the quality of the administered chemotherapy in the Ferme study although no data are supplied in the paper to judge this. It’s as if as long as the acronym is familiar just saying it was used is sufficient to assure the quality of administration. Administered in 92 different institutions it was only capable of achieving a complete remission in 64% of patients with early-stage disease, a substantially lower complete remission rate than everyone is achieving in patients with advanced disease who have a much larger tumor burden. When inadequate chemotherapy is used, radiotherapy always improves the outcome.
In 1964, John Platt coined the use of the term “strong inference” to describe a particular approach to research using an inductive reasoning process.(6) The process involves devising alternative hypotheses and designing experiment that will exclude a hypothesis. “How many of us,” he said, “focus on experiments to exclude a hypothesis? We measure, we define, we compute, and we analyze but we do not exclude." If one examines rapidly moving fields you will find that inductive reasoning is the backbone of the design of experiments. In the laboratory a hypothesis can be excluded in a week. Clinical investigators, because of the nature of their experimental subjects, have a special burden but also a special responsibility not to waste resources on experiments that take 5 years or more to complete that are not designed to exclude a hypothesis.
Strong inference and the inductive reasoning process have not been apparent in these large clinical studies of Hodgkin’s disease. The alternate hypothesis that standard and adequate chemotherapy, alone in early stage Hodgkin’s disease would be equivalent or better than combined modality therapy has not really been tested let alone excluded. We already know we can cure it. The reason for this is addressed in another posting to follow.
References
1.Fermé et al. (2007) Chemotherapy plus involved-field radiation in early-stage Hodgkin's disease. N Engl J Med 357: 1916–1927
2. DeVita VT Jr et al. (1970) Combination chemotherapy in the treatment of advanced Hodgkin's disease. Ann Intern Med 73: 881–895
3. DeVita VT Jr et al. (1980) Curability of advanced Hodgkin's disease with chemotherapy. Long-term follow-up of MOPP-treated patients at the National Cancer Institute. Ann Intern Med 92: 587–595
4. Longo DL et al. (1991) Radiation therapy vs. combination chemotherapy in the treatment of early stage Hodgkin's disease: Seven-year results of a prospective randomized trial. J Clin Oncol 9: 906–917
5. Longo DL et al. (2006) A prospective trial of radiation alone vs combination chemotherapy alone for early-stage Hodgkin’s disease: implications of 25-year follow-up to current combined modality therapy [abstract # 98]. Pro Am Soc Hem 108(11)
6. Platt JR (1964) Strong inference: certain systematic methods of scientific thinking may produce much more rapid progress than others Science 146: 347–353
Subscribe to:
Posts (Atom)