It may be apparent from my most recent postings that, in my opinion, cooperative oncology groups around the world rarely use the inductive reasoning process. In the midst of a molecular revolution the main instrument for translating research findings from the laboratory to the clinic is, therefore not functioning as efficiently as it should be. When designed properly clinical trials can address fundamental questions. The design of a good therapeutic experiment should be driven by knowledge of the biology and natural history of the disease. For example, if you want to design studies to treat Hodgkin's disease, you need to learn to think like a Reed-Sternberg cell.
I envisage a group of cancer cells sitting in a corner laughing hysterically at some of our approaches to experimental design. Surgeons only willing to study the operation they were trained to do; medical oncologists always giving chemotherapy on days 1 and 8 because that’s when we have clinics; radiation oncologists giving treatments only 5 days a week because it is difficult to work at weekends. Often none of the above are willing to even participate in a trial unless they are assured of being part of the outcome. Cancer cells don’t think that way.
When involved in oncology therapeutic research you should always have in mind the five most important questions that can be addressed in clinical trials relevant to the oncology field. It is also important to know and be able to use the most appropriate human model system to address these questions. Of course there are no clinical cooperative groups organized to do this but there should be. Most groups specialize in studying a particular disease, or a particular stage of a disease. They should always have in mind the five most important questions that can be addressed in a clinical trial in that disease, or that stage of the disease. And, the questions should not be limited to questions facing a particular specialty.
How do you discern the most important questions of the day? Any individual or group responsible for a substantial clinical trials program ought to periodically participate in state of the art exercises, where experts are asked to define the major questions. And, these experts should not be limited to one’s own group. The end result would be questions that will apply to anyone anywhere studying the disease in question. Each group would then be unique only in the resources it can bring to bear on addressing a question. If we did this, there would be fewer individual group and more intergroup studies.
The larger the group involved the less the tendency there is to go through this kind of exercise. Groups become attached to their own specialties or their own studies. The focus is not on the best way to address important therapeutic questions but the best way to prove their approach is superior to someone else’s.
And, there is a tendency to do studies that address less important questions so that a clinical trial is available for each disease or stage of a disease that falls within the purview of a group. This is useful as a measure of the activity of a group when we review each other’s grants. So while we have a myriad of regulatory agencies and committees enforcing burdensome regulations ostensibly to protect patients, we spend precious little effort assuring that a clinical trial is really addressing a fundamental question. If you want to really protect the interests of cancer patients, for every trial you are asked to review, ask yourself where is the hypothesis? And, will the proposed study test it or try to confirm it?
Wednesday, April 2, 2008
Tuesday, February 26, 2008
More on Strong Inference
I want to revisit the subject of my previous editorial on the subject “Strong inference.” It’s an important concept. That editorial focused on a specific set of clinical studies on the treatment of early-stage Hodgkin’s disease but the failure to use strong inference is generic in the design of clinical trials because of the nature of the participants; that is, human subjects. Asking fundamental questions is not easy in clinical trials.
John Platt, who coined the term strong inference, said it consists of applying the following three steps: the development of alternative hypotheses; the development of crucial experiments devised to exclude one or more hypotheses, and the execution of careful studies to obtain easily interpretable result. This process results in the minimum number of steps to solve a problem.
The studies on Hodgkin’s disease were not designed to exclude the hypothesis that an adequately delivered, standard chemotherapy program, not a favorite untested alternative, could perform as well alone as when combined with radiotherapy. These studies spanned four decades without answering definitively the question of the respective places of radiotherapy and chemotherapy in early-stage disease. The same is true of the testing of local and systemic treatments for localized breast cancer. All the necessary data and tools to test the alternate hypotheses that radical mastectomy was either too much for small tumors or too little for large tumors, were in place by the 1960s. The major reason for failure was heretofore unappreciated micrometastases. But, definitive clinical trials were not completed until the 1980s because studies not designed to exclude a hypothesis are often repetative. "We measure, we define, we compute, we analyze, be we do not exclude", Platt said.
By training, clinicians cannot alter their methods rapidly and they tend to be men and women of one method. Disproving a therapeutic hypothesis might also result in the shift of the major part of the management of a disease from one specialty to another, which is generally not well received in medicine; therefore, there is a tendency for specialty competition to dominate the design of clinical experiments. Management shifts eventually happened in the examples cited above but they took too long.
We need to increase the use of strong inference in the design of all of our clinical studies. Hypotheses need to be clearly visible and the experiments designed to exclude them rather than support them. It will redirect us to a problem rather than a method orientation. But, this requires investigators to be willing, repeatedly, to put aside their last methods and adopt new ones. Investigators should also be willing to design studies that may exclude their specialty from the management of the disease. When a fact fails to fit a hypothesis we should retain the fact and discard the hypothesis.
As we enter the arena of molecularly targeted therapy, we will, in my view, see a shift form doing large studies looking for small differences to doing small studies looking for large differences. We may also need to introduce these new treatments at earlier stages where they will necessarily compete with established treatments. The design of such trials will be daunting but important to capture the clinical value of the many new advances we see printed in this journal in every issue. The use of strong inference will guide us well. It is applicable to all research, both in a laboratory and in the clinic and it is what really distinguishes good from bad research regardless of the size of the particle under study. Try it, you’ll like it.
John Platt, who coined the term strong inference, said it consists of applying the following three steps: the development of alternative hypotheses; the development of crucial experiments devised to exclude one or more hypotheses, and the execution of careful studies to obtain easily interpretable result. This process results in the minimum number of steps to solve a problem.
The studies on Hodgkin’s disease were not designed to exclude the hypothesis that an adequately delivered, standard chemotherapy program, not a favorite untested alternative, could perform as well alone as when combined with radiotherapy. These studies spanned four decades without answering definitively the question of the respective places of radiotherapy and chemotherapy in early-stage disease. The same is true of the testing of local and systemic treatments for localized breast cancer. All the necessary data and tools to test the alternate hypotheses that radical mastectomy was either too much for small tumors or too little for large tumors, were in place by the 1960s. The major reason for failure was heretofore unappreciated micrometastases. But, definitive clinical trials were not completed until the 1980s because studies not designed to exclude a hypothesis are often repetative. "We measure, we define, we compute, we analyze, be we do not exclude", Platt said.
By training, clinicians cannot alter their methods rapidly and they tend to be men and women of one method. Disproving a therapeutic hypothesis might also result in the shift of the major part of the management of a disease from one specialty to another, which is generally not well received in medicine; therefore, there is a tendency for specialty competition to dominate the design of clinical experiments. Management shifts eventually happened in the examples cited above but they took too long.
We need to increase the use of strong inference in the design of all of our clinical studies. Hypotheses need to be clearly visible and the experiments designed to exclude them rather than support them. It will redirect us to a problem rather than a method orientation. But, this requires investigators to be willing, repeatedly, to put aside their last methods and adopt new ones. Investigators should also be willing to design studies that may exclude their specialty from the management of the disease. When a fact fails to fit a hypothesis we should retain the fact and discard the hypothesis.
As we enter the arena of molecularly targeted therapy, we will, in my view, see a shift form doing large studies looking for small differences to doing small studies looking for large differences. We may also need to introduce these new treatments at earlier stages where they will necessarily compete with established treatments. The design of such trials will be daunting but important to capture the clinical value of the many new advances we see printed in this journal in every issue. The use of strong inference will guide us well. It is applicable to all research, both in a laboratory and in the clinic and it is what really distinguishes good from bad research regardless of the size of the particle under study. Try it, you’ll like it.
Strong inference and Inductive Reasoning
In November, Ferme et al., (1)published an article on the treatment of early- stage Hodgkin’s disease and said the following; “Our study showed that a combination of chemotherapy and radiotherapy should now be the standard of care for all patients with localized stage, supradiaphragmatic Hodgkin’s disease” and, added somewhat amazingly, that “the remaining question now under investigation is whether early-stage Hodgkin’s disease can be cured by chemotherapy alone”.
I have to confess I was a bit stunned by this backward statement . And decided to editorialize about it in Nature CLinical Practice Oncology, where I serve as Editor in Chief. This , the initial editorial, will appear in the April issue. The one that will follow will appear in the May issue.
In 1970, the cure of advanced Hodgkin’s disease with combination chemotherapy was reported,(2 ) an observation that has been amply confirmed since then. The complete remissions in that study have remained durable over four decades. In the 1960s, Howard Skipper promulgated “the inverse rule”, which stated there is an invariable inverse relationship between the body burden of cancer cells and curability by chemotherapy in all experimental systems studied. In 1991, we published a randomized trial comparing MOPP chemotherapy to standard radiotherapy in poor prognosis early-stage Hodgkin’s disease.(3,4 ) Even after 25 years’ follow-up chemotherapy won hands down, as it has in all studies since, when compared with radiotherapy alone.(5 )The collective response and survival data from the chemotherapy arm of our 1991 study and others like it that use standard and adequate chemotherapy suggest the inverse rule is operative in human systems. No one has shown a survival advantage of adding radiotherapy to chemotherapy in early-stage disease compared with standard and adequate chemotherapy alone. Despite this, almost all large clinical trials have not tested a standard chemotherapy program alone compared with combined modality regimens. This is important because we know that radiotherapy is associated with very substantial late carcinogenic effects especially to the breast.
I used the words “adequate chemotherapy” repeatedly because one has to question the quality of the administered chemotherapy in the Ferme study although no data are supplied in the paper to judge this. It’s as if as long as the acronym is familiar just saying it was used is sufficient to assure the quality of administration. Administered in 92 different institutions it was only capable of achieving a complete remission in 64% of patients with early-stage disease, a substantially lower complete remission rate than everyone is achieving in patients with advanced disease who have a much larger tumor burden. When inadequate chemotherapy is used, radiotherapy always improves the outcome.
In 1964, John Platt coined the use of the term “strong inference” to describe a particular approach to research using an inductive reasoning process.(6) The process involves devising alternative hypotheses and designing experiment that will exclude a hypothesis. “How many of us,” he said, “focus on experiments to exclude a hypothesis? We measure, we define, we compute, and we analyze but we do not exclude." If one examines rapidly moving fields you will find that inductive reasoning is the backbone of the design of experiments. In the laboratory a hypothesis can be excluded in a week. Clinical investigators, because of the nature of their experimental subjects, have a special burden but also a special responsibility not to waste resources on experiments that take 5 years or more to complete that are not designed to exclude a hypothesis.
Strong inference and the inductive reasoning process have not been apparent in these large clinical studies of Hodgkin’s disease. The alternate hypothesis that standard and adequate chemotherapy, alone in early stage Hodgkin’s disease would be equivalent or better than combined modality therapy has not really been tested let alone excluded. We already know we can cure it. The reason for this is addressed in another posting to follow.
References
1.Fermé et al. (2007) Chemotherapy plus involved-field radiation in early-stage Hodgkin's disease. N Engl J Med 357: 1916–1927
2. DeVita VT Jr et al. (1970) Combination chemotherapy in the treatment of advanced Hodgkin's disease. Ann Intern Med 73: 881–895
3. DeVita VT Jr et al. (1980) Curability of advanced Hodgkin's disease with chemotherapy. Long-term follow-up of MOPP-treated patients at the National Cancer Institute. Ann Intern Med 92: 587–595
4. Longo DL et al. (1991) Radiation therapy vs. combination chemotherapy in the treatment of early stage Hodgkin's disease: Seven-year results of a prospective randomized trial. J Clin Oncol 9: 906–917
5. Longo DL et al. (2006) A prospective trial of radiation alone vs combination chemotherapy alone for early-stage Hodgkin’s disease: implications of 25-year follow-up to current combined modality therapy [abstract # 98]. Pro Am Soc Hem 108(11)
6. Platt JR (1964) Strong inference: certain systematic methods of scientific thinking may produce much more rapid progress than others Science 146: 347–353
I have to confess I was a bit stunned by this backward statement . And decided to editorialize about it in Nature CLinical Practice Oncology, where I serve as Editor in Chief. This , the initial editorial, will appear in the April issue. The one that will follow will appear in the May issue.
In 1970, the cure of advanced Hodgkin’s disease with combination chemotherapy was reported,(2 ) an observation that has been amply confirmed since then. The complete remissions in that study have remained durable over four decades. In the 1960s, Howard Skipper promulgated “the inverse rule”, which stated there is an invariable inverse relationship between the body burden of cancer cells and curability by chemotherapy in all experimental systems studied. In 1991, we published a randomized trial comparing MOPP chemotherapy to standard radiotherapy in poor prognosis early-stage Hodgkin’s disease.(3,4 ) Even after 25 years’ follow-up chemotherapy won hands down, as it has in all studies since, when compared with radiotherapy alone.(5 )The collective response and survival data from the chemotherapy arm of our 1991 study and others like it that use standard and adequate chemotherapy suggest the inverse rule is operative in human systems. No one has shown a survival advantage of adding radiotherapy to chemotherapy in early-stage disease compared with standard and adequate chemotherapy alone. Despite this, almost all large clinical trials have not tested a standard chemotherapy program alone compared with combined modality regimens. This is important because we know that radiotherapy is associated with very substantial late carcinogenic effects especially to the breast.
I used the words “adequate chemotherapy” repeatedly because one has to question the quality of the administered chemotherapy in the Ferme study although no data are supplied in the paper to judge this. It’s as if as long as the acronym is familiar just saying it was used is sufficient to assure the quality of administration. Administered in 92 different institutions it was only capable of achieving a complete remission in 64% of patients with early-stage disease, a substantially lower complete remission rate than everyone is achieving in patients with advanced disease who have a much larger tumor burden. When inadequate chemotherapy is used, radiotherapy always improves the outcome.
In 1964, John Platt coined the use of the term “strong inference” to describe a particular approach to research using an inductive reasoning process.(6) The process involves devising alternative hypotheses and designing experiment that will exclude a hypothesis. “How many of us,” he said, “focus on experiments to exclude a hypothesis? We measure, we define, we compute, and we analyze but we do not exclude." If one examines rapidly moving fields you will find that inductive reasoning is the backbone of the design of experiments. In the laboratory a hypothesis can be excluded in a week. Clinical investigators, because of the nature of their experimental subjects, have a special burden but also a special responsibility not to waste resources on experiments that take 5 years or more to complete that are not designed to exclude a hypothesis.
Strong inference and the inductive reasoning process have not been apparent in these large clinical studies of Hodgkin’s disease. The alternate hypothesis that standard and adequate chemotherapy, alone in early stage Hodgkin’s disease would be equivalent or better than combined modality therapy has not really been tested let alone excluded. We already know we can cure it. The reason for this is addressed in another posting to follow.
References
1.Fermé et al. (2007) Chemotherapy plus involved-field radiation in early-stage Hodgkin's disease. N Engl J Med 357: 1916–1927
2. DeVita VT Jr et al. (1970) Combination chemotherapy in the treatment of advanced Hodgkin's disease. Ann Intern Med 73: 881–895
3. DeVita VT Jr et al. (1980) Curability of advanced Hodgkin's disease with chemotherapy. Long-term follow-up of MOPP-treated patients at the National Cancer Institute. Ann Intern Med 92: 587–595
4. Longo DL et al. (1991) Radiation therapy vs. combination chemotherapy in the treatment of early stage Hodgkin's disease: Seven-year results of a prospective randomized trial. J Clin Oncol 9: 906–917
5. Longo DL et al. (2006) A prospective trial of radiation alone vs combination chemotherapy alone for early-stage Hodgkin’s disease: implications of 25-year follow-up to current combined modality therapy [abstract # 98]. Pro Am Soc Hem 108(11)
6. Platt JR (1964) Strong inference: certain systematic methods of scientific thinking may produce much more rapid progress than others Science 146: 347–353
Subscribe to:
Posts (Atom)