Monday, September 28, 2009

What happened to the merit sytem in NIH peer review?

There was an article in the New York Times on 22 September by Gardner Harris entitled “Debate flaring over Grants research”.It was based on a recently released report by the GAO on how NIH manages its grant program.

The essence of the NYTs articcle was whether or not NIH grants administrators are right to reach down to “make exceptions “to fund grants with worse scores, to support new young investigators. They say the average age of investigators has risen from 35 in 1980 to 41today, so such steps are necessary. The American Cancer Society goes even further. Anyone over the age of 45 need not apply.

 What happened to the merit system? If your goal is to find new knowledge that leads to the eradication of disease shouldn’t we be funding the best and the brightest whatever their age?

Harris says “ There has been a growing chorus of complaints over the years that the agency's scientific review process is deficient-that is fails to finance high risk research; that projects must effectively be half done before financing is approved; that cliques control the process; and that reviewers are rarely the field’s leading lights”.

Anyone involved in the NIH grant system knows all these complaints are true. I speak from experience; I ran the largest chunk of that grant program at the NCI for 15 years.

The essence of the problem is that universities are addicted to Ro-1 grants. The grant system has become an entitlement program. Without a Ro-1 grant it is difficult for an investigator to attain tenure. What has surprised me, even shocked me, and is that what an investigator does with those grants is often secondary to the fact that they got them. Supporting high-risk research is not the major goal.
The grant peer review process has become the major arbiter of tenure. And incumbents do have an unfair advantage. Peer review committees, each made up of grantees, give the edge to established investigators like themselves. And, scientists will always admit to other scientists, (but not in public) that they don’t submit their best and newest ideas in a grant but ideas that have some data to support them- . It’s a bit of grantsmanship.

Left to their own devices, young investigators do well on their own. Their ideas are often fresh and, in a purely merit system, they can out compete an incumbent. But the way the system is now constructed they are at a disadvantage but it is a disadvantage of NIH’s own making.

The NIH distorts the system even further. It decries “targeted research” but it regularly influences the research process by issuing Request for Applications in specific (targeted) areas with set aside funds. The areas selected are what the Congress or some NIH staffer, or a board of advisors, thinks is the best way to spend grants monies. I have watched young investigators change their research interest not because they thought an RFA identified an interesting area but because they needed to follow pools of money to get a grant. So much for NIH's storied primacy of “investigator initiated research”.

And more gamesmanship. To keep Congress anxious about how many grants are funded each year NIH artificially keeps the percentage of approved grants funded very low. NIH scores grants on a system of 1 to 5 with one being the best and 5 the worst score. A grant can also be disapproved. Few ever are. Many are , however, given scores of 3, 4 or 5 that indicate they shouldn’t be funded even if the investigator is young and money is available. Or expressed another way, no matter how much money we had, we could find better ways of spending it than funding grants with bad scores. Instead of funding only 21 % of all approved applications, many of which shouldn't be funded,  we are more often funding 40 to 50 % of the good applications. Not too bad really. Reaching down could get you into bad territory.
It would be interesting if data were available on the age distribution of the PI's of grants that score better than 2 compared to those that score worse than 2.

It is in the best interests of both universities and the NIH to leave the system as is. But, NIH has so distorted the peer review process that it is faced with dilemmas like funding young investigators just because they are young not because of the merit of their ideas. The grant system has become an end in itself instead of a means to an end.

A Churchill quote about Democracy is often paraphrased to defend the current system as “the worst system ever invented except for every other system”. This may be true for democracy but not the NIH peer review system.

Thursday, September 10, 2009


There was an exchange on the floor  of the house chamber between the President and Congressman Joe Wilson last night that bears on the figure of the number of people who are uninsured in this country.
The data used comes from the US census bureau and is 47 million.  I used the 47 million  figure to point out the various groups that make up this number using the same census bureau data. The 47 million figure includes 10 million people who , according to the Bureau , are " not American citizens".
Last night the  President vehemently said that his plan would not include the funding of health insurance for illegal alliens.
The President's speech should now automatically  lead everyone to reduce the figure of those without insurance  to 37 million..  Will it? I doubt it since it would tend to  reduce the urgency  to take down our entire health system for a mere 37 million uninsured, made up of four additional groups whose probelms can be solved by simple legislation. Actually , this morning he actually increased it!
As I said in my recent footnote to that blog, when it was reposted, I use these figures as a way to  judge whether any speaker actually knows what he or she is talking about vis a vis the health care issue..
So far it is slim pickings and it doesn't include the President or his advisors..

Thursday, September 3, 2009

Note: I worte the following post in 2007 using US census figures for that year but it is relavent today. I have found that anyone who uses the "47 million are uninsured "argument  is usually as wrong about everything else the say about the health care system so I have decided to re  post this.

 I am concerned about the direction of the current debate on the reform of our health care system. Most proposals for reform are based on two premises.

The first, is that our health care system is overrated and not equal to some that use a different model for health care , like the single payer systems of the United Kingdom and other European countries.
The second is that 47 million Americans are "uninsured".
Neither , in my view, is correct. Both come from the unfortunate tendency to tear something down when proposing something new. Afterall ,who will take a proposal to fix something seriously unless the something is totally broken.
It is the second premise I want to address here because the phrases "47 million Americans are uninsured", or " without insurance whatsoever" are used as if this was one homogeneous group to show that the american health care system is broken.
So , let's look at the makeup of the 47 million figure dervied from the US Census Bureau figures.
1. 27 % , or roughly 12.7 million people, are uninsured for only a part of the year in which they are counted but are ultimately insured. This is an issue or portability of health insurance. This group will need plan A to reduce their risk of becoming ill while not covered by insurance . But are they really " uninsured" in the way the term is usually used?
2. Roughly 10.3 million of the 47 million are listed as " not American citizens". They require Plan B which surely has more to do with immigration reform than reform of the health care system. Most proposals use the 47 million uninsujred total but ignore the fact that a substantial part of this group are illegal aliens.
3. The third group is made up of roughly 9 million people, half of which make between 50 and 75 thousand dollars a year and the other half more than 75 thousand dollars a year. Many of them are healthy young people who can afford insurance but do not wish, for various reasons, to buy it. This group, if they must be covered, would require a plan that required everyone to buy health insurance, say, Plan C.
4. In the 4th group, there are roughly 8 milion people of all ages, adults and children alike, who are actually eligible for health insurance under a variety of existing plans but don't take advantage of them , again, for a variety of reasons, sometimes out of ignorance. Surely we can solve the problems of educating people about the existence of these plans using Plan D.
That leaves the 5th group, roughly about 7 million people, who might be called the " hard core uninsured" or "without insurance whatsoever", certainly a tragedy for a country as rich as ours . This group will require Plan E.
But the 47 million uninsured figure is quoted as if all of them are hard core uninsured which is factually incorrect .
It is important , it seems to me, to examine the issues related to these groups separately, and to hear specifics about plans A through E so we do not apply " the general solution for the specific problem" and do more harm than good to the finest health care system in the world..

Monday, August 31, 2009

Speaking of New Orleans

There was a story today in the New York Times about how this magnificent city is clawing its way back trying  make itself better than it was before hurricane  Katrina and  with some success.
It reminded me of  a bad decision the leadership of the American Society of Clinical Oncology ( ASCO) made to  drop New Orleans from it's list of meeting sites.
When someone commented that New Orleans would no longer accommodate a meeting the size of ASCO, I checked with a colleague of mine in New Orleans and was assured it wasn't true. The convention center , hotels and New Orleans outstanding  eateries are well able to handle the meeting.

I was told by ASCO's  EVP Alan Lister, that it was primarily a financial decision . ASCO can make more money from the exhibits at the Chicago convention center than at New Orleans because of the way it's configured. But it turns out New Orleans is not suffering from discrimination .  ASCO is gong to be in  Chicago in perpetuity despite  good facilities at other cities as well.

Now don't get me wrong, I like Chicago-just not every year. Not to mention that ASCO could do it's part  by helping the economy of New Orleans.

I told the ASCO leaders that I thought if the ASCO members were polled  they would probably  have voted for  variety over Chicago in perpetuity.

Annoyance was obvious. I was told it was a board decision to make. As a member I disagree.
I was at the founding meeting of ASCO. The society was founded for the benefit of its members, and had as it's primary role  the dissemination of information, not to become a bank.
I don't suppose the annual budget to run  the society of 90 million dollars had anything to do with  this?

The Slippery slope

I read with great interest the story in the 30 August Sunday Magazine section of the New York Times  on the  tragedy at Memorial Hospital in New Orleans in the days following hurricane Katrina.
If the story is accurate , patients who would have been difficult to move or who had signed a " Do not Resuscitate Order" ( DNR) were euthanized by lethal injections on a grand scale.
The doctors involved justified it by saying the police told them if they were not out by 5 PM on that last day they would be left behind . And they wouldn't have survived anyhow.
There were clearly patients who were conscious and not ready to die  and those who had signed DNR orders as a statement, not because they were terminal.
Now the surgeon who administered the lethal doses of drugs is campaigning for laws that would exempt doctors from liability if they did these things under emergency conditions.
And, of course, the doctors were not indicted  even though euthanasia is illegal.  New Orleans politicians, it appears , had decided they had had enough bad  exposure.
I mention  this because it is an example of what can go wrong if you give doctors the right to judge who should live and who should die. I am a cancer doctor and know physicians who believe that patients who have metastatic cancer should be left to die anyhow, with dignity of course, but early. My worst nightmare has always been to wake up from a cancer operation and find one of these guys looking down on me. I would not have wanted to be a cancer patient at that  hospital after Katrina.
Giving doctors that power is a first step down a slippery slope, especially if we end up with a government run health care system and rationing.

Saturday, August 29, 2009

Jim Watson's editorial in the NYTs

I read Jim Watson's editorial in the New York Times   a few weeks ago about the war on cancer , and actually agree with his thoughts about our ability now to design smart drugs.
This vision is better than the one he had  that saw cancer cured by Judah Folkman 's antiangiogenesis factors "in two years," 8 years ago, a prediction he repeated to no avail two years later at a meeting in Aspen. But clinical cancer is not Jim's strong suit.
Everything else was  wrong , as well , especially about how the money allotted to the war on cancer was spent.
He was also wrong about  why he was dropped from the Presidentially appointed National Cancer Advisory Board (NCAB). I was there and watched his interaction with Cancer Panel Chair, Benno Schmidt, who attended every board meeting.
 Jim expressed nothing but contempt for the new war on cancer  and the NCAB and showed it by coming to the board meeting with the New York Times , putting his feet up on the table , reading the paper and ignoring what was going on. When somebody said something about a program he  didn't like he would lower the paper and spit out an expletive. " This is a pile of  s** " was his favorite.
He made the mistake one day of saying that about a program and forgetting he was a major beneficiary .  Benno , in a way only the eloquent Benno Schmidt could do, pointed this out. Since this was a public meeting, it made the news. Jim was quiet for the next few board meetings
The President's Cancer  Panel had oversight function for the NCI and the NCAB, and Schmidt thought his behavior  was rude and non contributory  and was  hurting the program . He  asked for his removal  two years into a six year term and the White House complied.
 In truth , while I suppose I may have missed it somewhere, I never heard Watson  utter a single positive suggestion about  anything  to do with the National Cancer Program.
His constructive criticism of the NCI  that led to his removal , he said in his editorial ,was to put all the money into basic research but "instead" he said, " they went clinical". 
Eighty five percent of the money did go onto basic research. He wasn't paying attention.
The remaining 15 % is responsible for the declining national mortality rates from cancer we have been witnessing since  1990, something else he may have  missed.
My guess is ,the hundreds of thousands of people whose lives were saved would agree that Benno  Schmidt did the right thing.
But hey, 1 out of 3  is considered good in some sports.

Thursday, July 30, 2009

On the Struggles of Cancer Centers
Vincent T. DeVita Jr. and Elizabeth DeVita Raeburn
My daughter, Elizabeth, and I are writing a book about the War on Cancer. So,I’ve been doing a lot of reflecting on how things have played out since the Cancer Act was passed in 1971. Cancer Centers were a major theme of the Cancer Act.
Before it was passed, we had three freestanding cancer centers. Now we have 63—the vast majority “matrix centers,” meaning they are integrated within the structure of university medical schools and hospitals. The matrix set-up helped us get new cancer centers up and running across the country much faster, and at less cost than building new, free-standing ones.
The downside, however, is that, historically, “matrix” centers have often run afoul of the departmental structure of medical schools. They are often centers in name only. More energy is often expended over turf struggles between the departments of medicine and cancer centers over the operation of the sections of Medical Oncology than over innovations in cancer care. This is a recurring theme that has dogged NCI cancer centers since their inception in 1972.
Was there a better model? Possibly. Columbia College of Physicians and Surgeons (C P&S) had the first University based freestanding cancer hospital. But even it was dogged by some of the problems that affect the matrix model.
In November of 2007, while researching the upcoming book, Elizabeth and I interviewed Dr. Alfred Gellhorn the first Director of the Francis Delafield Cancer Hospital.
Gellhorn was trained as a general surgeon but re trained himself in Internal Medicine. When we interviewed Dr. Gellhorn, he was involved in his fourth successful career since leaving the Delafield.
Here’s an excerpt of our conversation:
“Dr. Gellhorn how did the Delafield get started?”
“It started in 1952. A CP&S breast surgeon by the name of Christian Hagenson persuaded the city to build a cancer hospital. Hagenson could never find anybody in Internal Medicine who was interested in cancer because the Chairman of the Medicine, Robert Loeb, felt that this was a disease that a good internist would have nothing to do with, because you couldn’t cure it.
I had absolutely unbounded admiration for Bob Loeb. He was a magnificent clinician…and a wonderful teacher. I think he just had a blind spot when it came to cancer. I think that the reason that Medical Oncology got such a slow start had to do with the attitude of people in Internal Medicine.
Loeb would not consider rotating CP&S house staff through a cancer hospital, but we were able to eventually get approval for our own residency program. I was able to recruit some wonderful colleagues, like Paul Marks, Elliot Osserman, John Ultmann, Bernie Weinstein, Helen Ranney among others. And my first chief resident was Jim Holland.
I was then the Director of the Institute of Cancer Research. The Delafield was its clinical arm where we also had labs.
Loeb tolerated me. But he was fond of saying rather openly, “Alfred, you’re a part of the lunatic fringe.”
And then Loeb retired and an even more pessimistic and indifferent clinician came in as Chair of Medicine, Stanley Bradley. Stan Bradley sent two guys to replace me in ’68 who had no connection to cancer and it was closed shortly thereafter.”
Despite the obvious need for, and the success of the Delafield, two Chairs of Medicine closed it. Their actions set the template for most NCI matrix centers today. Delafield was a good model just not free standing enough.
CP&S received its NCI designation in 1974. Since then, like a lot of matrix centers, they have struggled to find their Cancer Center identity. I am sure, for those who knew of the Delafield, there have been many times when they wished they still had it.
Dr Gellhorn died peacefully in March 2008 at the age of 94. He was a great mentor and a pioneer in the cancer field. Not many people, having been told they were part of the “lunatic fringe” by one of the greats in medicine, would have had the fortitude to continue. He did. And for that, and all the many luminaries he trained, we all owe him a debt of gratitude. Thanks Alfred.
The Co-Directors of the Center for Management
Research in Healthcare at Vanderbilt,
David Dilts and Alan Sandler, reported a study
of an alarming nature to the National Cancer
Institute (NCI). The study was covered by The
Cancer Letter and documented that cancer
trials take an average of 800 days to start. The
clinical trials program in the US is broken and
apparently nobody has noticed: 800 days! Can
you imagine that? The response of the NCI
Director was a vow to cut the time to approval
in half. At 400 days the system would still be
broken. Dilts and Sandler reported that for
cooperative-group studies, a protocol takes
800 days from conception to activation, with an
additional 200 days at a cancer center. Even
an investigator-initiated trial at a cancer center
takes a median of 116 to 252 days, and if you
make amendments to a study it goes back
to the end of the line. Furthermore, the study
by Dilts and Sandler was conducted by two
cooperative groups at four centers selected
by NCI because they scored better than most
(in peer review) on the function of their clinical
trials programs. The solutions offered by
the authors of the study to rectify this situation
were not of much help (e.g. start fewer studies,
stop tweaking studies).
We live in an age where we are blessed with
bounteous information about critical steps
in the pathways that cancer cells use to outstrip
the growth of normal cells and information
about how their cell-death pathways are
broken. Although we have the specific agents
and the opportunity to capitalize on the whole
concept of ‘targeted therapy’ we find ourselves
unable to use these advances. If one looks
at past successes in treatment, they were
by the ability to test an idea
rapidly, and to tweak studies to allow swift
adjustment of protocols in response to events
happening in real time. I have not examined
the protocols in question directly, but I would
not hesitate to suggest that a study that takes
800 days to activate is hopelessly outdated
the day it starts. Furthermore, if investigators
are encouraged to avoid tweaking, then the
talents of our best investigators are either not
being used or are being wasted.
At my own institution I reside on many committees
involved in protocol review, and I can
say with authority that we are hopelessly
Although cancer centers are
not perfect and have many problems of their
own, they are still where the knowledge base
is. The requisite talent to know the right way
to design and modify an ongoing study does
not reside on remote review committees at the
NCI or the FDA, yet those are the places where
the delays are greatest. Too many cooks are
spoiling the broth.
A step towards fixing this problem would
be to delegate the entire review and approval
process, at least for phase I and II studies,
to NCI-designated cancer centers with NCIapproved
clinical trials programs, with the NCI
and FDA only retaining audit responsibilities.
This was the role envisioned for cancer centers
by those who framed the NCI Act of 1971.
Most of these regulations have been imposed
on us by the US congress in the name of
patients’ safety. Review boards set up and
codified in regulations to protect patients are
doing just the opposite. They prevent patients
from having access to the fruits of the war on
cancer and the best we have to offer.
We have met the enemy and he is us!

I would like to return to the subject of over regulation and how it is sometimes of our own doing. My editorial on the broken clinical trials program generated a fair amount of discussion, including at my own institution. In an article written about the editorial, Dr. Alan Sandler provided a startling additional piece of information. At one institution, 87 steps and 29 signatures were required to launch a trial (Helwick, C. Oncol. News Int. 18, [2009]). There are no regulations that require so many signatures. Since my editorial and the article were circulated to all relevant committee members at Yale, I was invited to attend a discussion of the problem at a Cancer Committee meeting. It was a lively group and they had a good discussion hoping to improve the protocol review and approval process at Yale and dearly hoping we didn’t require 29 signatures for approval (we didn’t). Near the end of the meeting, someone asked “Why do we have to submit cooperative group protocols to our Human Investigation Committee (HIC) when they have been reviewed by NCI’s central Institutional Review Board and many other committees and are essentially unchangeable by the time they arrive at our place anyhow?”
A member of our HIC rose to answer and said ”You know, we always wondered why you sent them to us to review because we don’t really have to review them.” A hush fell on the room. People looked at one another. Investigators had assumed these protocols needed HIC review and routinely sent them into the process. HIC members thought they didn’t but took the submissions as a request for a review. Neither one had said anything to the other. Both were doing a lot of unnecessary work.
In my April editorial on the use of approved drugs in non approved ways, I highlighted another problem that occurs in our highly regulated environment (DeVita Jr, V.T. Nat. Revs. Clin. Oncol. 6, 181 [2009]). That is the assumption that everything we do needs FDA input and approval. In that case, investigators were intending to ask the FDA for approval of a new use for drugs that have been around for many years and are safe. If you ask the FDA to define a strategy for you they will and they may define a complicated set of steps. But, you may not even need to ask. They actually have better things to do.
I came across a small article in Journal Watch General Medicine by Thomas L. Schwenk entitled “A critique of clinical practice guidelines” (Schwenk, T.L. Journal Watch General Medicine 1, 1 [2009]). I gravitated to it because I dislike practice guidelines. In my view, they are too restrictive in rapidly moving fields. Schwenk was commenting on guideless in the cardiovascular field not for cancer but the general principles are the same. He cited a study of the evidence underlying guidelines developed by committees appointed by the American Heart Association and the American College of Cardiology (Tricoci, P. et al. JAMA 301, 831–841 [2009]). In short, since 1984 there have been 7,196 recommendations on 22 topics for 56 clinical practice guidelines. The number of recommendations increases by almost 50% and, the levels of evidence for guidelines decrease with every iteration so that the majority of recommendation are now made on the basis of opinions and case studies. Schwenk says “Clinical practice guidelines, once spare and elegant in their creation, dissemination and application have become commonplace, tedious and of questionable clinical relevance.” The recommendation to correct this was to replace all members of the guideline committees with each iteration, especially the leadership.
These are all examples of things we do to ourselves. They brought to mind my favorite cartoon of all times “Pogo”. In one famous cartoon, Pogo was commenting on the politics of the 1970s in the US. He is standing on a platform in front of an inclined mirror looking at his image and appears startled - his hat is seen flying off. He says “We have met the enemy and he is us!”
In the three examples I used above the enemy is us. That’s the bad news. The good news is anything put together by us can also be disassembled by us if we are willing to ask ‘is this really necessary?’
The Most important news at ASCO 2009

Iam often asked my opinion of the most important presentation at the annual ASCOmeetings. Igave up long ago trying to explain that it is impossible to read the thousands of abstracts and rate them accordingly, but like everyone else Ido keep my eyes peeled to note something of special interest. In most years the pickings are actually fairly slim; breakthrough results rarely make it to ASCO. However, this year it was easy. Buried amongst all the other news tidbits emanating from the 2009 ASCOmeeting in Orlando, Florida, was the description of a momentous event. Iread about it in the Wall Street Journal (Winslow, R. AstraZeneca, Merck to Test Cancer Drugs in ‘Cocktail’. Wall Street Journal 1 June 09). The two drugs in question, Merck’s Akt inhibitor MK 2206 and AstraZeneca’s Mek inhibitor AZD 6244, are years away from any possible approval. This is what made the announcement so unusual. They would be tested together in the clinic by their respective companies to try to verify preclinical evidence of synergy.
Why does this announcement have such an important impact? First, it is axiomatic that systemic cancer cannot be cured by single agents. Virtually all past curative treatments consist of combinations of agents and, for targeted therapies, the redundancy in the signaling systems at the disposal of the cancer cells suggests the same will be true. Are there exceptions? Yes. But they are rare. Mother Nature has sent us a message and we should listen to it. So why persist in testing cancer drugs one at a time?
Second, we are faced with an interesting problem in the development of targeted therapies; the more specific the agent is against the target the less the toxic and therapeutic effects. We got what we wished for—clean hits on target. However, we are left with very little on which to base approval of individual targeted agents for clinical use, except that these newer agents are safer than older cytotoxic chemotherapy. Pharmaceutical companies face the dilemma that after investing millions of dollars on developing an agent that blocks a theoretically important pathway they either have to ditch it, on the basis of the usual standards of efficacy for drug approval, or wait another decade of testing in combination with other promising agents. Furthermore, patients with cancer have to wait with them. In fact, this is what has happened. Most of the new and useful targeted treatments would never have been marketed if they had not been tested in combination with another agent, usually a cytotoxic drug.
Third, most biopharmaceutical companies do not have the resources to develop multiple targeted therapies at the same time. They do believe, however, that it is in the best interests of their shareholders to focus on getting their drug approved, by itself, above all else. This usually does not involve testing it jointly with a drug from another company before it is approved.
So, we are at a time when we have identified many important biological pathways cancer cells use to divide and survive, and easily have the capacity to develop inhibitors of these pathways, but our old inflexible approach does not allow us to use our imagination and test these agents early on in ways that the best scientific minds think might achieve synergy. That is why the announcement by Merck and AstraZeneca is so important. It establishes a new paradigm. We need to push competitive interests back even further. Iwould suggest that the next step is to do the same thing as each drug actually reaches the clinic—combination phase Itrials, if you will. The problems this approach will create for pharmaceutical companies trying to sort out relative value for their shareholders, and for regulatory agencies worried about safety, should not be underestimated, but they too can be solved. It is truly the way of the future for cancer treatment.