Monday, September 28, 2009

What happened to the merit sytem in NIH peer review?

There was an article in the New York Times on 22 September by Gardner Harris entitled “Debate flaring over Grants research”.It was based on a recently released report by the GAO on how NIH manages its grant program.

The essence of the NYTs articcle was whether or not NIH grants administrators are right to reach down to “make exceptions “to fund grants with worse scores, to support new young investigators. They say the average age of investigators has risen from 35 in 1980 to 41today, so such steps are necessary. The American Cancer Society goes even further. Anyone over the age of 45 need not apply.

 What happened to the merit system? If your goal is to find new knowledge that leads to the eradication of disease shouldn’t we be funding the best and the brightest whatever their age?

Harris says “ There has been a growing chorus of complaints over the years that the agency's scientific review process is deficient-that is fails to finance high risk research; that projects must effectively be half done before financing is approved; that cliques control the process; and that reviewers are rarely the field’s leading lights”.

Anyone involved in the NIH grant system knows all these complaints are true. I speak from experience; I ran the largest chunk of that grant program at the NCI for 15 years.

The essence of the problem is that universities are addicted to Ro-1 grants. The grant system has become an entitlement program. Without a Ro-1 grant it is difficult for an investigator to attain tenure. What has surprised me, even shocked me, and is that what an investigator does with those grants is often secondary to the fact that they got them. Supporting high-risk research is not the major goal.
The grant peer review process has become the major arbiter of tenure. And incumbents do have an unfair advantage. Peer review committees, each made up of grantees, give the edge to established investigators like themselves. And, scientists will always admit to other scientists, (but not in public) that they don’t submit their best and newest ideas in a grant but ideas that have some data to support them- . It’s a bit of grantsmanship.

Left to their own devices, young investigators do well on their own. Their ideas are often fresh and, in a purely merit system, they can out compete an incumbent. But the way the system is now constructed they are at a disadvantage but it is a disadvantage of NIH’s own making.

The NIH distorts the system even further. It decries “targeted research” but it regularly influences the research process by issuing Request for Applications in specific (targeted) areas with set aside funds. The areas selected are what the Congress or some NIH staffer, or a board of advisors, thinks is the best way to spend grants monies. I have watched young investigators change their research interest not because they thought an RFA identified an interesting area but because they needed to follow pools of money to get a grant. So much for NIH's storied primacy of “investigator initiated research”.

And more gamesmanship. To keep Congress anxious about how many grants are funded each year NIH artificially keeps the percentage of approved grants funded very low. NIH scores grants on a system of 1 to 5 with one being the best and 5 the worst score. A grant can also be disapproved. Few ever are. Many are , however, given scores of 3, 4 or 5 that indicate they shouldn’t be funded even if the investigator is young and money is available. Or expressed another way, no matter how much money we had, we could find better ways of spending it than funding grants with bad scores. Instead of funding only 21 % of all approved applications, many of which shouldn't be funded,  we are more often funding 40 to 50 % of the good applications. Not too bad really. Reaching down could get you into bad territory.
It would be interesting if data were available on the age distribution of the PI's of grants that score better than 2 compared to those that score worse than 2.

It is in the best interests of both universities and the NIH to leave the system as is. But, NIH has so distorted the peer review process that it is faced with dilemmas like funding young investigators just because they are young not because of the merit of their ideas. The grant system has become an end in itself instead of a means to an end.

A Churchill quote about Democracy is often paraphrased to defend the current system as “the worst system ever invented except for every other system”. This may be true for democracy but not the NIH peer review system.

Thursday, September 10, 2009


There was an exchange on the floor  of the house chamber between the President and Congressman Joe Wilson last night that bears on the figure of the number of people who are uninsured in this country.
The data used comes from the US census bureau and is 47 million.  I used the 47 million  figure to point out the various groups that make up this number using the same census bureau data. The 47 million figure includes 10 million people who , according to the Bureau , are " not American citizens".
Last night the  President vehemently said that his plan would not include the funding of health insurance for illegal alliens.
The President's speech should now automatically  lead everyone to reduce the figure of those without insurance  to 37 million..  Will it? I doubt it since it would tend to  reduce the urgency  to take down our entire health system for a mere 37 million uninsured, made up of four additional groups whose probelms can be solved by simple legislation. Actually , this morning he actually increased it!
As I said in my recent footnote to that blog, when it was reposted, I use these figures as a way to  judge whether any speaker actually knows what he or she is talking about vis a vis the health care issue..
So far it is slim pickings and it doesn't include the President or his advisors..

Thursday, September 3, 2009

Note: I worte the following post in 2007 using US census figures for that year but it is relavent today. I have found that anyone who uses the "47 million are uninsured "argument  is usually as wrong about everything else the say about the health care system so I have decided to re  post this.

 I am concerned about the direction of the current debate on the reform of our health care system. Most proposals for reform are based on two premises.

The first, is that our health care system is overrated and not equal to some that use a different model for health care , like the single payer systems of the United Kingdom and other European countries.
The second is that 47 million Americans are "uninsured".
Neither , in my view, is correct. Both come from the unfortunate tendency to tear something down when proposing something new. Afterall ,who will take a proposal to fix something seriously unless the something is totally broken.
It is the second premise I want to address here because the phrases "47 million Americans are uninsured", or " without insurance whatsoever" are used as if this was one homogeneous group to show that the american health care system is broken.
So , let's look at the makeup of the 47 million figure dervied from the US Census Bureau figures.
1. 27 % , or roughly 12.7 million people, are uninsured for only a part of the year in which they are counted but are ultimately insured. This is an issue or portability of health insurance. This group will need plan A to reduce their risk of becoming ill while not covered by insurance . But are they really " uninsured" in the way the term is usually used?
2. Roughly 10.3 million of the 47 million are listed as " not American citizens". They require Plan B which surely has more to do with immigration reform than reform of the health care system. Most proposals use the 47 million uninsujred total but ignore the fact that a substantial part of this group are illegal aliens.
3. The third group is made up of roughly 9 million people, half of which make between 50 and 75 thousand dollars a year and the other half more than 75 thousand dollars a year. Many of them are healthy young people who can afford insurance but do not wish, for various reasons, to buy it. This group, if they must be covered, would require a plan that required everyone to buy health insurance, say, Plan C.
4. In the 4th group, there are roughly 8 milion people of all ages, adults and children alike, who are actually eligible for health insurance under a variety of existing plans but don't take advantage of them , again, for a variety of reasons, sometimes out of ignorance. Surely we can solve the problems of educating people about the existence of these plans using Plan D.
That leaves the 5th group, roughly about 7 million people, who might be called the " hard core uninsured" or "without insurance whatsoever", certainly a tragedy for a country as rich as ours . This group will require Plan E.
But the 47 million uninsured figure is quoted as if all of them are hard core uninsured which is factually incorrect .
It is important , it seems to me, to examine the issues related to these groups separately, and to hear specifics about plans A through E so we do not apply " the general solution for the specific problem" and do more harm than good to the finest health care system in the world..