Because NIH is back on again: I sometimes have trouble with separating Impact and Significance.

I sometimes have trouble with separating Impact and Significance.

Significance is (from NIAID):

  • How your research will advance your field.
  • How it will fill knowledge gaps or address opportunities or roadblocks in the field, and how it relates to research underway.
  • How the work is new and unique.
  • How it meets the NIH mission to improve health through science, by leading to cures, treatments, or preventions for human disease.

It is one of the five things you get ranked on:

  1. significance
  2. investigator
  3. innovation
  4. approach
  5. environment

Except as I have often said the ranking is more like:

  • 1. Significance/Approach
  • 1.01 Approach/Significance
  • 3. Innovation (or higher, in NIH’s eyes)
  • 13. Investigator
  • 14. Environment

These five things together make up your overall Impact. It’s not the average, because reviewers are free to rank the criteria as they chose. In short, In NIAID’s words:

An application does not need to be strong in all review criteria to get an outstanding overall impact score, though all the criteria can affect your score.

So… everything can count, but some animals are more equal than others. In today’s climate, everything is incredibly important.

The interplay among these factors (I’ve talked about it before, with lots of links to other sources, here). But one of the most succinct ways of putting it is (again, paraphrased from NIAID’s webpage) is that there are two uber-criteria (not really discussed anywhere) of Importance and Feasibility(likelihood). These two words get thrown around a lot in study section. Both are make a difference in the review, the score, and the ultimate disposition of your proposal. The significance and innovation (the difference between these two is discussed here) contribute to Importance, and approach, investigator and environment to feasibility.

Importance is are you going to move the field forward? Are you going to change things? Are you addressing the needs/goals/priorities of NIH (which is more than just the nation’s health, it behooves you to go look at what the IC(s) you are targeting want and have declared as priorities).

Feasibility is are you going to get the damn thing done? Do you have the ability to do this? A bit more subtle: will the experiments you propose answer the specific aims, are you hypotheses testable? And, if you get negative results, is it worth doing? This is more than do you have the right equipment.

A good summary (once again from my friends at NIAID) of IMPACT.

impact = function (importance, likelihood)

Breaking this down, but what is shown above;

impact = function {importance (significance, innovation), feasibility (approach, investigator, environment}

Understand that there are generalized Functions in this equation. It is not necessarily additive, and the weight of different components is quite variable. You still gotta write the best proposal you can, but it is important to write with the knowledge of the mindset of the reviewers who are going to be reading your proposal.

PS – usage clarification for sticklers: Its not a grant till you get the $$, up till then it is a proposal, as in “I propose to get great big gobs of money to do the coolest things possible, and by the way get tenure”.


2 thoughts on “Because NIH is back on again: I sometimes have trouble with separating Impact and Significance.

  1. The way I usually talk about this is partly visual. “Overall impact is a product of Significance (put out thumb), coupled with likelihood of success (put out all four other fingers). The other four review criteria speak to likelihood of success, the biggest predictor of which is the Approach.”

    I’m sure you saw the data where they correlated individual criterion scores with the Overall Impact score? Approach score was the biggest predictor for high Impact score. Doesn’t matter if it’s a significant problem if you can’t convince reviewers that your approach to solve it will work.

  2. Pingback: (more) NIH instructions on writing grant reviews | Mistress of the Animals

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s