The template for an NIH review is set, and reviews, in form, are standardized. This was not always true, and this change a significant improvement for both reviewers, as it was easier, and applicants, as the problems and strengths were easier to discern. I’ve put the headings from that form below (taken out white space). The overall impact (which is what you usually can count on the entire SS reading before voting) is the most important part. I’ve pasted a pastiche of overall impacts that I’ve written (removing identifying science and PI’s) to give you a sense of what a relatively bad one looks like.
The NIH instructions for Overall Impact are:
Reviewers will provide an overall impact score to reflect their assessment of the likelihood for the project to exert a sustained, powerful influence on the research field(s) involved, in consideration of the following five scored review criteria, and additional review criteria. An application does not need to be strong in all categories to be judged likely to have major scientific impact.
Here is the Overall Impact paragraph. This proposal would have been triaged (I’ve taken bits and pieces from reviews that did not get discussed).
This ESI has the background and environment necessary to carry out this project. MRU has invested in this investigator, and she has the protected time necessary to carry out this project. The project represents an interesting and novel concept based on the integration of two technologies for understanding problem X through development of a systems level neurological model. However, there are several significant weaknesses in design of the project. It is unclear what form the model will take. SA1 is to develop the procedures for constructing the models. What if this is not possible? A description of what specific variables and data will be obtained and how these data will be used to address SA 2 & 3 (to test the model) is lacking. How this model will be or even could be used applied to clinical therapy is not stated. The PI acknowledges that P and Q cannot be measured, and will need to be approximated. However, there is no mechanism or allowance for the impact of P and Q on the model. Significance and innovation are relatively strong because a quantified and “realizable” model of X has the potential to transform clinical practice for this problem. However, concerns about the research design limit enthusiasm.
One of the things that does reflect lots of reviews is the relative disorganization of this paragraph. I always try for clear writing, good organization, etc. But in the end, what is most important is what is said – in this case that it is the design that is significantly flawed.
Here are the headings for the review template. The links with each take you to a description of the questions important for that heading.
Scored Review Criteria
Overall Impact Write a paragraph summarizing the factors that informed your Overall Impact score.
Reviewers will consider each of the five review criteria below in the determination of scientific and technical merit, and give a separate score for each.