The NIH is pleased to announce the details of changes to the peer review process and application forms for fellowship applications, to be implemented for applications submitted for due dates on or after January 25^th, 2025. (See NOT-OD-24-107.) These changes are meant to facilitate the identification of the most promising candidates and the individualized training opportunities that will assist these researchers along their paths to careers in biomedical research. Read more in the full story and in the NIH post.

The changes to peer review and the fellowship application result from years of analysis and discussion. In response to continued concerns voiced by the extramural community that the current fellowship review process potentially disadvantages some highly qualified candidates, the NIH Center for Scientific Review (CSR) formed a CSR Advisory Council working group in Fall 2021, charged with evaluating the peer review process for NIH fellowship candidates. The working group recommendations evolved through input from the community, including that in response to a Request for Information, and input from NIH leadership.

The revisions to the NIH fellowship application and review process are meant to increase the chances that the most promising fellowship candidates will be consistently identified by scientific review panels. The changes are intended to:

1. Better focus reviewer attention on three key assessments: the fellowship candidate’s preparedness and potential, research training plan, and commitment to the candidate.
2. Ensure a broad range of candidates and research training contexts can be recognized as meritorious by clarifying and simplifying the language in the application and review criteria. 
3. Reduce bias in review by emphasizing the commitment to the candidate without undue consideration of sponsor and institutional reputation.

To achieve these goals, NIH is reorganizing and redefining the current five scored criteria into three:

• Candidate’s Goals, Preparedness and Potential (scored 1-9)
  • This criterion emphasizes the candidate’s potential to benefit from the fellowship research training plan considering factors such as their preparedness for the proposed training plan, their training stage, opportunities available to them, and qualities such as scientific understanding, creativity, and drive.
• Research Training Plan (scored 1-9)
  • This criterion emphasizes whether the research training plan identifies appropriate professional and scientific development goals, and the role of the research training project, mentor, and available resources in achieving those goals.
• Commitment to Candidate (scored 1-9)
  • This criterion emphasizes the role of the mentor and mentoring plan in the development of the candidate and considers whether the sponsor(s) commitment is appropriate and sufficient to support the candidate’s research training plan and career in the biomedical research workforce.

NIH is making multiple changes to the PHS Fellowship Supplemental Form to align the information collected with the revised review criteria, to reduce applicant burden, and to clarify which party – candidate or sponsor – should author each section of the application. Notable application changes include:

Candidate Section

• Grades will no longer be required or allowed.
• Candidates will be required to submit four personal statements: (1) a statement of professional and fellowship goals, (2) fellowship qualifications, (3) a self-assessment, and (4) scientific perspective

Research Training Plan

• The headings of some sections have been revised to emphasize the importance of training in the fellowship plan.
• The section “Selection of Sponsor and Institution” has been removed in favor of including the information elsewhere in the application.
• The sections include: (1) Training Activities and Timeline, (2) Research Training Project Specific Aims, (3) Research Training Project Strategy, including the Scientific Foundation & Rationale and Approach.

Commitment to Candidate, Mentoring, and Training Environment Section

• Sponsors and Co-sponsors will be required to submit five statements: (1) Mentoring Approach and Candidate Mentoring Plan; (2) Prior Commitment to Training and Mentoring; (3) Commitment to the Candidate’s Research Training Plan; (4) Research Training Environment; and (5) Candidate’s Potential. A sixth statement on Clinical Training will be required for candidates proposing to gain experience in a clinical trial as part of the research training plan.

NIH is currently considering multiple recommendations made by the NIH Advisory Committee to the Director on Re-envisioning NIH-Supported Postdoctoral Training. Among their recommendations, the committee suggested that NIH should hold institutions and investigators responsible for providing high quality training and mentorship to their fellows. NIH shares this interest, and one goal of the revised fellowship process is to ensure fellows receive the high-quality training they need to succeed in a biomedical research career.

In the coming years, we hope that this change simplifies and expands talented researchers’ access to NIH fellowships.  As always, we will keep you informed through our webpage, notices in the NIH Guide, and through the Review Matters and Open Mike blogs in the lead-up to implementation. We encourage you to register to join us for a public webinar on the revisions to fellowship application and review in September to learn more and have your questions answered by experts. We will announce additional webinars as we near January.

Strengthening the fellowship application and peer review process is just one way that NIH is working to empower the next generation of biomedical researchers.

On 5/15 from 9:00 AM – 10:00 AM, CDWG will host Jiahui Xu to present her research. Jiahui Xu is a Ph.D. candidate in Sociology and Demography at Pennsylvania State University. Her research interests lie in social inequality, quantitative methodology, and computational sociology. Her actively ongoing projects include: 1). adapting the generalized random forests for causal decomposition to investigate college returns; 2). combining machine learning and causal inference methods to decompose health disparities; 3). applying natural language processing models to autocode occupational text data. The event will occur in 223 Raitt (the Demography Lab) and on Zoom (register here). Learn more about the talk in the full story.

Title: From Job Descriptions to Occupations: New Natural Language Processing Models for Automated Coding

Abstract: Occupation is a fundamental concept in social and policy research, but classifying job descriptions into occupational categories can be challenging and susceptible to errors. Traditionally, this involved expert manual coding, translating detailed, often ambiguous job descriptions to standardized categories, a process both laborious and costly. However, recent advances in computational techniques offer efficient automated coding alternatives. Existing autocoding tools, including the O*NET-SOC AutoCoder, the NIOCCS AutoCoder, and the SOCcer AutoCoder, rely on supervised machine learning methods and string-matching algorithms. Yet these autocoders are not designed to understand semantic meanings in occupational write-in text. We develop a new autocoder based on Google’s Text-to-Text Transfer Transformer (T5) model. Like GPT and other large language models, T5 is pretrained on vast amounts of text data. We develop a T5-based occupational classifier (T5-OCC) model with fine-tuned model parameters and training data from occupation write-ins from the 2019 American Community Survey. By comparing our T5-OCC with existing methods, we show that the autocoding accuracy rate increases from 61.8% to 71.1%. Considering the rapid change in neural language models, we conclude by offering suggestions on how to adapt our method for the development of occupational autocoding models in future research.