Determining Residency Program Directors’ Opinions on Advanced Pharmacy Practice Grading Methods

Purpose

Residency programs’ perceptions of the different grading scales used to evaluate advanced pharmacy practice experiences (APPE) performance and reasons for the preferences were assessed.

Summary

The residency application process is becoming increasingly competitive as more students seek postgraduate training. Surveying current postgraduate year one (PGY1) residency program directors (RPDs) may provide insight into whether the type of APPE grading scale presented on a transcript is a consideration factor when selecting candidates to interview. A survey instrument was developed to gather information on residency programs’ opinions and preferences for the different APPE grading scales (letter, pass/fail, and pass/fail with honors/distinction). Surveys (n=1,203) were electronically distributed to PGY1 program RPDs. Of the 1,172 surveys successfully delivered, 225 were completed (response rate, 19.2%). Programs in 45 states were represented in the results. Most (80%) respondents preferred the letter APPE grading scale, with pass/fail with distinction/honors selected as the second most preferred scale. RPDs emphasized the ability to distinguish among candidates’ performance to be valuable. In general, many RPDs were unfamiliar with the pass/fail evaluation criteria and expressed that the lack of standardization made it difficult to compare candidates from different institutions.

Conclusion

The majority of RPDs surveyed preferred the letter APPE grading scale because they believed the grading scale was better able to differentiate among candidates and understood how grades were awarded. Colleges of pharmacy who use honors grading should consider providing details about how honors are assigned on the transcript.

Published in College of Pharmacy, Virtual Poster Session Spring 2020

Responses

  1. Did you ever look at students with lower passing letter grades compared to just passes? I noticed most of letter grades were still pretty high on average. Good work, and I I suspect it will have some practical impact on our own APPEs.

    1. Hi Dr. Henchey, thank you for your question! We did not provide a transcript with lower letter grades for a couple of reasons. We expected that a “Pass” would be preferred to a “C” and also used the letter grades to serve as a control as the grades of Candidate 4 should be preferred over those of Candidate 1 (we found that all RPDs preferred Candidate 4). We also wanted to limit the number of variables/comparison as well as the number of questions survey respondents would need to answer.

      ASHP has stated that residency programs should have implemented a standardized process to evaluate applicants from pharmacy schools using pass/fail grading systems, and it would be interesting to see if the current situation has any impact on the evaluation process!

  2. Young, this is an excellent poster and body of work. This is such important information for our College and others to know! I wonder if colleges in fact included information for how award of honors was decided if most people reviewing applications would really take the time to look at it?

    1. Hi Dr. Keefe, thank you for your question! That is an interesting and realistic point to consider, given the increasing number of applications residency programs must review every year! Based on the responses that we received, I believe that reviewers would review the information but may be more inclined to do so for cases where perhaps something seems inconsistent or when there are candidates of similar qualifications. RPD respondents mentioned a nationally standardized evaluation rubric would be ideal.

      I agree that this is an important consideration and expect there will probably be greater interest in this area of research given the current situation. More literature in this area would certainly be very helpful to guide colleges of pharmacies in the decision regarding which APPE grading scale to implement!

  3. Hi Young, great study I think this is a very interesting perspective. I think pass/fail with a ranking system could be a cool alternative to grades just due to the bias and problems with the grading scale for APPE rotations. Did your study look at if the current grading for APPE is bias or how different schools do these grades?

    1. Thanks, Taylor! We limited our focus to RPDs’ opinions regarding pass/fail grading scales and letter grading scale so we didn’t explicitly explore APPE grading bias or how different schools decide on which grading scale to implement. However, there have been some studies that I have come across while researching our topic that indicated there is APPE grade inflation as well as bias due to variability in evaluation rubrics, expectations of preceptors, etc. There weren’t many studies that I read which included information regarding how APPE grades were awarded except for one on pass with distinction/honors, which was done through nomination by the preceptor if he/she regarded the student to be one of the top 10% of students whom they had ever precepted and included a description explaining their nomination.

      What was interesting from our findings, though, was that many RPDs mentioned in the free text that they believed APPE grades are grossly inflated across the board (for all grading scales)! They also mentioned APPE grades could more significantly impact the evaluation process when they were either very good (pass with honors) or very bad (fail or “C” and lower).

  4. Hi Young! Awesome job on your project and poster. In your opinion, do you agree with the RPDs’ general preference for APPE letter grades over pass/fail?

    1. Hi Angie!! Thanks! I don’t know if I “agree” with the RPDs’ preference for letter grades but can understand why a majority of RPDs would prefer this scale based on the reasons they mentioned. I can see how it could be difficult to differentiate among and rank candidates with pass/fail. Many RPDs also were unsure of how pass/fail grades were awarded and also believed there would be a significant amount of variability in the evaluation (as with all APPE grading), which would add another dimension of confusion/uncertainty. I think it would be helpful for both students and programs if there was a way to minimize this confusion regarding pass/fail grading scales e.g., through a standardized national rubric!

  5. This is a really interesting finding and I am surprised by it. When I was interviewing, I had heard that more RPDs prefer pass/fail because you are less likely to “pad” your GPA with more A’s during clinical rotations! And that your GPA is more reflective of your classroom work.

    1. That’s so interesting to hear! In our study, the majority of RPDs stated preferring the letter grading scale mainly because they felt they were better able to understand how the candidates received the grades vs. pass/fail and due to the ability to distinguish among candidates (including eg, clinical competency). They did state they believed APPE grade inflation was pretty prevalent! I didn’t know that the RPDs viewed GPA that way – thanks for sharing!

  6. Hi Young, a very nice study that addresses a topic of interest to the Dean’s Office.

    1. Thank you, Dr. Herron! It was a great topic to explore, and I hope our findings can serve as a source of insight for our college!

  7. Young- great study and very interesting results. I would be curious to compare these results with the perspective of students and preceptors. Did you find any literature about what students and/or preceptors prefer?

    1. Hi Dr. Raber, thank you for your question!

      We found data in literature indicating that students preferred the pass/fail grading schemes vs. letter grades due to a reduction in perceived stress, leading to increased interactions on rotations and positive attitudes.

      On the other hand, a couple of studies indicated that preceptors generally favored the letter grading system (at least initially, prior to exposure to the pass/fail grading system):
      -One study carried out at a college of pharmacy found that preceptors were concerned that a pass/fail grading scale would lead to reduced student motivation and performance. However, after moving to a pass/fail system for one year, they found that student motivation and performance were not affected and that preceptors’ perceptions of student motivation did not change.
      -A 2018 study evaluating preceptors’ preferences for the grading scales found that a majority preferred the grading scale they had been evaluated with in pharmacy school, which was the letter grading scale, due to 2/3 of the colleges of pharmacy using letter grades for IPPEs/APPEs. This was mainly due to a familiarity with this system. Our RPD findings were pretty consistent with this study, although many responses seemed to suggest their preferences could change if they had a better understanding of the other scales or if evaluations were standardized nationally for all grading scales.

      Based on the limited data in literature, students seem to prefer the pass/fail grading scales and preceptors view the scales as acceptable after gaining an understanding of and exposure to them!

  8. Super interesting! As a student, I think these findings would lead me to be concerned about receiving pass/fail grades instead of letter grades for APPEs, unless it was something that was uniformly implemented across all colleges of pharmacy. It seems like the pass/fail system could put you at a significant disadvantage compared to the alternatives, regardless of your actual performance. I love this topic!

    1. Thanks, Brynne! It was a fascinating topic to explore! You hit the nail on the head – RPDs mentioned variability in the grading criteria across schools, a poor understanding of how grades were awarded/how students were evaluated, and that the pass/fail grading scale could make it more difficult to distinguish among candidates. Uniformity on some level (RPDs mentioned a standardized grading rubric) would definitely be very helpful!

  9. Congratulations on addressing this important issue. I think it is an important question that anyone who evaluates potential residents has encountered. You have done an amazing job not only on this project, but in your other important projects.

  10. Great work Young! This is very informative and a question that many have been or are curious to know.

Comments are closed.