Welcome to Science with Shrike! Today we will discuss one of the key aspects of scientific success: letters of recommendation. These are essential for careers and awards in the academic realm. Despite the critical role these documents play, there is very little training around writing them. This leads to some letters that are useless, despite the recommender’s best intentions.
A “useless” letter of recommendation
Letters of recommendation serve one of two functions, depending on your beliefs. On one hand, a letter helps the reader neutrally evaluate the candidate for the position and make an informed decision on hiring them/giving them an award. On the other hand, the letter sells the candidate to the evaluator to persuade the reader to hire them/give them an award. In practice, doing a good job of the latter will also convince people you did the former.
While these camps primarily disagree on how persuasively you should write, they can agree on the most useless letters of recommendation. These are the letters that do not give useful information about the candidate. There is nothing concrete for someone to evaluate, and without that concrete information, also fail to sell the candidate.
On the surface, these letters look positive. They use lots of good adjectives and try to hit all the key buzzwords. But when you’re done reading them, you realize the name and/or research project could have been cut and pasted in with no change. They’re generic. This appears to Shrike to be the standard for letters of recommendation coming out of India. They even bold name and project to make it completely clear they spent 2 sec copy/pasting for their letter of recommendation.
Generic letters of recommendation are not helpful.
So what makes the letters more helpful?
Sales.
Exactly what people in science never explicitly learn. People in the first camp shrink back in horror. A letter of recommendation …actually recommending someone? Nevermind that in most cases (tenure is the notable exception), the applicant solicits the referees. The only way an applicant gets a negative referent is if they are struggling to find references, or a prospective referent secretly dislikes the applicant and wants to tank their chances. In both cases, the letter is likely to be more generic, and/or subtly undermine the points the referent claims to be making. It’s more ethical just to say no if you do not plan to put the effort into selling the applicant.
We need to stop for a moment and define sales/selling. Sales to most academics calls to mind used car salesmen using every greasy trick in the book to get you to sign, or the scammer trying to make a huge profit at your expense. Shrike used to believe this, too. The Bowtied Jungle and some of the various sales experts within (eg @bowtiedbull, @bowtiedsalesguy) have helped provide key perspective on sales. Sales being something other than scammy, evil and/or dishonest work was a tough mindset shift for Shrike. Now Shrike believes making this shift to see sales as positive is a necessary adjustment to succeed.
This video helped Shrike with the change in perspective:
Sales is transferring emotions to the recipient. And humans are all emotional creatures. Regardless of your intent, you will convey some emotion in your letter, even if that emotion is ‘this person wasn’t worth my time to write for’. As a result, you will write stronger letters when you lean into this. When you end up evaluating letters, you will hear summaries of good letters like ‘the letter writers were enthusiastic’. We use emotion to summarize the quality of the letter… which should tell you the letter is informing an emotional decision. Write to that, anon.
Emotions for a letter of recommendation
For a letter of recommendation, you need to address the reviewers’ pain, be credible, and excite the reviewers. The pain is more challenging in many letter of recommendation settings since you will not know the specifics of the person reading the letter. You may or may not know the department or school. Also, addressing a medical school’s or prestigious award committee’s pain is more challenging. However, there’s a set of 15 core competencies for medical school that you can address.
The best solution for discovering the pain? Serve on the awards or admissions committee. Getting onto the top awards committees is not practical starting out, but you can start with departmental awards committees and graduate student selection committees, and move up to society/foundation review panels and awards committees. When you serve on the committee, you have a few key objectives.
1) When you review applications, analyze the extent to which the letter impacted your decision-making process. Did it strengthen the application, weaken it, or have no effect? Which parts of the letter sold (or failed to sell) it?
2) When you’ve read a ton of applications, what do you start skimming the letters for? If you can articulate what you are seeking, you will be able to review faster, but you’ll also know what you’re looking for, and which words you just start filtering out.
3) At the committee meeting, STFU and listen. When the letters and their quality are discussed, what do other people mention about the letter? If you want more clear data, ask a few probing questions about why they liked the letter. Similarly, use this inside look to see what features sell the winning candidates, and what process narrows the candidates to the short list. Look at what people put up as the surface justification, as well as read the applications. See which candidates get championed by members of the committee.
4) Combine all of the above with some self-reflection on your emotions. Focus on how you feel and what you think the main challenges facing your organization. What are the challenges your organization faces? For example, suppose you are on a graduate student selection committee. What needs does the program (or lab) have? Is there key expertise missing? What kind of grant proposals and/or research projects could this student successfully pull off? How much and what kind of training would need to be invested for the student to be successful? Will this student contribute to changing the departmental culture? How will this student facilitate the PI’s (and department’s) success? Which of these problems are unique to your organization/situation vs generalizable to others?
You can try to simulate all of this by pretending to be in their position. However, simulations often miss key factors taught by experience.
Once you have a list of pain points identified, now you figure out which ones the applicant’s skill set and experiences can address. For these parts, you want to give specific, concrete examples. Remember, generic statements are not helpful. You need to show (with imagery and words) how the applicant will address those pain points. For example, suppose you are writing a letter for a technician position. One pain point in labs is organization and keeping things clean. If the student color-codes their notes and keeps meticulous records of their experiments, specifically mention that, as well as how it averted any major disasters in the lab.
How the student addresses the pain points will be the majority of the letter. However, there are two other things that need to be done in the letter, too. First, you need to establish your credibility. A flight attendant who is a long-time family friend is not going to be a credible reference for medical school. Unless you are a Nobel Laureate or a member of the National Academy, it is unlikely that the reviewers will know who you are. So you need to establish your credibility. This includes both your qualifications to make the statements you will make, and your relationship to the applicant. The other aspect of credibility is whether the reader agrees with your statements/assessments. For example, if you told Shrike someone was the best student you ever had, yet their research project was inconclusive/poorly controlled and never presented at a local meeting, that tells Shrike you have no clue what you are talking about. This is where superlatives need to be toned down to match a realistic assessment of the candidate. If they are excellent, you have primed the reader for an example of their excellence. Failing to show one will disappoint.
Hence med schools recommend that you do not advocate for the candidate. If it comes off as sales, you’re doing it wrong. Instead, you need to establish your credibility and convince the reader to trust your assessment of the candidate. One way to do this is to paint the picture, and let the reader draw your desired conclusions themself. If they make their own conclusions, it doesn’t feel like sales, and they get convinced.
This is also why credibility is usually close to the beginning of the letter. If the reader isn’t sure that you are credible, your recommendation is done. Comparisons also help with credibility, which is why med schools like those. For example, “out of 500+ undergraduates I have taught, I would rank X in the top 5%” The number of undergrads gives credibility, as does showing where the student falls. On one hand, not every student can be top 1-10%. On the other hand, if you have 10% A’s in your class, your A students are in the top 10%. When you only write letters for the top 10% of your students, these claims become easier to make. For graduate students, it gets harder, but you can compare to the whole department if necessary.
Status also helps sell credibility. If Joe Goldstein and Mike Brown (earned the Nobel prize for cholesterol metabolism; built UT Southwestern to be world-leader for cholesterol research) say that a student is one of their top students, not much more needs to be said in the cholesterol field. They would likely still paint a picture of the student to illustrate their statements because they know how to write letters of recommendation. However. Their status does as much selling as the words they write.
Please note that the status must also be combined with knowing the candidate! If for some reason Goldstein and Brown wrote a vague letter for a student who started in another lab 2 months ago, their letter would not help as much. In fact, ‘writer does not appear to know candidate’ is one of the ways readers disqualify letters of recommendation. Along with vagueness, getting key facts wrong is another way to reduce credibility. If you say the student is a Biology major, when they are really a Biochemistry major, it doesn’t help your credibility. Make sure to check statements against applicant’s CV/resume and with the applicant.
Last, you also need to address readers’ potential objections to the candidate. Any personal issues should be approved by the candidate (and indicate to the reader that you have the candidate’s permission to discuss the issue). Also, avoid potential objections that are not relevant to the reader. The student always drives at least 20 mph above the speed limit is unlikely to be relevant to a student award (unless it is for safe driving). Finally, with grades, you need to be careful since federal law (FERPA) technically does not allow you to disclose their GPA or grade without written permission.
In general, if you were looking at this candidate and you saw red flags, or anything concerning, they need to be addressed by you. The applicant cannot address red flags without sounding shifty, but the credible letter writer can. Some of these can even be re-framed to be positives. Student took a year off, lived with family and helped support them? Shows emotional maturity and service orientation by seeing to his family’s needs, even at the cost of delaying his future by a year.
What about excitement, Shrike? That was on your list before!
Yes, it was. The strategy to driving reviewer excitement is to do all of the above. Credibly present the applicant as solving the institutional pain points, while handling objections. Done correctly, the reviewers become excited about the applicant. Once they are excited, they will push for the one they like at the meeting. All that stuff @bowtiedsalesguy says about the clients selling themselves? It plays out at the institutional level, too.
However. There may be multiple exciting candidates that the reviewers have to narrow down to one. The other reviewers will make the case for the candidates they like, and FUD the ones they don’t like. This is why you need to credibly address any concerns or red flags… they will likely come up in committee, and/or in later voting for things like faculty hires. If you have the justification already written, the reviewer is armed with a rebuttal to that criticism. If you have quotable snippets, the favorable reviewer can highlight that for the committee. In some ways, your letter of recommendation is a way to arm a third party to advocate for your applicant, and shield them from the FUD.
Reviewers are also fickle, and excitement can be easily extinguished. This is why addressing potential FUD is also important. Shrike was excited by an interview candidate’s talk…. then during the final discussion learned that the candidate had been a postdoc for a decade. While the talk was good for a 3-4 year postdoc, someone a decade in should have had a better story. Worse for the candidate, the search committee suggested a poor publication record was “very good”. Shrike went to the PI’s letter of recommendation to look for mitigants. Nothing. Candidate went from being an excellent choice to someone Shrike was on the fence about. To be clear: the candidate gave a good seminar (job talk). The committee oversold the record… if they’d just listed publication record as a negative, it would not have drawn attention.
Some of these dynamics you cannot do anything about as a letter writer (or as a candidate). There are also politics that come into play, which are also challenging/impossible to predict without inside knowledge. The applicant may also have core weaknesses that simply make them less competitive than others. This is why applicants should not limit themselves to one school, one application, or only take one shot at an award. Instead, you do the best you can, which is to provide a credible explanation of how the applicant will fit the institution or award while handling objections.
Great post and thanks for the video shoutout! /Hedgehog