Equitable performance reviews

During my time at Ramp, one of my proudest accomplishments was co-leading a revamp of the performance review cycle. At the time, I didn't find many examples of how other companies ran this process, so I'm writing this in the hopes this may help others.

Performance reviews at larger companies

At Facebook, we ran calibration / performance reviews every 6 months. In practice, this meant every half, the entire company would fully focus on this for a 2-3 week period. As an IC (individual contributor), I would write the following:

• Self review: This was a comprehensive listing of everything I worked on in the half, sorted by the impact those projects had. Most people agreed the best way to approach this was to maintain a document throughout the half (a brag sheet) that tracked all your projects and the corresponding impact, so you didn't have to dig up artifacts from months ago. Even with a brag sheet though, reviewing & editing this took painstakingly long.
• Peer reviews: Since I was a junior engineer, I'd only receive peer review requests from folks directly on my team. At the time, I typically wrote 1-2 peer reviews, with each taking a few hours. I also knew that my reviews weren't weighed as heavily as the more senior engineers on the team.
• Upward review for my manager: This was the simplest of the three – typically a few bullet points on what a manager did well, and what they could improve on.

For managers, this process was far more time-consuming. In addition to receiving significantly more peer review requests, managers would write "downward reviews". These were written after all other feedback for a given report was submitted, and meant to serve as a summary for everyone else in calibration. It wasn't until I started writing these myself later in my career that I realized how taxing these were, primarily because they were meant to encapsulate the entire packet, and because they were weighed the most heavily.

Finally, after all feedback was collated, the "calibration" process began – a process meant to ensure that regardless of which part of the organization you were in, the performance criteria you were evaluated on for promotions and ratings was consistent.

At Facebook, these meetings were limited to managers & the most senior ICs on each team. Each org would run its own calibration process, then the org leads would meet, recursively moving up the ladder until all of leadership had signed off.

Performance reviews at startups

At startups, the picture is far different. Standardized performance reviews are unnecessary if a startup is small. It isn't worth standardizing levels & compensation until you hit a certain scale, and introducing them earlier is a distraction from the most important problem, derisking the business as a whole. This doesn't mean that ignoring performance – instead managing performance at a small startup is typically done on a person-by-person basis.

Somewhere in the 20-100 person range, it probably makes sense to introduce more standardized compensation bands, levels, and a lightweight performance review process. There's a few problems this looks to solve:

• Pockets of the org developing different philosophies on performance.
• New hires wanting the "same level" as other offers.
• Compensating those around the same skill level within a consistent band.

In terms of content, self / peer / upward reviews cover the majority of bases for a holistic understanding of performance. For the calibration commmittee, the goal is to make this group as small as possible. This encourages deeper discussions any fewer decisions by committee. For any given department between 20-50 people, a group of 4-5 leaders can reasonable review all the packets in a timely manner.

Finally, with all the context set, this brings us to Ramp.

Ramp

Ramp started out with the lightweight process described above – self/peer/upward reviews, with a calibration group consisting of all M7 (Director) + IC7 (Principal Engineer)s. This process was sustainable when Ramp was smaller, but by the time I was promoted to Director, there were 200-300 engineers at Ramp, and the calibration group was also roughly 20 people.

The contents of the packets weren't an issue – self/downward/upward reviews are standard, and the packets themselves were comprehensive. The problems arose when "live calibration" happened, or when packets were discussed live. There were roughly ~2 weeks budgeted for the live calibration process, with a day or two budgeted for each level.

Given thoroughly reading all the packets (200-300) was near-impossible, discussions were shallow and we routinely ran out of time for a given level. There was also significant time wasted on questions about a packet that could've been ironed out first if there had been a pre-review process.

Alongside Lisa (on the People team) & Max (my manager), we brainstormed a new process that would scale as the engineering org grew at Ramp:

1. Self/upward/peer reviews are submitted
2. Manager reviews are submitted, serving as an executive summary of the entire packet
3. Async calibration begins, with level leads for each level. Two level leads are assigned to each packet.
4. Live calibration is reserved only for packets where the level leads cannot align with the manager, and the specific disagreements are brought to a larger group.

Steps (1) & (2) are fairly standard and commonplace at nearly all companies. (3) contains the bulk of the changes, which we'll delve into below.

Level leads & async calibration

One of the terms we introduced was a "level lead", defined as a person would be responsible for calibrating packets at a given level (alongside others). The first cycle we implemented the process, we had roughly ~4 level leads per level. To make this more concrete:

• IC3: A, B, C, D
• IC4: E, F, G, H
• IC5: ...

Then, for a given IC3 packet, two level leads would be selected to review the packet's contents. Whenever possible, we'd avoid the packet's management chain, but this wasn't always possible for more senior candidates (IC6+). Each packet would have a combination of [rating + promo decision] – so for an IC3 it could be [meets expectations + no promo], or [greatly exceeds expectations + promo].

Each level lead would read through the packet, either agree with the decision, or ask the manager / relevant director additional questions to align. If both level leads agreed with the manager's recommendation, calibration for this packet would end. If alignment couldn't be reached, the specific questions would be flagged for live calibration.

This system works because each level lead is only reviewing packets for a specific level, ensuring the bar we set for a given level is consistent. There aren't any single points of failure since each packet has to go through two approvals, and in working through the entire set of people, level leads reach consensus with each other on what each rating corresponds to (the pairings are random, so A would work with all of B/C/D throughout the process).

You may be curious how this system handles promo decisions, which are the most contentious & impactful decisions that are made during the calibration process. At Ramp and at many companies, a minimum rating is required to be eligible for promotion. Concretely, an engineer underperforming at their level isn't eligible for promotion.

Therefore, for someone recommended for promotion from IC3 -> IC4 would first need their rating approved by the IC3 level leads, then their packet reviewed by the IC4 level leads (or N+1 level leads) to ensure they meet the bar at the next level. This codifies the expectation that promotion candidates must be operating at the next level to be promoted, which can be inconsistent in other calibration processes.

Live calibration

Finally, for cases that cannot be fully resolved by the async process, they are brought to live discussion. When we ran this process at Ramp, this was typically <10% of cases for a given level. Then, for an engineering team of 200 people, only ~20 cases would be discussed live, split across a few days.

As part of the changes we made, we also limited the live calibration group to ensure discussion was substantive. Attendance was limited to senior directors (M8+), the relevant level leads, and the org lead for the packet. The packet & relevant comments were a pre-read, and the resulting discussion was far more specific and productive.

Closing thoughts

I'd again like to thank Lisa & Max for their partnership in coming up with this, which was one of my proudest accomplishments during my time at Ramp. I hope this can be a helpful resource to any other companies & leaders looking to improve their calibration process.