The managing partner asked one question in the first meeting. Which of our partners are costing us associates?
The firm had just over 900 lawyers across a dozen offices, 47 partners carrying real supervisory load, and four consecutive years of engagement survey data. Nobody in the room could answer the question. Not because the data was hidden, but because the instrument had never been built to answer it.
What the firm already had
An annual engagement survey, run by the professional development team on a platform the firm licensed. Firm-wide results, cut by office and practice group. Participation sitting at 54 percent, which leadership read as respectable for a voluntary survey.
Two problems sat inside that 54 percent, and neither was visible from the results deck.
The first is arithmetic. A survey with 54 percent participation is not a smaller version of the same population. The people who decline are not a random half. In upward review work the associates least likely to respond are consistently those with the most specific criticism, because they are the ones with the most to lose from being identified. What comes back skews toward the partners who were already fine.
The second is structural. The platform was licensed by the firm, administered by the firm's own staff, and hosted on infrastructure the firm's IT function could reach. Associates were told their answers were confidential. Every one of them understood that confidentiality was a policy the firm had written about data the firm held. A diplomatic answer is the rational response to that arrangement, and so is no answer at all.
The first independent cycle
The change in the first cycle was custody, not question design. Responses went to SRA's infrastructure rather than the firm's. The invitation said so plainly, named who would hold the data, and said what the firm would and would not receive.
| Measure | In-house administration | First SRA cycle |
|---|---|---|
| Associate participation | 54% | 87% |
| Free-text comment volume | Baseline | 2.4x |
| Partners with enough responses to report individually | Under half | Nearly all |
The participation figure matters for a specific reason. Below a certain number of respondents per partner, no individual report can be issued without making those respondents identifiable. At 54 percent, more than half the partnership sat under that threshold, which is why four years of surveys had never produced a single partner-level finding. At 87 percent, most partners cleared it.
Comment volume is the more useful of the two numbers. Ratings show where a problem sits; comments show what it is. A 2.4x increase in free text meant people had decided the process was worth their time.
What the data said
Three findings shaped everything that followed.
Feedback quality was the largest retention gap. Measured as importance to retention minus the mean score, the benchmark ranking across participating Am Law firms puts it first:
| Dimension | Gap |
|---|---|
| Quality of partner feedback | -28 |
| Clarity of path to partnership | -24 |
| Work allocation fairness | -19 |
| Action taken on survey results | -17 |
| Compensation transparency | -11 |
Compensation transparency sat at the bottom. That surprised the management committee, which had spent the previous eighteen months on compensation communication. The firm had been solving its fifth-largest problem with considerable energy.
The tenure curve showed where the risk sat. SRA indexes engagement by tenure band across participating client firms, which gave the firm a shape to read its own results against:
| Tenure | Indexed engagement |
|---|---|
| Year 1 | 82 |
| Year 2 | 68 |
| Years 3 to 4 | 47 |
| Years 5 to 6 | 55 |
| Partners | 74 |
The trough falls exactly where an associate has become useful. They can run a matter day to day, they carry the working knowledge, and they have now worked for enough partners to compare them. They are also the most expensive people in the building to replace. BigHand's 2025 research across more than 800 firm leaders put the cost of replacing a third-year associate above one million dollars once recruitment, lost billable revenue and training are counted.
The firm average concealed the distribution. This was the finding that changed the conversation. Firm-wide, feedback quality scored close to the benchmark. Partner by partner, the spread ran nearly forty points. A small number of partners were carrying scores that would be indefensible in any other part of the business, and their practice groups showed the attrition to match. Several were strong originators, which is why the pattern had survived four years of surveys.
What the firm changed
Nothing moved until each finding had someone accountable for acting on it.
| Change | Owner | What it replaced |
|---|---|---|
| Every partner report goes to the partner and to their practice group leader | Practice group leaders | Reports sent to the partner alone |
| Firm-average benchmark printed beside every score | Professional development | Raw scores with no reference point |
| Supervisory dimensions enter the annual compensation discussion | Management committee | Financial contribution only |
| Two named behaviors per partner, revisited at the next cycle | Partner and group leader | General encouragement to improve |
| Counsel included in the reviewed population | Professional development | Associates and partners only |
The third row was the difficult one. Putting supervisory behavior into the compensation conversation was contested for a full cycle before it was adopted. The objection was not that supervision does not matter. It was that the data would not survive a challenge from a partner who disagreed with it.
That objection is reasonable and it is the reason methodology matters more than reporting design. What made the data defensible was that it was administered independently, that participation was high enough for individual reporting to clear the threshold, that the threshold was agreed before fielding rather than after results, and that cuts falling below it were withheld rather than estimated. A partner disputing a score was disputing a documented instrument rather than an opinion.
Once the upward review findings entered the compensation conversation, partner take-up of voluntary leadership development rose 50 percent. Nobody was compelled into coaching. The conversation stopped being optional, and development became the obvious response.
Coaching against something specific
The practical difference showed up in the conversations practice group leaders had to have.
Before, a leader could tell a partner that the feedback had been mixed. That is not actionable and it invites a defensive response, because it sounds like a judgment rather than an observation.
After, the same conversation started from a score, a benchmark and two behaviors. Feedback timing sits at 41 against a firm average of 63. Associates report that assignments arrive without context. Here are the two things to change before the next cycle. A partner can argue with a characterization. A distribution is harder to argue with.
Practice group leaders reported that these conversations became shorter and less adversarial once the benchmark was on the page. That determines whether the conversation happens at all, since leaders avoid conversations they expect to go badly.
Counsel, and the population nobody reviews
Counsel are frequently invisible in law firm review architecture. They are not on the partnership track in the conventional sense, they often supervise substantially, and many review systems treat them as neither reviewer nor reviewed.
Adding counsel to the reviewed population had two effects. It gave the firm supervision data on a group carrying real day-to-day responsibility for associates. And it gave counsel a development track with documented criteria, which several had been asking for without a mechanism to request it.
Where only partners are reviewed, counsel absorb the supervision the partners have stopped doing, and the firm has no measurement of how well that goes.
What it cost to run
A fully managed cycle removed instrument build, distribution, reminder management, threshold application, suppression checks and report production from the professional development team. Firms running this way report completing a full cycle in roughly a fifth of the internal time the previous arrangement required.
For a team of three managing an annual cycle across a dozen offices, that is the difference between a program that runs every year and one that slips to every other year, then stops.
What it was worth
Set the measured outcomes against the published cost of the problem, and keep the two separate.
The measured outcomes are participation rising 33 points, comment volume rising 2.4x, and voluntary leadership development take-up rising 50 percent once findings reached the compensation discussion. Those are figures from SRA engagements, measured against each firm's own baseline before SRA took over administration.
The cost of the problem comes from published research. BigHand put firm-wide lawyer attrition at 27 percent and third-year replacement cost above one million dollars. The NALP Foundation's most recent data, released in April 2026 across 141 firms, found 83 percent of departures happening within five years of hire. Thomson Reuters found 61 percent of associates receiving useful feedback only a few times a year.
A firm of this size losing a handful of mid-level associates a year to supervision problems it cannot see is carrying seven figures of avoidable cost. That is arithmetic on published numbers rather than a measured result, and should be read as arithmetic. What it establishes is the order of magnitude the program is measured against.
The revenue effect runs through the same mechanism and is harder to isolate. Associates who stay bill more, carry matter knowledge that would otherwise be rebuilt at the firm's expense, and give clients the staffing continuity that outside counsel guidelines increasingly require in writing. None of that is attributable to a survey in a clean way, and any vendor presenting it as such is overreaching.
What made it work
Four things made this work, and the order matters.
Custody sat outside the firm. Everything downstream depends on associates answering honestly, and that is decided before a single question is written.
Someone other than the partner received the report. A finding delivered only to the person it concerns is a finding with no owner.
The benchmark was printed beside the score. A partner told their feedback timing scores 41 learns nothing. A partner told it scores 41 against a firm average of 63 learns where they stand.
The cycle came back round. Annual is the minimum cadence at which a partner can see whether a change registered. On a longer interval the exercise reads as an audit rather than as development.
What nearly derailed it
The suppression threshold derailed it twice.
The first time, at design, when a practice group leader wanted individual reporting on a group of four and could not have it. The second time, at readout, when the same leader asked for a breakdown that would have made three respondents identifiable.
Both times the answer was no. A threshold that moves under pressure is not a threshold, and associates compare notes on what came back to whom. The cost of bending it once is paid in the following year's response rate.
Agreeing the threshold in writing before fielding, sized against each population rather than set at a convenient round number, is what makes the refusal survivable. The conversation happens once, in advance, when nobody is arguing about a specific result.
What a firm should take from this
The instrument was not the hard part. Question design for upward reviews is well understood, and a competent professional development team can write a serviceable one.
The hard parts were custody, thresholds, ownership and the decision to let supervision data into a conversation where real money is decided. Three of those four are governance decisions rather than survey decisions, and a firm can make them before it selects a vendor.
The managing partner's question had an answer the whole time. Producing it required changing who held the responses, not what the responses were asked.
The client is not identified, at the firm's request. Outcome figures are approved SRA engagement metrics held under client contract, measured against the firm's own pre-engagement baseline and published anonymized. Where a table is labeled as a benchmark, the figures are SRA cross-firm data across participating Am Law firms rather than this engagement. Third-party figures are cited inline.
Sources: BigHand, "Navigating the Million Dollar Problem," 2025. NALP Foundation associate attrition data, released April 2026. Thomson Reuters, "Legal Talent and Career Development Report," 2024. SRA benchmark and engagement metrics, 2023 to 2025.