Software & TechnicalFree
Add Docstrings Where Reviewers Already Asked
River mines your review threads for the questions people actually asked about the code, then drafts the comments those questions were asking for.
Search this and page one is a run of guides agreeing with each other: point a model at the repository, generate the module documentation first, then function-level docstrings, review the drafts, wire drift detection into the pipeline. It is reasonable advice and it ends with a docstring on every function. Ravensley already had one on 2,671 of its 3,180 functions, coverage read 84% against an 80% gate, and nobody in the building could tell you what the rate card function did.
Ravensley is an illustrative freight booking platform. Four years of its pull request threads hold 21,400 review comments, 3,120 of them questions, each one blamed back to the function it was asked about. Forty-seven functions carry 1,402 of those questions. That is 45% of every question the codebase has ever drawn, sitting on 1.5% of it. And 1,180 of the 3,120 had already been answered on the same function, in an older thread.
Built for whoever inherited the code, and for the reviewer answering the same question a third time. Run it on a service, on one package, or on the twenty files a migration is about to touch. What the whole estate does is an inherited codebase documentation pass, the tables underneath it are a schema documentation pass, and the diff in front of you is a change summary. The public surface is its own job, which is an API documentation pack.
Why a docstring on every function makes it worse
The 84% is a number a coverage tool can produce, and what it counts is presence. Cross each docstring against the identifier and the signature directly above it, and 1,908 of Ravensley's 2,671 add nothing either of those had not already said. The one-line docstring should not be a signature reiterating the function parameters, which can be obtained by introspection, and PEP 257 said so decades ago. Substantive coverage at Ravensley is 763 functions. That is 24%.
Complexity and churn are the standard ranking, and they point the right way. Across 39 proprietary codebases and 30,737 files, resolving issues in low quality code takes on average 124% more time in development. Used alone, though, the product of the two put generated protocol buffer stubs, schema migrations and test fixtures into eight of Ravensley's top ten, because all of those churn hard and none of them is read by a human. Readership is the missing term.
The comment worth writing states something the code cannot. Ravensley has 337 of those sites: 214 numeric literals outside any named constant, 63 retry counts and sleeps, 41 handlers catching a named exception and carrying on, and 19 orderings that matter with nothing saying why. Thirty-nine of the 47 hot functions hold at least one. A drafted docstring describes the parameters of a function like that and leaves the question that was actually asked standing.
How it works
Bring the code
The repository, or one package of it. Tests are worth including, since they answer some questions.
Add the threads
Pull request history, four years of it if you have that. The evidence of confusion lives here.
Read the ranking
Functions ordered by questions asked, with the repeats and the reasons the code cannot state.
Review the drafts
One comment at a time, beside the thread it came from, so you are checking a claim.
What you get
- Every review question blamed back to the function it was asked about, then counted
- Functions ranked by questions asked, so effort lands where the confusion is documented
- Questions already answered in an older thread, flagged as the answer worth writing down
- Docstrings restating the name and signature counted apart from the ones saying something
- Generated code, migrations and fixtures held out, because churn there is not readership
- Magic numbers, retries, swallowed exceptions and orderings listed as reasons code cannot state
- Drafted comments written against the thread that asked, crediting the reviewer who answered
- Complexity and churn kept as a second ranking, for code no reviewer has questioned yet
Common questions
Can we not just generate one for everything?
You can, and Ravensley did. Reviewing one drafted comment per function at six minutes each is 318 hours of somebody's attention, and it reproduces the problem, because a draft starting from the signature says what the signature said. The 47 cost 4.7 hours and answer 45% of every question the codebase has drawn.
Our review history is somewhere you cannot read.
Then the ranking falls back to complexity, churn and distinct authors, and the report names which ranking it used. That is a weaker signal and worth saying out loud. On Ravensley the fallback put eight generated or fixture files in its top ten, so the filter that holds those out is the part worth keeping.
The comments will go stale.
Some will, and the ones that rot fastest describe behaviour, which is why this writes as few of those as it can get away with. A comment recording why a retry count is seven, or which upstream bug a handler exists for, survives a refactor intact. The code was never going to say either of those things.
Is this not what code review is for?
It is, and Ravensley's reviewers did it 3,120 times. The answer went into a thread, the thread got merged, and 1,180 of those questions were asked again on the same function. Review reliably produces the answer. Nothing in the process then puts it where the next reader will look, which is next to the code.
We are not a Python codebase.
The mechanism does not care, because it runs on review threads and blame rather than on syntax. What changes is the convention the output is written to, so a TypeScript repository gets TSDoc and a Go one gets doc comments. The pass follows whatever the repository already does rather than importing a house style with it.
What stops it inventing a reason?
Every drafted comment carries the thread it came from and the reviewer who answered, so you are checking a claim against a source rather than trusting a summary. Where no thread explains a magic number, it writes the question rather than an answer and names who would know. An unanswered question like that belongs in the debt register rather than in a comment.
What do we actually get back?
A sheet with one row per function carrying questions asked, distinct askers, complexity, changes, authors and a priority, plus a document holding the drafted comments beside their evidence. Ravensley's head of the list drew 31 questions from 14 engineers at complexity 34 across 96 changes, and had no docstring at all.
Add Docstrings Where Reviewers Already Asked
Fill in the form and your workspace opens with the work already underway.