Writing & MediaFree
Documentation Audit Checklist and Report
River checks every command, setting and endpoint your docs name against the product's reference and release notes, then ranks fixes by how many readers hit each.
River takes the sources your docs cannot check themselves against: the product's own reference (its help output, API description or settings file), the release notes for the period, and, if you have it, a pageviews export. Then it joins them. Every command, setting and endpoint your pages name goes against what the product reports about itself, and three counts fall out that are facts rather than judgements: documented and real, documented and gone, real and undocumented. On the worked set that came out 195, 26 and 23.
Then the second join, which is what makes the backlog tractable. A finding on a page nobody reads and a finding on the quickstart are the same row in every audit checklist and not the same problem. Ordering the 38 wrong pages by monthly reads instead of by last-modified date changes what you fix first: the ten stalest pages carry 1 percent of the wrong reads, the ten busiest wrong ones carry 79 percent.
Written for whoever owns a documentation set that has a product moving underneath it: technical writers, developer-experience teams, and the engineer who inherited the docs folder. Run it after a release, or before you promise a rewrite. It reads the pages you publish; the spec gap workup reads the document those pages are generated from and finds fields nobody ever filled. The two findings look alike in a Sheet and have opposite fixes: one is a correction, the other is a first draft.
Accuracy is the one thing on a docs page you do not have to score
Most documentation audit checklists ask you to rate accuracy on a scale of one to five, then multiply that by an impact score and divide by effort. Scoring is the right move when nobody can check the underlying fact. Reference material is the case where somebody can, and Diátaxis puts it plainly: reference is led by the product it describes, and its only purpose is to describe it. Accuracy there is not a score. It is a lookup. Either the name is still there or it is not.
The lookup exists because many products describe themselves. Python's argparse generates a command-line tool's help text from the code, so the help output cannot drift from the product the way a page can. The changelog is the second source: a file following Keep a Changelog already separates what was removed from what changed, which is the list of things your pages may now be wrong about. If yours is a dump of commit messages, write it for a reader first.
Which is why the date column misleads. On the worked set the stalest third of the pages carried a falsehood 26 percent of the time and the freshest third 29 percent. The pages that get edited most are the pages the product keeps changing underneath. 22 of the 38 wrong pages had been edited inside ninety days, and those 22 carried 90 percent of every wrong read in the set.
How it works
Send the docs
Attach the pages or give the public docs address, plus a pageviews export if you have one.
Add the outside sources
Add the product's own reference, such as help output or an API description, and the release notes.
Run the joins
Every name the docs assert is looked up, and every finding is tied to the release behind it.
Work the queue
A Sheet with one row per page, and a Doc putting the fixes in reader order.
What you get
- One row per page with its findings, its monthly reads and the release behind each finding
- The name join in three numbers: documented and real, documented and gone, real and undocumented
- Findings split into falsehoods a reader acts on and absences that only fail to help
- Monthly reads landing on a page that carries at least one false statement, as a count
- A remediation order by reads rather than by date, with what the first ten pages clear
- Each finding traced to the release note behind it, so the correction arrives with its value
- The pages that check out clean, named, so nobody audits them again next quarter
- The sweep's own coverage: what was checked, what could not be, and which areas went unread
Common questions
Do we need pageview data for this?
No, and it changes what you get. Without it you get the findings and a severity order, which puts a removed flag on a quickstart above a stale default in an appendix. With it you get the reads landing on each wrong page, which is the number that settles arguments about what to fix first. The output says which one it used.
Why not just sort by last updated?
Because that column records editing, not checking, and the two come apart. On the worked set the freshest third of the pages carried a falsehood slightly more often than the stalest third. The ten oldest pages held 1 percent of the wrong reads. The ten busiest wrong pages held 79 percent.
Is this the same as a content audit?
It shares the inventory step and nothing after it. A content audit scores pages on clarity, structure and search performance, all of which are judgements. This one asks whether the names on the page still exist, which has an answer. The style pack handles the prose side, and which noun means which concept is the terminology and glossary pack.
What if the product has no command-line tool?
Then the reference of record is whatever describes the product without a person writing it: an OpenAPI description of the API, a generated client library, or a settings schema. The join is the same shape. If that reference is itself thin, the spec gap workup comes first, because checking pages against an incomplete reference only moves the gap.
Why not check the pages by hand?
Because most of the work confirms nothing is wrong. The worked set held 812 checkable statements; at three minutes each to settle against the product or a release note, that is 41 hours before a word is corrected, and 91 percent of it finds no problem. The join does that checking and leaves the writer the 38 pages that need fixing.
Does it rewrite the pages it finds?
Not in the audit. It reports each finding with the current value and the release note behind it, which is what a writer needs to make the correction; afterwards you can ask River to draft the fixes. On the worked set that was 38 pages, about 16 hours of writing against 41 hours of checking. The docs as code workflow pack keeps them from drifting again.
How often should this run?
After any release that removed or renamed something, which semantic versioning says is a major bump. Between those, a quarterly pass over the busiest fifty pages catches most of it. Better than any cadence is not needing one: the release notes pack maps each change to the pages it breaks while the diff is still open.
Sources
Built by
Chandler Supple
Co-founder & CEO, River
Chandler has spent the last six years working in AI, building tools for millions of users and Fortune 100 companies. In 2025, he co-founded River to build a new kind of operating system for AI agents.
Documentation Audit Checklist and Report
Describe what you need, add any files you have, and your workspace opens with the work already underway.