Writing & MediaFree
Documentation Audit Checklist and Report
River checks every name your docs assert against the product's own help output and changelog, then orders the fixes by how many readers hit each one.
River takes three things the docs cannot check themselves: the product's own help output, the changelog for the window, and ninety days of pageviews. Then it joins them. Every flag, command and endpoint your pages name goes against the surface the binary reports, and the three counts that fall out are facts rather than judgements: documented and real, documented and gone, real and undocumented. On the worked set that came out 195, 26 and 23.
Then the second join, which is what makes the backlog tractable. A finding on a page nobody reads and a finding on the quickstart are the same row in every audit checklist and not the same problem. Ordering the 38 wrong pages by monthly reads instead of by last-modified date changes what you fix first: the ten stalest pages carry 1 percent of the wrong reads, the ten busiest wrong ones carry 79 percent.
Written for whoever owns a documentation set that has a product moving underneath it: technical writers, developer-experience teams, and the engineer who inherited the docs folder. Run it after a release, or before you promise a rewrite. It reads the pages you publish; the spec gap workup reads the document those pages are generated from and finds fields nobody ever filled. The two findings look alike in a Sheet and have opposite fixes: one is a correction, the other is a first draft.
Accuracy is the one thing on a docs page you do not have to score
Most documentation audit checklists ask you to rate accuracy on a scale of one to five, then multiply that by an impact score and divide by effort. Scoring is the right move when nobody can check the underlying fact. Reference material is the case where somebody can, and Diátaxis puts it plainly: reference is led by the product it describes, and its only purpose is to describe it. Accuracy there is not a score. It is a lookup. Either the name is still there or it is not.
The lookup exists because a command-line tool describes itself. Python's argparse generates the help text from the parser, so the help output cannot drift from the binary the way a page can. The changelog is the second source: a file following Keep a Changelog already separates what was removed from what changed, which is the list of things your pages may now be wrong about. If yours is a dump of commit messages, write it for a reader first. Ordering a new integration's setup steps is the developer guide and integration pack.
Which is why the date column misleads. On the worked set the stalest third of the pages carried a falsehood 26 percent of the time and the freshest third 29 percent. The pages that get edited most are the pages the product keeps changing underneath. 22 of the 38 wrong pages had been edited inside ninety days, and those 22 carried 90 percent of every wrong read in the set.
How it works
Send the docs
Paste your page inventory with URLs, or point River at the repository the documentation lives in.
Add the outside sources
The help output or spec, the changelog for the window, and a pageviews export if you have one.
Run the joins
Every name the docs assert is looked up, and every finding is tied to the release behind it.
Work the queue
A Sheet with one row per page, and a Doc putting the fixes in reader order.
What you get
- One row per page with its findings, its monthly reads and the release that caused each
- The name join in three numbers: documented and real, documented and gone, real and undocumented
- Findings split into falsehoods a reader acts on and absences that only fail to help
- Monthly reads landing on a page that carries at least one false statement, as a count
- A remediation order by reads rather than by date, with what the first ten pages clear
- Each finding traced to the release note behind it, so the correction arrives with its value
- The pages that check out clean, named, so nobody audits them again next quarter
- The sweep's own coverage: what was checked, what could not be, and which areas went unread
Common questions
Do we need pageview data for this?
No, and it changes what you get. Without it you get the findings and a severity order, which puts a removed flag on a quickstart above a stale default in an appendix. With it you get the reads landing on each wrong page, which is the number that settles arguments about what to fix first. The output says which one it used.
Why not just sort by last updated?
Because that column records editing, not checking, and the two come apart. On the worked set the freshest third of the pages carried a falsehood slightly more often than the stalest third. The ten oldest pages held 1 percent of the wrong reads. The ten busiest wrong pages held 79 percent.
Is this the same as a content audit?
It shares the inventory step and nothing after it. A content audit scores pages on clarity, structure and search performance, all of which are judgements. This one asks whether the names on the page still exist, which has an answer. The style pack handles the prose side, and which noun means which concept is the terminology and glossary pack.
What if the product has no command-line tool?
Then the surface of record is whatever describes it without a human in the loop: an OpenAPI document, a generated client, a configuration schema. The join is the same shape. If that surface is itself thin, the spec gap workup is the prior step, because auditing pages against an incomplete document only moves the gap.
How long does the audit itself take?
By hand, longer than anyone budgets. The worked set held 812 checkable assertions, three minutes each to settle against the binary or a release note, so 41 hours before a word is written. 91 percent of that confirms nothing is wrong. The review that let them ship is the docs as code workflow pack.
Does it rewrite the pages it finds?
No. It reports the finding, the current value and the release note behind it, which is what a writer needs to make the correction. On the worked set that was 38 pages, about 16 hours of writing against 41 hours of checking. Publishing the corrected set is the API documentation pack.
How often should this run?
After any release that removed or renamed something, which semantic versioning says is a major bump. Between those, a quarterly pass over the busiest fifty pages catches most of it. Better than any cadence is not needing one: the release notes pack maps each change to the pages it breaks while the diff is still open.
Documentation Audit Checklist and Report
Fill in the form and your workspace opens with the work already underway.