River
Y CombinatorBacked by Y Combinator

Software & TechnicalFree

API Design Review Checklist for OpenAPI

River reads the specification and sorts every finding by what fixing it later costs: free, free until the beta ships, or a new version.

Context to provide

River reads the specification rather than a description of it, then puts a clock on every finding. Three answers only: fixable at any time, fixable free until the endpoint has clients, or already needing a new version. That last column is what separates a design review from a style guide, because a finding with no deadline is a preference and a finding with one is a decision you are making by not making it. Every finding names the endpoints it touches.

Take an illustrative example: a scheduling platform we will call Netherby, with 84 operations. It came back with 47 findings, and the split was the useful part: 16 could be fixed whenever, 9 were free only until the private beta went public, and 22 had already missed the moment. A checklist would have ranked all 47 by severity and put the cheapest permanent risk on the list near the bottom, which is where it had been sitting for nineteen months. The 9 with a deadline are the ones worth an argument this week.

Built for the engineer proposing a v1 that will outlive them, and for the reviewer asked to approve one. Run it before the first external client, and again before any beta goes public. What the specification fails to document is a separate question, answered by the spec gap workup. A permanent finding you decide to live with is a decision and belongs in an architecture decision record, and a v2 worth designing properly starts as an RFC.

Findings with a deadline you did not write down

Some fixes stop being available and there is no warning when they do. Google's API design guidance says collections must provide pagination at the outset, because adding it to an existing method is a backwards-incompatible change. The reason is quieter than a broken client. An old caller that expected everything gets a first page instead, does not know it, and reports a number that is wrong rather than an error that is loud. In the Netherby example there were three unpaginated collections. Two were public.

The same guidance is explicit about the rest. It says an API must not change the format or the algorithm behind an existing field's value, even one the server alone populates, and that doing so requires a new version. It also rules out adding a required field to an existing request. In the Netherby example, 61 timestamp fields ran in two formats, 44 as readable date strings and 17 as raw second counts. Every one of the 17 sat on a public endpoint, so the inconsistency is now a property of the API.

The cheapest fix on the list is usually a sentence. A list of allowed values in a response, such as an order status, can only grow safely if callers were told from the start to expect new values. Otherwise a caller that handles every listed value breaks on the first new one. In the Netherby example there were 23 such lists and the documentation said this about none of them, though 14 appear in responses. Error bodies were worse: four shapes across 84 operations, and none using the standard problem details format.

How it works

  1. Send the spec

    Paste or attach the OpenAPI or Swagger file itself, whole, since the counting runs across all of it.

  2. Mark what shipped

    Say which paths are public and which are not, because that is where each deadline comes from.

  3. Say who calls it

    Internal callers, named partners or an open developer programme, since each changes what breaking means.

  4. Work the deadlines

    The findings with a date first, then the permanent ones you are choosing to accept.

What you get

  • Every finding classified by what fixing it later costs, not by severity
  • The findings that are free today and a new version once the beta ships
  • Each inconsistency counted across the whole document rather than asserted as a preference
  • The majority convention named, with every operation that departs from it listed
  • Lists of allowed values split by whether callers receive them, where adding a value breaks callers
  • Endpoints affected per finding, so the size of the migration is visible
  • A findings sheet showing which fixes would break existing callers, with every endpoint affected

Common questions

We already run a spec linter.

Then the style rules are handled and the question this asks is not one a linter has an opinion on. A linter tells you 29 fields are in the wrong case. It cannot tell you that 23 of them are on endpoints with clients, so renaming them is a version, and the other 6 are free until next month.

How does it know what has clients?

You tell it, at the path level, and that answer does most of the work. If you do not say, it treats anything published as having clients and says which paths that assumption covers. Traffic logs sharpen it further, because an endpoint documented for two years and never called is closer to free than to permanent.

Consistency is subjective though.

The preference is. The count is not. In the Netherby example there were three pagination shapes across 24 collection endpoints, 15 cursor, 6 offset and 3 with none at all. Nobody chose that, and nobody could see it, because you only see it by reading all 24 at once. The review names the majority and lists the departures.

What do we do about the permanent ones?

Three options and the review costs each. Accept it and write down why, which makes it a decision instead of an oversight. Add a corrected sibling and deprecate the old shape, which the release notes then have to announce. Or bank it for a v2, in which case it goes on a list nobody has started yet.

Our API is internal. Does any of this apply?

Less of it, and the review asks so it can say so. An internal API you deploy with its callers can change a field name on a Tuesday, so most of the permanent column collapses into free. What survives is the counting: three pagination shapes are three shapes to whoever has to use them. When the callers are another team, what breaks on their side is a design review.

Does it rewrite the spec?

It writes the finding register and the document that argues the ranking, and it proposes the corrected shape for each finding as a fragment you can paste. It does not hand back a rewritten specification, because a specification you did not read is worse than the one you had.

We are designing v1 now and have no spec yet.

Then this is the cheapest hour you will spend on it, because every finding is in the free column and stays there until you publish. Sketch the resources and the collection shapes and run it on that. The reference documentation comes after the shape settles, not before.

API Design Review Checklist for OpenAPI

Describe what you need, add any files you have, and your workspace opens with the work already underway.