River
Y CombinatorBacked by Y Combinator

Software & TechnicalFree

API Design Review Checklist for OpenAPI

River reads the specification and sorts every finding by what fixing it later costs: free, free until the beta ships, or a new version.

Start here

River reads the specification rather than a description of it, then puts a clock on every finding. Three answers only: fixable at any time, fixable free until the endpoint has clients, or already needing a new version. That last column is what separates a design review from a style guide, because a finding with no deadline is a preference and a finding with one is a decision you are making by not making it. Every finding names the endpoints it touches.

Netherby, a workforce-scheduling platform, ran its 84-operation specification through this. It came back with 47 findings, and the split was the useful part: 16 could be fixed whenever, 9 were free only until the private beta went public, and 22 had already missed the moment. A checklist would have ranked all 47 by severity and put the cheapest permanent risk on the list near the bottom, which is where it had been sitting for nineteen months. The 9 with a deadline are the ones worth an argument this week.

Built for the engineer proposing a v1 that will outlive them, and for the reviewer asked to approve one. Run it before the first external client, and again before any beta goes public. What the specification fails to document is a separate question, answered by the spec gap workup. A permanent finding you decide to live with is a decision and belongs in an architecture decision record, and a v2 worth designing properly starts as an RFC.

Findings with a deadline you did not write down

Some fixes stop being available and there is no warning when they do. Google's API design guidance says collections must provide pagination at the outset, because adding it to an existing method is a backwards-incompatible change. The reason is quieter than a broken client. An old caller that expected everything gets a first page instead, does not know it, and reports a number that is wrong rather than an error that is loud. Netherby had three unpaginated collections. Two were public.

The same guidance is explicit about the rest. It says an API must not change the format or the algorithm behind an existing field's value, even one the server alone populates, and that doing so requires a new version. It also rules out adding a required field to an existing request. Netherby's 61 timestamp fields ran in two formats, 44 as RFC 3339 strings and 17 as Unix integers. Every one of the 17 sat on a public endpoint, so the inconsistency is now a property of the API.

The cheapest fix on the list is usually a sentence. Stripe documents whether each of its enums is open or closed, and says new values can be added to an open one without an API version. Netherby declared 23 enums and said this about none of them, though 14 appear in responses where a client may already be switching exhaustively. The error bodies were worse: four shapes across 84 operations, and none using the registered problem media type.

How it works

  1. Send the spec

    The OpenAPI or Swagger document itself, whole, since the counting runs across all of it.

  2. Mark what shipped

    Which paths are public and which are not, because that is where the deadline comes from.

  3. Say who calls it

    Internal callers, named partners or an open developer programme, since each changes what breaking means.

  4. Work the deadlines

    The findings with a date first, then the permanent ones you are choosing to accept.

What you get

  • Every finding classified by what fixing it later costs, not by severity
  • The findings that are free today and a new version once the beta ships
  • Each inconsistency counted across the whole document rather than asserted as a preference
  • The majority convention named, with every operation that departs from it listed
  • Enums split by whether they appear in a response, where adding values breaks callers
  • Endpoints affected per finding, so the size of the migration is visible
  • A finding register carrying the breaking-change classification and the affected endpoint list

Common questions

We already run a spec linter.

Then the style rules are handled and the question this asks is not one a linter has an opinion on. A linter tells you 29 fields are in the wrong case. It cannot tell you that 23 of them are on endpoints with clients, so renaming them is a version, and the other 6 are free until next month.

How does it know what has clients?

You tell it, at the path level, and that answer does most of the work. If you cannot say, it treats anything published as having clients and says which paths that assumption covers. Traffic logs sharpen it further, because an endpoint documented for two years and never called is closer to free than to permanent.

Consistency is subjective though.

The preference is. The count is not. Netherby had three pagination shapes across 24 collection endpoints, 15 cursor, 6 offset and 3 with none at all. Nobody chose that, and nobody could see it, because you only see it by reading all 24 at once. The review names the majority and lists the departures.

What do we do about the permanent ones?

Three options and the review costs each. Accept it and write down why, which makes it a decision instead of an oversight. Add a corrected sibling and deprecate the old shape, which the release notes then have to announce. Or bank it for a v2, in which case it goes on a list nobody has started yet.

Our API is internal. Does any of this apply?

Less of it, and the review asks so it can say so. An internal API you deploy with its callers can change a field name on a Tuesday, so most of the permanent column collapses into free. What survives is the counting: three pagination shapes are three shapes to whoever has to use them. When the callers are another team, what breaks on their side is a design review.

Does it rewrite the spec?

It writes the finding register and the document that argues the ranking, and it proposes the corrected shape for each finding as a fragment you can paste. It does not hand back a rewritten specification, because a specification you did not read is worse than the one you had.

We are designing v1 now and have no spec yet.

Then this is the cheapest hour you will spend on it, because every finding is in the free column and stays there until you publish. Sketch the resources and the collection shapes and run it on that. The reference documentation comes after the shape settles, not before.

API Design Review Checklist for OpenAPI

Fill in the form and your workspace opens with the work already underway.