River
Y CombinatorBacked by Y Combinator
Log InStart Free
FREE TEMPLATE

Product Metrics Definition Document

A metric is defined when you can name the events that compute it, the filters that run, and the platforms those events actually reach.

Edit with AIDownload Word and CSV

Free download  ·  No account needed

What you get

Pack of 6 files: 3 documents, 3 spreadsheets

Product Metrics Definition Document
  • How a Metric Is Traced to Its EventsDocument
  • Definition StandardDocument
  • Change PolicyDocument
  • Metric RegisterSpreadsheet
  • Instrumentation CoverageSpreadsheet
  • Known Discrepancy LogSpreadsheet

Two dashboards publish Weekly Active Teams for the same week and return 4,812 and 5,376. The usual response is a meeting about what active means, and it never works, because both teams already agree on the sentence. What they disagree on is which events count. This pack settles that the only way it can be settled, by writing every metric as an expression in event names rather than as a description, and then going and checking what those events actually do.

Every row in the register carries two columns a metrics document never has. Coverage is the share of your users on which every event a metric needs actually fires, so a metric whose event set skips Android is not undercounting Android, it cannot see it. Comparable From is the date the last of those events landed everywhere inside that coverage set. Before that date the line on the chart contains a rollout, and a rollout looks exactly like growth.

Send the dashboards, the weekly report and the board pack, plus whatever describes your instrumentation and roughly how your sessions split across platforms. Back come a register, an event-to-platform map and a log of every pair of surfaces that disagree, each gap decomposed to the one clause causing it. The pack assumes your events exist and mostly work. If you are not sure they do, audit the events first, because this reads instrumentation rather than repairing it.

Eighteen published metrics, traced to 41 events

Metric Register, the event-to-platform map underneath it, and every pair of surfaces that disagree with the gap decomposed.

Product Metrics Definition Document
RiverMetric Register

Metric Register

18 metrics, 6 dashboards, 41 events

Illustrative, for a fictional collaboration product called Amberline. Sessions in the week of 9 Feb 2026: 214,000 across web 51.2%, Android 18.4%, iOS 17.9%, desktop 12.5%.

MetricEvent expressionAll events fire onCoverageComparable fromVerdict
Paid Conversionssubscription_startedserver100%11 Apr 2023Clean
Support Contact Ratesupport_ticket_createdserver100%11 Apr 2023Clean
Weekly Active Teamsworkspace_opened OR doc_edited OR task_completed OR comment_posted OR mention_createdweb, desktop63.7%17 Sep 2024Partial coverage
Collaboration Depthcomment_posted, mention_created per teamweb, desktop63.7%17 Sep 2024Partial coverage
Notification Click-throughnotification_opened / notification_sentweb51.2%22 Jan 2025Partial coverage
Docs Created per Active Teamdoc_created / weekly active teamsweb, Android, desktop82.1%4 Jun 2025Partial coverage
Onboarding Completionsetup_finishedrenamed 2 Oct 2025--Dead event
Activation Rateactivated / signups, denominator disputedserver100%11 Apr 2023Two definitions

The count is the finding

Seven of eighteen traced clean, and every one of the seven is written server-side. Six carry a coverage gap, three depend on an event that was renamed or stopped firing, and two are charted from before they were comparable. Nothing computed from a client SDK survived the pass.

RiverInstrumentation Coverage

Instrumentation Coverage

Keyed by event, so a rename is a filter

One row per event rather than per metric. The question asked in practice is “what breaks if we rename this”, and keying by metric turns that into an investigation.

EventWebiOSAndroidDesktopCoverageMetrics
subscription_startedserverserverserverserver100%M-03
workspace_opened6 Jun 202314 Nov 202314 Nov 20236 Jun 2023100%M-08, M-12
task_completed17 Sep 202428 Jan 202512 Mar 202517 Sep 2024100%M-08, M-11
comment_posted17 Sep 2024nevernever17 Sep 202463.7%M-08, M-09
mention_created17 Sep 2024nevernever17 Sep 202463.7%M-08, M-09
notification_opened22 Jan 2025nevernevernever51.2%M-14
setup_finishedstopped 2 Oct 2025stopped 2 Oct 2025stopped 2 Oct 2025stopped 2 Oct 20250%M-15

Why the dates matter more than the ticks

Weekly Active Teams counts five events. Four of them reached every platform, but comment_posted and mention_created never shipped on mobile, so the whole metric drops to web and desktop: 136,318 of 214,000 sessions, 63.7%. It became comparable on 17 Sep 2024, and the exec chart starts on 1 Feb 2024.

RiverDiscrepancy Log

Known Discrepancy Log

Every gap decomposed to one clause

Closed rows stay. A closed row is the reason a metric is clean today, and deleting it leaves the next reader with no idea the two surfaces ever disagreed.

MetricSurface ASurface BGapThe one clause they differ onStatus
Weekly Active TeamsExec 4,812Growth 5,376564, 11.7%The event set. Exec counts three events, Growth counts fiveOpen
Activation RateExec 38.4%Growth 44.1%5.7 ptsThe denominator. All signups against verified signupsOpen
Time to First ValueExec 4.2 daysProduct 6.8 days2.6 daysThe aggregation. Median against mean on an identical populationOpen
Seats ProvisionedExec 18,204Revenue 17,916288, 1.6%Revocation. Gross assignments against assignments minus revocationsClosed 14 Aug 2025

Weekly Active Teams, 564 teams, both halves named

421 teams whose only qualifying action that week was a comment or a mention, which the three-event Exec expression does not count. Plus 143 teams dropped by an Exec filter on a surface property Android never sends. 421 and 143 make the whole 564, and neither half is a wording problem.

The +7.1% week nobody questioned

Weekly Active Teams moved from 4,364 to 4,676 in the week of 16 Mar 2025, the largest week-over-week jump in a 24-month series. task_completed shipped on Android v4.18 on 12 Mar 2025.

Edit with AI

What is in the pack

01

Metric Register

One row per published metric: the aggregation, the event expression, the filter clauses, the platforms every required event fires on, coverage, and the date it became comparable. The expression is written in event names, so it can be checked and it can be wrong.

02

Instrumentation Coverage

One row per event, not per metric, with first-fired dates by platform and the metrics that depend on it. Even a vendor's own built-in events are annotated by platform, which is why this has to be a real map rather than an assumption.

03

Known Discrepancy Log

Every pair of surfaces publishing different numbers for the same metric, with the gap decomposed to the clause causing it and the arithmetic closing. Closed rows stay, because a closed row is the reason a metric agrees with itself today.

04

Definition Standard

The shape a definition has to take before it counts as one: expression, filters, window, population, coverage and comparable-from date. Enough that a second team implementing it independently gets the same number, which is the only test that matters.

05

Change Policy

What happens the next time an event is renamed, a filter is added or a platform ships late: who is told, which series get restated, and how a comparable-from date moves. Written before the next change rather than argued about after it.

06

How a Metric Is Traced to Its Events

The method, plus ten prompts that run it: read every surface publishing a number, trace, map events to platforms, compute coverage, recompute the metrics that appear twice, and answer the which number is right question when it arrives.

How it works

  1. 1

    Send every surface that publishes a number

    Dashboard exports, the weekly report, the board pack, alerting thresholds, lifecycle audiences. The metric list comes off the surfaces people actually read, not out of the document that claims to define them, because those two lists are never the same.

  2. 2

    Each metric becomes an expression

    River rewrites every metric as an event expression with its filter clauses, window and population made explicit. Where a metric cannot be traced, it is recorded as untraced rather than guessed at, because a guessed expression is a wrong answer wearing a right one's clothes.

  3. 3

    Events get mapped to platforms and dated

    Every required event is mapped to the clients it fires on and the date it started. Coverage is the intersection expressed against your session split; comparable-from is the last of those dates. Both land in the register beside the metric they qualify.

  4. 4

    The disagreements get decomposed

    Any metric appearing on two surfaces is recomputed both ways and the difference is split into named parts that add up. You get the standard and the change policy alongside, so the same argument does not restart the next time an event moves.

Frequently asked questions

How is this different from a KPI dictionary?

A dictionary holds a sentence per metric: name, owner, plain-language definition, formula, source, refresh cadence. Two teams can implement the same agreed sentence faithfully and still publish different numbers, because the sentence never named the events. This register holds expressions in event names, and adds the two columns that decide whether an expression is even computable.

What does Coverage actually measure?

The share of the population a metric claims to describe on which every one of its required events fires. Any missing event removes the whole platform, because a team there cannot satisfy the expression no matter what they do. It is measured against the claimed population, so a deliberately mobile-only metric sits at 100%.

Why date a metric from when it became comparable?

Because instrumentation ships gradually and charts do not. Apple releases an update to 1% of users on day one and 100% on day seven, so an event added in a mobile release lands over a week rather than at once. Before the last platform is in, the series mixes behaviour with rollout, and a rollout looks like growth.

Our metrics doc already lists formulas. Is that enough?

Check whether the formulas name events. A formula written as activated divided by signups is a shape, not a definition, and Amberline's two dashboards both implemented it correctly to get 38.4% and 44.1%. The numerator was identical at 3,164 accounts. The 1,064 unverified signups in one denominator were the entire gap.

Will this fix our broken events?

No, and it should not pretend to. This reads instrumentation and tells you what your current numbers can and cannot support. Fixing the events themselves is a separate pass over volumes, property completeness and duplicates, which is what the instrumentation audit does.

What if a metric turns out to be uncomputable?

It gets recorded as untraced with the specific event that would fix it, rather than filled in with something plausible. In the worked example one event, shipping on two mobile clients, would have taken six partial metrics to full coverage. That is a roadmap item with a number attached, not a caveat.

Where does this sit next to our OKRs and experiments?

Upstream of both. An OKR written on a metric with 63.7% coverage is measuring two thirds of the company, and an experiment readout on a metric that changed definition mid-flight is reading a rollout. Trace first, then set targets against something that holds still.

Sources

  1. Google HelpAutomatically collected events (opens in a new tab)
  2. Apple DeveloperRelease a version update in phases — Update your app — App Store Connect (opens in a new tab)

Written by

Chandler Supple

Co-founder & CEO, River

Chandler has spent the last six years working in AI, building tools for millions of users and Fortune 100 companies. In 2025, he co-founded River to build a new kind of operating system for AI agents.

LinkedInGitHubX

Reviewed by

Brian Joseff, Co-founder, River

LinkedInX

Related

Analytics instrumentation audit

The pass underneath this one: which events are broken, duplicated or unused, before you build definitions on top of them.

Funnel and retention analysis

What you do once the definitions hold still, and the first place a coverage gap shows up as a step that looks worse than it is.

OKR and outcome pack

Where the traced metrics end up as targets, with the coverage figure carried across rather than quietly dropped.

More free templates

Accessibility Audit Remediation Plan

Free remediation plan that regroups your axe or WAVE export by the component that rendered each selector, then ranks the fixes by what they actually block.

Beta Program Plan Template

Free beta program plan template that counts who can actually produce the evidence each exit criterion asks for, then turns every gap into a date or a rewrite.

Concept Testing Plan Template

Free concept testing plan template that freezes the kill criteria before the first session and scores what participants did rather than what they said.

Continuous Discovery Habits Template

Free continuous discovery template built on one insight log that never resets, so week twenty adds to week one instead of rediscovering it.

Find out how many of your metrics trace clean

Send the dashboards and whatever describes your instrumentation. The first thing back is the count, and the metric with the worst coverage.

Trace my metrics