River
Y CombinatorBacked by Y Combinator
FREE TEMPLATE

Product Analytics Export Runbook

Four documents and four sheets that prove an event export is complete, keyed to the clock each platform actually measures its window against.

Free download  ·  No account needed

Completeness Check Method

An export that finished is not an export that is complete. Four steps, or the verdict is unverified.

1. Get the platform's own count for the window

Ask the tool how many events it thinks are in the window, on the same clock the export used. Record the surface you read it from. A count you cannot source is not a control total.

2. Count the exported rows the way the format is shaped

Newline delimited JSON counts by line, after decompression. An archive of many files counts as the sum across every file, because one absent file reads as a smaller number rather than as an error.

3. Decompose the difference, and do not net it

Rows missing from the export and rows extra in the export are two counts. Their sum is gross misplaced and it sets the verdict. Their difference is the net, reported beside the gross and never on its own.

4. Set the verdict

VerdictWhen
CompleteGross misplaced is zero against a sourced count, and the second count is in
ShortGross misplaced is above zero, with a gap open and a named owner
UnverifiableThe tool gives no count on the clock the export used

There is no fourth verdict. A tolerance band is how a five percent boundary error becomes permanent.

Every guide to pulling event data out of a product analytics tool covers the same ground: authenticate, page through the results, retry on a rate limit. What actually breaks the load sits upstream of all of it. A date range is not a set of events, because each export surface measures its window against a clock, and three of the four surfaces this pack documents measure it against arrival rather than against the event. Nothing in the response tells you which one you got.

The differences are documented, just never in one place. Amplitude's Export API states that its date range refers to when the data was uploaded to its servers. PostHog's batch exports use the time a row landed in the database, and explain that an event arriving late would otherwise never be exported by any run. GA4's BigQuery export dates events properly and then keeps editing a daily table for two calendar days plus today. Only Mixpanel windows purely on event time.

So the reconciliation carries the anchor timestamp on every row, and it compares like against like. An arrival windowed export is measured against an arrival windowed count, never against the dashboard number somebody quoted in a meeting. That is the same discipline the metric definitions pack applies to a metric and this pack applies to a window, and it is why a verdict here can honestly read Unverifiable instead of green.

The week that reconciled and was still wrong

Row Count Reconciliation, the Export Log behind it, and the Platform Limits every count is checked against.

Row Count Reconciliation

Illustrative rows for a fictional fitness app, Ridgeline. One week, 7 to 13 September.

SurfaceAnchorPlatformExportedNetMissingExtraGrossVerdict
MixpanelEvent, Denver742,318694,659-47,65947,659047,659Short
MixpanelEvent, Denver742,318742,3180000Complete
MixpanelEvent, UTC738,632742,318+3,68618,20421,89040,094Short
AmplitudeUpload time388,440388,4400000Complete
AmplitudeEvent, redistributed391,204383,915-7,2897,28907,289Short
GA4 dailyEvent date2,671,5302,664,108-7,4227,42207,422Short
GA4 dailyEvent date2,671,5302,671,5300000Complete

Row three is the reason this sheet never nets. The net is 3,686 rows, half a percent, and it passes any tolerance check. The gross is 40,094 rows on the wrong day.

Export Log

One row per run, including the runs that were superseded.

RunWindowTZOffsetChunkRequestsRowsErrors
MX-0914-0109-07 to 09-13Denver-6One day7694,6590
MX-0914-0209-07 to 09-13Denver-6Half day14742,3180
MX-0914-0309-06 refillDenver-6Half day218,2040
AM-0914-0109-07 to 09-13UTC0Six hours28388,4401 timeout
GA-0914-0109-07 to 09-13Denver-6One table72,664,1080
GA-0917-0109-07 to 09-13Denver-6One table72,671,5300

The first Mixpanel run set a row limit of 100,000 in the belief it was a page size. Five of the seven days ran past it and each came back at exactly 100,000.

Platform Limits

Read from each vendor's own documentation, with the date it was read.

SurfaceAnchor timestampPer requestPaginationLate window
Mixpanel raw exportEvent time, project TZ or UTC by project agelimit max 100,000NoneNot published
Amplitude Export APIserver_upload_time4GB, then 400NoneAbout 2 hours
GA4 BigQuery dailyEvent date1M events/day standardNot applicable2 days plus today
GA4 streamingArrivalNo volume limitNot applicableBest effort, gaps
PostHog batch exportTime the row landed in the databaseDestination setNot applicableLands in a later run

Three of these five window on arrival, and each names a different arrival clock. That is why the anchor column exists before any count does.

What you get

01

Platform Limits

One row per export surface, carrying the window parameter, the anchor timestamp, the per-request ceiling, whether pagination exists at all, the rate limit and the documented late window, each with the date you read the vendor's docs.

02

Row Count Reconciliation

One row per window per surface per count. Rows missing and rows extra are separate columns whose sum sets the verdict, and the net sits beside the gross rather than replacing it.

03

Export Log

One row per run, with the window timezone and its UTC offset as a signed number. A run you superseded stays in the sheet, because it is what explains a refill six weeks later.

04

Gap Register

Every Short verdict attributed to a documented limit rather than a guess, with a Recurs Until column for the gaps that are the shape of the surface and will never close.

05

Export Procedure

The sequence per surface, written from each vendor's current documentation: Mixpanel, the Amplitude Export API, GA4's BigQuery export and PostHog batch exports, with the ceilings that bite on each.

06

Which Clock the Window Uses

Event time against arrival time, what each surface actually windows on, and why an export and a dashboard number can be several percent apart with nothing broken.

07

Completeness Check Method

Four steps and three verdicts, with the boundary error worked through in full. Complete, Short and Unverifiable are the only allowed answers, and there is no tolerance band.

08

When a Recount Is Required

The late arrival interval per surface, the event time redistribution step for arrival windowed exports, and the rule that a second count never edits the first.

How it works

  1. 1

    Send your export setup

    The script, the scheduled job, a sample of the output, or just the tool name and the error. River fills Platform Limits from each vendor's own current documentation, one row per surface.

  2. 2

    Settle the anchor timestamp

    The one column everything downstream depends on. A range that refers to data being received, uploaded, ingested or landed is an arrival window, whatever the parameter happens to be named.

  3. 3

    Plan the chunk against the peak

    Chunk size and rate limit are one decision. River sizes the chunk against your busiest period rather than the average, then costs the request count in wall clock time before anything runs.

  4. 4

    Reconcile, then recount

    The four step check, with the difference decomposed into rows missing and rows extra. Then a second count after the surface's late window closes, which is what turns a verdict into a closed one.

Frequently asked questions

Is this free, and what format are the downloaded files?

Free, and no account is needed for the download. Documents arrive as .docx and sheets as .csv, so they open in Word, Pages, Google Docs, Excel, Numbers or Sheets with no conversion step. Edit with AI is the other button: it installs the same pack as a private space and fills it in from your own export setup.

How do I chunk a Mixpanel export without losing events?

By narrowing the window, because there is nothing else to narrow. Mixpanel's raw export takes a `limit` whose value cannot be over 100000, and its parameter list carries no cursor, offset or page parameter to walk that limit forward with. Leave the limit unset and size the date chunk so the busiest period fits under it.

Why don't my exported row counts match the numbers in the tool?

Usually because the two were measured against different clocks. Most export surfaces window on when data arrived, while every dashboard is built on when the event happened, so the two populations genuinely differ. The pack records the anchor on both counts and refuses to compare across them, the way the conflicting report tool refuses to average two disagreeing reports.

Why decompose the difference instead of just reporting it?

Because two errors in opposite directions cancel. In the worked week, a six hour timezone offset left 18,204 events missing from one end of the window and 21,890 extra at the other. The net is 3,686 rows, half a percent, and it passes. The gross is 40,094 rows on the wrong day.

How long should I wait before taking a second count?

It depends on the surface, and the pack lists what each one publishes. A GA4 daily table gets updated for up to two calendar days plus today, so the fourth day is the earliest honest count. Amplitude data becomes exportable roughly two hours after its servers receive it. Mixpanel publishes nothing, so measure it on your own project.

Does this space build the dashboard the data feeds?

No, and that boundary is deliberate. This one ends at a verdict on a window. Naming the metrics comes first, in the metric definitions pack, and writing the requirements for what gets built comes after, in the dashboard spec tool. Putting a recurring check on the window belongs in the data quality monitoring pack.

Find out whether last week's export was complete

Send your export setup and one control total. River records which clock each surface windows on, decomposes the difference, and gives the window a verdict you can defend.

Edit with AI