Privacy

Raw conversations never leave the device.

This benchmark needs two numbers per interval: how many tokens were spent, and how far the quota moved. Neither requires knowing what anyone was working on, and the collector is built so that it cannot find out.

During the validation experiment, the collector uploads nothing automatically. It appends to a file in the home directory and stops there. The only data that has ever left the machine is the aggregate published on the live page.

Recorded locally

Appended to ~/.subworth/snapshots.jsonl, one line per status line refresh.

Timestamp of the reading
Quota window name, percentage used, and reset timestamp
Session identifier, used only to keep one session's readings together
Model id and display name
Client version

Uploaded, only on request

When contribution ships, sending data will be a separate, explicit command — never automatic.

provider, plan, model
input_tokens, output_tokens, cache_tokens
quota_before, quota_after, quota_window, resets_at
client_version, timestamp

Never collected, never uploaded

Prompts, responses, or any conversation content
Source code, file contents, file paths, project or directory names
Email addresses, account identifiers, OAuth tokens, API keys

This is enforced by construction, not by policy: the collector copies an explicit list of fields out of the data the client hands it, rather than storing what it was given. A field added upstream is dropped by default — it has to be allowed in deliberately.

This website

What the published file contains

The site is static. It is built from one generated JSON file, which holds only aggregates:

Hourly percentage-used curve per quota window
Snapshot and session counts, as counts
Aggregate window statistics: range, distinct values, smallest step, session spread, resets crossed
Experiment day, collection span, client versions, model names
Rolling dollar-estimate aggregates per window: sample count, mean and spread of dollars per percentage point

It contains no session identifier — not even a hashed one — no path, and no free text of any kind. Every string in the file is a timestamp, a window name, a client version, or a model name. That property is checked by a test in the exporter, so it cannot quietly stop being true.

Website traffic analytics

This website uses Vercel Web Analytics to send page-view data to Vercel and measure visitors, page views, referral sources, and aggregate device and geographic information. These traffic statistics are separate from the benchmark collector and contain no prompts, conversations, or local usage logs. See the Vercel Web Analytics privacy documentation for details.

Contribution

When telemetry opens up

Community measurement is what makes a benchmark more than one person’s account, but it is an addition, not a prerequisite — the first numbers will come from probe accounts we pay for ourselves.

Contributors will be identified by a random identifier generated on the device and rotated through a salt before it is sent, which is enough to group one person’s samples during aggregation and not enough to link them to anything else. Passive collection will be opt-in at install time and visible in the status line the whole time it runs.