Token benchmarks & methodology

Every percentage on this site comes from the benchmark below — one corpus, one tokenizer, defined baselines, per-file results.

Everything here is measured

How we count

Counts use OpenAI’s o200k_base tokenizer (via tiktoken, offline). The raw per-file numbers behind every table on this page — token counts, page counts, source hashes and the derived percentages — are published as benchmark-v1-results.json. One caveat up front: o200k is OpenAI’s tokenizer, not Anthropic’s — Claude tokenizes differently, and absolute counts vary by model. We use one tokenizer as a consistent estimate because the ratios between formats are far more stable than the absolute counts, and ratios are what we publish. The token badge in the app is an estimate for the same reason.

A second caveat: with API prompt caching, repeated reads of the same attachment get cheaper, so the dollar saving shrinks on later turns. The context-window saving — how much of the model’s working memory your document occupies — does not shrink.

A third, for the many people who never see a per-token bill: on a fixed monthly plan the saving is not money at all. Plans ration tokens rather than charge for them, so a smaller document buys headroom — more documents before a usage limit, longer conversations before the window fills. Every percentage on this page is a token reduction, and tokens are what the app counts.

Web pages: 72–95% fewer tokens

Baseline: the page’s raw HTML source, as fetched — what you’d paste if you saved the page. Five real public pages, chosen to span the range from lean text to pages that are mostly interface:

Lean documentation pages sit near 72%; pages that are mostly interface, like GitHub, approach 95%. If you paste browser-selected text instead of page source, your baseline — and your saving — lands somewhere in between.

PDFs: a measured 71–84% fewer tokens vs native upload

Baseline: attaching the PDF natively, priced by Anthropic itself. Each corpus PDF was sent to Anthropic’s token-counting endpoint, which returns the real input-token count for a document without running inference. Both sides of the comparison are counted, not assumed. Measured against claude-opus-5 on 13 August 2026; token counts are model-dependent, so the model and the date are part of the number. Raw results: benchmark-v1-measured-pdf.json.

Text-sparse pages save more, because the fixed per-page cost dominates. Scanned PDFs go through on-device OCR first and then follow the same arithmetic. Across the corpus the measured overhead works out at roughly 1,700–1,880 tokens per page beyond the Markdown text — the page image, plus whatever the extractor emits that our Markdown does not.

Spreadsheets: CSV beats Markdown tables by 12–29%

Baseline: identical cell data as a Markdown pipe table vs as CSV. This is the format where Markdown loses: the benchmark measures 12–29% fewer tokens for CSV over the identical cell data. Three generated sheets, each built from a fixed seed so every run produces exactly the same data:

Numeric-dense grids save the most; text-heavy tables the least. Milldown’s CSV output mode exists exactly for this: Markdown tables when a human reads them, CSV when a model does.

Word & PowerPoint: no saving claimed

You will see other tools claim ~97% savings for Office files by comparing Markdown against the raw OOXML inside the file. Our corpus reproduces that number — 94.1% and 96.8% on two real documents — and we still don’t claim it, because no chat workflow pays that baseline: every tool extracts the text before the model sees it. Against what you’d actually otherwise send, a .docx costs about the same in Markdown. What Milldown adds for Office files is structure (real headings and tables), cleanup, and measurement — not a token discount.