Token benchmarks & methodology

Every percentage on this site comes from the benchmark below — one corpus, one tokenizer, defined baselines, per-file results. Version 1, measured 8 August 2026. When we re-run it, this page and every number that cites it change together.

How we count

Counts use OpenAI’s o200k_base tokenizer (via tiktoken, offline). One honest caveat up front: o200k is OpenAI’s tokenizer, not Anthropic’s — Claude tokenizes differently, and absolute counts vary by model. We use one tokenizer as a consistent estimate because the ratios between formats are far more stable than the absolute counts, and ratios are what we publish. The token badge in the app is an estimate for the same reason.

A second caveat: with API prompt caching, repeated reads of the same attachment get cheaper, so the dollar saving shrinks on later turns. The context-window saving — how much of the model’s working memory your document occupies — does not shrink.

Web pages: 72–95% fewer tokens

Baseline: the page’s raw HTML source, as fetched — what you’d paste if you saved the page. Five real public pages, chosen for chrome-to-content diversity:

Page	Raw HTML tokens	Markdown tokens	Saving
GitHub — microsoft/markitdown	143,818	6,670	95.4%
Wikipedia — Markdown	97,305	17,771	81.7%
Wikipedia — Switzerland	542,470	134,106	75.3%
MDN — HTTP Overview	61,212	16,911	72.4%
Python docs — Tutorial	21,924	6,180	71.8%

Lean documentation pages sit near 72%; app-shell-heavy pages like GitHub approach 95%. If you paste browser-selected text instead of page source, your baseline — and your saving — lands somewhere in between.

PDFs: 68–90% fewer tokens vs native upload

Baseline: attaching the PDF natively, which providers bill as extracted text plus a per-page image — Anthropic documents roughly 1,500–3,000 tokens per page. The Markdown carries the same words as the PDF’s text layer, so the saving is that per-page image cost; we report both bounds rather than picking the flattering one.

Document	Pages	Markdown tokens	Native upload (est.)	Saving
arXiv 2506.16035 (research paper)	21	14,816	46,316–77,816	68.0–81.0%
Milldown technical design doc	20	9,406	39,406–69,406	76.1–86.4%
Keydock technical design doc	18	6,135	33,135–60,135	81.5–89.8%
Pricing research report	9	4,405	17,905–31,405	75.4–86.0%

Text-sparse pages save more, because the fixed per-page image cost dominates. Scanned PDFs go through on-device OCR first and then follow the same arithmetic.

Spreadsheets: CSV beats Markdown tables by 12–29%

Baseline: identical cell data as a Markdown pipe table vs as CSV. This is the format where Markdown loses — and where we correct ourselves: we previously published 32–39% for this saving; the v1 benchmark measures 12–29%, and the smaller number is the one we now claim. Three generated sheets (deterministic, shapes disclosed in the repo manifest):

Sheet	Markdown table tokens	CSV tokens	CSV saving
Transaction log (400×8)	15,298	11,901	22.2%
KPI grid (60×14, numeric)	5,108	3,607	29.4%
Personnel directory (150×6)	3,945	3,466	12.1%

Numeric-dense grids save the most; text-heavy tables the least. Milldown’s CSV output mode exists exactly for this: Markdown tables when a human reads them, CSV when a model does.

Word & PowerPoint: no saving claimed

You will see other tools claim ~97% savings for Office files by comparing Markdown against the raw OOXML inside the file. Our corpus reproduces that number — 94.1% and 96.8% on two real documents — and we still don’t claim it, because no chat workflow pays that baseline: every tool extracts the text before the model sees it. Against what you’d actually otherwise send, a .docx costs about the same in Markdown. What Milldown adds for Office files is structure (real headings and tables), cleanup, and measurement — not a token discount.