ttok 1.0
ttok 1.0 ships with GPT-5/6 tokenizer default after community experiment confirms GPT-6 tokenizer unchanged
“All seven GPT models (5.5, 5.6 Sol/Terra/Luna, 6 Astra/Sol/Luna) report 44,794 tokens and match each other on every one of the 31 fixtures. GPT-6 introduces no input-count change on this corpus.”
Simon Willison released ttok 1.0, a CLI token-counting tool, updating its default tokenizer from GPT-4 to the GPT-5/6 family. The release surfaces a community-run experiment suggesting GPT-6 uses the same tokenizer as GPT-5 variants, a detail OpenAI has not officially confirmed. This is a minor developer tooling release with a small but useful side finding about GPT-6 tokenizer continuity.