What Is a Token in AI? A Plain-English Guide
Tokens explained simply: what they are, how ChatGPT, Claude and Gemini count them, and why they decide both your cost and your context limit.
Tokens in one sentence
Language models do not read letters or words; they read tokens, short pieces of text such as 'pre', 'par', 'ation'. Common words are often one token, rare words and code are split into several.
Why tokens matter to you
Every model has a context window measured in tokens and every API bills per token. Long prompts full of whitespace, repeated headers, or HTML tags burn through both without adding meaning.
- Cost: more tokens, higher bill.
- Limits: more tokens, sooner you hit the context ceiling.
- Quality: more noise tokens, less attention on what matters.
Measure before you send
Paste any text into the Token Counter to see its exact count for modern GPT tokenizers. If the number surprises you, the Context Compressor can usually cut 15 to 40 percent without changing the meaning.
Try it free in your browser
- Context Compressor — Reduce token usage with compression strategies tuned for LLMs.
- Token Counter — Live token, character, and context-window meter for any prompt. Uses official tiktoken encoders.
Tools mentioned
Frequently asked
How many words is 1,000 tokens?
Roughly 750 English words, though code, numbers, and other languages use more tokens per word.
Do spaces and punctuation count?
Yes. Whitespace, punctuation, and formatting characters all become tokens, which is why messy text costs more.
Keep reading
How to Prepare PDFs for ChatGPT, Claude, and Gemini
A practical guide to extracting clean, AI-ready text from PDFs — born-digital and scanned — so ChatGPT, Claude, and Gemini answer accurately and don't waste tokens on headers, footers, and page numbers.
The Best Way to Feed Long Documents to Claude (and Other Long-Context Models)
Claude's 200K-token context is generous, but you'll still want to clean, compress, and structure long documents before sending them. Here's a step-by-step playbook.
Cleaning Whisper Transcripts for AI Summaries
OpenAI Whisper, Otter, and YouTube transcripts are full of timestamps, filler words, and speaker noise. Here's how to strip them before sending to ChatGPT or Claude — and why it matters.