A Token Is Not a Word, and That Distinction Matters
When an AI model reads your prompt, it doesn’t see words the way you do. It breaks text into chunks called tokens, which are often smaller than a whole word. The word “unbelievable” might split into three tokens: “un,” “believ,” and “able.” Common short words like “the” or “is” are usually a single token each. On average, in English, one token works out to roughly four characters, or about three-quarters of a word. That means 100 words of text is typically closer to 130-140 tokens, not 100.
Why This Is the Unit Everything Gets Priced In
AI providers charge per token, not per word or per question, because tokens are what the underlying model actually processes, regardless of language or formatting. This is also why a prompt in a language that needs more tokens per word (some non-English languages split less efficiently) can cost more to run than the English equivalent, even for the same meaning. If you’ve noticed API costs or usage limits that seem to shift depending on wording or language, token counting is almost always the reason.
The ‘Context Window’ Is Really a Token Budget
When a tool advertises a “128,000 token context window,” that’s the maximum combined size of everything the model can consider at once: your entire conversation history, any documents you’ve pasted in, and the response it’s about to generate. Once a conversation’s total token count approaches that ceiling, something has to give, usually the oldest messages get dropped or summarized, which is why long chats sometimes seem to “forget” something you said earlier. Pasting a long document eats into this same budget, leaving less room for conversation.
Why Terse Prompts Aren’t Always Cheaper or Better
It’s tempting to assume fewer words always means fewer tokens and a faster, cheaper response, but that’s only sometimes true. A dense, jargon-heavy sentence can tokenize into more pieces than a slightly longer, plain-English version, because uncommon words and unusual punctuation often split into more tokens than common ones. Code, non-English text, and emoji in particular tend to be token-expensive. If you’re optimizing for cost or speed at scale, testing actual token counts (most provider dashboards show this) beats guessing from word count alone.
One Practical Habit This Explains
If an AI tool starts giving worse answers partway through a very long conversation, the usual fix isn’t “try harder,” it’s to start a new conversation and re-paste only the essential context. You’re not fighting the model’s intelligence at that point, you’re fighting a token budget that’s already full of less relevant earlier messages.
