What is a token in AI?
A token is the small unit of text that an AI model reads and writes. A token is often a short word or a part of a longer word.
Key takeaways
- Models count tokens, not words or pages.
- In English, 100 tokens are roughly 75 words.
- Usage limits and prices for AI services are set in tokens.
How it works
Before a model reads your message, the tool cuts the text into tokens. Common words are one token. Rare or long words become two or more tokens. The model then works only with those tokens, both for what it reads and for what it writes.
Example
A 1,500-word report is roughly 2,000 tokens. If you paste it into a chat and ask for a 300-word summary, the request uses about 2,000 tokens of input and about 400 tokens of output.
Why it matters
Tokens explain two things you see in daily use. They set how much text fits in one conversation, and they set what a company pays when it uses a model through an API.
Common mistake
The rule of 75 words per 100 tokens is an estimate for English. Other languages, code and tables often use more tokens for the same content.
Related terms
See all terms in the AI Encyclopedia, the numbers on the AI statistics pages, or build a prompt with the prompt builder.