Core Concepts

Token

Definition

The basic chunk of text that language models read and generate, roughly a word fragment rather than a full word.

In Plain English

The small pieces of text AI counts, reads, and bills you for.

See Also