Technical Terms
Attention
Definition
The mechanism that lets a model weigh which other tokens matter most when updating a token representation or predicting the next token.
In Plain English
How the model decides what parts of the text to pay attention to right now.