A practical lab · about 18 minutes
Interactive learning lab
How AI works · guide 5 of 8
The attention architecture observatory
0/5 core signals
Saved only on this device
Architecture lab · BriefingScene 1 / 6
Every token asks what other tokens matter
Select a query token and inspect one illustrative head at a time.
Every token asks what other tokens matter ready.
Thefastestwaytoimprove→?
Self-attention creates token-specific weighted combinations of other token representations. Multiple heads and layers learn different patterns; the final model behaviour cannot be reduced to one heatmap.
Q asks; K matches; V carries information.
Heads specialise differently.
Position must be represented explicitly.