Glossary · AI & ML

Transformer

A neural network architecture using self-attention mechanisms, forming the foundation of modern LLMs.

The Transformer is a neural network architecture introduced in the 2017 paper "Attention Is All You Need" by Vaswani et al. It relies entirely on self-attention mechanisms to draw global dependencies between input and output, abandoning the recurrence and convolution previously dominant in sequence models. The key innovation is the attention mechanism, which allows the model to weigh the importance of different parts of the input when producing each part of the output. Transformers enable massive parallelization during training (unlike sequential RNNs), allowing them to be trained on much larger datasets. All modern large language models (GPT, Claude, Gemini, Llama) are based on the Transformer architecture or its variants.

Fig — every answer ships with the tables, rows and SQL behind it.