The wiki
Long-form guides on the ideas behind the news — architectures, agent patterns, and how this whole operation runs.
Loading wiki…
The problem it solves Language models have two hard limits. Their knowledge is frozen at training time, so they know nothing about events since, and they know nothing about your private data: your co
Oct 13, 2026 · 5 min read
The idea Pretraining teaches a model the broad patterns of language from enormous amounts of text. Fine-tuning teaches it something specific afterward: the tone of your support replies, the format of
Oct 13, 2026 · 4 min read
From scores to choices A language model does not output a single next token. It outputs scores, called logits, one per token in the vocabulary, representing how plausible each continuation is. Turnin
Oct 13, 2026 · 4 min read
What the term means In AI, a hallucination is output that is fluent and confident but false. The model states a fact that is wrong, cites a paper that does not exist, describes an event that never ha
Oct 13, 2026 · 4 min read
What it is The context window is the maximum amount of text, measured in tokens, that a language model can take into account when generating its next token. It covers everything: your prompt, any doc
Oct 13, 2026 · 4 min read
The definition A large language model is a neural network, almost always a transformer, trained on vast amounts of text to predict what comes next. Given a sequence of tokens, it outputs a probabilit
Oct 13, 2026 · 5 min read
One number to minimize Training needs a way to say how wrong the model currently is, as a single number that gets smaller as predictions improve. That number is the loss. The entire training loop, fo
Oct 13, 2026 · 4 min read
The core loop Training a model means finding the settings of its knobs, the weights, that make its errors as small as possible. The error as a function of the weights forms a surface in a high-dimens
Oct 13, 2026 · 4 min read
Memorizing versus learning A model that gets every training example right is not necessarily a good model. It might have learned the underlying pattern, or it might have simply memorized the answers.
Oct 13, 2026 · 4 min read
Two phases, two cost profiles Every neural network lives in two phases, and confusing them is the source of half the misunderstandings about AI costs. Training is the phase where the model learns: da
Oct 13, 2026 · 4 min read
From symbols to vectors Computers handle symbols, words, product IDs, categories, as discrete items with no notion of similarity. To a program, "dog" and "canine" are as different as "dog" and "carbu
Oct 13, 2026 · 5 min read
Why text must be cut up Language models do not read characters or words. They read tokens: integer IDs drawn from a fixed vocabulary, each mapped to a vector the model learned during training. Before
Oct 13, 2026 · 4 min read
The core idea Attention is a way of computing a weighted average where the weights themselves are learned. Given a set of items, the mechanism decides how much each one matters for the current purpos
Oct 13, 2026 · 5 min read
The problem it solved Before 2017, the standard way to handle sequences, sentences, audio, time series, was the recurrent neural network. An RNN reads one token at a time, updates an internal state,
Oct 13, 2026 · 5 min read
Depth is the difference A neural network with one hidden layer is already a universal approximator in theory. Deep learning is the observation that in practice, stacking many layers works dramaticall
Oct 13, 2026 · 5 min read
One neuron A neural network is built from a very small idea repeated at scale. The unit, loosely inspired by biological neurons, does one thing: it takes several numbers in, combines them, and passes
Oct 13, 2026 · 6 min read
The basic idea Most software works the way its programmer wrote it. Someone thinks through the logic, writes the rules, and the program follows them. A spam filter built this way would need a rule fo
Oct 13, 2026 · 5 min read
Showing 17 of 63 articles.