What is Generative AI, and why did it only become viable now?

Ask a recommendation system what you will drink on Friday and it will answer with something you have already bought. Ask a generative model for a short story about that Friday and it will hand you a text that never existed. The two systems share most of their mathematics, and yet they answer different questions. This article explains what changed to make the second kind viable, where the line between predicting and generating really lies, and what the technology can and cannot be trusted to do today.^[4]

1. Why now: three factors that converged

Generative AI became viable when three independent lines of progress met. None of them is new on its own; what is new is that all three matured at the same time.

• Compute. Training a neural network is mostly matrix multiplication, and graphics processors were built to do exactly that in parallel. When a convolutional network trained on two consumer GPUs won the 2012 ImageNet competition by a wide margin, operations that had been too slow and too expensive to scale became routine.^[2]
• Data. The volume of digital text, images and code grew, but the decisive change was form: data organized, labelled or at least crawlable in a way that a training pipeline can consume. ImageNet itself — millions of images with structured labels — was the template for that shift.^[3]
• Architectures. Recurrent networks could model sequences but processed them one step at a time and struggled with long contexts. The Transformer replaced recurrence with attention, which looks at the whole context at once and parallelizes across the sequence. Language and context could finally be modelled at scale.^[1]

Once the three were in place, a further empirical fact took over: performance improves predictably as model size, data and compute grow together — the so-called scaling laws.^[5] That is why the field moved so quickly from a research curiosity to products.

Practise: open in the Lab: Exercise 1, Three factors

2. From traditional AI to generative AI

Traditional AI works mainly with pattern recognition, classification and prediction over data that already exists. A recommendation system learns a user's habits and estimates the next probable action — wine on cold days, beer on Friday. The output is a choice among known items, or a number attached to one of them.

Generative AI changes the goal. Instead of predicting an existing item, it synthesizes a new output from the patterns it learned: a paragraph, an image, a function. The machinery underneath is often the same family of neural networks; what differs is what the model is asked to produce and how its output is judged. That change of objective is what drew so much attention to the field.

Table 1 — The evaluation row is the one that matters in practice: a predictive system can be scored automatically against ground truth; a generative one produces things for which no ground truth exists, which is why human judgement re-enters the loop (section 7).

Practise: open in the Lab: Exercise 2, Predict or generate

3. Generation is not copying

Generating does not mean cutting and pasting examples from the training set. It means producing a new combination that is coherent with the request. If someone asks for a book “based on” many earlier books, the system should not return a copied passage; it should return a new artifact guided by the instruction.

The originality here is statistical and compositional, not creation from nothing. A language model has learned a probability distribution over sequences of words; generating means walking through that distribution one step at a time, conditioned on everything that came before — including the prompt:

Each factor on the right is a distribution over the next token given the prompt and the tokens already produced. The model never stores sentences to retrieve; it stores the parameters that shape these distributions. Verbatim reproduction can still happen when a passage appeared many times in training — a known and studied failure mode^[8] — but it is the exception the mechanism has to be pushed into, not its normal behaviour.

This also brings the discussion closer to how people create. Human creation normally reorganizes references, influences and repertoire; a novelist has read novels, a designer has seen designs. The difference is one of degree and of intention, not a clean line between “original” and “derived”.

4. Creation, intelligence and the philosophical debate

Calling these systems “not intelligent” because they do not create the way a human artist does mixes philosophical criteria with operational ones. In everyday use, intelligence is inferred from behaviour: the ability to learn, to apply patterns, and to produce answers appropriate to a context. That is exactly the kind of behaviour these systems exhibit — and it is, not by coincidence, the operational test Turing proposed when he replaced “can machines think?” with a question about behaviour.^[9]

Recognizing that does not make the system equivalent to a human being. It has no goals of its own, no experience of the world beyond its training data, and no way to check its output against reality unless a tool gives it one. But it also prevents the opposite mistake: reducing its behaviour to mechanical automation without learning. The useful discussion, therefore, is not whether the system is capable, but which kind of intelligence and which kind of creation it manifests — and where the responsibility for the result stays.

5. The prompt as the control mechanism

The prompt is the input specification: it tells the model what to generate, in what style and under which constraints. Because the prompt is part of the conditioning in equation (1), it shapes every token that follows. The clearer the request, the better the system can align its output with the goal — for text, image or code alike.

In real applications, “knowing how to prompt” is not a superficial trick. It is the skill of turning human intention into an operational instruction for the model: audience, format, length, tone, what to include, what to leave out, and examples of the expected result. The observation that a few worked examples inside the prompt are enough to steer a large model to a new task — few-shot prompting — was one of the findings that made the current generation of models useful in practice.^[4] A poor prompt produces vague or wrong outputs; a well-formed one raises usefulness, coherence and adherence to context.

Practise: open in the Lab: Exercise 3, The control mechanism

6. Where it already works: text, image, video and code

Generative models already produce books, e-mails, automatic replies and other textual artifacts with good coherence and consistent style. The practical value lies in a useful first draft, adapted to the desired format and tone. Because the output depends on the prompt, strategy, structure and level of formality can all be steered. Text tends to come out more consistently than other media, but it still requires human validation.

Image generation moved from visibly defective results to convincing compositions in a few years, largely through diffusion models that learn to turn noise into an image guided by a text description.^[6] A short prompt can now produce covers, scenes and visual concepts close to known references; models ship with guardrails against certain misuse, such as deceptive images or real people in problematic contexts. Video quality has advanced quickly too, yet finishing errors, wrong proportions and inconsistent details still appear. Those errors show that an output can be impressive without being automatically reliable.

Code generators — Copilot, Devin and their peers — already produce complete snippets of real practical use. A model trained on public code solves a meaningful share of programming tasks from a natural-language description alone.^[7] This does not remove the need to know how to program: whoever does not understand the problem tends to ask for the wrong thing and to accept incorrect solutions. In software development the model accelerates implementation, exploration and boilerplate; the review of logic, security, architecture and adherence to requirements stays with the engineer. Unreviewed generated code is operational risk, not productivity.

Table 2 — Maturity describes how often the first output is usable, not how often it is correct. The last column is the point of section 7: the more consequential the medium, the less optional the review.

7. Human review is not optional

Generative outputs can look correct and still contain subtle errors. In images this shows up as text with wrong accents, strange proportions or inconsistent anatomical details; in video, as motion and finishing artifacts; in text and code, as incoherence or answers that are plausible but wrong. The failure is rarely loud. It is fluent.

The correct workflow, then, is not “generate and publish” but “generate, inspect and correct”. Generative AI is most useful when it enters the process as a co-producer under human supervision: it drafts, the person judges, and the correction feeds the next request. The gain in productivity is real, but it is realized in the inspection step, not in the generation step.

Practise: open in the Lab: Exercise 4, Human review

8. Under the hood: deep learning, attention and context

Deep learning lets a system learn complex representations from large volumes of data instead of depending only on explicit rules: nobody writes down what makes a sentence coherent or an image plausible; the network learns it from examples, layer by layer.^[10]

In modern models, attention mechanisms decide which parts of the context matter most for generating the next part of the output.^[1] That is why the system can tell apart the senses of one word — bank as the edge of a river or as a financial institution — by weighing the rest of the sentence. Context and attention are what turn a local prediction (the next token) into a coherent generation (a paragraph that keeps its subject).

Equation (1) and Figure 4 meet in a small piece of code. Given the scores a model assigns to every possible next token, this is all that generation does at each step:

import numpy as np

def sample_next_token(logits, temperature=1.0, rng=np.random.default_rng()):
    """logits: one score per vocabulary entry, produced by the model for the current context.
    Returns the index of the token chosen for this step."""
    # 1. temperature reshapes the distribution: < 1 sharpens it, > 1 flattens it
    z = logits / max(temperature, 1e-6)

    # 2. softmax turns scores into probabilities that sum to 1 (this is p(x_t | prompt, x_<t))
    z = z - z.max()                       # numerical stability
    p = np.exp(z) / np.exp(z).sum()

    # 3. sampling: the same context can yield different continuations
    return rng.choice(len(p), p=p)


# Generating a sequence is this step in a loop: each chosen token joins the context of the next one.
def generate(model, prompt_tokens, max_new_tokens, temperature=0.8):
    tokens = list(prompt_tokens)
    for _ in range(max_new_tokens):
        logits = model(tokens)            # scores for the next token, given everything so far
        tokens.append(sample_next_token(logits, temperature))
    return tokens

Practise: open in the Lab: Exercise 5, Sampling the next token

9. Conclusion

Generative AI is not magic and not a copying machine. It is a learned probability distribution over artifacts, sampled one step at a time under the guidance of a prompt — made viable by the convergence of parallel compute, organized data and an architecture that reads context at scale. Its outputs are new, coherent and useful, and also fluent when they are wrong; that is why the prompt carries the intent and the human carries the judgement. The next article draws that line in detail: what traditional AI does with labeled data, why its answer space is closed, and where generative composition begins.