AI before LLMs
Why language models are a recent chapter in a larger story
Understand that AI did not begin with modern chatbots: before LLMs, there were decades of attempts based on logic, rules, search, statistics and neural networks.
AI was not born conversing
Today many people first encounter AI in conversational form: a chatbot that answers questions, writes texts, summarises documents and helps with code. This gives the impression that artificial intelligence was always fluid natural language. It was not.
For much of AI's history, the dream was different: to make machines solve problems, follow rules, prove theorems, plan actions, play chess, recognise patterns, or emulate human experts in specific domains.
First major vision: intelligence as symbolic reasoning
One of the earliest strong bets was symbolic AI. The idea was to represent knowledge using symbols, rules and logic. Instead of learning from millions of examples, the machine would receive explicit rules about the world.
This kind of approach matches statements like: if a person is human, they are mortal; Socrates is human; therefore Socrates is mortal. The machine manipulates symbols according to formal rules. It seems elegant — and it is. But the real world tends to be far messier than a logic exercise.
Expert systems: machines with an expert's manual
Then came expert systems: programs that tried to capture the knowledge of doctors, engineers, analysts or other specialists in a large collection of rules.
They could work well in narrow domains when the rules were clear. But they struggled with exceptions, ambiguity, common sense and new situations. It was like trying to fit the whole world into a filing cabinet of rules. Courageous, but slightly insane — in the best academic sense.
Search: intelligence as exploring possibilities
Another important AI tradition is search. Imagine a game, a maze or a planning problem. The machine needs to explore possible paths until it finds a good solution.
Instead of "understanding" the world the way a person does, it can test states: if I do this, I end up there; if I choose another path, I might arrive better. This view was very important in games, planning and problem solving.
Graphs: knowledge as a network of relations
Graphs appear naturally when we want to represent relations: one thing linked to another. People connected in a social network, cities connected by roads, concepts connected by meaning, pages connected by links.
In AI, graphs help represent knowledge and search. If "coffee" is linked to "drink", "caffeine" and "cup", the machine can navigate these relations. This is powerful for explicit knowledge, but still does not by itself solve the complexity of human language.
The statistical turn: learning patterns from data
Over time, one idea gained force: instead of writing all the rules by hand, we can give examples to the machine and let it learn patterns.
This statistical turn shifts the centre of AI. The question changes from "which rules should we write?" to "which patterns appear in the data?". This opens the way for classification, prediction, speech recognition, machine translation and, later, modern language models.
Neural networks: adjustable connections
Neural networks enter this story as systems with many adjustable connections. They receive an input, transform it through layers and produce an output. During training, the weights of these connections are adjusted to reduce error.
It is tempting to say they "mimic the brain", but this metaphor should be used carefully. For this course, it is better to think of them as flexible mathematical machines that learn transformations from examples.
Why LLMs are a meeting of several traditions
Large language models did not emerge from nowhere. They combine several historical threads: statistics to predict patterns, neural networks to learn representations, tokens to transform text into manipulable units, attention to connect parts of the context, and powerful hardware to do all of this at scale.
When we look at an LLM, we are seeing a recent chapter in a larger story. It does not replace all prior AI; it inherits ideas, solves some old problems and creates new ones. A complete package: brilliant, expensive and occasionally dramatic.
Initial mental map
Write three different ways to imagine "intelligence" in a machine: following rules, searching for paths and learning patterns. Then give a simple example for each.
See expected answer
Following rules: if it is raining, take an umbrella. Searching for paths: find the shortest route on the map. Learning patterns: see many sentences and notice that after "good" often comes "morning". These three ideas appear at different phases of AI.
LLMs are recent but are part of a long history. Before them, AI explored rules, logic, expert systems, search, graphs, statistics and neural networks.