Somewhere in a data center right now, OpenAI is running a cluster that costs more per hour to operate than most startups spend in a year. They’re training the next version of GPT. Anthropic is running an equivalent job for Claude. DeepSeek is running one too, at a fraction of the compute cost. Between the three of them: different bets on architecture, different training philosophies, tens of billions of dollars. From the outside, it looks like the deepest trade secret in tech.
A dozen years ago, I learned the terms perceptron and neural network in university. Back then, the discipline felt mechanical, not mysterious. A set of steps you worked through on paper. Today, most people use AI daily without understanding how it works, and somewhere along the way that gap started to feel intentional.
In university, we calculated neural networks by hand. Full forward and backward passes, on paper, one weight at a time, before ever touching code that would do it for us. Those pages held the exact computation running inside GPT, Claude, and DeepSeek today, executed at a scale of hundreds of billions of parameters instead of a handful. The math itself (backpropagation, gradient descent) has been public and peer-reviewed for decades, long before any of these companies existed. Only the size of the paper has changed. So let’s go through the whole calculation, start to finish, to reveal what’s hiding behind the AI giants: simple math that anyone can do, knowing only multiplication and subtraction.
The problem: scoring sales leads with AI
Every SaaS founder knows this problem. A trial user signs up. Two weeks later, they either convert to paid or disappear. Somewhere in between, someone on the team looks at the account and makes a judgment call: is this lead worth a demo call, a follow-up email, a discount offer, or nothing? Most teams solve it with gut feeling. A sales rep glances at the account activity and scores the lead on an internal scale, say 0 to 5. It works, but it doesn’t scale, and it walks out the door when the rep does.
We’re going to build the model behind that gut feeling. A small neural network that predicts the lead score from two numbers every SaaS product already tracks: days active during the trial, and retention time, meaning minutes the user spends inside the product per session. The network will be small enough to calculate by hand, and structurally identical to what runs inside the AI giants.
📌 Note - the four terms we'll useWeight - a number the model multiplies an input by. It answers one question: how much does this input matter? The only thing a network actually learns. Set to a small random value before training. Small on purpose: large starting weights make numbers grow out of control from layer to layer, and training breaks down before it begins.
Neuron - one unit of computation: multiply each input by its weight, sum the results. Several neurons side by side form a layer; stacked layers form a network.
Activation function - a filter on a neuron’s output. In our example we’ll use ReLU, and behind the intimidating name hides a simple max function:
max(0, z). Negative numbers become zero, positive numbers pass unchanged. This is what lets a network learn curves instead of straight lines. We apply it only on the hidden neurons - the output neuron just sums and reports its number raw.Error - the gap between the prediction and reality. This single number drives all learning.
The network on paper
Here’s the whole model. Two inputs, two hidden neurons, one output. Six weights total, and every weight starts as a small random number. These are ours:
One real example from historical data: a user was active 10 days, with an average retention of 2 minutes per session. A sales rep scored this lead a 3 on the 0 – 5 scale. The rep’s score is our target: the answer the network should learn to produce on its own.
Forward pass: the network makes it’s first guess
The forward pass is one motion, left to right: multiply, sum, filter, pass along. Neuron A takes both inputs, multiplies each by its weight, and sums:
The result, 1.4, is positive, so ReLU passes it through untouched. Neuron A is active. Neuron B runs the identical operation with its own weights and lands negative:
ReLU cuts -0.3 down to zero and neuron B goes silent. Nothing broke. With these weights and these inputs, this neuron simply has nothing to contribute yet. Remember this silence, because it has a consequence in the backward pass. The output neuron collects what the hidden layer produced:
The network’s first guess is 0.7. The rep said 3. The error is -2.3, so the model badly underestimated this lead. Every number in that gap is about to be put to work.
Backward pass: the error teaches the weights
Now the same path in reverse. The error travels right to left, and every weight gets one question: how much did you
contribute to this miss, and in which direction should you move? The rule for each weight is single-line: correction = error × the input that weight was multiplying. For the weight connecting A to the output, that input was A’s
activation, 1.4:
Δ(wAy) = -2.3 × 1.4 = -3.22
For the weight connecting B to the output, that input was B’s activation, which was zero:
Δ(wBy) = -2.3 × 0 = 0
Look at that second line. The correction isn’t small. It’s exactly zero, a direct consequence of B’s silence. Nobody
wrote special logic for inactive neurons; multiplication by zero did it on its own. The error then flows one layer deeper,
and here ReLU plays its second role. On the way forward, it passed A’s positive value through unchanged; on the way back,
it passes the correction through unchanged too. Neuron A’s share of the blame is the error times the weight that carried
its signal forward: -2.3 × 0.5 = -1.15. That share gets multiplied by each of A’s inputs to correct the first-layer weights:
Δ(w1A) = -1.15 × 10 = -11.5
Δ(w2A) = -1.15 × 2 = -2.3
For neuron B, ReLU does the opposite. It blocked B’s negative value on the way forward, so it blocks the correction on the way back. B’s weights receive nothing. Apply every correction (scaled down by a learning rate of 0.01, so no single example can yank the weights too far), and here is the network after one full training step:
Three weights moved and three stayed exactly where they were. That’s one complete training step, done with nothing but multiplication and subtraction. Run this loop across thousands of historical leads, and the six random numbers we started with converge into a model that scores new trials the way your best rep would. Without the rep.
What actually differs between GPT, Claude, and DeepSeek
Here’s the claim worth sitting with: the AI giants do not run a different algorithm than the six-weight network above. They run the identical forward pass and backward pass, at a scale of hundreds of billions of weights instead of six, on trillions of training examples instead of one.
The differences live in the engineering on top. Anthropic reportedly keeps every weight active on every request, a design called dense architecture. OpenAI and DeepSeek increasingly route each request through only a fraction of their parameters using Mixture-of-Experts, which activates a small set of specialized sub-networks per query instead of the whole model. DeepSeek prioritizes training efficiency and publishes its weights openly; Anthropic trains with Constitutional AI, where the model critiques its own outputs against a written set of principles. Different bets, same math underneath.
No magic, just data
Walk back through everything we calculated. At no point did the network understand what a sales lead is. It multiplied, summed, and corrected. That’s the complete list. Everything it “knows” came from the historical examples we fed it. The model took the rep’s judgment and compressed it. No intelligence was added anywhere in the process. Scale this up and nothing changes. GPT, Claude, and DeepSeek are shaped entirely by their training data: text and code written by people, judgments made by people. When the output looks brilliant, you’re looking at a reflection of human brilliance.
That has a hard consequence. High-quality human data is finite, and when models feed on their own outputs without control, quality degrades generation by generation. This is a documented effect called model collapse. The real bottleneck of this industry isn’t compute. It’s fresh human input.
Yes, in narrow domains with verifiable answers, like protein structures and mathematical proofs, models have found results no human computed before. That’s real. But it’s search over patterns, running on the math we just did by hand. The questions worth asking still come from us. So the next time an AI giant looks like it’s guarding an unknowable secret, remember what’s running inside: multiplication, subtraction, and a mirror pointed at human knowledge. If you can multiply and subtract, you can build one yourself.