History Technology

History of Deep Learning: From 1943 Neuron to ChatGPT 2026

September 22, 2026

Introduction: History of Deep Learning

History of deep learning.

When I first heard “deep learning” in 2015, I thought it was new.

My professor said “This is the future.”
I Googled it. Top result said “2012 AlexNet”.

I thought, okay, 2012. Not that old.

Then I kept digging.

Turns out the history of deep learning starts in 1943.
During World War 2. On paper. No computers.

80 years. 2 AI winters. Billions of dollars wasted.
And then ChatGPT.

This is the full history of deep learning. Not the textbook version.
The real version. With failures, fights, and comebacks.

History of Deep Learning Begins in 1943

McCulloch and Pitts Draw The First Artificial Brain

Year: 1943. World War 2.

Chicago.

Warren McCulloch: Neuroscientist. 45 years old.
Walter Pitts: 19 year old kid. Dropped out of high school. Genius.

They asked: “What if we can build a brain with math?”

They wrote a 5 page paper. Nobody read it.

But inside was the first artificial neuron.

Their idea:
A neuron gets signals. If signals are strong enough, it fires. 1 or 0.

They even proved it could do logic. AND, OR, NOT.

This is where the history of deep learning begins.
Not with code. With a pencil.

Problem: No computers existed to run it.

History of Deep Learning Year 1957

Frank Rosenblatt Builds The Perceptron Machine

1957. Cold War.

Cornell University.

Frank Rosenblatt was tired of theory. He wanted a real machine.

He built “The Perceptron”.

It was huge. Looked like a robot.
400 photocells as eyes. Wires everywhere.

And it could learn.

You show it a picture. It guesses.
Wrong? It changes internal numbers called “weights”.
Show it again. It gets better.

After 1000 pictures, it could tell circle from square.

New York Times headline: “Navy Unveils Machine That Learns”

Rosenblatt did TV interviews. He said “In 10 years, machines will walk and talk.”

Government gave millions.

For 5 years, everyone thought the history of deep learning would end here.
With success.

History of Deep Learning Year 1969

The Book That Killed AI For 10 Years

1969. MIT.

Marvin Minsky and Seymour Papert. Two very smart guys.

They wrote a book. “Perceptrons”.

Inside they proved: “A single layer perceptron cannot solve XOR.”

XOR is simple: 1,1 = 0. 0,1 = 1.
The perceptron couldn’t do it.

What did that mean? It meant the perceptron was too dumb.

Government read the book. Stopped funding.
Companies stopped hiring.

This was the “First AI Winter”. 1969 to 1980.

For 11 years, if you said “neural networks” on your resume, you didn’t get a job.

I talked to a professor who lived through it.
He said “We had to call it statistics to get funding.”

So the Development of neural networks almost died in 1969.

History of Deep Learning Year 1986

Backpropagation: The Algorithm That Saved Everything

1986. 17 years after the winter.

3 researchers.
David Rumelhart, Geoffrey Hinton, Ronald Williams.

They published a paper. 15 pages.

“Learning Representations by Back-Propagating Errors”

This was the breakthrough.

The idea:
1. Let network guess
2. Check how wrong
3. Send error backwards
4. Tweak all weights a little

Now you could train networks with 3, 4, 5 layers.

Hinton later said “We knew this was big. But nobody cared for 10 years.”

Why? Computers in 1986 were slower than your calculator.
Training took 1 month.

But this 1986 paper is the real start of modern deep learning.
Without this, no ChatGPT.

Geoffrey Hinton is now called “Godfather of Deep Learning”.

Histoy of Deep Learning Year 1998

LeNet-5 Reads Your Handwriting

1998.

Yann LeCun at Bell Labs.

Banks had a problem. Millions of checks. Need to read handwriting.

LeCun built “LeNet-5”. A Convolutional Neural Network. CNN.

It could read zip codes with 99% accuracy.

This was the first time deep learning made money.

Banks used it. Made billions.

But outside banks, nobody cared.

The How deep learning developed in the 90s was boring.
Just banks and research labs.

Geoffrey Hinton Brings Neural Nets Back

2006.

Hinton was tired of people saying “neural nets don’t work.”

He published 2 papers. “Deep Belief Networks”.

He showed you could train 5 layers deep and it actually worked.

He called it “Deep Learning” to sound new.

Slowly, people started listening.

But still, no one outside academia cared.

History of Deep Learning Year 2012

AlexNet and The Day Everything Changed

2012. ImageNet Competition.

Task: Classify 1.2 million images. 1000 categories.

For 5 years, best error rate was 25%.

Then 3 guys from Toronto entered.
Alex Krizhevsky, Ilya Sutskever, Geoffrey Hinton.

They built “AlexNet”. 8 layers.
Trained on 2 gaming GPUs for 1 week.

Their error rate: 15%.
Second place: 26%.

The room went silent. Then applause.

Why did it work in 2012 and not 1986? 3 things:

1. Backpropagation from 1986
2. GPUs from Nvidia
3. Big data from the internet

After that day, Google hired the team.
Facebook hired Yann LeCun.
Microsoft hired everyone.

This is the year most people think the history of deep learning started.
Because this is when it started working.

History of Deep Learning Year 2014

GANs: Machines That Can Create

2014.

Ian Goodfellow, a PhD student.

He invented “GAN”. Generative Adversarial Network.

Two networks fight each other.
One makes fake images. The other tries to catch it.

After training, the first network gets so good it can make real-looking faces.

This is how deepfakes started.

Suddenly deep learning wasn’t just about classification.
It could create.

History of Deep Learning Year 2017

Transormer: The Paper That Created ChatGPT

2017. Google Brain. 8 researchers.

They were frustrated. Old models were too slow.

They wrote a paper. 8 pages.
“Attention Is All You Need”

They invented “Transformer”.

Old models read word by word.
Transformers read the whole sentence and decide “what to pay attention to”.

Example: “The bank was by the river bank”
It knows which “bank” means what.

1 year later OpenAI used this for GPT-1.
2019: GPT-2
2020: GPT-3
2022: ChatGPT

100 million users in 2 months.

This is the biggest moment in the history of deep learning.

History of Deep Learning Year 2020 to 2026

From GPT-3 to GPT-5 and Beyond

2020: GPT-3. 175 billion parameters.
2022: ChatGPT. Everyone uses it.
2023: GPT-4. Can see images.
2024: Gemini, Claude, Llama 3. Competition.
2025: GPT-5. Can code, reason, remember.
2026: AI agents. Deep learning runs your email, calendar, phone.

What changed?
Data got bigger. GPUs got faster. Models got smarter.

The Evolution of deep learning went from 5 pages in 1943 to trillions of parameters in 2026.

The 7 People Who Shaped History of Deep Learning

You can’t tell this story without them:

1. Warren McCulloch and Walter Pitts 1943: First neuron
2. Frank Rosenblatt 1957: First learning machine
3. Geoffrey Hinton 1986: Backpropagation
4. Yann LeCun 1998: CNN
5. Alex Krizhevsky 2012: AlexNet
6. Ian Goodfellow 2014: GAN
7. Ashish Vaswani 2017: Transformer

Without these 7, you wouldn’t be talking to me.

Why Did History of Deep Learning Take 80 Years

3 reasons.

Reason 1: No Data

In 1950 there was no data.
In 2020 we had all of Wikipedia, Reddit, books.

Reason 2: No Compute

In 1950 a computer cost $2 million.
In 2024 a $2000 GPU is 1 million times faster.

Reason 3: No Algorithm

We needed 40 years to invent backprop and transformers.

So the history of deep learning is really a story of waiting for hardware.

Full Timeline of History of Deep Learning

FAQ About History of Deep Learning

1. When id history of deep learning start

1943 with McCulloch and Pitts. But modern deep learning started 2012.

2. Who father of deep learning

Geoffrey Hinton. He invented backprop and brought neural nets back.

3. What caused 2012 deep learning boom

GPUs plus Big Data plus AlexNet.

4. Is ChatGPT the end of history of deep learning

No. We are probably at chapter 5 of 20.

Final Thoughts: History of Deep Learning

I started researching this for a class assignment.

80 years later I’m still shocked.

The history of deep learning is not about computers.
It’s about people who didn’t give up.

Rosenblatt was laughed at in 1969.
Hinton was ignored in 1986.
Krizhevsky almost quit in 2011.

But they kept going.

Now in 2026, deep learning writes emails, diagnoses cancer, and talks to us.

The guys in 1943 dreamed of a thinking machine.
We are 80 years in. And we’re close.

What part of the history of deep learning surprised you most?
Comment below. I read every one.

Stanford CS231n Course Notes