Karpathy on Software 3.0 and the Attention Breakthrough
Andrej Karpathy has a way of making a decade of AI feel like one straight line. If you build with LLMs, generative AI, agents, or transformers, his breakdown of how programming got here is the framing worth keeping. It is short, concrete, and it explains why the work looks different than it did five years ago.
There is nothing to install here. What you get is the mental model, plus one line you will probably want to remember.

What Happened
Karpathy traces the path in five stops: traditional software, neural networks, transformers, GPTs, and what he calls Software 3.0.
The turning point was "Attention Is All You Need", the 2017 paper that changed the trajectory of AI. The idea sounds almost too plain. Instead of building ever more complex architectures on top of RNNs, you remove the RNN and keep the attention. That opened the door to the transformer era we are living through now.
He then reframes what programming itself is:
- Software 1.0 — design the algorithm.
- Software 2.0 — design the dataset.
- Software 3.0 — design the prompt.
And the line worth bookmarking: "The hottest new programming language is English."
Why It Matters
A prompt can increasingly be treated as a program, with the LLM acting as the system that interprets and runs the instructions. That matters because transformers work across text, images, and speech, and GPTs start to behave less like one tool and more like a general-purpose computer.
If natural language is the new interface, then knowing what you want and saying it precisely is the skill. Frames like this one are how you keep up with where all of it is heading.
More shares you might like
AIStand With Jensen Petition Passes 89,000 SignaturesJen · 14h · 22 views
AIHow to Break Down Any Video With rea-videoJen · 1d · 37 views
AIHow to Claim 1 Month of SuperGrok for FreeJen · 3d · 68 views
News500+ AI Agent Projects Land on GitHub With 39K StarsJen · 3d · 53 views
ToolsHow to Sign Up for Amazon Associates on Your PhoneJen · 4d · 49 views
AIHow to Claim 20,000 Free Clocked AI API CreditsJen · 4d · 70 views
Comments
Join the conversation — sign in to comment.
No comments yet — start the conversation!