About Andrej Karpathy
Andrej Karpathy is one of AI's most respected researchers and educators. He was a founding member of OpenAI, then led Tesla's Autopilot vision team, and has become famous for making deep learning accessible through YouTube videos and open-source projects.
Career Highlights
- Eureka Labs (2024-present): Founder, AI education startup
- Tesla (2017-2022): Director of AI, led Autopilot computer vision
- OpenAI (2015-2017): Founding member and research scientist
- Stanford PhD: Studied under Fei-Fei Li on image captioning
- nanoGPT: Created minimal GPT implementation for education
Notable Positions
On LLMs as "Ghosts"
Karpathy's most provocative framing:
"LLMs are 'ethereal spirit entities' - fully digital, mimicking humans, starting from a completely different point in the space of possible intelligences. We're building ghosts, not animals."
Animals evolved with hardcoded hardware. A zebra runs minutes after birth. LLMs emerged from imitating text - a fundamentally different optimization process.
On Agent Timelines
A reality check on hype:
"Decade of agents, not year of agents. When would you actually hire Claude as an intern? You wouldn't today because it just doesn't work reliably enough."
On Context vs Weights
Technical insight on how LLMs work:
"The KV cache stores 320 KB per token vs 0.7 bits in weights - a 35 million fold difference. Anything in context is working memory; anything in weights is hazy recollection."
Key Quotes
- "We're building ghosts, not animals."
- "Decade of agents, not year of agents."
- "Pre-training is crappy evolution."
Related Reading
- Scaling Laws - The paradigm Karpathy helped establish
- Pre-training - What Karpathy calls "crappy evolution"
Video Appearances
Ghosts not animals thesis
LLMs are 'ethereal spirit entities' - fully digital, mimicking humans, starting from a completely different point in the space of possible intelligences. We're building ghosts, not animals.
at 00:08:00
Agent timeline reality check
Decade of agents, not year of agents. When would you actually hire Claude as an intern? You wouldn't today because it just doesn't work reliably enough.
at 00:15:00
Context vs weights compression
The KV cache stores 320 KB per token vs 0.7 bits in weights - a 35 million fold difference. Anything in context is working memory; anything in weights is hazy recollection.
at 00:22:00
AI education
Intro to Large Language Models in 30 minutes - making complex AI concepts accessible through clear explanations.
at 00:00:00
Software development evolution
Referenced for his talk on software development paradigms: Software 1.0 (code rules), 2.0 (ML learns rules), 3.0 (English language programming with LLMs). 'Software development was stable for 70 years and saw two vast changes in the last two decades.'
at 01:50:00