About Jeff Dean
Jeff Dean is the Chief Scientist of Google DeepMind and Google Research, often called the "Chuck Norris of Computer Science" for his legendary contributions to systems and AI. He has been instrumental in designing many of Google's foundational systems and is co-lead of the Gemini project.
Career Highlights
- Google (1999-present): Built foundational systems including MapReduce, BigTable, Spanner, and TensorFlow
- TPU Development: Led the development of Google's Tensor Processing Units
- Gemini: Co-lead of Google's flagship AI model
- Academic Contributions: Co-authored influential papers on distributed systems and machine learning
Technical Contributions
TPU Development
Dean oversees Google's TPU development, now in its seventh generation ("Ironwood"):
"Every generation of TPU we really try to take advantage of the co-design opportunities we have with having a lot of researchers thinking about where ML computations we're going to want to run 2.5 to 6 years from now."
This involves a fascinating forecasting exercise - predicting the future of ML to design hardware years in advance.
Systems That Shaped the Industry
Dean's systems work at Google created templates for the entire industry:
- MapReduce - Inspired Hadoop and modern data processing
- BigTable - Template for NoSQL databases
- Spanner - Globally distributed database
- TensorFlow - Open-source ML framework
Notable Positions
On Academic Research Funding
Dean is a strong advocate for public funding of academic research:
"I feel like it's really important to have a vibrant academic research ecosystem in the US and also in the world because those early stage creative ideas are often the things that lead to major breakthroughs and innovations."
He notes that the deep learning revolution was built on academic research from 30-40 years ago, and Google itself was built on academic work including TCP/IP and the Stanford Digital Library Project that funded PageRank.
On Research Moonshots
Dean advocates for 3-5 year research moonshots with mixed teams:
"I really like the 3 to 5 year time horizon kind of thing with an ambitious set of people around a particular thing they're trying to achieve because it's not so distant that it won't have impact, but it's not so short that you can't conceive of doing something ambitious."
Key Quotes
- "Trying to predict a very fast-moving field is not a very easy thing."
- "The returns are quite large to society" (on academic research funding)
- "3 to 5 years is a delightful time range to consider" (on research planning)
Related Reading
- 15 Years of ML Advances at Google - Dean's definitive history of deep learning's rise
- TPU - The custom AI chips Dean helped develop
- AI Infrastructure - The systems layer Dean has shaped
Video Appearances
Early scaling work
I built data parallelism and model parallelism into my 1990 undergrad thesis. I made a huge mistake - I didn't increase model size as I added processors.
at 00:05:00
Informal scaling laws
We had a great saying at Google Brain: bigger model, more data, more compute. We didn't formalize it as scaling laws but we knew.
at 00:20:00
Compute multiplication
Algorithmic improvements like transformers and sparse models multiply together with hardware. We're doing billions of times more compute than 10 years ago.
at 00:35:00
Google TPU Ironwood announcement
This is Ironwood, the seventh generation of TPUs. It has quite a lot of new capabilities, connected together into very large configurations we call pods - 9216 chips per pod.
at 00:04:00
Hardware-software co-design
Every generation of TPU we really try to take advantage of the co-design opportunities... having a lot of researchers thinking about where ML computations we're going to want to run 2.5 to 6 years from now.
at 00:05:30
Advocacy for academic research funding
I feel like it's really important to have a vibrant academic research ecosystem because those early stage creative ideas are often the things that lead to major breakthroughs.
at 00:07:00