Andrej Karpathy
Building a kind of JARVIS @ OреոΑӏ. Previously Director of AI @ Tesla, CS231n, PhD @ Stanford. I like to train large deep neural nets 🧠🤖💥
عرض في 𝕏سلاسل التغريدات
"Yes AlphaGo only won by 0.5, but it was not at all a close game. It's an artifact of its training objective."
Next frontier of prompt engineering imo: "AutoGPTs" . 1 GPT call is just like 1 instruction on a computer. They can be strung together into programs. Use prompt to define I/O devic...
More good read/discussion on psychology of LLMs. I don't follow in full but imo it is barking up the right tree w.r.t. a framework for analysis. https://t.co/gh9X65r22E
🔥 New (1h56m) video lecture: "Let's build GPT: from scratch, in code, spelled out." https://t.co/2pKsvgi3dE We build and train a Transformer following the "Attention Is All You Ne...
Didn't tweet nanoGPT yet (quietly getting it to good shape) but it's trending on HN so here it is :) : https://t.co/qouvC6xuXq Aspires to be simplest, fastest repo for training/fin...
Dreambooth (stable diffusion finetuning for personal profile pictures) has been going viral last few days as well, for good reasons it's super fun; Unlike other places https://t.co...
An interesting historical note is that neural language models have actually been around for a very long time but noone really cared anywhere near today's extent. LMs were thought o...
Is it the number of examples that matters or the number of presentations to the model during training? E.g. humans used spaced repetition to memorize facts but there are no equival...
The Transformer is a magnificient neural network architecture because it is a general-purpose differentiable computer. It is simultaneously: 1) expressive (in the forward pass) 2)...
🥷New (1h55m) Lecture #5: "Becoming a Backprop Ninja" https://t.co/ekZgAQON3O We take the 2-layer MLP from last lecture and backprop through all of it manually: cross entropy loss,...
@hardmaru @StabilityAI I wish! I can't make the GPUs come out very well sad :) https://t.co/Elk7J95qGv
I actually mostly built Lexicap so I could share a few snippets of Nick Lane ep :). (I already read the books so I'm ~familiar with the topics, these snippets are just personally n...