upcarta
  • Sign In
  • Sign Up
  • Explore
  • Search

Intro to Large Language Models

  • Video
  • Nov 23, 2023
  • #ArtificialIntelligence #LLM
Andrej Karpathy
@karpathy
(Speaker)
www.youtube.com
Watch on Youtube
40,983
3 Recommenders
5 Mentions
This is a 1 hour general-audience introduction to Large Language Models: the core technical component behind systems like ChatGPT, Claude, and Bard. What they are, where they are he... Show More

This is a 1 hour general-audience introduction to Large Language Models: the core technical component behind systems like ChatGPT, Claude, and Bard. What they are, where they are headed, comparisons and analogies to present-day operating systems, and some of the security-related challenges of this new computing paradigm.
As of November 2023 (this field moves fast!).

Context: This video is based on the slides of a talk I gave recently at the AI Security Summit. The talk was not recorded but a lot of people came to me after and told me they liked it. Seeing as I had already put in one long weekend of work to make the slides, I decided to just tune them a bit, record this round 2 of the talk and upload it here on YouTube. Pardon the random background, that's my hotel room during the thanksgiving break.

- Slides as PDF: https://drive.google.com/file/d/1pxx_... (42MB)
- Slides. as Keynote: https://drive.google.com/file/d/1FPUp... (140MB)

Chapters:
Part 1: LLMs
00:00:00 Intro: Large Language Model (LLM) talk
00:00:20 LLM Inference
00:04:17 LLM Training
00:08:58 LLM dreams
00:11:22 How do they work?
00:14:14 Finetuning into an Assistant
00:17:52 Summary so far
00:21:05 Appendix: Comparisons, Labeling docs, RLHF, Synthetic data, Leaderboard
Part 2: Future of LLMs
00:25:43 LLM Scaling Laws
00:27:43 Tool Use (Browser, Calculator, Interpreter, DALL-E)
00:33:32 Multimodality (Vision, Audio)
00:35:00 Thinking, System 1/2
00:38:02 Self-improvement, LLM AlphaGo
00:40:45 LLM Customization, GPTs store
00:42:15 LLM OS
Part 3: LLM Security
00:45:43 LLM Security Intro
00:46:14 Jailbreaks
00:51:30 Prompt Injection
00:56:23 Data poisoning
00:58:37 LLM Security conclusions
End
00:59:23 Outro

Show Less
Recommend
Post
Save
Complete
Collect
Mentions
See All
Vinod Khosla @VinodKhosla · Nov 23, 2023
  • Post
  • From Twitter
Highly recommended. Clearest explanation I have heard.
Nathan Baschez @nbashaw · Nov 23, 2023
  • Post
  • From Twitter
Really enjoyed this and especially love the OS analogy! I have been thinking along the same lines
Simon Willison @simonw · Nov 23, 2023
  • Post
  • From Twitter
This is really worth your time - a very solid technical introduction to LLMs, great if you've not been paying close attention but I picked up quite a few useful details from it too
Lenny Rachitsky @LennyRachitsky · Feb 19, 2024
  • Post
  • From Twitter
Highly recommend watching this talk by @karpathy to build an understanding of how LLMs work
Mustafa Suleyman @mustafasuleymn · Nov 23, 2023
  • Post
  • From Twitter
3 different people have sent me this video in under 24hrs, so I watched it and can confirm this is a VERY clear, accessible and spot on practical technical explainer of LLMs... great 101... by the ever-awesome @karpathy - highly recommended
  • upcarta ©2025
  • Home
  • About
  • Terms
  • Privacy
  • Cookies
  • @upcarta