Skip to content
Artwork for MIT OpenCourseWare: Free Courses from the institute
MIT OpenCourseWare: Free Courses from the institute · October 14 · 1 hr 21 min

Lec 08. Architectures: Transformers

MIT 6.7960 Deep Learning, Fall 2024 Instructor: Phillip Isola View the complete course: https://ocw.mit.edu/courses/6-7960-deep-learning-fall-2024/ YouTube Playlist: https://www.youtube.com/playlist?list=PLUl4u3cNGP63URZnh5iqBzDTDYPUTQT-8 Uploaded 11-02-26 This video introduces transformers, focusing on three key ideas: tokens, attention, and positional codes. It also explores how transformers relate to MLPs, GNNs, and CNNs as variations on common principles. License: Creative Commons BY-NC-SA More information at https://ocw.mit.edu/terms More courses at https://ocw.mit.edu Support OCW at http://ow.ly/a1If50zVRlQ We encourage constructive comments and discussion on OCW’s YouTube and other social media channels. Personal attacks, hate speech, trolling, and inappropriate comments are not allowed and may be removed. More details at https://ocw.mit.edu/comments. Learn more about your ad choices. Visit megaphone.fm/adchoices Learn more about your ad choices. Visit megaphone.fm/adchoices

0:00-1:21:19

transcript

No transcript — this publisher did not publish one.

show notes

MIT 6.7960 Deep Learning, Fall 2024
Instructor: Phillip Isola
View the complete course: https://ocw.mit.edu/courses/6-7960-deep-learning-fall-2024/
YouTube Playlist: https://www.youtube.com/playlist?list=PLUl4u3cNGP63URZnh5iqBzDTDYPUTQT-8

Uploaded 11-02-26

This video introduces transformers, focusing on three key ideas: tokens, attention, and positional codes. It also explores how transformers relate to MLPs, GNNs, and CNNs as variations on common principles.

License: Creative Commons BY-NC-SA
More information at https://ocw.mit.edu/terms
More courses at https://ocw.mit.edu
Support OCW at http://ow.ly/a1If50zVRlQ

We encourage constructive comments and discussion on OCW’s YouTube and other social media channels. Personal attacks, hate speech, trolling, and inappropriate comments are not allowed and may be removed. More details at https://ocw.mit.edu/comments.



Learn more about your ad choices. Visit megaphone.fm/adchoices

Learn more about your ad choices. Visit megaphone.fm/adchoices

links7