Meta
Llama Videos
Understanding the Llama 3 Tokenizer
​
Back to Llama Videos

April 10, 2025

Understanding the Llama 3 Tokenizer

Aston Zhang, research scientist working on Llama at Meta discusses the new tokenizer in Meta Llama 3.

Explore more
Running Llama 3 on Linux
Running Llama 3 on Linux
Running Llama 3 on Mac
Running Llama 3 on Mac
Break Free from Closed Models: Unlock the Power of Open Source Now
Break Free from Closed Models: Unlock the Power of Open Source Now
Meta
FacebookXYouTubeLinkedIn
Documentation
OverviewModels Getting the Models Running Llama How-To Guides Integration Guides Community Support
Community
Community StoriesOpen Innovation AI Research CommunityLlama Impact Grants
Resources
AI at Meta BlogMeta NewsroomFAQPrivacy PolicyTermsCookies
Llama Protections
OverviewDeveloper Use Guide
Documentation
Overview
Models
Getting the Models
Running Llama
How-To Guides
Integration Guides
Community Support
Community
Community Stories
Open Innovation AI Research Community
Llama Impact Grants
Resources
AI at Meta Blog
Meta Newsroom
FAQ
Privacy Policy
Terms
Cookies
Llama Protections
Overview
Developer Use Guide
Documentation
Overview
Models
Getting the Models
Running Llama
How-To Guides
Integration Guides
Community Support
Community
Community Stories
Open Innovation AI Research Community
Llama Impact Grants
Resources
AI at Meta Blog
Meta Newsroom
FAQ
Privacy Policy
Terms
Cookies
Llama Protections
Overview
Developer Use Guide
Documentation
Overview
Models
Getting the Models
Running Llama
How-To Guides
Integration Guides
Community Support
Community
Community Stories
Open Innovation AI Research Community
Llama Impact Grants
Resources
AI at Meta Blog
Meta Newsroom
FAQ
Privacy Policy
Terms
Cookies
Llama Protections
Overview
Developer Use Guide
Skip to main content
Meta
Models & Products
Docs
Community
Resources
Try Llama
Download models