Skip to content
Sign in
Artwork for My Weird Prompts
My Weird Prompts · September 21 · 21 min

Xet, Buckets, and Auto-Pulling Model Weights

Hugging Face quietly migrated 500,000 repositories and 20 petabytes to Xet, a chunk-level dedup engine that replaced file-level Git LFS. This episode unpacks how that migration stayed invisible, why content-defined chunking matters for quantization-heavy model repos, and what Storage Buckets actually are versus repositories. Then it tackles the real question: can a bucket be configured to automatically pull new upstream variants? The answer is no, and the reason is structural, not a missing feature. Episode #260264 — open it directly at myweirdprompts.com/260264

0:00 · Intro-21:44
Play in
Transcript file

transcript

No transcript — this publisher did not publish one.

show notes

Hugging Face quietly migrated 500,000 repositories and 20 petabytes to Xet, a chunk-level dedup engine that replaced file-level Git LFS. This episode unpacks how that migration stayed invisible, why content-defined chunking matters for quantization-heavy model repos, and what Storage Buckets actually are versus repositories. Then it tackles the real question: can a bucket be configured to automatically pull new upstream variants? The answer is no, and the reason is structural, not a missing feature.

Episode #260264 — open it directly at myweirdprompts.com/260264

chapters

5 chapters

more episodes

All episodes