Skip to content
Artwork for Domesticating AI
Domesticating AI · August 14 · 41 min

Open Source Projects Every AI Homelabber Should Know

Everyone has a list of open-source AI projects they want to try. But installing repos is not the same as building a stack. Ryan Russon joins Miriah, Matt, and Chris to talk through the projects that actually teach you how AI homelabs work: Unsloth for training, Ollama and llama.cpp for local inference, Open WebUI for a usable front door, K3s for hosting, Longhorn and SeaweedFS for storage, and Prometheus/Grafana for observability. We also get into build-vs-buy decisions, speech models like Whisper.cpp, Parakeet, and Granite, why state is where homelabs stop being cute, and why the best open-source projects are the ones that teach you what breaks next. Links mentioned: https://github.com/unslothai/unsloth https://ollama.com/ https://github.com/ggml-org/llama.cpp https://github.com/open-webui/open-webui https://k3s.io/ https://longhorn.io/ https://github.com/seaweedfs/seaweedfs https://prometheus.io/ https://grafana.com/ https://github.com/Arize-ai/phoenix https://github.com/langfuse/langfuse Support the show: https://patreon.com/DomesticatingAIPodcast Keep your AI on a leash.

0:00-41:18

transcript

No transcript — this publisher did not publish one.

show notes

Everyone has a list of open-source AI projects they want to try. But installing repos is not the same as building a stack.

Ryan Russon joins Miriah, Matt, and Chris to talk through the projects that actually teach you how AI homelabs work: Unsloth for training, Ollama and llama.cpp for local inference, Open WebUI for a usable front door, K3s for hosting, Longhorn and SeaweedFS for storage, and Prometheus/Grafana for observability.

We also get into build-vs-buy decisions, speech models like Whisper.cpp, Parakeet, and Granite, why state is where homelabs stop being cute, and why the best open-source projects are the ones that teach you what breaks next.

Links mentioned:
https://github.com/unslothai/unsloth
https://ollama.com/
https://github.com/ggml-org/llama.cpp
https://github.com/open-webui/open-webui
https://k3s.io/
https://longhorn.io/
https://github.com/seaweedfs/seaweedfs
https://prometheus.io/
https://grafana.com/
https://github.com/Arize-ai/phoenix
https://github.com/langfuse/langfuse

Support the show:
https://patreon.com/DomesticatingAIPodcast

Keep your AI on a leash.

links12