Skip to content
Artwork for Tech Rants
Tech Rants · Yesterday · 1 hr 29 min

LLM Speed Secrets

Llama server is faster. It uses KV cache. PDF loading is slow. The CLI is the problem. Consumer GPUs are best. They have faster cores. Blog: https://ojambo.com/high-speed-llm-secrets-consumer-gpus-and-llama-server-power Amazon Storefront: https://amzn.to/4xevbes - Explore the hardware, tools, and gear I use and recommend. Video: https://youtube.com/live/3o7c5MO1GIc Books: https://www.amazon.com/stores/Edward-Ojambo/author/B0D94QM76N Woodworking Blueprints: https://ojamboshop.com Tutorials: https://ojambo.com/contact Consultations: https://ojamboservices.com/contact Amazon Fire HD 8: https://amzn.to/4g5DOk1 TP Link Deco: https://amzn.to/4gu1IXL Kingston NV3: https://amzn.to/4xyZg8c #LocalAI #LlamaCPP #TechHacks

0:00-1:29:31

transcript

No transcript — this publisher did not publish one.

show notes

more episodes

All episodes