Skip to content
Sign in
Artwork for cloud2030
cloud2030 · September 4 · 39 min

Token Cost Optimization

In this episode, we focus on token cost and token optimization in generative AI, and we discuss why token use needs budgets and controls rather than being treated as unlimited. We also talk about how repeated prompting, testing, and downstream work can quickly increase token consumption. We also talk about where AI agents make sense and where cheaper models or existing data should be used instead, including a bug triage example where systems should gather history and reproduce issues before escalating. We close by discussing responsibility, accuracy, and governance, including marking unverified claims clearly and using strict prompts and review passes. We also note the need for better expectations around safety and liability in AI systems. Transcript here: https://otter.ai/u/0Y-EBKoKsN7UWxC6YzMkb44ZN6w?utm_source=copy_url

0:00-39:04
Play in

transcript

No transcript — this publisher did not publish one.

show notes

In this episode, we focus on token cost and token optimization in generative AI, and we discuss why token use needs budgets and controls rather than being treated as unlimited. We also talk about how repeated prompting, testing, and downstream work can quickly increase token consumption. We also talk about where AI agents make sense and where cheaper models or existing data should be used instead, including a bug triage example where systems should gather history and reproduce issues before escalating. We close by discussing responsibility, accuracy, and governance, including marking unverified claims clearly and using strict prompts and review passes. We also note the need for better expectations around safety and liability in AI systems.

Transcript here: https://otter.ai/u/0Y-EBKoKsN7UWxC6YzMkb44ZN6w?utm_source=copy_url

more episodes

All episodes