Skip to content
Artwork for My Weird Prompts
My Weird Prompts · Saturday · 21 min

Which AI Models Actually Generate Good Training Data?

Synthetic training data lives or dies on the generator behind it. This episode compares the instruction models actually available for the job — Qwen, DeepSeek, Llama, Mistral, and Gemma on the open-weight side, and the Claude Opus/Sonnet/Haiku and GPT-5.6 Sol/Terra/Luna tiers on the closed side — with real pricing per million tokens and what each licence actually permits. Then it digs into the scores people cite to prove a model follows instructions: what IFEval really measures, why models near two billion parameters now clear eighty percent on it, and what IFBench's fifty-eight new constraints reveal when the test stops looking familiar. The throughline: producing one acceptable answer is not the same as producing thousands of useful training examples. Key sources: • OpenRouter model listings — https://openrouter.ai/anthropic/claude-opus-5.5 • OpenRouter model listings — https://openrouter.ai/openai/gpt-5.6-sol • IFBench — https://arxiv.org/abs/2507.02833 Episode 6238 — myweirdprompts.com/n/6238 (permanent short link: myweirdprompts.com/542975)

0:00 · Intro-21:45

transcript

No transcript — this publisher did not publish one.

show notes

Synthetic training data lives or dies on the generator behind it. This episode compares the instruction models actually available for the job — Qwen, DeepSeek, Llama, Mistral, and Gemma on the open-weight side, and the Claude Opus/Sonnet/Haiku and GPT-5.6 Sol/Terra/Luna tiers on the closed side — with real pricing per million tokens and what each licence actually permits. Then it digs into the scores people cite to prove a model follows instructions: what IFEval really measures, why models near two billion parameters now clear eighty percent on it, and what IFBench's fifty-eight new constraints reveal when the test stops looking familiar. The throughline: producing one acceptable answer is not the same as producing thousands of useful training examples.

Key sources:
• OpenRouter model listings — https://openrouter.ai/anthropic/claude-opus-5.5
• OpenRouter model listings — https://openrouter.ai/openai/gpt-5.6-sol
• IFBench — https://arxiv.org/abs/2507.02833

Episode 6238 — myweirdprompts.com/n/6238 (permanent short link: myweirdprompts.com/542975)

chapters

6 chapters