Skip to content
Sign in
Artwork for My Weird Prompts
My Weird Prompts · September 22 · 20 min

Small Models, Big Schemas: When JSON Constraints Backfire

Everyone building agents reaches for the biggest model and wraps every endpoint as a tool. But there's a quieter pattern: tiny text-to-text models doing one narrow structured job. Daniel wrote in asking whether small models doing predictable structured output inside MCP pipelines actually ship in production, or whether everyone just lets a general-purpose model emit tool calls. The answer runs through Todoist's MCP servers, a 7B structured-object model from Amazon Science, and a May 2026 study called The Constraint Tax, which found that hard answer-only schemas pushed schema validity to 100% while answer accuracy fell from 19.7% to 11%. Then FlowMCP flips the whole premise: maybe the win isn't a smaller model, it's a smaller surface. Episode #666624 — open it directly at myweirdprompts.com/666624

0:00 · Intro-20:21
Play in
Transcript file

transcript

No transcript — this publisher did not publish one.

show notes

Everyone building agents reaches for the biggest model and wraps every endpoint as a tool. But there's a quieter pattern: tiny text-to-text models doing one narrow structured job. Daniel wrote in asking whether small models doing predictable structured output inside MCP pipelines actually ship in production, or whether everyone just lets a general-purpose model emit tool calls. The answer runs through Todoist's MCP servers, a 7B structured-object model from Amazon Science, and a May 2026 study called The Constraint Tax, which found that hard answer-only schemas pushed schema validity to 100% while answer accuracy fell from 19.7% to 11%. Then FlowMCP flips the whole premise: maybe the win isn't a smaller model, it's a smaller surface.

Episode #666624 — open it directly at myweirdprompts.com/666624

chapters

5 chapters

more episodes

All episodes