Skip to content
Sign in
Artwork for Models & Agents
Models & Agents · October 12 · 11 min

Ep 201: Self-scaling coding benchmarks just arrived, with difficulty now set by the agents…

Models & Agents Self-scaling coding benchmarks just arrived, with difficulty now set by the agents themselves rather than fixed tasks. What You Need to Know: SWE-Duel pits coding agents against each other in mirrored Red-Blue turns on real repositories, generating fresh tasks and oracles at evaluation time so leakage is impossible. GPT-5.5 with Codex CLI led as the strongest solver while Claude Opus 4.8 with mini-swe-agent led as task author. ... Sources: arxiv.org AI Disclosure: This podcast is curated by Patrick but uses AI-generated voice synthesis for audio production. 🎬 Watch on YouTube: https://www.youtube.com/watch?v=99xagD2tn8c If this episode was useful, a rating or a short review on Apple Podcasts or Spotify is how the next listener finds the show — and following it in your app means the next episode is there when you are. 📝 Full show notes, transcript & sources: read the episode page 🌐 Part of the Nerra Network — explore every show at nerranetwork.com.

0:00 · Introduction-11:30
Play in
Transcript file

transcript

No transcript — this publisher did not publish one.

show notes

Models & Agents Self-scaling coding benchmarks just arrived, with difficulty now set by the agents themselves rather than fixed tasks. What You Need to Know: SWE-Duel pits coding agents against each other in mirrored Red-Blue turns on real repositories, generating fresh tasks and oracles at evaluation time so leakage is impossible. GPT-5.5 with Codex CLI led as the strongest solver while Claude Opus 4.8 with mini-swe-agent led as task author. ... Sources: arxiv.org AI Disclosure: This podcast is curated by Patrick but uses AI-generated voice synthesis for audio production. 🎬 Watch on YouTube: https://www.youtube.com/watch?v=99xagD2tn8c If this episode was useful, a rating or a short review on Apple Podcasts or Spotify is how the next listener finds the show — and following it in your app means the next episode is there when you are. 📝 Full show notes, transcript & sources: read the episode page 🌐 Part of the Nerra Network — explore every show at nerranetwork.com.
links3

chapters

9 chapters

more episodes

All episodes