Sumeet Vaidya | AI Driven SDLC
Guest: Sumeet Vaidya, Co-founder & CEO, Crafting (previously Facebook, Uber, Discord) Sumeet joins Manav to argue that the industry is optimizing the wrong half of the software lifecycle. Code generation is nearly free; validation is where the real engineering now lives. What we cover: The model debate is a distraction. Claude Code vs. Codex vs. Cursor matters tactically for an IC picking a tool today, but it's the wrong question for an engineering leader deciding where to invest. The right question is resilience — can your organization absorb the next wave without being locked in? Resilience has two meanings. First, the boring one: if your primary provider goes down, does your engineering org stop? We learned this lesson with cloud providers and auth providers; AI infrastructure deserves the same treatment. Second, the harder one: if engineers increasingly don't touch code, how do you retain a deterministic way to evaluate and resolve failures in a probabilistic system? Harnesses are mostly markdown. Sumeet's sharpest critique: the average "harness" is a pile of markdown files with no real contract, no true guardrails, and no answer for credentials, dependencies, or access. Loop engineering has the same gap — "check if it works" is doing enormous heavy lifting and getting glossed over. Spec-driven development, honestly assessed. It serves the purpose architecture docs used to: forcing the orchestrator to refine the ask. It does not put hard guardrails on the agent. A beautiful spec that passes human scrutiny can still fall apart in execution. Useful, not a silver bullet. The test taxonomy collapses. With the right infrastructure, the distinction between unit test, integration test, and CI/CD stops mattering — a test simply verifies something works, whether that spans one service or a hundred. Manav pushes on a pattern he sees repeatedly: engineers fooling themselves with high unit-test coverage while integration remains the weak point. The compounding math. Even if loops cut regression probability in half, 10x more code shipped means roughly 5x more outages. Production-monitoring agents that detect and patch errors are another band-aid. Every step of that chain gets resolved upstream by real infrastructure investment. Enterprise reality. Crafting deploys into customer clouds — no data leaves. Shared pre-production clusters, sanitized DB snapshots, credential injection via Vault, network interception for shared staging. Named customers include Brex, Fair, and Webflow; larger regulated logos are in flight. What enterprises actually ask about: security, visibility, and — increasingly — cost. Token maxing is blowing through budgets across the board. Productivity, measured soberly. Not 100–200% gains. Double digits. Anyone counting PRs has lost the plot. Sumeet expects a winnowing of the middle layer of engineering, particularly outside Silicon Valley, alongside genuinely new demand — every dental office and clinic plausibly supports one engineer. For new grads: CS isn't dead, it's changing. Someone who has relearned AI tooling four times in six months may be more adaptable than a senior engineer optimized for the old stack. The closing question for your organization: When an agent's change passes every check and is still wrong in production — what's your blast radius, and who's accountable? If you can't answer cleanly, that's your roadmap.
- Transcript