Post-Editing vs Critique: Fine-Tuning Data

When you fine-tune a model on style and personality, the training loop is the easy part — the dataset is where you're on your own. This episode works through two competing approaches: post-editing, where a human rewrites the model's output directly, and critique, where a human annotates what's wrong and a model produces the corrected version. Drawing on a head-to-head study out of Columbia, Microsoft and Johns Hopkins on Llama 3.1 8B Instruct, we walk through an Elo ladder that puts full-rewrite and stepwise-edit preference pairs hundreds of points above the standard A/B ranking everyone treats as default — and then hit the number that cuts the other way, because the critique protocol generates more than nine times as many preference pairs per human-hour. There's also a warning about vanilla DPO on small local edits, and why style fine-tunes are exactly the case where it bites. Episode #036601 — open it directly at myweirdprompts.com/036601