Can CriticGPT Replace Hand-Designed Rewards for Robot Manipulation?

Can a multimodal language model replace hand-designed rewards for robot manipulation? This episode examines what CriticGPT actually learns, how its video preferences become training rewards, and where the experimental evidence stops. We cover three questions: whether strong trajectory-ranking accuracy survives ambiguous comparisons; whether the reported policy improvements are convincing given run-to-run variation; and what transfers when the critic is reused but a new reward model still has to be trained. The method adapts LLaVA-1.5 using automatically generated supervision from Meta-World, then connects pairwise feedback to reward modeling and DrQ-v2 policy learning. The experimental platform is a simulated tabletop manipulation arm — not a physical-robot deployment. Along the way, we explain why preference accuracy does not guarantee useful dense credit assignment, why baseline adaptation budgets matter, and how newer stage-aware and history-conditioned reward approaches change the research context. The central limitation is the gap between evaluating recorded behavior and supplying a trustworthy objective for an optimizing controller. For practitioners, the episode offers concrete audits for clip-order consistency, ambiguous outcomes, reward-model extrapolation, and end-to-end reproduction. If your policy becomes very good at maximizing the learned reward, will it also become better at the task?