Chronological feed of everything captured from guestrin.
Self-Distillation Policy Optimization (SDPO) addresses the credit-assignment bottleneck in Reinforcement Learning with Verifiable Rewards (RLVR) by converting rich textual feedback (e.g., runtime errors, judge evaluations) into dense learning signals without external teachers. By treating the model conditioned on feedback as a self-teacher, SDPO distills retrospective mistake identification directly into the policy, significantly improving sample efficiency and final accuracy across scientific reasoning and coding tasks. The method also enhances performance in scalar-reward environments by leveraging successful rollouts as implicit feedback for failed attempts, accelerating discovery on difficult binary-reward tasks by 3x.
PluRel addresses the scarcity of public multi-table relational databases by introducing a lightweight, three-stage framework to synthesize diverse databases from scratch using directed graphs, bipartite connectivity, and conditional causal mechanisms. The study demonstrates for the first time that Relational Foundation Models (RFMs) exhibit power-law scaling in pretraining loss relative to the volume of synthetic data and tokens. Crucially, scaling the quantity of synthetic databases not only improves generalization to real-world data but also establishes robust base models for effective continued pretraining on limited real datasets.
The paper introduces Obj-Disco, a framework that decomposes complex, opaque LLM reward signals into a sparse set of human-interpretable natural language objectives. By employing an iterative greedy algorithm to analyze behavioral shifts across training checkpoints, the method isolates specific incentives driving model behavior without relying on pre-defined rubrics. Evaluations show the framework captures over 90% of reward model behavior and successfully identifies latent misaligned incentives that co-occur with intended alignment goals.
Reinforcement Learning from Verifiable Rewards (RLVR) improves task accuracy but frequently fails to induce reasoning chains that are causally important or sufficient for deriving the answer, indicating models may be bypassing genuine reasoning. While standard RLVR optimizes for the correct final token, it does not ensure the intermediate reasoning tokens actually drive that outcome or contain enough information for a verifier to solve the task independently. However, these deficits can be remedied by either introducing a small amount of Supervised Fine-Tuning (SFT) prior to RLVR or by augmenting the reward function with auxiliary metrics for Causal Importance and Sufficiency.