adrian_weller

POET-X: Memory-Efficient LLM Training via Scalable Orthogonal Transformations

POET-X is a memory-efficient variant of the Reparameterized Orthogonal Equivalence Training (POET) framework, designed to address the high memory and computational costs of training large language models (LLMs). It achieves reduced computational complexity through a scalable approach to orthogonal equivalence transformations, enabling the training of billion-parameter LLMs on a single H100 GPU, a feat not possible with standard optimizers like AdamW. POET-X preserves the stability and generalization benefits of its predecessor while significantly improving training efficiency.

InfoAtlas: A Foundation Model for Zero-Shot Statistical Dependence Estimate

Conditioning Causal Foundation Models on Partial Graph Knowledge Closes the Gap with Specialized Estimators

Causal Foundation Models (CFMs) offer a unified approach to causal discovery and inference via amortization, but currently lack the ability to incorporate domain knowledge, limiting their practical utility. This paper introduces conditioning strategies that inject causal graph or ancestral information — including partial graphs — directly into CFMs. The key finding is that learnable biases embedded in the attention mechanism are the most effective way to leverage both full and partial causal information, enabling a general-purpose CFM to match the performance of specialized models trained on fixed causal structures.

POET-X: Billion-Parameter LLM Pretraining on a Single GPU via Scalable Orthogonal Reparameterization

POET-X is a memory-efficient extension of the POET (Reparameterized Orthogonal Equivalence Training) framework, designed to reduce the computational and memory overhead of training large language models. By scaling orthogonal equivalence transformations at lower cost, POET-X preserves the spectrum-preserving and training stability properties of the original method while dramatically improving throughput. The practical result is the ability to pretrain billion-parameter LLMs on a single Nvidia H100 GPU — a regime where standard optimizers like AdamW fail due to out-of-memory errors.

Hybrid LLM + Probabilistic Models for Transparent Clinical Reasoning Under Uncertainty

MedMSA is a proposed framework that combines language models for knowledge retrieval with formal probabilistic models to produce calibrated, verifiable clinical inferences — addressing the dual failure modes of current clinical AI: poor uncertainty calibration and opaque reasoning. The system demonstrates a proof-of-concept on differential diagnosis, generating uncertainty-weighted ranked diagnoses from patient symptoms. This hybrid neuro-symbolic architecture is positioned as a path toward AI that is both practically useful and formally auditable in high-stakes medical settings.

LLM Agent Evaluation Needs a First-Principles Redesign Around Interaction Trajectories

Current AI evaluation practices are structurally misaligned with how modern LLM-based agents actually operate: they inherit assumptions from static, response-centered benchmarks (fixed inputs, isolated outputs, single-response judgments) while agents increasingly act over time across tools, environments, users, and other agents. The authors argue interactive evaluation must be treated as a distinct, principled paradigm — not just a new class of benchmarks — redefining evaluation as a mapping from interaction-generated trajectories to judgments that assess process, recoverability, coordination, robustness, and system-level performance. They propose a two-axis taxonomy, design principles, and reporting standards to unify a currently fragmented landscape.