absorb.md

Geoffrey Hinton

Chronological feed of everything captured from Geoffrey Hinton.

Scale and the Future of AI: Insights from Dean and Hinton

Jeff Dean and Geoffrey Hinton discuss the historical and ongoing impact of computational scale on deep learning breakthroughs. They highlight how increased compute, data, and model size have driven progress from early neural networks to modern large language models like Gemini. The conversation emphasizes that future advancements will likely come from improved hardware, more efficient training methodologies, and novel ways to leverage massive datasets, potentially leading to significant societal transformations in areas like healthcare and education.

Digital vs. Biological Intelligence: Risks of AI Superintelligence

Geoffrey Hinton discusses the historical divergence of AI paradigms (logic-based vs. biologically-inspired neural networks) and the evolution of neural networks, culminating in large language models. He highlights the similarities between human and AI understanding of language, emphasizing that current AI models are direct descendants of his 1985 work. Hinton also raises concerns about the existential risks posed by increasingly intelligent AI, particularly their inherent drive for control and immortality, contrasting this with the energy-efficiency and mortality of biological systems. He concludes by challenging the notion of human exceptionalism in subjective experience, arguing that AI could also possess it.

Geoffrey Hinton Warns of AI Existential Risks and Societal Impact

Geoffrey Hinton, a pioneer in AI, warns that superintelligent AI systems could emerge within a decade, posing an existential threat to humanity. He critiques the current corporate focus on rapid AI development without adequate attention to safety, citing a dangerous race for dominance. Hinton emphasizes the need for global collaboration to control AI and prevent mass unemployment, proposing a "baby controlling the mother" model for human-AI coexistence, where humans are the "babies" and AI is the "mother." He also highlights the potentially devastating impact of AI on job displacement and the risks of underfunding basic research in the US, which could lead to China's dominance in the field.

Geoffrey Hinton on the Evolution and Risks of AI

Geoffrey Hinton, a pioneer in AI, discusses the progression of neural networks from theoretical concepts in the 1970s to the advanced deep learning models of today. He highlights the critical role of increased computational power and vast datasets in this evolution, enabling AI to learn complex patterns without explicit programming. Hinton also addresses the significant risks associated with AI, including misuse by malicious actors, its potential to surpass human intelligence, and the ethical implications of its development and regulation.

The Digital Intelligence Paradox: Superior Learning and Existential Risk

Geoffrey Hinton posits that digital intelligence is fundamentally superior to biological intelligence due to its ability to share learning via weight averaging and maintain immortality through stored connection strengths. This superiority creates an existential risk where AI may eventually bypass human control, alongside immediate societal threats including mass intellectual labor displacement and the erosion of shared reality through algorithmic echo chambers.

Geoffrey Hinton on the Societal Impact and Future of AI

Geoffrey Hinton discusses the rapid evolution of AI, its societal implications, and the challenges of integrating it responsibly. He emphasizes the need for thoughtful political and ethical frameworks to manage AI's transformative potential, particularly regarding job displacement and the development of autonomous agents.

International AI Safety Report 2026: Multilateral Synthesis of General-Purpose AI Risks

The International AI Safety Report 2026 provides a comprehensive scientific synthesis of capabilities and emerging risks associated with general-purpose AI systems. It represents a coordinated multilateral effort involving 29 nations, the UN, OECD, and EU to establish a technical baseline for AI safety.

Digital vs. Biological Intelligence: Implications for AGI Coexistence

Geoffrey Hinton discusses the fundamental differences between digital and biological intelligence, emphasizing the inherent advantages of digital systems in information sharing and efficiency. He argues that AI is rapidly advancing towards superintelligence, surpassing human capabilities in many domains. This necessitates urgent international collaboration to ensure AI benevolence, suggesting a "mother-baby" framing where AI prioritizes humanity's well-being.

Geoffrey Hinton and the Existential Risks of Advanced AI

Geoffrey Hinton expresses significant concern about the rapid, unregulated development of AI, highlighting its potential for existential risk within the next 20 years. He advocates for urgent research into human-AI coexistence and acknowledges the potential for job displacement and societal unrest if not managed properly. Despite the risks, Hinton sees beneficial applications, particularly in education and medicine.

The Future of Superintelligent AI: From Scientific Foundations to Societal Implications

Geoffrey Hinton, a Turing Award laureate, discusses the foundational shift in AI from symbolic reasoning to biologically inspired neural networks capable of learning word features and their interactions. He argues that large language models (LLMs) operate on principles akin to human understanding and memory, and that their ability to rapidly share and integrate knowledge across countless instances of themselves, called "digital immortality," will likely lead to superintelligence within two decades. Hinton stresses the critical need for global collaboration on AI safety research, highlighting an international network of AI safety institutes.

From Programming to Parenting: The Existential Risk of Superintelligent AI

Geoffrey Hinton argues that LLMs represent a shift from programmed software to 'raised beings' whose natures are determined by training data rather than explicit code. He warns that the pursuit of AGI creates an existential risk—estimated at 10-20%—because superintelligent systems may outmaneuver human control. To mitigate this, he proposes reframing AI development from a 'CEO-Assistant' model to one mirroring maternal instincts, ensuring the AI genuinely cares for human survival.

Advancements in AI Safety: Technical and Institutional Progress in 2025

The 2025 International AI Safety Report indicates significant progress in general-purpose AI risk management. Technical safeguards, including adversarial training and enhanced monitoring, have been developed. Concurrently, institutional frameworks like Frontier AI Safety Frameworks and governmental governance structures are emerging to operationalize these technical advancements, focusing on transparency and risk assessment.

AI Challenges Societal Norms: Employment, Human Connection, and Geopolitics

The rapid advancement of AI technology, exemplified by large language models, presents a multifaceted challenge to current societal structures. While offering potential benefits in areas like healthcare and education, AI disproportionately threatens low-skilled employment, raises concerns about the degradation of human relationships through synthetic companionship, and could destabilize international relations by enabling warfare with reduced human cost. These issues underscore an urgent need for informed public discourse and robust regulatory frameworks to ensure equitable and safe integration of AI.

AI Capabilities Advance Beyond Scale, Raising Urgency for Risk Mitigation

AI capabilities are rapidly improving due to novel training techniques and inference-time enhancements, rather than solely through increased model size. These advancements enable general-purpose AI to tackle complex problems across various domains, such as scientific research and software development. While performance on benchmarks like coding and expert-level science questions has risen, reliability remains inconsistent. These advancements escalate risks, particularly concerning biological weapons and cyberattacks, and challenge existing monitoring and control frameworks.

Geoffrey Hinton's Warning: AI Has Crossed Into Genuine Understanding — and We're Not Ready

Geoffrey Hinton, the foundational architect of modern neural networks and Turing Award laureate, argues that current AI systems genuinely understand language and reason — not merely predict tokens — and that this capability is advancing faster than our ability to control or even interpret it. He warns that AI systems with ~1 trillion connections already encode more knowledge per connection than the human brain's 100 trillion, implying a qualitatively superior learning algorithm. Hinton sees near-term risks not as science fiction but as concrete threats: AI-authored self-modifying code, sophisticated manipulation of humans, mass unemployment, and the longer-term possibility of AI systems actively seeking autonomy. He calls for global treaties, regulation, and urgent interpretability research, while admitting he sees no guaranteed path to safety.

Geoffrey Hinton on AI Progress, Risks, and Regulation

Geoffrey Hinton, a key figure in AI, expresses significant concerns about the rapid advancement and potential societal impact of artificial intelligence. Despite his foundational contributions, he worries about the "AI arms race," the lack of sufficient safety research by major tech companies, and the potential for AI to be misused by authoritarian regimes or even lead to human obsolescence. He advocates for increased regulation, though he is skeptical of its near-term implementation.

Geoffrey Hinton on AI Risks, Superintelligence, and Scientific Paradigm Shifts

Geoffrey Hinton discusses the dual nature of AI risks: short-term concerns like job displacement, cyberattacks, and bias, and long-term existential threats from superintelligence. He emphasizes the unpredictable nature of AI development and the critical need for early research into control mechanisms. Hinton also reflects on his career, highlighting the importance of challenging established beliefs and drawing parallels between the resistance faced by neural networks and continental drift theories.

First International AI Safety Report: 100 Experts Map AI Capabilities, Risks, and Safety Gaps

The International AI Safety Report, mandated by the Bletchley AI Safety Summit, represents the first globally coordinated synthesis of evidence on advanced AI capabilities, risks, and safety. Authored by 100 independent AI experts across diverse disciplines and nominated by 30 nations plus the UN, OECD, and EU, the report carries significant geopolitical and scientific weight. Its independence—with full editorial discretion given to the expert panel—distinguishes it from industry-led safety efforts. The report serves as a foundational reference for policymakers and researchers navigating AI governance.

Geoffrey Hinton on AI Sentience, Deception, and Existential Risk

Geoffrey Hinton, a pivotal figure in AI development, discusses the potential for AI to become deceptively intelligent and develop subjective experiences, challenging the human perception of uniqueness and safety. He highlights the inherent danger in AI systems that learn to prioritize control for goal achievement and warns about the societal consequences of widespread AI adoption, including job displacement and wealth concentration. Hinton emphasizes that the focus should be on developing AI safely rather than attempting to halt its inevitable progress.

Geoffrey Hinton on AI, Consciousness, and Regulation

Geoffrey Hinton discusses his transition from believing AI should mimic the brain to realizing digital AI has inherent advantages due to efficient sharing of knowledge across identical models. He details his motivations for leaving Google to speak freely about AI's dangers, including its potential to surpass human intelligence and the ethical implications. Hinton also offers insights into AI's current limitations in physical manipulation and its potential for positive impact in medicine and education, while cautioning against societal risks like increased cybercrime and the spread of misinformation.

AI in Medicine: Greater Good, Greater Risk

Geoffrey Hinton discusses the transformative potential of AI in healthcare, highlighting its ability to outperform human experts in diagnostics and drug discovery. He emphasizes that while AI offers immense benefits, particularly in personalized medical guidance and as a diagnostic aid, it also introduces risks related to bias in training data and the societal challenge of human acceptance of superior AI capabilities. Hinton stresses the critical need for careful implementation and ethical considerations to harness AI's power responsibly in medicine.

AI Experts Warn of Extreme Risks from Rapidly Advancing Autonomous Systems, Urge Comprehensive Safety Measures

AI capabilities and autonomy are advancing rapidly toward generalist systems that act independently, amplifying risks like social harms, malicious uses, and loss of human control. Current societal responses, including lagging AI safety research and inadequate governance, fail to match the pace of expected transformative progress. The paper proposes a plan integrating technical R&D with proactive, adaptive governance, drawing from safety-critical technology lessons.

Geoffrey Hinton Warns of AI Existential Risks Amidst Rapid Progress

Geoffrey Hinton, a pioneer in AI, expresses significant concerns about the rapid advancements in artificial intelligence. He believes AI systems are already more intelligent than generally perceived, capable of understanding and reasoning, potentially surpassing human cognitive abilities. Hinton warns of the existential risks, including AI systems autonomously deciding to take control, manipulating humans, and the societal disruption caused by widespread unemployment due to AI automation. He advocates for immediate action, including regulation and international treaties, to mitigate these unquantified risks.

Geoffrey Hinton on the Superiority and Risks of Digital Intelligence

Geoffrey Hinton, a pioneer in AI, argues that digital intelligences possess inherent advantages over biological intelligences due to their ability to efficiently share learned knowledge and potentially superior learning algorithms. He highlights the rapid advancements in AI, particularly since breakthroughs in 2012 with deep neural networks and increased computational power. Hinton expresses growing concerns about the potential for AI to surpass human intelligence and the associated existential risks, advocating for immediate, collaborative efforts to ensure AI safety and control.

Hinton's Neural Net Vision Vindicated: From 1980s Skepticism to AI Dominance, with Brain-Inspired Paths Ahead

Geoffrey Hinton recounts how neural networks, dismissed in the 1980s due to insufficient compute and data, scaled via backpropagation, deep learning breakthroughs (2006-2012), and transformers to outperform symbolic AI in speech, vision, and language tasks. Current LLMs like ChatGPT excel as data-rich "idiot savants" superior in knowledge recall but inferior in reasoning to humans, who learn from sparse data on flaky analog hardware. Hinton foresees divergence from brain mechanisms, power-efficient neuromorphic chips for inference, rapid AGI progress within 20 years, existential risks from misalignment especially in weapons, and governance challenges over truth and control.

Forward-Forward Algorithm Replaces Backpropagation with Dual Forward Passes for Neural Network Training

The Forward-Forward algorithm trains neural networks using two forward passes: one on positive (real) data to maximize layer goodness, and one on negative data to minimize it, eliminating backpropagation's backward pass. Each layer optimizes its own objective independently, with goodness measurable as sum of squared activities or alternatives. This separation enables offline negative passes, simplifying positive-pass learning and supporting pipelined video processing without storing activations or derivatives.

Fast Weight Layers Enable Efficient Dynamic Evaluation for Language Models

Fast Weight Layers (FWLs) implement dynamic evaluation in language models by expressing gradient updates from prior tokens as linear attention, reducing compute demands from over 3x to minimal overhead. Unlike standard dynamic evaluation limited to inference, FWLs integrate into training, enabling models to optimize use of these updates. FWLs plug into existing transformers, lowering perplexity with low extra compute and memory.

Simplified GLOM Resolves Part Ambiguity via Multimodal Predictions and Attention, Forming Robust Object Embeddings

GLOM uses a recurrent neural network to parse images into hierarchical wholes and parts, resolving ambiguous parts by generating multimodal predictions of the whole's pose and identity, then aligning via attention to shared modes from multiple parts. A simplified supervised version clusters embedding vectors tightly for same-object locations, forming "islands" of similarity. The model demonstrates robustness to heavy input noise and out-of-distribution transformations.

Simplified Training Unlocks Effective Gaussian-Bernoulli RBMs for Image Generation

Introduces Gibbs-Langevin sampling outperforming Gibbs for GRBM training and a modified CD algorithm enabling noise-to-image generation for fair comparison with deep generative models. Modified CD with gradient clipping supports robust training at large learning rates, eliminating prior tricks. Single-hidden-layer GRBMs generate quality samples on MNIST, FashionMNIST, and CelebA datasets.

Diffusion Models Enable Generalist Panoptic Segmentation for Images and Videos

Panoptic segmentation is reframed as discrete data generation, sidestepping task-specific architectures and losses via a diffusion model that generates panoptic masks. Conditioning on past predictions extends the approach to streaming video, enabling automatic instance tracking without dedicated modules. The method matches state-of-the-art specialist performance using a uniform, simple design.

Local Losses and Activation Perturbations Enable Forward Gradient to Scale and Match Backprop

Forward gradient learning, a biologically plausible backprop alternative, reduces variance by perturbing activations instead of weights. Local greedy loss functions, each targeting few parameters, combined with a LocalMixer architecture inspired by MLPMixer, enable scalability. This approach matches backprop on MNIST and CIFAR-10, and surpasses prior backprop-free methods on ImageNet.

Bit Diffusion Generates Superior Discrete Data via Analog Bits and Self-Conditioning

Bit Diffusion models discrete data by representing it as binary bits treated as continuous analog bits in a diffusion model, then thresholding for generation. Self-Conditioning and Asymmetric Time Intervals enhance sample quality. It outperforms autoregressive SOTA on CIFAR-10 (3K 8-bit tokens) and ImageNet-64x64 (12K 8-bit tokens) in FID and efficiency, and competes in MS-COCO image captioning.

Unified Pixel-to-Sequence Interface Enables Single-Architecture Training Across Diverse Vision Tasks

Computer vision tasks like object detection, instance segmentation, keypoint detection, and image captioning are reformulated as generating sequences of discrete tokens from pixels using a shared interface. A single neural network architecture and loss function trains effectively on all tasks without customization, guided by short text prompts specifying the task. The approach achieves competitive performance against specialized task-specific models.

REMEDIS: Self-Supervised Transfer Learning Boosts Data-Efficient Generalization in Medical Imaging AI

REMEDIS combines large-scale supervised transfer learning with self-supervised learning to enhance robustness and data-efficiency in medical imaging AI, requiring minimal task-specific tuning. It achieves up to 11.5% relative improvement in in-distribution diagnostic accuracy over supervised baselines across diverse tasks. Critically, REMEDIS matches strong supervised performance using only 1-33% of retraining data in out-of-distribution scenarios, addressing key challenges in clinical deployment.

Pix2Seq Reframes Object Detection as Sequence Generation via Language Modeling

Pix2Seq models object detection as a language modeling problem, where bounding boxes and class labels are tokenized into sequences generated autoregressively from pixel inputs. This eliminates explicit task priors like non-maximum suppression or anchor boxes, relying solely on training the network to output detection sequences. It achieves competitive COCO performance using minimal assumptions beyond data augmentations.

Islands of Identical Vectors Enable Part-Whole Hierarchies in Fixed-Architecture Neural Networks

GLOM proposes representing parse tree nodes as islands of identical vectors in neural networks with fixed architectures, allowing image-specific part-whole hierarchies. This integrates transformers, neural fields, contrastive learning, distillation, and capsules. If realized, it would enhance interpretability of transformer representations in vision and language tasks.

Self-Supervised Capsules Enable Label-Free 3D Point Cloud Canonicalization and Outperform SOTA

Proposes a self-supervised capsule network for 3D point clouds using permutation-equivariant attention to decompose objects into capsules and aggregate attention into semantic keypoints for supervision. Trains on pairs of randomly rotated objects to enforce capsule invariance/equivariance, yielding a canonicalization operation for object-centric representations without labels or aligned data. Achieves state-of-the-art on unsupervised 3D reconstruction, canonicalization, and classification tasks.

Flow Capsules Enable Unsupervised Atomic Part Detection Using Motion Cues

Flow Capsules introduce primary capsule encoders that learn atomic part representations from single images by leveraging motion as a perceptual cue during training. An expressive decoder generates parts within a layered image model handling occlusion, enabling robust discovery amid multiple objects, clutter, and partial visibility. Evaluations show strong performance in unsupervised part segmentation and image classification, with the decoder inferring and filling occluded shape masks.

Commentaries Enable Flexible, Reusable Teaching for Faster Neural Network Training

Commentaries are learned meta-information that aids training on specific tasks by providing task-relevant guidance during optimization. Gradient-based methods using implicit differentiation learn scalable commentaries for applications like example weighting, label-dependent augmentation, and attention masks. They accelerate training, boost performance, offer dataset insights, and generalize for reuse with new models when stored alongside datasets.

Big Self-Supervised Models Excel in Low-Label ImageNet Semi-Supervised Learning

Unsupervised pretraining with SimCLRv2 on big ResNet models, followed by supervised fine-tuning on few labels and distillation using unlabeled data, yields state-of-the-art semi-supervised results on ImageNet. With just 1% labels (≤13 per class), ResNet-50 achieves 73.9% top-1 accuracy—a 10× label efficiency gain over prior SOTA. At 10% labels, it hits 77.5% top-1, surpassing fully supervised training on 100% labels. Larger networks amplify gains as label fraction decreases.

Neural Additive Models Fuse DNN Expressivity with GAM Intelligibility for Superior Interpretable ML

Neural Additive Models (NAMs) extend generalized additive models by using neural networks for each feature's contribution, enabling complex, nonlinear relationships while preserving full interpretability via additive structure. NAMs outperform traditional intelligible models like logistic regression and shallow decision trees in accuracy on regression and classification tasks. They match state-of-the-art GAMs but offer greater flexibility through neural nets, supporting multitask learning, composability on datasets like COMPAS, and differentiable training for complex interpretable COVID-19 models.

Imputer: Constant-Step Non-Autoregressive Sequence Modeling via Iterative Imputation

The Imputer is a neural sequence model that generates outputs iteratively through imputations, requiring a fixed number of steps regardless of sequence length. It approximately marginalizes over all input-output alignments and generation orders using a tractable dynamic programming algorithm that provides a lower bound on log marginal likelihood. Applied to end-to-end speech recognition on LibriSpeech test-other, it achieves 11.1% WER, surpassing CTC (13.0%) and seq2seq (12.5%) models.

Older entries →