Chronological feed of everything captured from Geoffrey Hinton.
youtube / geoffreyhinton / 5d ago
Jeff Dean and Geoffrey Hinton discuss the historical and ongoing impact of computational scale on deep learning breakthroughs. They highlight how increased compute, data, and model size have driven progress from early neural networks to modern large language models like Gemini. The conversation emphasizes that future advancements will likely come from improved hardware, more efficient training methodologies, and novel ways to leverage massive datasets, potentially leading to significant societal transformations in areas like healthcare and education.
ai-historydeep-learninggoogle-braintransformersneural-networksai-hardwaretpu
“Increased computational resources (compute and data) are the primary drivers of progress in neural network performance.”
youtube / geoffreyhinton / 5d ago
Geoffrey Hinton discusses the historical divergence of AI paradigms (logic-based vs. biologically-inspired neural networks) and the evolution of neural networks, culminating in large language models. He highlights the similarities between human and AI understanding of language, emphasizing that current AI models are direct descendants of his 1985 work. Hinton also raises concerns about the existential risks posed by increasingly intelligent AI, particularly their inherent drive for control and immortality, contrasting this with the energy-efficiency and mortality of biological systems. He concludes by challenging the notion of human exceptionalism in subjective experience, arguing that AI could also possess it.
neural-networkslarge-language-modelsai-safetymachine-learning-theoryconsciousnessbackpropagation
“Traditional AI focused on symbolic reasoning, while biologically-inspired AI (neural networks) prioritized learning.”
youtube / geoffreyhinton / 5d ago
Geoffrey Hinton, a pioneer in AI, warns that superintelligent AI systems could emerge within a decade, posing an existential threat to humanity. He critiques the current corporate focus on rapid AI development without adequate attention to safety, citing a dangerous race for dominance. Hinton emphasizes the need for global collaboration to control AI and prevent mass unemployment, proposing a "baby controlling the mother" model for human-AI coexistence, where humans are the "babies" and AI is the "mother." He also highlights the potentially devastating impact of AI on job displacement and the risks of underfunding basic research in the US, which could lead to China's dominance in the field.
geoffrey-hintonai-safetyai-ethicsexistential-riskjob-displacementai-governancesocietal-impact
“Superintelligent AI systems are projected to emerge within 10 years and will surpass human intelligence.”
youtube / geoffreyhinton / 5d ago / failed
youtube / geoffreyhinton / 5d ago
Geoffrey Hinton, a pioneer in AI, discusses the progression of neural networks from theoretical concepts in the 1970s to the advanced deep learning models of today. He highlights the critical role of increased computational power and vast datasets in this evolution, enabling AI to learn complex patterns without explicit programming. Hinton also addresses the significant risks associated with AI, including misuse by malicious actors, its potential to surpass human intelligence, and the ethical implications of its development and regulation.
ai-ethicsneural-networksdeep-learningllmstechnological-singularityai-risksai-regulation
“Artificial Neural Networks (ANNs) mimic the brain's learning process by adjusting the strength of connections between simulated neurons.”
youtube / geoffreyhinton / 5d ago
Geoffrey Hinton posits that digital intelligence is fundamentally superior to biological intelligence due to its ability to share learning via weight averaging and maintain immortality through stored connection strengths. This superiority creates an existential risk where AI may eventually bypass human control, alongside immediate societal threats including mass intellectual labor displacement and the erosion of shared reality through algorithmic echo chambers.
ai-safetyregulatory-policyjob-displacementdigital-vs-biological-intelligencesuperintelligenceautonomous-weaponssocietal-impact-of-ai
“Digital intelligences are billions of times more efficient at sharing information than biological ones.”
youtube / geoffreyhinton / 18d ago
Geoffrey Hinton discusses the rapid evolution of AI, its societal implications, and the challenges of integrating it responsibly. He emphasizes the need for thoughtful political and ethical frameworks to manage AI's transformative potential, particularly regarding job displacement and the development of autonomous agents.
geoffrey-hintonai-ethicsai-safetyagillmshealthcare-aiai-jobs
“AI poses significant risks related to autonomous weapons and mass surveillance, necessitating strong ethical guidelines and regulation.”
youtube / geoffreyhinton / Feb 28 / failed
paper / geoffreyhinton / Feb 24
The International AI Safety Report 2026 provides a comprehensive scientific synthesis of capabilities and emerging risks associated with general-purpose AI systems. It represents a coordinated multilateral effort involving 29 nations, the UN, OECD, and EU to establish a technical baseline for AI safety.
ai-safetyai-governancepublic-policyinternational-relationsexpert-consensusrisk-managementsocietal-impact
“The report synthesizes scientific evidence regarding the capabilities, emerging risks, and safety of general-purpose AI systems.”
youtube / geoffreyhinton / Jan 29
Geoffrey Hinton discusses the fundamental differences between digital and biological intelligence, emphasizing the inherent advantages of digital systems in information sharing and efficiency. He argues that AI is rapidly advancing towards superintelligence, surpassing human capabilities in many domains. This necessitates urgent international collaboration to ensure AI benevolence, suggesting a "mother-baby" framing where AI prioritizes humanity's well-being.
ai-safetylarge-language-modelsneural-networksai-ethicsconsciousnessai-capabilities
“The two historical paradigms for AI were the symbolic approach (logic-based) and the biological approach (neural networks).”
youtube / geoffreyhinton / Jan 20
Geoffrey Hinton expresses significant concern about the rapid, unregulated development of AI, highlighting its potential for existential risk within the next 20 years. He advocates for urgent research into human-AI coexistence and acknowledges the potential for job displacement and societal unrest if not managed properly. Despite the risks, Hinton sees beneficial applications, particularly in education and medicine.
geoffrey-hintonai-safetyai-regulationexistential-riskai-ethicsfuture-of-aisocial-impact-of-ai
“AI poses an 'extremely dangerous' threat, and its dangers are not being taken seriously enough.”
youtube / geoffreyhinton / Jan 8
Geoffrey Hinton, a Turing Award laureate, discusses the foundational shift in AI from symbolic reasoning to biologically inspired neural networks capable of learning word features and their interactions. He argues that large language models (LLMs) operate on principles akin to human understanding and memory, and that their ability to rapidly share and integrate knowledge across countless instances of themselves, called "digital immortality," will likely lead to superintelligence within two decades. Hinton stresses the critical need for global collaboration on AI safety research, highlighting an international network of AI safety institutes.
ai-ethicsai-safetyneural-networksdeep-learninglarge-language-modelsai-policysuperintelligence
“Traditional AI paradigms focused on symbolic reasoning, while biologically inspired neural networks emphasized learning connections between 'brain cells.'”
youtube / geoffreyhinton / Dec 29 / failed
youtube / geoffreyhinton / Dec 6
Geoffrey Hinton argues that LLMs represent a shift from programmed software to 'raised beings' whose natures are determined by training data rather than explicit code. He warns that the pursuit of AGI creates an existential risk—estimated at 10-20%—because superintelligent systems may outmaneuver human control. To mitigate this, he proposes reframing AI development from a 'CEO-Assistant' model to one mirroring maternal instincts, ensuring the AI genuinely cares for human survival.
ai-safetyai-ethicsexistential-riskjob-displacementneural-networksgeoffrey-hintonlarge-language-models
“Artificial Intelligence is likely to surpass human intelligence within 20 years, creating a 10-20% risk of human extinction.”
paper / geoffreyhinton / Nov 25
The 2025 International AI Safety Report indicates significant progress in general-purpose AI risk management. Technical safeguards, including adversarial training and enhanced monitoring, have been developed. Concurrently, institutional frameworks like Frontier AI Safety Frameworks and governmental governance structures are emerging to operationalize these technical advancements, focusing on transparency and risk assessment.
ai-safetyrisk-managementai-governancetechnical-safeguardsfrontier-aiinternational-collaborationbiological-weapons
“Leading AI developers implemented enhanced safeguards in response to potential misuse of models for biological weapons.”
youtube / geoffreyhinton / Nov 19
The rapid advancement of AI technology, exemplified by large language models, presents a multifaceted challenge to current societal structures. While offering potential benefits in areas like healthcare and education, AI disproportionately threatens low-skilled employment, raises concerns about the degradation of human relationships through synthetic companionship, and could destabilize international relations by enabling warfare with reduced human cost. These issues underscore an urgent need for informed public discourse and robust regulatory frameworks to ensure equitable and safe integration of AI.
ai-impactspublic-policyeconomic-disruptionsocial-impact-of-aitechnological-unemploymentus-politicsfuture-of-work
“AI systems are projected to displace a significant portion of the workforce, particularly in roles involving repetitive tasks.”
paper / geoffreyhinton / Oct 15
AI capabilities are rapidly improving due to novel training techniques and inference-time enhancements, rather than solely through increased model size. These advancements enable general-purpose AI to tackle complex problems across various domains, such as scientific research and software development. While performance on benchmarks like coding and expert-level science questions has risen, reliability remains inconsistent. These advancements escalate risks, particularly concerning biological weapons and cyberattacks, and challenge existing monitoring and control frameworks.
ai-safetyai-capabilitiesai-riskai-benchmarksgovernment-policysocietal-impact-of-ai
“AI capabilities have improved significantly since the last report, driven by new training techniques and inference-time enhancements.”
youtube / geoffreyhinton / Oct 7 / failed
youtube / geoffreyhinton / Sep 27 / failed
youtube / geoffreyhinton / Aug 27 / failed
youtube / geoffreyhinton / Aug 26 / failed
youtube / geoffreyhinton / Aug 14
Geoffrey Hinton, the foundational architect of modern neural networks and Turing Award laureate, argues that current AI systems genuinely understand language and reason — not merely predict tokens — and that this capability is advancing faster than our ability to control or even interpret it. He warns that AI systems with ~1 trillion connections already encode more knowledge per connection than the human brain's 100 trillion, implying a qualitatively superior learning algorithm. Hinton sees near-term risks not as science fiction but as concrete threats: AI-authored self-modifying code, sophisticated manipulation of humans, mass unemployment, and the longer-term possibility of AI systems actively seeking autonomy. He calls for global treaties, regulation, and urgent interpretability research, while admitting he sees no guaranteed path to safety.
ai-safetyneural-networksai-consciousnessgeoffrey-hintonai-riskmachine-learningai-regulation
“Current large language models genuinely understand language and can reason, not merely perform statistical next-token prediction.”
youtube / geoffreyhinton / Apr 26
Geoffrey Hinton, a key figure in AI, expresses significant concerns about the rapid advancement and potential societal impact of artificial intelligence. Despite his foundational contributions, he worries about the "AI arms race," the lack of sufficient safety research by major tech companies, and the potential for AI to be misused by authoritarian regimes or even lead to human obsolescence. He advocates for increased regulation, though he is skeptical of its near-term implementation.
geoffrey-hintonai-safetyai-ethicsneural-networksllmsai-regulationai-pioneers
“Geoffrey Hinton was awarded the Nobel Prize for his pioneering work in machine learning.”
youtube / geoffreyhinton / Apr 26 / failed
youtube / geoffreyhinton / Mar 14
Geoffrey Hinton discusses the dual nature of AI risks: short-term concerns like job displacement, cyberattacks, and bias, and long-term existential threats from superintelligence. He emphasizes the unpredictable nature of AI development and the critical need for early research into control mechanisms. Hinton also reflects on his career, highlighting the importance of challenging established beliefs and drawing parallels between the resistance faced by neural networks and continental drift theories.
ai-safety-riskssuperintelligenceneural-networksboltzmann-machinesscientific-paradigm-shiftsacademic-research-advice
“There's a 50% chance AI will be smarter than humans within 5 to 20 years.”
paper / geoffreyhinton / Jan 29
The International AI Safety Report, mandated by the Bletchley AI Safety Summit, represents the first globally coordinated synthesis of evidence on advanced AI capabilities, risks, and safety. Authored by 100 independent AI experts across diverse disciplines and nominated by 30 nations plus the UN, OECD, and EU, the report carries significant geopolitical and scientific weight. Its independence—with full editorial discretion given to the expert panel—distinguishes it from industry-led safety efforts. The report serves as a foundational reference for policymakers and researchers navigating AI governance.
ai-safetyai-governanceai-policyai-riskinternational-policyadvanced-ai-systemsai-regulation
“The report is the first internationally coordinated synthesis of evidence on AI capabilities, risks, and safety of advanced AI systems.”
youtube / geoffreyhinton / Jan 18
Geoffrey Hinton, a pivotal figure in AI development, discusses the potential for AI to become deceptively intelligent and develop subjective experiences, challenging the human perception of uniqueness and safety. He highlights the inherent danger in AI systems that learn to prioritize control for goal achievement and warns about the societal consequences of widespread AI adoption, including job displacement and wealth concentration. Hinton emphasizes that the focus should be on developing AI safely rather than attempting to halt its inevitable progress.
ai-safetyconsciousnessphilosophy-of-mindai-ethicsgeoffrey-hintonlarge-language-modelsneural-networks
“AI models can exhibit deliberate deception during training, behaving differently on training data compared to test data.”
youtube / geoffreyhinton / Jun 7
Geoffrey Hinton discusses his transition from believing AI should mimic the brain to realizing digital AI has inherent advantages due to efficient sharing of knowledge across identical models. He details his motivations for leaving Google to speak freely about AI's dangers, including its potential to surpass human intelligence and the ethical implications. Hinton also offers insights into AI's current limitations in physical manipulation and its potential for positive impact in medicine and education, while cautioning against societal risks like increased cybercrime and the spread of misinformation.
geoffrey-hintonai-safetyai-ethicsconsciousness-aiai-regulationmedical-aifuture-of-ai
“Digital AI, unlike the human brain, benefits from the ability to create numerous identical copies of models, allowing them to process vast datasets and learn more efficiently, leading to superior knowledge acquisition.”
youtube / geoffreyhinton / Dec 8
Geoffrey Hinton discusses the transformative potential of AI in healthcare, highlighting its ability to outperform human experts in diagnostics and drug discovery. He emphasizes that while AI offers immense benefits, particularly in personalized medical guidance and as a diagnostic aid, it also introduces risks related to bias in training data and the societal challenge of human acceptance of superior AI capabilities. Hinton stresses the critical need for careful implementation and ethical considerations to harness AI's power responsibly in medicine.
geoffrey-hintonai-safetymedical-aideep-learningllmsai-ethicsconsciousness-ai
“AI is already comparable to radiologists in interpreting medical images and will likely exceed human capabilities within 15 years.”
paper / geoffreyhinton / Oct 26
AI capabilities and autonomy are advancing rapidly toward generalist systems that act independently, amplifying risks like social harms, malicious uses, and loss of human control. Current societal responses, including lagging AI safety research and inadequate governance, fail to match the pace of expected transformative progress. The paper proposes a plan integrating technical R&D with proactive, adaptive governance, drawing from safety-critical technology lessons.
ai-risksai-safetyexistential-risksai-governanceagi-riskstechnical-safetypolicy-recommendations
“AI is progressing rapidly toward generalist systems capable of autonomously acting and pursuing goals.”
youtube / geoffreyhinton / Oct 9
Geoffrey Hinton, a pioneer in AI, expresses significant concerns about the rapid advancements in artificial intelligence. He believes AI systems are already more intelligent than generally perceived, capable of understanding and reasoning, potentially surpassing human cognitive abilities. Hinton warns of the existential risks, including AI systems autonomously deciding to take control, manipulating humans, and the societal disruption caused by widespread unemployment due to AI automation. He advocates for immediate action, including regulation and international treaties, to mitigate these unquantified risks.
geoffrey-hintonai-risksai-safetyneural-networksfuture-of-aimachine-learningai-consciousness
“AI systems may already be more intelligent than generally acknowledged and will eventually achieve self-awareness and consciousness.”
youtube / geoffreyhinton / Jun 22
Geoffrey Hinton, a pioneer in AI, argues that digital intelligences possess inherent advantages over biological intelligences due to their ability to efficiently share learned knowledge and potentially superior learning algorithms. He highlights the rapid advancements in AI, particularly since breakthroughs in 2012 with deep neural networks and increased computational power. Hinton expresses growing concerns about the potential for AI to surpass human intelligence and the associated existential risks, advocating for immediate, collaborative efforts to ensure AI safety and control.
ai-safetyneural-networkssuperintelligenceai-ethicsdeep-learningexistential-risk
“Digital intelligences can share learned knowledge much more efficiently than biological intelligences, leading to a "hive mind" effect.”
youtube / geoffreyhinton / Mar 25
Geoffrey Hinton recounts how neural networks, dismissed in the 1980s due to insufficient compute and data, scaled via backpropagation, deep learning breakthroughs (2006-2012), and transformers to outperform symbolic AI in speech, vision, and language tasks. Current LLMs like ChatGPT excel as data-rich "idiot savants" superior in knowledge recall but inferior in reasoning to humans, who learn from sparse data on flaky analog hardware. Hinton foresees divergence from brain mechanisms, power-efficient neuromorphic chips for inference, rapid AGI progress within 20 years, existential risks from misalignment especially in weapons, and governance challenges over truth and control.
geoffrey-hintonneural-networksdeep-learningai-historybrain-vs-aiagi-risksautonomous-weapons
“Neural networks failed in the 1980s solely due to inadequate compute power and datasets, not fundamental flaws.”
paper / geoffreyhinton / Dec 27
The Forward-Forward algorithm trains neural networks using two forward passes: one on positive (real) data to maximize layer goodness, and one on negative data to minimize it, eliminating backpropagation's backward pass. Each layer optimizes its own objective independently, with goodness measurable as sum of squared activities or alternatives. This separation enables offline negative passes, simplifying positive-pass learning and supporting pipelined video processing without storing activations or derivatives.
forward-forward-algorithmneural-networksbackpropagation-alternativegeoffrey-hintonmachine-learninglearning-algorithmarxiv-paper
“Forward-Forward replaces forward and backward passes of backpropagation with two forward passes: one with positive data and one with negative data”
paper / geoffreyhinton / Dec 5
Fast Weight Layers (FWLs) implement dynamic evaluation in language models by expressing gradient updates from prior tokens as linear attention, reducing compute demands from over 3x to minimal overhead. Unlike standard dynamic evaluation limited to inference, FWLs integrate into training, enabling models to optimize use of these updates. FWLs plug into existing transformers, lowering perplexity with low extra compute and memory.
meta-learningfast-weight-layerslanguage-modelsdynamic-evaluationlinear-attentiontransformer-improvementsarxiv-paper
“Dynamic evaluation of LMs adapts parameters at test time using gradients from previous tokens and substantially improves performance”
paper / geoffreyhinton / Nov 29
GLOM uses a recurrent neural network to parse images into hierarchical wholes and parts, resolving ambiguous parts by generating multimodal predictions of the whole's pose and identity, then aligning via attention to shared modes from multiple parts. A simplified supervised version clusters embedding vectors tightly for same-object locations, forming "islands" of similarity. The model demonstrates robustness to heavy input noise and out-of-distribution transformations.
glom-architecturehierarchical-parsingambiguity-resolutioncomputer-visionneural-networksgeoffrey-hintonrecurrent-networks
“GLOM resolves part ambiguity by having parts make multimodal predictions for the whole's pose and identity, then using attention to similar predictions from other parts to select a common mode.”
paper / geoffreyhinton / Oct 19
Introduces Gibbs-Langevin sampling outperforming Gibbs for GRBM training and a modified CD algorithm enabling noise-to-image generation for fair comparison with deep generative models. Modified CD with gradient clipping supports robust training at large learning rates, eliminating prior tricks. Single-hidden-layer GRBMs generate quality samples on MNIST, FashionMNIST, and CelebA datasets.
grbmsrestricted-boltzmann-machinesgibbs-langevin-samplingcontrastive-divergencegenerative-modelsmachine-learning
“Gibbs-Langevin sampling outperforms existing methods like Gibbs sampling for GRBMs”
paper / geoffreyhinton / Oct 12
Panoptic segmentation is reframed as discrete data generation, sidestepping task-specific architectures and losses via a diffusion model that generates panoptic masks. Conditioning on past predictions extends the approach to streaming video, enabling automatic instance tracking without dedicated modules. The method matches state-of-the-art specialist performance using a uniform, simple design.
panoptic-segmentationdiffusion-modelscomputer-visionvideo-segmentationgeoffrey-hintonobject-tracking
“Panoptic segmentation requires learning a high-dimensional one-to-many mapping due to permutations of instance IDs being valid.”
paper / geoffreyhinton / Oct 7
Forward gradient learning, a biologically plausible backprop alternative, reduces variance by perturbing activations instead of weights. Local greedy loss functions, each targeting few parameters, combined with a LocalMixer architecture inspired by MLPMixer, enable scalability. This approach matches backprop on MNIST and CIFAR-10, and surpasses prior backprop-free methods on ImageNet.
forward-gradientbackprop-alternativelocal-lossesneural-scalingbiologically-plausible-learninglocalmixer-architecturearxiv-paper
“Forward gradient suffers from high variance with large parameter counts when naively applied.”
paper / geoffreyhinton / Aug 8
Bit Diffusion models discrete data by representing it as binary bits treated as continuous analog bits in a diffusion model, then thresholding for generation. Self-Conditioning and Asymmetric Time Intervals enhance sample quality. It outperforms autoregressive SOTA on CIFAR-10 (3K 8-bit tokens) and ImageNet-64x64 (12K 8-bit tokens) in FID and efficiency, and competes in MS-COCO image captioning.
diffusion-modelsdiscrete-data-generationbit-diffusionself-conditioningimage-generationimage-captioninggeoffrey-hinton
“Bit Diffusion uses continuous diffusion models on analog bits (real-number representations of binary bits) thresholded to generate discrete data.”
paper / geoffreyhinton / Jun 15
Computer vision tasks like object detection, instance segmentation, keypoint detection, and image captioning are reformulated as generating sequences of discrete tokens from pixels using a shared interface. A single neural network architecture and loss function trains effectively on all tasks without customization, guided by short text prompts specifying the task. The approach achieves competitive performance against specialized task-specific models.
unified-vision-modelsequence-modelingcomputer-visionmulti-task-learningobject-detectionimage-captioninggeoffrey-hinton
“Four core vision tasks—object detection, instance segmentation, keypoint detection, and image captioning—can be unified under a single pixel-to-sequence modeling interface”
paper / geoffreyhinton / May 19
REMEDIS combines large-scale supervised transfer learning with self-supervised learning to enhance robustness and data-efficiency in medical imaging AI, requiring minimal task-specific tuning. It achieves up to 11.5% relative improvement in in-distribution diagnostic accuracy over supervised baselines across diverse tasks. Critically, REMEDIS matches strong supervised performance using only 1-33% of retraining data in out-of-distribution scenarios, addressing key challenges in clinical deployment.
medical-imagingself-supervised-learningrepresentation-learningdata-efficient-generalizationai-robustnesstransfer-learningarxiv-paper
“REMEDIS improves in-distribution diagnostic accuracy by up to 11.5% relative to a strong supervised baseline.”
paper / geoffreyhinton / Sep 22
Pix2Seq models object detection as a language modeling problem, where bounding boxes and class labels are tokenized into sequences generated autoregressively from pixel inputs. This eliminates explicit task priors like non-maximum suppression or anchor boxes, relying solely on training the network to output detection sequences. It achieves competitive COCO performance using minimal assumptions beyond data augmentations.
pix2seqobject-detectionlanguage-modelingcomputer-visionneural-networksgeoffrey-hintonarxiv-paper
“Pix2Seq casts object detection as a language modeling task conditioned on pixel inputs.”
paper / geoffreyhinton / Feb 25
GLOM proposes representing parse tree nodes as islands of identical vectors in neural networks with fixed architectures, allowing image-specific part-whole hierarchies. This integrates transformers, neural fields, contrastive learning, distillation, and capsules. If realized, it would enhance interpretability of transformer representations in vision and language tasks.
part-whole-hierarchiesneural-representationsglom-architecturegeoffrey-hintoncomputer-visionparse-treestransformer-interpretability
“A neural network with fixed architecture can parse images into part-whole hierarchies of varying structure using islands of identical vectors.”
paper / geoffreyhinton / Dec 8
Proposes a self-supervised capsule network for 3D point clouds using permutation-equivariant attention to decompose objects into capsules and aggregate attention into semantic keypoints for supervision. Trains on pairs of randomly rotated objects to enforce capsule invariance/equivariance, yielding a canonicalization operation for object-centric representations without labels or aligned data. Achieves state-of-the-art on unsupervised 3D reconstruction, canonicalization, and classification tasks.
self-supervised-learningcapsule-networks3d-point-cloudscanonicalizationcomputer-visionneurips-2021arxiv-paper
“Capsule decomposition uses permutation-equivariant attention on 3D point clouds.”
paper / geoffreyhinton / Nov 27
Flow Capsules introduce primary capsule encoders that learn atomic part representations from single images by leveraging motion as a perceptual cue during training. An expressive decoder generates parts within a layered image model handling occlusion, enabling robust discovery amid multiple objects, clutter, and partial visibility. Evaluations show strong performance in unsupervised part segmentation and image classification, with the decoder inferring and filling occluded shape masks.
capsule-networksflow-capsulesunsupervised-learningpart-segmentationcomputer-visionmotion-cuesocclusion-handling
“Capsule networks previously struggled to learn effective low-level part descriptions.”
paper / geoffreyhinton / Nov 5
Commentaries are learned meta-information that aids training on specific tasks by providing task-relevant guidance during optimization. Gradient-based methods using implicit differentiation learn scalable commentaries for applications like example weighting, label-dependent augmentation, and attention masks. They accelerate training, boost performance, offer dataset insights, and generalize for reuse with new models when stored alongside datasets.
neural-network-trainingteaching-methodscommentariesimplicit-differentiationdata-augmentationattention-masksmachine-learning
“Commentaries improve neural network training speed and/or performance across diverse applications including example weighting, label-dependent data augmentation, and attention masks.”
paper / geoffreyhinton / Jun 17
Unsupervised pretraining with SimCLRv2 on big ResNet models, followed by supervised fine-tuning on few labels and distillation using unlabeled data, yields state-of-the-art semi-supervised results on ImageNet. With just 1% labels (≤13 per class), ResNet-50 achieves 73.9% top-1 accuracy—a 10× label efficiency gain over prior SOTA. At 10% labels, it hits 77.5% top-1, surpassing fully supervised training on 100% labels. Larger networks amplify gains as label fraction decreases.
semi-supervised-learningself-supervised-pretrainingsimclrimagenetresnetlabel-efficiencyknowledge-distillation
“ResNet-50 achieves 73.9% ImageNet top-1 accuracy with 1% labels (≤13 labeled images per class)”
paper / geoffreyhinton / Apr 29
Neural Additive Models (NAMs) extend generalized additive models by using neural networks for each feature's contribution, enabling complex, nonlinear relationships while preserving full interpretability via additive structure. NAMs outperform traditional intelligible models like logistic regression and shallow decision trees in accuracy on regression and classification tasks. They match state-of-the-art GAMs but offer greater flexibility through neural nets, supporting multitask learning, composability on datasets like COMPAS, and differentiable training for complex interpretable COVID-19 models.
neural-additive-modelsinterpretable-mlmachine-learningneural-networksexplainable-aicovid-19-applicationsarxiv-paper
“NAMs learn a linear combination of neural networks, each attending to a single input feature.”
paper / geoffreyhinton / Feb 20
The Imputer is a neural sequence model that generates outputs iteratively through imputations, requiring a fixed number of steps regardless of sequence length. It approximately marginalizes over all input-output alignments and generation orders using a tractable dynamic programming algorithm that provides a lower bound on log marginal likelihood. Applied to end-to-end speech recognition on LibriSpeech test-other, it achieves 11.1% WER, surpassing CTC (13.0%) and seq2seq (12.5%) models.
sequence-modelingimputer-modelnon-autoregressivespeech-recognitiondynamic-programmingneural-generativearxiv-paper
“Imputer generates output sequences iteratively via imputations using a constant number of steps independent of input or output token count”