[siteorigin_widget class=”thinkup_builder_divider”][/siteorigin_widget]
NeurIPS 2022 declared 13 submissions as outstanding papers from its main track.
- Is Out-of-distribution Detection Learnable?
- Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding
- Elucidating the Design Space of Diffusion-Based Generative Models
- ProcTHOR: Large-Scale Embodied AI Using Procedural Generation
- Using Natural Language and Program Abstractions to Instill Human Inductive Biases in Machines
- A Neural Corpus Indexer for Document Retrieval
- High-dimensional Limit Theorems for SGD: Effective Dynamics and Critical Scaling
- Gradient Descent: The Ultimate Optimizer
- Riemannian Score-Based Generative Modelling
- Gradient Estimation with Discrete Stein Operators
- An Empirical Analysis of Compute-optimal Large Language Model Training
- Beyond Neural Scaling Laws: Beating Power Law Scaling via Data Pruning
- On-Demand Sampling: Learning Optimally from Multiple Distributions
Here are my top two picks from this list. The criteria for consideration are perceived utility, and ease of application in real-life scenarios.
[1] A Neural Corpus Indexer for Document Retrieval
Why do I consider it important? Index-Retrieve is arguably the most widely-followed approach in document retrieval. This paper demonstrates how a sequence-to-sequence network called Neural Corpus Indexer (NCI) unifies training and indexing to significantly improve upon conventional systems, particularly those based on Inverted Index and Dense Retrieval approaches. A key aspect of this paper is to leverage a novel decoder architecture called Prefix-Aware Weight-Adaptive (PAWA) to generate document identifiers.
[2] Gradient Descent: The Ultimate Optimizer
Why do I consider it important? This paper demonstrates how to automatically compute hypergradients through the backpropagation approach. The technique published by the researchers enables gradient descent-based optimizers to tune hyperparameters through automatic differentiation.