Recent advancements in artificial intelligence have fueled considerable excitement around what many call “AI Chips”, specialized hardware tailored to optimize and accelerate AI workloads. Amidst the noise, many companies appear to be branding nearly every processor enhancement or hardware upgrade as an AI chip, blurring
Mathematical optimization drives complex decision-making across a diverse range of problems – e.g., energy management, inventory planning, network design, pricing & revenue management, production planning & scheduling, supplier selection, and transportation planning. This field has significantly evolved over decades, from its formative years around World
A recent paper jointly published by DeepMind, LG, and KAIST researchers introduced a new innovation for Transformer models: the global-to-local Block Architecture. The paper highlights up to 20x improvement in inference throughput vis-à-vis conventional transformer models. While this claim needs to be empirically validated by
MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts This paper combines State Space Modeling (SSM) with the Mixture of Experts (MoE) approach, and introduces the MoE-Mamba model in which every other Mamba layer is replaced with a MoE feed-forward layer based on the
A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models This paper highlights 32 techniques to mitigate hallucination in LLMs, including a well-defined taxonomy to categorize these methods. AlphaGeometry: An Olympiad-level AI system for geometry DeepMind introduced its AI system for solving Olympiad
Deep Learning-based architectures have had a significant impact on computer vision, natural language processing, and other machine learning areas. However, the scenario is not so straightforward when it comes to Forecasting, an area where statistical and traditional machine learning models have generally outperformed other types
Google introduced Spotlight, a foundational model for mobile UI modeling, particularly for tasks like command grounding, screen summarization, tappability prediction, and widget captioning. Traditional mobile UI design often uses the concept of view hierarchy information, but these view hierarchies are sometimes either not available, or
Microsoft introduced VALL-E, its neural codec language model for zero-shot Text-to-Speech Synthesis (TTS) that generates high-quality audio/speech with only a 3-second acoustic prompt (i.e., voice recording.) Unlike conventional models that consider TTS a continuous signal regression task, VALL-E approaches this as a conditional language modeling
Large Language Models (LLMs) are state-of-the-art today, and generally perform well for simple and low-interaction tasks, such as single-turn conversations, and command-and-response systems. However, their direct use is generally limited in the case of applications with complex and high-interaction tasks, such as multi-turn dialogue systems,
Generative AI technologies have garnered much attention over the last 24 to 30 months. Innovations like GPT-3 and 3.5 (ChatGPT), DALL-E2, and Stable Diffusion have occupied a large part of the public discourse. The developers of these technologies, and the entire ecosystems around them are
[siteorigin_widget class=”thinkup_builder_divider”][/siteorigin_widget] NeurIPS 2022 declared 13 submissions as outstanding papers from its main track. Is Out-of-distribution Detection Learnable? Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding Elucidating the Design Space of Diffusion-Based Generative Models ProcTHOR: Large-Scale Embodied AI Using Procedural Generation Using Natural Language and
3D Vision with Transformers: A Survey: https://arxiv.org/pdf/2208.04309v1.pdf Unifying Visual Perception by Dispersible Points Learning: https://arxiv.org/pdf/2208.08630v1.pdf ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wild: https://arxiv.org/pdf/2208.11547v1.pdf ROLAND: Graph Learning Framework for Dynamic Graphs: https://arxiv.org/pdf/2208.07239v1.pdf Investigating Efficiently Extending Transformers for Long Input Summarization: https://arxiv.org/pdf/2208.04347v1.pdf Semantic-Aligned Matching
High-Performance GPU-to-CPU Transpilation and Optimization via High-Level Parallel Constructs: https://arxiv.org/pdf/2207.00257.pdf Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understanding: https://arxiv.org/pdf/2207.02971v1.pdf More ConvNets in the 2020s: Scaling up Kernels Beyond 51 × 51 using Sparsity: https://arxiv.org/pdf/2207.03620v1.pdf Softmax-free Linear Transformers: https://arxiv.org/pdf/2207.03341v1.pdf
Scaling Vision Transformers: https://arxiv.org/pdf/2106.04560.pdf Learning Efficient Vision Transformers via Fine-Grained Manifold Distillation: https://arxiv.org/pdf/2107.01378.pdf Risk-averse autonomous systems: A brief history and recent developments from the perspective of optimal control: https://arxiv.org/pdf/2109.08947.pdf LightSeq2: Accelerated Training for Transformer-based Models on GPUs: https://arxiv.org/pdf/2110.05722.pdf Conditionally Elicitable Dynamic Risk Measures For Deep
Computational Storytelling And Emotions: A Survey: https://arxiv.org/pdf/2205.10967.pdf Are Large Pre-Trained Language Models Leaking Your Personal Information?: https://arxiv.org/pdf/2205.12628.pdf FreDo: Frequency Domain-based Long-Term Time Series Forecasting: https://arxiv.org/pdf/2205.12301.pdf A Survey on Long-tailed Visual Recognition: https://arxiv.org/pdf/2205.13775.pdf Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors: https://arxiv.org/pdf/2205.12854.pdf On the
Multiview Transformers for Video Recognition: https://arxiv.org/pdf/2201.04288.pdf ViNTER: Image Narrative Generation with Emotion-Arc-Aware Transformer: https://arxiv.org/pdf/2202.07305.pdf Privacy-preserving Anomaly Detection in Cloud Manufacturing via Federated Transformer: https://arxiv.org/pdf/2204.00843.pdf A Tour of Visualization Techniques for Computer Vision Datasets: https://arxiv.org/pdf/2204.08601.pdf Transfer Attacks Revisited: A Large-Scale Empirical Study in Real Computer Vision
Near-optimal Offline Reinforcement Learning with Linear Representation: Leveraging Variance Information with Pessimism: https://arxiv.org/pdf/2203.05804.pdf Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic Interfaces: https://arxiv.org/pdf/2203.03254.pdf A Fast and Convergent Proximal Algorithm for Regularized Nonconvex and Nonsmooth Bi-level Optimization: https://arxiv.org/pdf/2203.16615.pdf Monte Carlo
Is Neuro-Symbolic AI Meeting its Promise in Natural Language Processing? A Structured Review: https://arxiv.org/pdf/2202.12205.pdf NeuralFusion: Neural Volumetric Rendering under Human-object Interactions: https://arxiv.org/pdf/2202.12825.pdf Deep Generative model with Hierarchical Latent Factors for Time Series Anomaly Detection: https://arxiv.org/pdf/2202.07586.pdf Deep Recurrent Modelling of Granger Causality with Latent Confounding: https://arxiv.org/pdf/2202.11286.pdf
LaMDA: Language Models for Dialog Applications: https://arxiv.org/pdf/2201.08239v3.pdf Data-Driven Offline Optimization For Architecting Hardware Accelerators: https://arxiv.org/pdf/2110.11346v3.pdf Don’t Lie to Me! Robust and Efficient Explainability with Verified Perturbation Analysis: https://arxiv.org/pdf/2202.07728v1.pdf Block-NeRF: Scalable Large Scene Neural View Synthesis: https://arxiv.org/pdf/2202.05263v1.pdf Maintaining fairness across distribution shift: do we have viable
A Systematic Exploration Of Reservoir Computing For Forecasting Complex Spatiotemporal Dynamics: https://arxiv.org/pdf/2201.08910.pdf FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting: https://arxiv.org/pdf/2201.12740.pdf Quantifying Epistemic Uncertainty in Deep Learning: https://arxiv.org/pdf/2110.12122.pdf What’s Wrong With Deep Learning In Tree Search For Combinatorial Optimization: https://arxiv.org/pdf/2201.10494.pdf A Leap among Quantum
Beyond Simple Meta-Learning: Multi-Purpose Models for Multi-Domain, Active and Continual Few-Shot Learning: https://arxiv.org/pdf/2201.05151.pdf A unified software/hardware scalable architecture for brain-inspired computing based on self-organizing neural models: https://arxiv.org/pdf/2201.02262v1.pdf MGAE: Masked Autoencoders for Self-Supervised Learning on Graphs: https://arxiv.org/pdf/2201.02534v1.pdf Applications of Signature Methods to Market Anomaly Detection: https://arxiv.org/pdf/2201.02441v1.pdf
A Globally Convergent Distributed Jacobi Scheme for Block-Structured Non-convex Constrained Optimization Problems: https://arxiv.org/pdf/2112.09027.pdf A Robust Optimization Approach to Deep Learning: https://arxiv.org/pdf/2112.09279.pdf A Simple Single-Scale Vision Transformer for Object Localization and Instance Segmentation: https://arxiv.org/pdf/2112.09747.pdf A Survey of Natural Language Generation: https://arxiv.org/pdf/2112.11739.pdf Are Large-scale Datasets Necessary for
Simulation Intelligence: Towards A New Generation Of Scientific Methods: https://arxiv.org/pdf/2112.03235v1.pdf Information is Power: Intrinsic Control via Information Capture: https://arxiv.org/pdf/2112.03899v1.pdf GLaM: Efficient Scaling of Language Models with Mixture-of-Experts: https://arxiv.org/pdf/2112.06905v1.pdf Creating Multimodal Interactive Agents with Imitation and Self-Supervised Learning: https://arxiv.org/pdf/2112.03763v1.pdf Efficient Geometry-aware 3D Generative Adversarial Networks: https://arxiv.org/pdf/2112.07945v1.pdf
Covariate Shift in High-Dimensional Random Feature Regression: https://arxiv.org/pdf/2111.08234v1.pdf Improving Transferability of Representations via Augmentation-Aware Self-Supervision: https://arxiv.org/pdf/2111.09613v1.pdf GFlowNet Foundations: https://arxiv.org/pdf/2111.09266v1.pdf Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferability: https://arxiv.org/pdf/2111.10752v1.pdf Benchmarking Detection Transfer Learning with Vision Transformers: https://arxiv.org/pdf/2111.11429v1.pdf Improved Knowledge Distillation via Adversarial Collaboration:
Gradients are Not All You Need: https://arxiv.org/pdf/2111.05803v1.pdf RAVE: A variational autoencoder for fast and high-quality neural audio synthesis: https://arxiv.org/pdf/2111.05011v1.pdf NLP From Scratch Without Large-Scale Pretraining: A Simple and Efficient Framework: https://arxiv.org/pdf/2111.04130v1.pdf A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis: https://arxiv.org/pdf/2110.15678v2.pdf On Representation Knowledge
Shaking the foundations: delusions in sequence models for interaction and control: https://arxiv.org/pdf/2110.10819v1.pdf Understanding Dimensional Collapse in Contrastive Self-supervised Learning: https://arxiv.org/pdf/2110.09348v1.pdf SOFT: Softmax-free Transformer with Linear Complexity: https://arxiv.org/pdf/2110.11945v2.pdf Understanding How Encoder-Decoder Architectures Attend: https://arxiv.org/pdf/2110.15253v1.pdf Parameter Prediction for Unseen Deep Architectures: https://arxiv.org/pdf/2110.13100v1.pdf From Machine Learning to Robotics: Challenges
Multitask prompted training enables zero-shot task generalization: https://arxiv.org/pdf/2110.08207v1.pdf DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries: https://arxiv.org/pdf/2110.06922v1.pdf Object DGCNN: 3D Object Detection using Dynamic Graphs: https://arxiv.org/pdf/2110.06923v1.pdf Symbolic Knowledge Distillation: from General Language Models to Common sense Models: https://arxiv.org/pdf/2110.07178v1.pdf Graph Neural Networks with Learnable
It is common knowledge that efficient architecture design is a key aspect of building any product/solution, including AI applications. However, in reality, it is often observed that companies pay limited attention to developing robust end-to-end architectures before initiating AI application development. This leads to several
Human perception is multimodal. We make sense of objects and events through multiple modalities (sensory organs), and that is why we excel in our understanding of the world around us. Similarly, in many real-world problems, Artificial Intelligence systems become more efficient when they process inputs
Researchers from MIT & IBM recently introduced CRL, a new self-supervised framework that learns task-agnostic visual representations in embodied environments. This approach is able to construct representations not only from unlabeled datasets, but also from environments. The CRL framework jointly learns a reinforcement learning policy,
The European Union has just released its first legal framework for Artificial Intelligence. It covers a wide range of areas, including:▪︎ Defining the fundamental notion of an AI system.▪︎ Laying down the requirements for high risk AI systems, and obligations of their operators.▪︎ Prohibiting certain
The Beijing Academy of Artificial Intelligence (BAAI) released China’s first super-scale intelligence model system: WuDao 1.0. This is a combination of four very large-scale NLP models. WenYuan: China’s largest pre-training language model (supporting Chinese & English) for text categorization, sentiment analysis, reading comprehension, etc. It
Automated Machine Learning has come a long way since Google Brain introduced NAS in early 2017. Amazon’s AutoGluon, Google’s AutoMLZero, Salesforce’s TransmogrifAI, the AutoML features of Azure, H2O, Scikit-learn, Keras & others (TPOT, DataRobot, etc.) are witnessing increased adoption. Modern AutoML systems generally focus on hyper-parameter optimization (HPO), neural architecture search (NAS), model
Earlier this year, 55 researchers from 44 global institutions proposed GEM (Generation, Evaluation & Metrics), a new benchmark environment for Natural Language Generation. It evaluates models through an interactive result exploration system. This enables a much better understanding of model limitations & improvement opportunities, and
Design Patterns are reusable, formalized constructs that serve as templates to address common problems in designing efficient systems. These enable the development of high-performance, resilient & robust applications. Widely-used design patterns, especially from the object-oriented paradigm, include:▪︎ Behavioral Patterns: Command, Mediator, Memento, Observer, Visitor, etc.▪︎ Creational Patterns:
Object detection is one of the most fundamental problems in Computer Vision, and has powered many of the significant advances in this field. It has applications in a wide range of areas such as advertising, driverless cars, healthcare, robotics, security, and others. This quasi-technical paper
Emotion AI, also known as Affective Computing or Artificial Emotional Intelligence, is an interdisciplinary field that operates at the intersection of Behavioral Science, Cognitive Computing, Computer Science, Machine Learning, Neuroscience, Psychology, Signal Processing, and others. This is one of the rapidly evolving areas of AI
Philip E. Tetlock, one of the foremost global experts in forecasting behavior, conducted a comprehensive 20 year study of expert judgment in scientific literature, and concluded that the average expert performance was around the same as random guessing. Since this study was conducted between 1984
The COVID-19 pandemic is an earth-shattering, black swan event. Personal lives, societies and businesses are getting severely impacted. As governments, companies, institutions, and other organizations around the world shift their focus and resource allocation from their regular objectives to controlling this pandemic and its impact,
“You see, but you do not observe.” Sherlock Holmes, A Scandal in Bohemia. Knowledge, not mere data, is the real deal. Data is abundant in organizations, but not always in the structure and form that can be easily processed, analyzed and leveraged. As a result, companies
A common grievance of most enterprises is that while data is abundant, there is not enough knowledge. Data is the symbolic representation of the observable properties of real-world entities and, on its own, yields limited practical value. Knowledge, on the other hand, is ‘meaningful data’
An Asian publishing company embarked on a major transformation program to deploy AI-based analytics and business intelligence solutions across all their strategic business units. The program witnessed initial success during the proof-of-concept and early validation stages. This encouraged the company to initiate a phase-wise enterprise
“If you torture the data long enough, it will confess.” Ronald Coase, Nobel Prize Winner in Economics, 1991. Imagine a below-average shooter (the original story has a Texan) randomly firing at the side of a barn, and then painting bull’s-eyes around the tightest clusters of
Strategic Foresight refers to an organization’s ability to identify key drivers of change, and effectively prepare for multiple plausible alternative futures. The focus is not on predicting a singular future outcome, but on understanding how different versions of the future might shape up, and optimally
The Fourth Industrial Revolution (4IR) has begun. We are already witnessing massive disruptions in different forms and levels in most aspects of our businesses and lives. The determinants of business success are changing. Incremental and Non-R&D innovation will not be adequate for business leadership in
The broad theoretical foundations of Deep Learning were formed in the seventies and eighties, and evolved in the nineties and early 2000s. However, it was only in the early part of this decade that Deep Learning started emerging out of the confines of academia and