Introduction The rapid rise of Generative AI has often been driven by a straightforward principle: scale to make models become more capable. For much of the past decade, this formula of more parameters, more data, and more GPUs has held true, reinforcing the notion that performance is a nearly mechanical function of size. Scaling laws…
Introduction The competitive advantage in AI is shifting from model performance to systems infrastructure. As model capabilities converge and access to powerful foundation models becomes commoditized, the true moat is the agentic infrastructure built around them. This stack encompasses orchestration frameworks, memory systems, secure tooling, observability mechanisms, and evaluation loops, and transforms static models into…
The classical theory of generalization in statistical learning has largely revolved around complexity measures such as the Vapnik-Chervonenkis (VC) Dimension and Rademacher Complexity. While these tools have been instrumental in providing theoretical guarantees, they often rely on worst-case analyses and may not fully capture the intrinsic structure of real-world learning problems. In this paper, we…
Introduction: Over the past few years, Agentic AI has been generating significant excitement, and deservedly so. In tandem with Generative AI, it represents the new frontier in Artificial Intelligence. While agent-based systems have existed for decades, it is only now that their capabilities are capturing mainstream attention. Organizations, ranging from startups to tech giants, are…
Introduction Recent advancements in artificial intelligence have fueled considerable excitement around what many call “AI Chips”, specialized hardware tailored to optimize and accelerate AI workloads. Amidst the noise, many companies appear to be branding nearly every processor enhancement or hardware upgrade as an AI chip, blurring the lines between genuine AI advancements and general performance…
Abstract Mathematical optimization drives complex decision-making across a diverse range of problems – e.g., energy management, inventory planning, network design, pricing & revenue management, production planning & scheduling, supplier selection, and transportation planning. This field has significantly evolved over decades, from its formative years around World War II to the modern age. While conventional optimization…
A recent paper jointly published by DeepMind, LG, and KAIST researchers introduced a new innovation for Transformer models: the global-to-local Block Architecture. The paper highlights up to 20x improvement in inference throughput vis-à-vis conventional transformer models. While this claim needs to be empirically validated by early adopters, the ideas posited in the paper do seem…
MoE-Mamba: Efficient Selective State Space Models with Mixture of Experts This paper combines State Space Modeling (SSM) with the Mixture of Experts (MoE) approach, and introduces the MoE-Mamba model in which every other Mamba layer is replaced with a MoE feed-forward layer based on the Switch transformer. MoE-Mamba is shown to not only outperform both…
A Comprehensive Survey of Hallucination Mitigation Techniques in Large Language Models This paper highlights 32 techniques to mitigate hallucination in LLMs, including a well-defined taxonomy to categorize these methods. AlphaGeometry: An Olympiad-level AI system for geometry DeepMind introduced its AI system for solving Olympiad geometry problems. Trained exclusively on synthetic data, AlphaGeometry uses a…
Deep Learning-based architectures have had a significant impact on computer vision, natural language processing, and other machine learning areas. However, the scenario is not so straightforward when it comes to Forecasting, an area where statistical and traditional machine learning models have generally outperformed other types of models. In recent years, Transformer architectures (e.g., Google’s Temporal…
Google introduced Spotlight, a foundational model for mobile UI modeling, particularly for tasks like command grounding, screen summarization, tappability prediction, and widget captioning. Traditional mobile UI design often uses the concept of view hierarchy information, but these view hierarchies are sometimes either not available, or corrupted. Spotlight not only bypasses the need for view hierarchies,…
Microsoft introduced VALL-E, its neural codec language model for zero-shot Text-to-Speech Synthesis (TTS) that generates high-quality audio/speech with only a 3-second acoustic prompt (i.e., voice recording.) Unlike conventional models that consider TTS a continuous signal regression task, VALL-E approaches this as a conditional language modeling problem. Trained on LibriLight’s 60K+ audio hours, VALL-E was shown…
Large Language Models (LLMs) are state-of-the-art today, and generally perform well for simple and low-interaction tasks, such as single-turn conversations, and command-and-response systems. However, their direct use is generally limited in the case of applications with complex and high-interaction tasks, such as multi-turn dialogue systems, and enterprise-grade chatbots. Most real-world applications are complex, and are…
Generative AI technologies have garnered much attention over the last 24 to 30 months. Innovations like GPT-3 and 3.5 (ChatGPT), DALL-E2, and Stable Diffusion have occupied a large part of the public discourse. The developers of these technologies, and the entire ecosystems around them are naturally creating a lot of hype. Others are adding to…
[siteorigin_widget class=”thinkup_builder_divider”][/siteorigin_widget] NeurIPS 2022 declared 13 submissions as outstanding papers from its main track. Is Out-of-distribution Detection Learnable? Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding Elucidating the Design Space of Diffusion-Based Generative Models ProcTHOR: Large-Scale Embodied AI Using Procedural Generation Using Natural Language and Program Abstractions to Instill Human Inductive Biases in Machines A…
3D Vision with Transformers: A Survey: https://arxiv.org/pdf/2208.04309v1.pdf Unifying Visual Perception by Dispersible Points Learning: https://arxiv.org/pdf/2208.08630v1.pdf ZoomNAS: Searching for Whole-body Human Pose Estimation in the Wild: https://arxiv.org/pdf/2208.11547v1.pdf ROLAND: Graph Learning Framework for Dynamic Graphs: https://arxiv.org/pdf/2208.07239v1.pdf Investigating Efficiently Extending Transformers for Long Input Summarization: https://arxiv.org/pdf/2208.04347v1.pdf Semantic-Aligned Matching for Enhanced DETR Convergence and Multi-Scale Feature Fusion: https://arxiv.org/pdf/2207.14172v1.pdf TransNorm:…
High-Performance GPU-to-CPU Transpilation and Optimization via High-Level Parallel Constructs: https://arxiv.org/pdf/2207.00257.pdf Branchformer: Parallel MLP-Attention Architectures to Capture Local and Global Context for Speech Recognition and Understanding: https://arxiv.org/pdf/2207.02971v1.pdf More ConvNets in the 2020s: Scaling up Kernels Beyond 51 × 51 using Sparsity: https://arxiv.org/pdf/2207.03620v1.pdf Softmax-free Linear Transformers: https://arxiv.org/pdf/2207.03341v1.pdf Learning Quality-aware Dynamic Memory for Video Object Segmentation: https://arxiv.org/pdf/2207.07922v1.pdf 3D…
Scaling Vision Transformers: https://arxiv.org/pdf/2106.04560.pdf Learning Efficient Vision Transformers via Fine-Grained Manifold Distillation: https://arxiv.org/pdf/2107.01378.pdf Risk-averse autonomous systems: A brief history and recent developments from the perspective of optimal control: https://arxiv.org/pdf/2109.08947.pdf LightSeq2: Accelerated Training for Transformer-based Models on GPUs: https://arxiv.org/pdf/2110.05722.pdf Conditionally Elicitable Dynamic Risk Measures For Deep Reinforcement Learning: https://arxiv.org/pdf/2206.14666.pdf Cooperative Retriever and Ranker in Deep Recommenders:…
Computational Storytelling And Emotions: A Survey: https://arxiv.org/pdf/2205.10967.pdf Are Large Pre-Trained Language Models Leaking Your Personal Information?: https://arxiv.org/pdf/2205.12628.pdf FreDo: Frequency Domain-based Long-Term Time Series Forecasting: https://arxiv.org/pdf/2205.12301.pdf A Survey on Long-tailed Visual Recognition: https://arxiv.org/pdf/2205.13775.pdf Understanding Factual Errors in Summarization: Errors, Summarizers, Datasets, Error Detectors: https://arxiv.org/pdf/2205.12854.pdf On the Robustness of Safe Reinforcement Learning under Observational Perturbations: https://arxiv.org/pdf/2205.14691.pdf Nesterov’s…
Multiview Transformers for Video Recognition: https://arxiv.org/pdf/2201.04288.pdf ViNTER: Image Narrative Generation with Emotion-Arc-Aware Transformer: https://arxiv.org/pdf/2202.07305.pdf Privacy-preserving Anomaly Detection in Cloud Manufacturing via Federated Transformer: https://arxiv.org/pdf/2204.00843.pdf A Tour of Visualization Techniques for Computer Vision Datasets: https://arxiv.org/pdf/2204.08601.pdf Transfer Attacks Revisited: A Large-Scale Empirical Study in Real Computer Vision Settings: https://arxiv.org/pdf/2204.04063.pdf Data Distributional Properties Drive Emergent Few-Shot Learning in…
Near-optimal Offline Reinforcement Learning with Linear Representation: Leveraging Variance Information with Pessimism: https://arxiv.org/pdf/2203.05804.pdf Augmented Reality and Robotics: A Survey and Taxonomy for AR-enhanced Human-Robot Interaction and Robotic Interfaces: https://arxiv.org/pdf/2203.03254.pdf A Fast and Convergent Proximal Algorithm for Regularized Nonconvex and Nonsmooth Bi-level Optimization: https://arxiv.org/pdf/2203.16615.pdf Monte Carlo Tree Search based Hybrid Optimization of Variational Quantum Circuits: https://arxiv.org/pdf/2203.16707.pdf…
Is Neuro-Symbolic AI Meeting its Promise in Natural Language Processing? A Structured Review: https://arxiv.org/pdf/2202.12205.pdf NeuralFusion: Neural Volumetric Rendering under Human-object Interactions: https://arxiv.org/pdf/2202.12825.pdf Deep Generative model with Hierarchical Latent Factors for Time Series Anomaly Detection: https://arxiv.org/pdf/2202.07586.pdf Deep Recurrent Modelling of Granger Causality with Latent Confounding: https://arxiv.org/pdf/2202.11286.pdf Generalizable Information Theoretic Causal Representation: https://arxiv.org/pdf/2202.08388.pdf Artificial Intelligence for the…
LaMDA: Language Models for Dialog Applications: https://arxiv.org/pdf/2201.08239v3.pdf Data-Driven Offline Optimization For Architecting Hardware Accelerators: https://arxiv.org/pdf/2110.11346v3.pdf Don’t Lie to Me! Robust and Efficient Explainability with Verified Perturbation Analysis: https://arxiv.org/pdf/2202.07728v1.pdf Block-NeRF: Scalable Large Scene Neural View Synthesis: https://arxiv.org/pdf/2202.05263v1.pdf Maintaining fairness across distribution shift: do we have viable solutions for real-world applications?: https://arxiv.org/pdf/2202.01034v1.pdf Transformers Can Do Bayesian Inference:…
A Systematic Exploration Of Reservoir Computing For Forecasting Complex Spatiotemporal Dynamics: https://arxiv.org/pdf/2201.08910.pdf FEDformer: Frequency Enhanced Decomposed Transformer for Long-term Series Forecasting: https://arxiv.org/pdf/2201.12740.pdf Quantifying Epistemic Uncertainty in Deep Learning: https://arxiv.org/pdf/2110.12122.pdf What’s Wrong With Deep Learning In Tree Search For Combinatorial Optimization: https://arxiv.org/pdf/2201.10494.pdf A Leap among Quantum Computing and Quantum Neural Networks: A Survey: https://arxiv.org/pdf/2107.03313.pdf Causality And…
Beyond Simple Meta-Learning: Multi-Purpose Models for Multi-Domain, Active and Continual Few-Shot Learning: https://arxiv.org/pdf/2201.05151.pdf A unified software/hardware scalable architecture for brain-inspired computing based on self-organizing neural models: https://arxiv.org/pdf/2201.02262v1.pdf MGAE: Masked Autoencoders for Self-Supervised Learning on Graphs: https://arxiv.org/pdf/2201.02534v1.pdf Applications of Signature Methods to Market Anomaly Detection: https://arxiv.org/pdf/2201.02441v1.pdf Does entity abstraction help generative Transformers reason?: https://arxiv.org/pdf/2201.01787v1.pdf Debiased Learning…
A Globally Convergent Distributed Jacobi Scheme for Block-Structured Non-convex Constrained Optimization Problems: https://arxiv.org/pdf/2112.09027.pdf A Robust Optimization Approach to Deep Learning: https://arxiv.org/pdf/2112.09279.pdf A Simple Single-Scale Vision Transformer for Object Localization and Instance Segmentation: https://arxiv.org/pdf/2112.09747.pdf A Survey of Natural Language Generation: https://arxiv.org/pdf/2112.11739.pdf Are Large-scale Datasets Necessary for Self-Supervised Pre-training?: https://arxiv.org/pdf/2112.10740.pdf A Survey on Gender Bias in Natural…
Simulation Intelligence: Towards A New Generation Of Scientific Methods: https://arxiv.org/pdf/2112.03235v1.pdf Information is Power: Intrinsic Control via Information Capture: https://arxiv.org/pdf/2112.03899v1.pdf GLaM: Efficient Scaling of Language Models with Mixture-of-Experts: https://arxiv.org/pdf/2112.06905v1.pdf Creating Multimodal Interactive Agents with Imitation and Self-Supervised Learning: https://arxiv.org/pdf/2112.03763v1.pdf Efficient Geometry-aware 3D Generative Adversarial Networks: https://arxiv.org/pdf/2112.07945v1.pdf GAN-Supervised Dense Visual Alignment: https://arxiv.org/pdf/2112.05143v1.pdf BEVT: BERT Pretraining of Video…
Covariate Shift in High-Dimensional Random Feature Regression: https://arxiv.org/pdf/2111.08234v1.pdf Improving Transferability of Representations via Augmentation-Aware Self-Supervision: https://arxiv.org/pdf/2111.09613v1.pdf GFlowNet Foundations: https://arxiv.org/pdf/2111.09266v1.pdf Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferability: https://arxiv.org/pdf/2111.10752v1.pdf Benchmarking Detection Transfer Learning with Vision Transformers: https://arxiv.org/pdf/2111.11429v1.pdf Improved Knowledge Distillation via Adversarial Collaboration: https://arxiv.org/pdf/2111.14356v1.pdf End-to-End Referring Video Object Segmentation with Multimodal Transformers: https://arxiv.org/pdf/2111.14821v1.pdf…
Gradients are Not All You Need: https://arxiv.org/pdf/2111.05803v1.pdf RAVE: A variational autoencoder for fast and high-quality neural audio synthesis: https://arxiv.org/pdf/2111.05011v1.pdf NLP From Scratch Without Large-Scale Pretraining: A Simple and Efficient Framework: https://arxiv.org/pdf/2111.04130v1.pdf A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesis: https://arxiv.org/pdf/2110.15678v2.pdf On Representation Knowledge Distillation for Graph Neural Networks: https://arxiv.org/pdf/2111.04964v1.pdf Meta-Learning to Improve Pre-Training:…
Shaking the foundations: delusions in sequence models for interaction and control: https://arxiv.org/pdf/2110.10819v1.pdf Understanding Dimensional Collapse in Contrastive Self-supervised Learning: https://arxiv.org/pdf/2110.09348v1.pdf SOFT: Softmax-free Transformer with Linear Complexity: https://arxiv.org/pdf/2110.11945v2.pdf Understanding How Encoder-Decoder Architectures Attend: https://arxiv.org/pdf/2110.15253v1.pdf Parameter Prediction for Unseen Deep Architectures: https://arxiv.org/pdf/2110.13100v1.pdf From Machine Learning to Robotics: Challenges and Opportunities for Embodied Intelligence: https://arxiv.org/pdf/2110.15245v1.pdf Implicit MLE: Backpropagating Through…
Multitask prompted training enables zero-shot task generalization: https://arxiv.org/pdf/2110.08207v1.pdf DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries: https://arxiv.org/pdf/2110.06922v1.pdf Object DGCNN: 3D Object Detection using Dynamic Graphs: https://arxiv.org/pdf/2110.06923v1.pdf Symbolic Knowledge Distillation: from General Language Models to Common sense Models: https://arxiv.org/pdf/2110.07178v1.pdf Graph Neural Networks with Learnable Structural and Positional Representations: https://arxiv.org/pdf/2110.07875.pdf Human-Robot Collaboration and Machine Learning:…
It is common knowledge that efficient architecture design is a key aspect of building any product/solution, including AI applications. However, in reality, it is often observed that companies pay limited attention to developing robust end-to-end architectures before initiating AI application development. This leads to several problems like schedule/cost overruns, automation job failures, interoperability & scalability…
Human perception is multimodal. We make sense of objects and events through multiple modalities (sensory organs), and that is why we excel in our understanding of the world around us. Similarly, in many real-world problems, Artificial Intelligence systems become more efficient when they process inputs (signals) from multiple modalities, and then generate the outputs (prediction.)…
Researchers from MIT & IBM recently introduced CRL, a new self-supervised framework that learns task-agnostic visual representations in embodied environments. This approach is able to construct representations not only from unlabeled datasets, but also from environments. The CRL framework jointly learns a reinforcement learning policy, and visual representation model. The policy tries to maximize the…
The European Union has just released its first legal framework for Artificial Intelligence. It covers a wide range of areas, including:▪︎ Defining the fundamental notion of an AI system.▪︎ Laying down the requirements for high risk AI systems, and obligations of their operators.▪︎ Prohibiting certain AI practices, e.g., attempts to distort human behavior; real-time remote…
The Beijing Academy of Artificial Intelligence (BAAI) released China’s first super-scale intelligence model system: WuDao 1.0. This is a combination of four very large-scale NLP models. WenYuan: China’s largest pre-training language model (supporting Chinese & English) for text categorization, sentiment analysis, reading comprehension, etc. It claims GPT-3 comparable performance on several important NLU tasks. WenLan:…
Automated Machine Learning has come a long way since Google Brain introduced NAS in early 2017. Amazon’s AutoGluon, Google’s AutoMLZero, Salesforce’s TransmogrifAI, the AutoML features of Azure, H2O, Scikit-learn, Keras & others (TPOT, DataRobot, etc.) are witnessing increased adoption. Modern AutoML systems generally focus on hyper-parameter optimization (HPO), neural architecture search (NAS), model selection & compression, and to a certain extent, meta-learning. They also…
Earlier this year, 55 researchers from 44 global institutions proposed GEM (Generation, Evaluation & Metrics), a new benchmark environment for Natural Language Generation. It evaluates models through an interactive result exploration system. This enables a much better understanding of model limitations & improvement opportunities, and does not misrepresent the complex interactions of individual measures. This…
Design Patterns are reusable, formalized constructs that serve as templates to address common problems in designing efficient systems. These enable the development of high-performance, resilient & robust applications. Widely-used design patterns, especially from the object-oriented paradigm, include:▪︎ Behavioral Patterns: Command, Mediator, Memento, Observer, Visitor, etc.▪︎ Creational Patterns: Builder, Factory, Prototype, Singleton, etc.▪︎ Structural Patterns: Adapter, Bridge, Composite, Decorator,…
Object detection is one of the most fundamental problems in Computer Vision, and has powered many of the significant advances in this field. It has applications in a wide range of areas such as advertising, driverless cars, healthcare, robotics, security, and others. This quasi-technical paper discusses the evolution, critical areas of research, and recent innovations…
Emotion AI, also known as Affective Computing or Artificial Emotional Intelligence, is an interdisciplinary field that operates at the intersection of Behavioral Science, Cognitive Computing, Computer Science, Machine Learning, Neuroscience, Psychology, Signal Processing, and others. This is one of the rapidly evolving areas of AI research today. At the basic level, Emotion AI refers to…
Philip E. Tetlock, one of the foremost global experts in forecasting behavior, conducted a comprehensive 20 year study of expert judgment in scientific literature, and concluded that the average expert performance was around the same as random guessing. Since this study was conducted between 1984 and 2004 – before the advent of Big Data, Deep…
The COVID-19 pandemic is an earth-shattering, black swan event. Personal lives, societies and businesses are getting severely impacted. As governments, companies, institutions, and other organizations around the world shift their focus and resource allocation from their regular objectives to controlling this pandemic and its impact, it also increases the risk of serious cyber-attacks from rogue…
“You see, but you do not observe.” Sherlock Holmes, A Scandal in Bohemia. Knowledge, not mere data, is the real deal. Data is abundant in organizations, but not always in the structure and form that can be easily processed, analyzed and leveraged. As a result, companies end up ignoring, intentionally or otherwise, most of the data…
A common grievance of most enterprises is that while data is abundant, there is not enough knowledge. Data is the symbolic representation of the observable properties of real-world entities and, on its own, yields limited practical value. Knowledge, on the other hand, is ‘meaningful data’ created through cognitive processing mechanisms. Generating actionable knowledge from raw…
An Asian publishing company embarked on a major transformation program to deploy AI-based analytics and business intelligence solutions across all their strategic business units. The program witnessed initial success during the proof-of-concept and early validation stages. This encouraged the company to initiate a phase-wise enterprise roll-out. However, the roll-out turned out to be a major…
“If you torture the data long enough, it will confess.” Ronald Coase, Nobel Prize Winner in Economics, 1991. Imagine a below-average shooter (the original story has a Texan) randomly firing at the side of a barn, and then painting bull’s-eyes around the tightest clusters of holes made by his gunshots. He then showcases his ‘marksmanship’…
Strategic Foresight refers to an organization’s ability to identify key drivers of change, and effectively prepare for multiple plausible alternative futures. The focus is not on predicting a singular future outcome, but on understanding how different versions of the future might shape up, and optimally channeling that knowledge to manage risk, create competitive advantage, and…
The Fourth Industrial Revolution (4IR) has begun. We are already witnessing massive disruptions in different forms and levels in most aspects of our businesses and lives. The determinants of business success are changing. Incremental and Non-R&D innovation will not be adequate for business leadership in this new industrial age. There is a compelling need to…
The broad theoretical foundations of Deep Learning were formed in the seventies and eighties, and evolved in the nineties and early 2000s. However, it was only in the early part of this decade that Deep Learning started emerging out of the confines of academia and research units, and into the world of enterprise adoption. The…