Strategic Foresight refers to an organization’s ability to identify key drivers of change, and effectively prepare for multiple plausible alternative futures. The focus is not on predicting a singular future outcome, but on understanding how different versions of the future might shape up, and optimally channeling that knowledge to manage risk, create competitive advantage, and meet business objectives. Strategic Foresight differs from traditional Strategic Planning or Strategic Management on account of its special emphasis on understanding and addressing non-linearities like discontinuities, game-changers, weak signals, wildcards, outliers, and others.

This paper proposes a new way of developing Strategic Foresight by combining the forces of Horizon Scanning, Artificial Intelligence, and modern Software Engineering. It is based on an independent project being conducted by the author on predicting global economic recessions through Artificial Intelligence.

[siteorigin_widget class=”thinkup_builder_divider”][/siteorigin_widget]

The Need for Strategic Foresight

Strategic Foresight, as a field of Futures Studies, started forming its roots in the 1940s. Today, it is a well-established discipline that enables organizations to answer strategic questions pertaining to the future. These questions address factors over a broad spectrum, such as macroeconomics, geopolitics, technology, global challenges, competitive landscapes, and others. Some examples of such questions are:

• Which country in the European Union will be the next to leave it? What will be the impact?
• How will the boundaries of certain industries change in the next 10 to 20 years?
• What new capabilities are needed by my business to achieve $100 billion of revenue?
• Will the rise of China affect climate change and other environmental issues?
• How will global Supply Chain evolve in the foreseeable future?
• What new types of competitors will my company face in the next 5 years? How can I prepare today?

Future events are the result of convergence of multiple factors ….. this is what makes it exceedingly difficult for even the best of experts to forecast and create a ‘future baseline’, especially in a dynamic VUCA (Volatile, Uncertain, Complex, and Ambiguous) world.

Traditional strategic planning techniques and systems are sub-optimally designed to identify and address complex, dynamic and highly unpredictable patterns. For instance, one of the key factors for the phenomenal rise of computing and smartphones is radio waves, an old technology that helped transmit binary information. Go back several decades, and even the best of foresight experts would have failed to predict that this re-purposing of an old technology would have such a massive global impact.

Corporate strategic planning, even in the best of companies, have often ‘missed the signal from the noise’ that resulted in sub-optimality in future-readiness. Firstly, there has been a lot of inconsistency in the manner companies have conducted these exercises. Some follow standard foresight methods, some follow project management guidelines, some leverage software development and business intelligence methods, and others bank on company-specific practices and individual judgements of key business leaders. Secondly, and more importantly, traditional strategic planning exercises are often plagued with inherent limitations that lead to long planning cycles, impractical strategy formulations, and inability to proactively recognize outliers and shocks. The results are far from ideal. Some well-known examples are cited below.

• Microsoft failed to understand the impact of mobile technologies in the 2000s, and significantly missed out on that opportunity to Apple, Google and other players.
• IBM and HP misread the sustaining impact of Cloud technologies in the late 2000s and early 2010s, and by the time they realized that, the world had moved ahead.
• Yahoo’s failure to truly embrace mobile and data technologies, even after these technologies became mainstream, was an automatic recipe for decline.
• Blackberry’s lack of understanding of long-term mobile trends, developer ecosystems, and evolving consumer tastes are well documented.

… the list goes on and on.

The common approach in Strategic Foresight development is to first identify probable drivers of change in STEEPLE (Social, Technological, Environmental, Economic, Political, Legal and Ethical) Factors. The next step is to develop multiple future megatrends, events or scenarios through various plausible combinations of these change drivers. These future versions are not fads, but those with high probabilities of occurrences that can lead to systemic and fundamental changes with global implications over the next 5 to 10 years (mid-range future) or even 10 to 30 years (long-range future). Finally, suitable responses to each of these future versions are determined along with phase-wise action plans.

Several qualitative, quantitative and mixed methods have been used in Strategic Foresight development over the years. Some examples are mentioned below.

• Qualitative Methods: Delphi Techniques, Environmental Scanning, Field Anomaly Relaxation, Futures Wheel, Visioning.
• Quantitative Methods: Econometric & Statistical Modeling, Cross-Impact Analysis, Trend Impact Analysis.
• Hybrid/Integrated Methods: Scenario Planning, Strategic Roadmapping, Structural Analysis, Systems Modeling, Text Mining.

Qualitative methods have been extensively used in foresight generation in the last century and the early years of the present one; whereas Quantitative and Mixed methods have generally been preferred since the 2008 global recession. However, these methods are not always efficient, and sometimes lack a holistic approach to application. Inefficiencies are often observed while integrating multiple data types from disparate sources, capturing and analyzing data in real-time, tracking outlier events in a scalable manner, executing foresight projects over multiple years, or seeking answers to deep level questions. This is where AI-based Horizon Scanning steps in to play a path-breaking role.

[siteorigin_widget class=”thinkup_builder_divider”][/siteorigin_widget]

Horizon Scanning for Strategic Foresight Development

Horizon Scanning, as defined by UK’s DEFRA organization, is the systematic examination of potential hazards, opportunities and likely future developments which are at the margins of current thinking and planning. Broadly speaking, it is a scientific way of exploring what is happening in the world around us, including unexpected issues, emerging trends, early signals, persistent problems, etc. Traditionally, this technique has been used by governments and public institutions in Europe and USA for policy-making in the areas of healthcare, education, biodiversity and conservation, and military studies. In the past few years, it has started getting adopted, in different forms and variants, by some corporations.

Horizon Scanning generally serves two purposes:

1. Proactively anticipates emerging issues, events, patterns, trends, and drivers of change.
2. Creates new issues, events, patterns, trends, and drivers of change by combining the original ones in different ways.

The European Commission carried out a multi-year Horizon Scanning program called SESTI (Scanning for Emerging Science and Technology Issues) from 2008 to 2011 that has almost become a gold standard for most corporate scanning exercises today. There are two principal approaches to designing and developing Horizon Scanning programs: (a) Exploratory, and (b) Issue Centered. The objective of the first approach is to assemble existing, emerging and potential issues from a wide spectrum, and cluster them into groups. The goal of the second one is to validate (or discard) these clustered issues, and generate signals from them. These signals are subsequently tracked through time, and action plans are determined so that they can be brought-to-action if the intensities of some signals increase. Both these approaches are complimentary in nature: the first one focuses on information discovery; the second one focuses on knowledge validation and action planning.

The biggest advantage of Horizon Scanning is that it provides a robust framework to identify outliers, weak signals and wild cards across the global landscape, including those with low-probability but high-impact characteristics (Black Swans.) Moreover, it offers an organized way to validate these wild cards and weak signals, and analyze them at deep levels for adequate understanding. While it is true that all future shifts, shocks and turbulence are impossible to forecast, Horizon Scanning does offer a mechanism to at least identify some early signs of some of those future effects, and make sense out of them. It also supports adaptive foresight, a key requirement for any kind of strategic futures analysis in an uncertain world.

Philip Tetlock, the renowned political scientist and best-selling author, conducted forecasting tournaments between 1984 and 2003 where non-experts and experts competed to predict the future on a wide range of areas. His study concluded that non-experts were better predictors than experts, especially in politics and economics. The reason attributed to this was that non-experts generally considered a broad view of multiple factors in their prediction process, whereas experts often considered a narrow scope of things. Horizon Scanning offers a practical, scalable and optimal way to mine the opinions of both the general public (non-experts) and experts. Moreover, it allows scope for even conflicting ideas to be considered, thus offering a highly integral approach to strategic foresight development.

[siteorigin_widget class=”thinkup_builder_divider”][/siteorigin_widget]

The Role of Artificial Intelligence in Horizon Scanning

Horizon Scanning involves searching and mining vast amounts of web and social media data, analyzing them in real-time, and generating intelligence. This is achieved through web scraping, statistical techniques, and expert judgments. However, this execution approach is often sub-optimal because of data losses, low fault-tolerance, high latency, batch-mode operations, shallow data mining, opinion biases, average predictive power, and others.

A new approach that I propose is to integrate AI and modern software engineering within the Horizon Scanning framework to develop real-time, more accurate, more actionable, and deeper level of strategic foresight.

Artificial Intelligence technologies such as Machine Learning, Deep Learning, Computer Vision, and Natural Language Processing (NLP) significantly score over traditional statistical techniques and expert judgments to achieve both superior accuracy (i.e. predictive power) and granularity (i.e. depth of analysis). New-age software engineering technologies like Big Data and Semantic Web Mining offer great improvements in collecting, storing and processing data. Semantic Web Mining scores over traditional web mining by virtue of its ontology-based approach that offers greater efficiencies while setting-up (and also scaling-up) the structure and scope for mining web content. Big Data, especially Event Stream Processing and Kappa & Lambda Architectures, offer significant improvements over traditional data processing techniques like low latency, high fault-tolerance, real-time analysis, high scalability and extensibility.

Here are five major ways in which AI enhances the efficiency and value of Horizon Scanning.

1. Developing strategic foresight necessitates the identification and monitoring of outliers, potential shocks, weak signals, wildcards, etc. This is where Deep Learning is much better than other known forms of monitoring and analysis. Deep Neural Networks and Advanced Optimization algorithms are the best ways to study outlier distributions, irregular patterns and discontinuities.
2. A significant chunk of the efforts in Horizon Scanning are linked to Text Mining, such as sentiment analysis, subjectivity detection, opinion identification, text summarization, contextual topic modeling, open-ended coding, etc. Deep NLP, particularly those based on Recurrent Neural Network (RNN), Gated Recurrent Unit (GRU), Long Short-Term Memory (LSTM), Encoder-Decoder, and Sequence-to-Sequence architectures, generally produce the best results for such use cases.
3. The joint forces of Unsupervised Learning and Reinforcement Learning offer a scalable way to develop multiple versions of the future. More importantly, Reinforcement Learning can enable the development of optimal action plans for each of these future versions in a scalable and objective manner. This is, in my opinion, the most optimal way to prepare for the future.
4. Pattern mining from audio/speech, images and videos can be effectively conducted only through Computer Perception (Audition & Vision) – there is no other alternate viable solution at the moment.
5. Finally, AI offers a mechanism to automate the end-to-end Strategic Foresight (Horizon Scanning) process; thus ensuring greater consistency, more repeatability, low long-term costs, lesser errors and reduced cycle time for implementation.

Closing Comments

Cognitive Algorithmic Foresight Generation is not perfect, but it is our best bet in an increasingly unpredictable world.

Complex systems are difficult to understand due to the presence of multiple chaotic and non-linear phases throughout their life cycles, thus causing practical challenges in modeling, forecasting and estimation. Nevertheless, modern computing and technological advances have now started providing us with the tools and techniques to generate strategic insights on these complex, dynamic and unpredictable systems. Even though there is a long way to go, we now have several underlying technological frameworks that support the approximation of these chaotic phases with a reasonable degree of accuracy; thereby enabling the study of such systems. Strategic Foresight Development is a great use case for this. Current and past baselines of the future are likely to be wrong, especially as the forecasting period increases. The understanding of the interplay of multiple factors, particularly those of the expected and unexpected aberrations, hold the key. AI-based Horizon Scanning is a scalable and efficient way to develop strategic foresight in today’s age. And it is only going to get better with time.

Note: The concepts of Strategic Foresight and Horizon Scanning are based on the works of several esteemed researchers in this field. Special mention to the SESTI Project and Jerome Glenn’s Futures Research Methodology 3.0 that have deeply influenced my thinking. My focus (and contribution) is on the adoption of AI and modern software engineering to make this field more efficient, implementable, scalable and technology-driven.