The Analytics-Innovation Paradox: Why Firms Invest in AI But Innovation Is Slowing

SERIES 1 — UNDERSTANDING THE RESEARCH LANDSCAPE  •  ARTICLE 2 OF 50

Companies are spending billions on artificial intelligence. Yet aggregate innovation metrics are disappointing. This article explains the central puzzle that motivates the entire field of AI innovation research — and what the latest evidence tells us about why it exists.

The Numbers That Don’t Add Up

If you have noticed, Something strange has been happening in the global economy for the past fifteen years. Between 2010 and 2020, global corporate investment in artificial intelligence grew from a negligible figure to hundreds of billions of dollars annually. The consulting firm McKinsey estimated that AI investment across industries exceeded $500 billion globally by 2021. Every major firm in every major industry launched an AI strategy. Chief AI officers became standard additions to corporate leadership teams. Business schools redesigned curricula around AI. The narrative was simple and powerful: AI was the new electricity — a general-purpose technology that would transform every corner of the economy.

We are investing more in AI than ever before. We are deploying more researchers than ever before. And yet the breakthroughs per dollar and per researcher are declining. This is the analytics-innovation paradox — and it is one of the most important unresolved questions in modern economics.

What the Paradox Actually Measures

Before explaining why the paradox exists, it is important to be precise about what it actually measures. The paradox is not simply that AI has failed to improve innovation. In many narrow and specific ways, AI has been enormously beneficial for innovation. Drug discovery timelines have shortened. Materials science has accelerated. Climate modeling has improved dramatically. The paradox operates at a more aggregate level — and it involves a specific type of innovation that researchers call explorative innovation.

The paradox is specifically about exploration — the bold, uncertain, potentially transformative type of innovation that creates new industries and solves problems that seemed unsolvable. This is the type of innovation that AI was supposed to accelerate most dramatically. And this is precisely where the evidence is most mixed.

THE KEY DISTINCTION The paradox is not that AI fails to help firms improve existing products — it clearly does. The paradox is that AI investment does not automatically translate into more explorative innovation — the kind that creates genuinely new knowledge, enters new technology domains, and produces breakthrough discoveries. Understanding why requires understanding what AI actually does — and what it cannot do.

Five Explanations for the Paradox

AI innovation research has produced five distinct candidate explanations for the analytics-innovation paradox. The honest answer is that all five are probably partially correct — operating in different proportions for different firms and industries. But understanding each one separately is essential for understanding the field.

Explanation 1 — AI Helps Recombination But Not Radical Invention

The intuition is straightforward once you understand what machine learning actually does. AI systems are extraordinarily powerful at scanning vast bodies of existing knowledge, identifying non-obvious patterns and connections, and suggesting combinations that human researchers might never have found on their own. This is recombination — and it is genuinely valuable. Many important innovations are recombinations of existing ideas applied in new contexts.

But AI is fundamentally less powerful at the kind of intuitive leap that produces truly radical discoveries — the moments when a scientist has an insight that no prior knowledge could have predicted, that comes from nowhere, that creates an entirely new category of understanding. This type of radical invention remains stubbornly human, dependent on the kind of creative intuition that current AI systems do not replicate. The aggregate implication is significant. If every firm in the economy uses AI to accelerate recombinative innovation while radical invention remains as difficult as ever, the distribution of innovation shifts. We get more combinations. We get fewer breakthroughs. The aggregate metrics that measure truly transformative discovery show stagnation or decline — even as firms report genuine productivity improvements from AI-enabled recombination.

---

Explanation 2 — The Intermediate Novelty Ceiling

Their research shows that AI investment specifically helps firms pursue exploration at intermediate levels of novelty — bold enough to represent a genuine departure from prior knowledge, but grounded enough in existing understanding that AI can provide meaningful guidance. At the extremes, purely incremental exploitation on one end, and radically novel invention with no prior knowledge foundation on the other, AI’s effect weakens substantially.

Think of it as a Goldilocks zone for AI-enabled exploration. Too close to existing knowledge — AI helps, but the innovation is not transformative. Too far from existing knowledge — AI cannot guide the journey because there is no existing map to draw on. In the middle — AI is most powerful. The paradox emerges because aggregate innovation metrics are disproportionately driven by the most radical innovations — the genuine breakthroughs that redefine fields. If AI’s effect is concentrated in the intermediate novelty zone and weak at the radical end, then aggregate metrics will undercount AI’s contribution even when firms are genuinely benefiting from it in the commercially most important novelty range.

---

Explanation 3 — Organizational Misalignment

The intuition is that AI acts as a coordination mechanism. In a decentralized firm, knowledge is dispersed across many people and teams who rarely interact. AI can connect this dispersed knowledge — identifying when a researcher in one team is working on a problem that another team has already solved, or when insights from one division are directly applicable to challenges in another. In a centralized firm where everyone already communicates directly, AI’s coordination value is lower. The paradoxical implication is stark. Most firms investing in AI are doing so without simultaneously restructuring their organizations to capture AI’s exploration benefits. They are plugging a powerful new tool into an organizational structure designed for a different era. The tool is capable. The structure is misaligned. The result is that AI investment produces operational efficiency gains — which are real and valuable — but does not produce the exploration acceleration that the investment theoretically enables.

---

Explanation 4 — Leadership Mismatch

The underlying mechanism connects to the upper echelons theory, established by Hambrick and Mason in their foundational 1984 paper in the Academy of Management Review. Their framework argues that CEO background, experience, and cognitive orientation directly shape strategic decisions. A STEM-background CEO understands at a deep level what AI can and cannot do, how to direct it toward genuine exploration rather than efficiency improvement, and how to communicate the value of AI-guided exploration to boards and investors who are naturally skeptical of long-horizon bets. Non-STEM CEOs are not incapable of leveraging AI for innovation — but they face a steeper learning curve, are more likely to use AI primarily for operational purposes, and may be less comfortable with the uncertainty inherent in genuine exploration. The result is that the same level of AI investment produces different innovation outcomes depending on who is directing it.

At the aggregate level, this creates a systematic mismatch. The majority of large-firm CEOs do not have STEM backgrounds. Most AI investment is therefore being directed by leaders who are not optimally positioned to convert it into exploration. The investment is real. The capability is real. But the leadership to deploy it toward transformative innovation is unevenly distributed — and the aggregate metrics reflect this mismatch.

Explanation 5 — AI Compresses the Innovation Distribution

The fifth explanation is the most recent and perhaps the most provocative. It has not yet been fully tested empirically — but it is theoretically compelling and connects directly to the aggregate innovation slowdown. The argument is that AI investment changes not just the average quality of innovation but the entire distribution of innovation outcomes. AI is fundamentally an optimization and risk management tool. It helps firms identify which exploration directions are promising and which are likely to be dead ends, before investing heavily in either. This means AI should systematically reduce the variance of innovation outcomes: fewer spectacular failures, but also fewer spectacular breakthroughs.

Think of what AI does to a pharmaceutical firm’s drug discovery process. AI can eliminate compounds that are almost certainly toxic or ineffective before expensive clinical trials. This reduces failures dramatically — a real and valuable benefit. But it also filters out the long-shot compounds that have a small probability of being transformative. The drugs that get through AI screening are reliable, safe, and moderately effective. The drugs that might have been revolutionary, but looked unpromising in early screening, are quietly eliminated.

If this variance compression effect is operating across all firms simultaneously — which AI adoption at scale implies — then aggregate innovation metrics that are disproportionately driven by the rare spectacular breakthrough will show stagnation or decline even as average innovation quality rises. The paradox is not that AI fails. It is that AI succeeds in a way that the metrics are not designed to capture.

The analytics-innovation paradox may not be a paradox at all. It may be a measurement problem. We are measuring innovation with metrics designed to capture breakthroughs — and AI is changing the distribution in ways that improve the average while reducing the breakthroughs. The metrics see stagnation. The reality is transformation — just not the kind we were looking for.

Explanation	Core Argument	Primary Evidence	Implication for Firms
AI helps recombination not radical invention	AI excels at combining existing knowledge — not at genuinely novel discovery	Wu, Hitt & Lou (2020, MS)	AI accelerates recombinative innovation but does not substitute for radical creative insight
Intermediate novelty ceiling	AI effect is strongest in the middle of the novelty spectrum — not at the radical end	Wu, Lou & Hitt (2025, MS); Lou, Ma & Wu	Firms should target AI at moderately novel exploration — not expect it to produce radical breakthroughs
Organizational misalignment	Firms invest in AI without restructuring organizations to exploit it	Wu, Lou & Hitt (2019, MS); Wang & Wu (2026, MS)	AI investment must be paired with organizational redesign — decentralization and lean practices
Leadership mismatch	Non-STEM CEOs cannot fully direct AI toward exploration	Lou, Ma & Wu; Hambrick & Mason (1984, AMR)	STEM leadership amplifies AI’s exploration benefits — a critical and undervalued complementarity
Innovation distribution compression	AI reduces variance — eliminating both failures and breakthroughs	Theoretical — under empirical investigation	Aggregate metrics undercount AI’s contribution — new measurement frameworks needed

The Productivity Paradox — A Historical Parallel

The analytics-innovation paradox may be following the same pattern. AI has been widely deployed for less than a decade. The organizational restructuring, leadership development, and process redesign required to fully exploit AI for exploration are still in early stages. The measurement frameworks designed to capture AI’s impact were built for a different innovation regime. The research suggests that firms that have made these complementary investments — decentralizing their innovation structures, adopting AI-compatible organizational practices, and developing STEM leadership — are already seeing exploration benefits that the aggregate statistics cannot yet capture. The paradox may be temporary — a lag effect rather than a permanent limitation.

THE IT PRODUCTIVITY PARALLEL Brynjolfsson and McElheran (2016) showed that IT productivity benefits arrived only after firms restructured their organizations to exploit the technology. The same logic almost certainly applies to AI and innovation. The firms that restructure now — decentralizing innovation, developing STEM leadership, adopting complementary practices — will likely capture exploration benefits that current aggregate metrics cannot yet see.

What This Means for How We Measure Innovation

One underappreciated dimension of the analytics-innovation paradox is that it may partly reflect a failure of our measurement frameworks rather than a failure of AI.

If AI is shifting the type of innovation from discrete product inventions toward process improvements and knowledge recombinations — which is exactly what the Lin and Maruping research suggests — then the standard metrics will systematically undercount AI’s innovation contribution. The innovation is real. The measurement system is not designed to see it.

This measurement challenge is itself an open research question — and one of the most important unsolved problems in the field. Developing better metrics for recombinative and process innovation, and better frameworks for understanding how AI changes the distribution rather than just the average of innovation quality, is among the highest-priority challenges for the next generation of AI innovation researchers.

The Competitive Dimension — A Hidden Piece of the Puzzle

There is one dimension of the paradox that existing research has not yet fully addressed — and it may be among the most important.

Every study of AI and innovation focuses on what happens inside a single firm. Does this firm’s AI investment affect this firm’s exploration? This is the natural starting point — but it misses a crucial dynamic.

In reality firms do not innovate in isolation. They watch each other constantly. When a rival firm surges in AI investment and begins filing patents in new technology domains, competitors face a choice: respond by exploring new territory themselves, or risk being left behind. This competitive pressure to explore — driven by rival AI investment rather than one’s own — is entirely absent from the existing literature.

If rival AI investment forces firms to explore — but those firms lack their own AI investment to guide that forced exploration intelligently — the result could be a wave of unfocused, low-quality exploration that inflates patent counts without producing genuinely valuable innovation. Firms are exploring. But they are exploring blindly. The quantity of exploration rises. The quality falls. The aggregate metrics see only the quantity — and report stagnation.

This competitive dimension of the analytics-innovation paradox is one of the most important open questions in the field — and one that the next generation of AI innovation research is beginning to address.

What Resolves the Paradox — The Research Consensus

Drawing together the five explanations and the historical parallel, the research consensus suggests that the analytics-innovation paradox resolves when firms do four things simultaneously:

→  Invest in machine learning specifically — not just general AI. Wu, Lou, and Hitt (2025, Management Science) and Lou, Ma, and Wu show consistently that ML-specific investment has stronger exploration effects than broad AI investment. The exploration premium comes from machine learning’s ability to identify non-obvious connections across vast knowledge domains — not from chatbots or process automation.

→  Restructure for exploration — AI complements decentralized innovation structures (Wu, Lou & Hitt 2019, Management Science) and lean organizational practices (Wang & Wu 2026, Management Science). Investment without restructuring produces efficiency gains but not exploration acceleration.

→  Develop STEM leadership — Lou, Ma, and Wu demonstrate that STEM-background CEOs amplify AI’s exploration effects substantially. Building STEM leadership — through hiring, development, and board composition — is as important as the AI investment itself.

→  Target intermediate novelty — The most productive use of AI for exploration is in the intermediate novelty zone — bold departures from existing knowledge that AI can still guide. Firms that try to use AI to produce radical breakthroughs misunderstand its comparative advantage.

The paradox is not a condemnation of AI. It is a description of where most firms are in the process of learning to use it. The firms that resolve the paradox — by making the right complementary investments in organization, leadership, and strategy — are not yet numerous enough to move the aggregate needle. But they exist. And the research is documenting how they do it.

What This Means for Researchers

For researchers entering this field the analytics-innovation paradox is not a discouraging finding — it is an invitation. Every one of the five explanations described in this article is an active research frontier with important unanswered questions.

Open Research Question	Why It Matters	Key Papers to Read
Does AI compress the innovation distribution — reducing both failures and breakthroughs?	Would explain why aggregate metrics show stagnation despite real firm-level benefits	Hall et al. (2001); Wu et al. (2020)
Does rival AI investment force unfocused exploration that drives down aggregate quality?	Would explain competitive dynamics component of paradox	Chen (1996); Bloom et al. (2013)
How do organizational structure and AI interact to determine exploration quality?	Would specify which restructuring investments matter most	Wu et al. (2019); Wang & Wu (2026)
Does the paradox resolve over time as complementary investments accumulate?	Would predict when and for whom AI’s exploration benefits will become visible in aggregate data	Brynjolfsson & McElheran (2016)
Are current patent-based metrics adequate for measuring AI-enabled innovation?	Would motivate development of new measurement frameworks appropriate for the AI era	Griliches (1990); Lin & Maruping

Next: Article 3 — What Is Explorative vs Exploitative Innovation and Why the Difference Matters