AI and The Serendipity Economy: Measuring Value Beyond Productivity and Narrow Returns
AI is not failing. It is behaving exactly like every other general‑purpose technology when it first collides with industrial‑age expectations: it creates value in ways reporting, dashboards and metrics built for previous technologies are blind to. But businesses keep insisting on asking the wrong questions because business models and mindsets change more slowly than technology, and in the case of AI, without sufficient investment in ongoing competency building, what capabilities change daily, and limitations are often illusory, even if the risks are not.
In Welcome to The Serendipity Economy, I argued that much of the value created in a networked world arrives off‑schedule, off‑ledger, and certainly off‑slide. We live in an economy of facilitated accidents, where value emerges from unexpected interactions such as people meeting, ideas recombining, technologies bumping into contexts their designers never imagined. Traditional productivity metrics do not see this kind of value; in fact, they are designed not to (see How IT Professionals Can Embrace the Serendipity Economy, HBR).
Fast forward to AI in 2026. MIT’s NANDA initiative reports that only about 5% of generative AI projects are producing tangible business results, while the other 95% languish in pilots or get shut down before they ever touch the balance sheet. At first glance, that sounds like failure. From a Serendipity Economy perspective, it looks very different. It looks like inventory.
AI and The Serendipity Economy: The 95% That “Failed” – Or Did They?
We evaluate most AI initiatives as if they were industrial products. We fund a project, define a use case, build or buy a solution, and then ask: did it reduce cost by X percent or increase revenue by Y percent, on schedule? If not, we declare the initiative a failure, cut the budget, and move on to the next, perhaps not so shiny thing.
That is not how knowledge work, networks, or serendipity operate.
In The Serendipity Economy, the process of creation is distinct from value realization. Building an AI copilot, fine‑tuning a model, or experimenting with new workflows is creation. The value often appears later, perhaps even in unexpected places. It shows up when a customer behaves differently, when an employee reuses a pattern in an unrelated context, or when a partner builds on your discarded experiment. The time between those events can be months or years.
Practitioners who look past the headline “95% failure” see something different. They find that early pilots create data, patterns, and organizational learning that are reused in second‑ or third‑wave initiatives, and that ecosystems and partners dramatically increase the chance that pilots eventually produce value. In other words: serendipity accumulates, even if the initial project never ships.
A Use Case: AI as a Serendipity Engine for Knowledge Work
Consider a global professional services firm rolling out a generative AI assistant across its knowledge workers. The official promise is typical: “We will reduce proposal development time by 20%.” That is a fine aspiration, but it is only one dimension of value, and a narrow one.
Here is what actually happens in organizations that treat AI as more than a typing aid.
A consultant in São Paulo asks the AI about a tough client problem. A cognitive search system, not unlike those described in work on reimagining enterprise search, goes beyond keyword matching and retrieves a mix of sales data, customer feedback, market trends, and even competitor performance. Generative AI–driven knowledge systems, such as Watson‑based solutions and newer watsonx.ai offerings, routinely cut retrieval time dramatically, but more importantly, they surface content from “far away” in the organization such as whitepapers from other regions, old prototypes, and niche case studies that would never have made it into a static portal.
The idea that emerges from that synthesis becomes the foundation for a new global offering. It was never in the original business case. No one forecast it. The serendipitous value appears because:
- The knowledge network exists: documents, experts, telemetry, and prior experiments are connected.
- AI reduces friction across that network, turning search into discovery and suggestion into recombination
- Someone is curious enough to ask a question and follow unusual, “non‑obvious” results that trigger learning and creativity.
Research on AI‑mediated “information encounters” shows this is not an anecdote. A 2025 study on Serendipitous sparks: AI information encounter, cognitive flexibility and creativity finds that generative systems increase the likelihood of valuable, unexpected insights while users pursue other goals. These encounters: an unasked‑for theoretical perspective, an interdisciplinary example, or a surprising method, predict higher creativity, with cognitive flexibility mediating the effect. That is The Serendipity Economy made visible: the value is not the time saved on a single query, but the downstream work that query enables.
On a quarterly dashboard focused on hours saved, this doesn’t register. Through a Serendipity Economy lens, it is the most important kind of outcome: emergent, unplanned, and disproportionately impactful.
Concrete Enterprise Examples: Search, KM, and “Unexpected Reuse”
If you want to see these dynamics in living systems rather than abstractions, look at what is happening in AI‑driven knowledge management and enterprise search.
- IBM Watson and AI‑driven KM
Case studies of Watson‑powered knowledge management describe reductions of around 40% in information retrieval time, alongside broader effects that are harder to quantify: legal analyses reused across regions, research synthesized across domains, and previously siloed content becoming discoverable in new contexts. IBM now pushes this further with watsonx.ai, positioning generative AI and retrieval‑augmented generation as a way to unify enterprise search, assistance, and decision support across documents, tables, and images (overview, product, embedded solutions case). The immediate value is efficiency; the longer‑term value is the web of recombinations that happen when answers draw from surprising places.ibm+3 - Cognitive enterprise search
Work on “cognitive enterprise search” emphasizes that the most powerful systems don’t just retrieve “relevant” results; they provide unusual juxtapositions—tying a product performance query to customer sentiment, external market signals, and competitor moves. Practitioners writing about reimagined enterprise search repeatedly report that subject matter experts find their existing search tools “relevant but not interesting”; what they crave are the provocative, unusual associations that prompt them to rethink the problem (see Cognitive Enterprise Search: The Renaissance of Knowledge Management). That is engineered serendipity. - Generative AI encounters and creativity
The “Serendipitous sparks” study mentioned earlier gives us empirical grounding for this: AI‑generated variety and unpredictability disrupt fixed models of information seeking and engage people in more dynamic, iterative cognitive processes. Unexpected, multifaceted information transforms individual knowledge representations, fostering flexibility and, ultimately, creativity. In Serendipity Economy terms, the AI becomes a “facilitated accident” engine embedded in everyday knowledge work.
These examples provide organizations with a proof starter when a CFO asks, “But where is the value?” The answer: some of it is in the time saved; much more sits in the reconfigured network of ideas and relationships those systems enable over time. In the best organizations, this will challenge the CFO to reconsider how value is measured.
Beyond the Firewall: Netflix, Spotify, and Algorithmic Serendipity
If executives remain skeptical, you can shift the lens to platforms they use every day.
Spotify’s Discover Weekly has become a global case study in algorithmic serendipity. Analyses like “AI and Serendipity: When Machines Help Us Discover the Unexpected” explain how it doesn’t just give listeners more of what they already listen to; it deliberately introduces tracks just outside their comfort zone, using collaborative filtering and content analysis to surface songs that feel like discoveries. Pieces on “AI‑powered serendipity” and “algorithmic serendipity” make the same argument.
Netflix operates similarly. Articles exploring “how Netflix and Spotify know what you’ll love” show how recommendation systems occasionally surface obscure titles and micro‑genres to test engagement, intentionally injecting novelty rather than collapsing entirely into predictable fare (GAiF overview). Commentary on “the end of serendipity” warns that if we over‑optimize for prediction, we risk losing this exploratory function, underscoring that serendipity has to be designed in, not assumed (See: The End Of Serendipity: What Happens When AI Predicts Every Choice?).
These systems embody several Serendipity Economy principles:
- Value is not fully forecastable: no one could have specified in advance which obscure show would become the next cultural phenomenon, or which micro‑genre would build a devoted following (AI-Powered Serendipity: Curating the Perfectly Unexpected, Just for You!).
- Value requires external validation: the worth of a recommendation is decided by listening, watching, sharing, and the emergent behaviors that ripple through culture.(see: Five worthy reads: Algorithmic serendipity—can AI bring back discovery?)
- Networks over lines: a tweak to a recommendation model influences millions of users, which influences artists’ trajectories, advertising spend, and even which projects get green‑lit.
When you look at Netflix and Spotify, you are not just seeing personalization. You are seeing industrial‑scale serendipity, quietly normalized into everyday experience, not as a gimmick but as a value engine.
Scientific Discovery: AI, Deep Learning, and Unplanned Breakthroughs
Scientific AI gives us another, high‑stakes glimpse into the The Serendipity Economy.
Work on AI and serendipity in scientific discovery highlights systems that surface promising candidate molecules, unexpected correlations, and novel hypotheses that human researchers had not prioritized. Articles like “AI and Serendipity: When Machines Help Us Discover the Unexpected” point to DeepMind and IBM Watson as examples in protein folding, materials science, and drug discovery, where models sift through enormous combinatorial spaces and flag options that often run counter to human intuition—but later prove fruitful.
Projects like Sakana AI’s “AI Scientist” push this further, automating parts of the research cycle: proposing ideas, running experiments, and iterating on results. Some of the most valuable outcomes are not the ones engineers aimed for, but side‑paths the AI explored along the way, which humans then pick up and extend.
In Serendipity Economy terms:
- The models and pipelines are the creation; the value is realized when a later lab, in a different context, uses their outputs as stepping stones to new knowledge.
- The payoff is displaced in time and space, appearing in patents, therapies, or materials far removed from the original grant proposal.
- The research ecosystem: the journals, datasets, conferences, and collaborations act as the value web along which these serendipitous sparks travel.
When AI is framed this way, it stops being a lab curiosity or an automation tool. It becomes the infrastructure for structured accidents in science.
Designing AI for Serendipity, Not Just Productivity
If we accept that AI lives inside The Serendipity Economy, then leaders need to stop treating AI solely as a productivity play and start designing it as a serendipity engine.
That means reframing core questions:
- Instead of “How fast will this pay back?” ask “What new options does this create, and how will we stay attached to those options over time?”
- Instead of “What is the single ROI metric?” ask “What families of value, such as reputation, learning, relationship depth, infrastructure, are we willing to track even if they do not hit net income this year?”
- Instead of “Did this pilot work?” ask “What did this pilot teach us, what data did it create, what behaviors did it expose, and where might those matter next?”
Practically, this suggests different instrumentation:
- Keep a “value diary” for AI assets, recording surprising uses, cross‑team adoption, and derivative projects.
- Tag major wins with their serendipitous AI origins: a partnership that started as a chatbot experiment, a product idea that came from a failed internal tool, a new market discovered in the logs of an abandoned prototype.
- Hold periodic “serendipity reviews” where old AI experiments are revisited as potential data and insight troves, not as dead projects.
We have been here before. Solow’s productivity paradox taught us that it can take decades for general‑purpose technologies to show up in traditional productivity statistics, even as they radically reshape work and competition. AI is at a similar stage. If we keep looking only where the light is good, at short‑term productivity, we will miss the more interesting, and ultimately more consequential, value quietly compounding in the dark.
For more serious insights on AI, click here.
For more serious insights on strategy and scenarios, click here.
Did you find AI and The Serendipity Economy useful? If so, please like, share or comment. Thank you!
All images generated via AI from prompts by the author.
Leave a Reply