Please register for the CV & Online Identity Workshop below.

Is innovation dominated by individual actions or should it be understood as a collective process to make the whole greater than the sum of its parts? Innovation that brings about new knowledge is often described as extraordinary individuals’ search processes, successfully exploiting and exploring a complex landscape of concepts, ideas, and items to find their novel connections. Then, where does the complex landscape come from? We search the space individually, but the landscape is collectively structured, maintained by, and shared with the society as we create and structure our findings. The very landscape will further down the road to open up new adjacent possible or close down existing possibilities to individuals’ future invention activities. Such co-evolution of constraints explains the full of contingent and idiosyncratic cases we often aspire to, and at the same time, puts us in despair of the unpredictable nature of innovation. Innovation is path dependent. However, what if past-paths are well-understood? Does this mean we can predict the future innovation from the past innovation? It is indeed the case that there has been evidences of predictive innovations that contradict the very premises of innovation. Here we construct a stochastic model of co-evolution between individual actions and the landscape. The landscape is defined as an accumulated construction of our collective past paths. We show individuals who have incentive to be aligned create a convention toward a paradigm, becoming an increasing inertia moving toward a particular dynamic direction at a larger scale than that of individual dynamics. As a result, the landscape becomes more modular and rugged structure that further constrain the individuals’ future degree of freedom. By constructing an innovation phase diagram of counter-factual spaces, and by analyzing two independent empirical datasets – almost two centuries of the U.S. patents and four decades of scientific publications – we demonstrate that both science and technology evolve at the edge between the exploitation strategy and exploration strategy. Our framework explains predictable innovation. The individual innovative process shapes the underlying knowledge landscape, but also their future actions are shaped by the very landscape they have been shaping. Such co-evolution processes between individuals and the collective structure seem to leave and pave the path-dependent trajectory of innovation, which explains predictable innovation.