We have consistently overestimated what computation is capable of throughout history, whether computation was seen as an algorithmic method executed by humans, or a process of automated deduction realised by a machine. The fictional record is crystal clear on this point.
Instead of imagining machines that can do a task better than we can, we imagine machines that can do it in the best possible way. When we ask why, the answer is invariably some variant upon: it is a machine and therefore must be infallible.
This is absurd enough in certain specific cases: what could a ‘best possible poem’ even be? There is no well-ordering of all possible poems, only ever a complex partial order whose rankings unravel as the many purposes of poetry diverge from one another.
However, the deep, and seemingly coherent computational illusion is that there is not just a best solution to every problem, but that there is a best way of finding such bests in every circumstance. This implicitly equates true AGI with the Godhead.
We hold, first, that the “religion of the Singularity” is not new—it must be understood as a symptom of neoliberal rationality in the Information Age. Second, we argue that the same neoliberal logic is exemplified by recent developments in the urban process, its value flows, and its associated forms of governance. Finally, we conclude that to surpass the contradictions of info-capitalism that unfold in the ideology of the Singularity and of tech-infused urban life, we can turn to alternative models of ownership. Only by wresting back control of information and space can we begin to build radical alternatives to Singularitarian reduction.Claudel & Shafer (2019), “A Rumble in the Taupe Hum of Info-Capital: On Reduction and the Neoliberal City“. Journal of Design and Science.
“For Singularity to have a positive outcome requires a belief that, given enough power, the system will somehow figure out how to regulate itself. The final outcome would be so complex that while we humans couldn’t understand it now, “it” would understand and “solve” itself. Some believe in something that looks a bit like the former Soviet Union’s master planning but with full information and unlimited power. Others have a more sophisticated view of a distributed system, but at some level, all Singularitarians believe that with enough power and control, the world is “tamable.” Not all who believe in Singularity worship it as a positive transcendence bringing immortality and abundance, but they do believe that a judgment day is coming when all curves go vertical.
Whether you are on an S-curve or a bell curve, the beginning of the slope looks a lot like an exponential curve. An exponential curve to systems dynamics people shows self-reinforcement, i.e., a positive feedback curve without limits. Maybe this is what excites Singularitarians and scares systems people. Most people outside the Singularity bubble believe in S-curves: nature adapts and self-regulates, and, for example, when a pandemic has run its course, growth slows and things adapt. They may not be in the same state, and a phase change could occur, but the notion of Singularity—especially as some sort of savior or judgment day that will allow us to transcend the messy, mortal suffering of our human existence—is fundamentally a flawed one.“