Utopia

Extropian Dreams, Paperclip Nightmares

In the early days of Silicon Valley, it was actually quite common (at least among far-thinking futurist types who populated the Bay Area) to believe that the coming Singularity would be the single greatest event in the history of the universe, the culmination of the techno-capitalist project that has already done so much good for humanity. In the 1980s and 1990s, the “extropian” community in the Bay Area (a futurist collective committed to fighting entropy and death, which organized around local meetups, an academic institute and a popular mailing list) eagerly awaited the Singularity, believing that a glorious transhuman future — free from all death, suffering, and scarcity — awaited them on the other side. The extropians thought that the interim state of life on Earth — a hotbed of torture, disease, and needless hunger — was a moral catastrophe, and believed that humans had a moral obligation to start the Singularity as soon as possible — in order to bring this nightmare to an end.

But beginning in the early 2000s, the assumption that the Singularity will be a good thing — for human beings, and in general — began to fade. Eliezer Yudkowsky, an autodidact AI researcher, was the first to popularize the argument that superintelligent AI, far from ushering in an era of limitless abundance, would likely optimize away everything that humans care about (including humans) in favor of cold, senseless, alien goals. Yudkowsky believed that humans were anthropomorphizing AI goals, which could be stranger, and less desirable, than we would naturally assume. In a classic Yudkowskian parable, a paperclip factory owner creates a superintelligent AI and tells it to “make as many paperclips as possible.” The AI then proceeds to turn the entire universe, including the factory owner, and all of the eight billion other people on Earth, into paperclips.

Ginerva Davis, The Silicon Man source

This is the dream time

So I agree with Robin Hanson: This is the dream time. This is a rare confluence of circumstances where the we are unusually safe from multipolar traps, and as such weird things like art and science and philosophy and love can flourish.

As technological advance increases, the rare confluence will come to an end. New opportunities to throw values under the bus for increased competitiveness will arise. New ways of copying agents to increase the population will soak up our excess resources and resurrect Malthus’ unquiet spirit. Capitalism and democracy, previously our protectors, will figure out ways to route around their inconvenient dependence on human values. And our coordination power will not be nearly up to the task, assuming something much more powerful than all of us combined doesn’t show up and crush our combined efforts with a wave of its paw.

Scott Alexander, Meditations on Moloch source

The u-via banishes the question of meaning

But Bauman is wary of the utopian perspective that the hunt offers, as he concludes his musings by saying that ‘the utopia, or “u-via”, of hunters, the utopia of life revolving around the pursuit of constantly elusive fashion, does not give sense to life, whether authentic or fake. It merely helps to banish the question of life’s meaning from our minds’ (Culture 30).

Lars Schmeink, Dystopia, Science Fiction, Posthumanism, and Liquid Modernity

The telos of man and capturing Gnon

And then there’s us. Man has his own telos, when he is allowed the security to act and the clarity to reason out the consequences of his actions. When unafflicted by coordination problems and unthreatened by superior forces, able to act as a gardener rather than just another subject of the law of the jungle, he tends to build and guide a wonderful world for himself. He tends to favor good things and avoid bad, to create secure civilizations with polished sidewalks, beautiful art, happy families, and glorious adventures. I will take it as a given that this telos is identical with “good” and “should”.

Thus we have our wildcard and the big question of futurism. Will the future be ruled by the usual four horsemen of Gnon for a future of meaningless gleaming techno-​progress burning the cosmos or a future of dysgenic, insane, hungry, and bloody dark ages; or will the telos of man prevail for a future of meaningful art, science, spirituality, and greatness?

Franklin continues:

The project of civilization [is] for man to graduate from the metaphorical savage, subject to the law of the jungle, to the civilized gardener who, while theoretically still subject to the law of the jungle, is so dominant as to limit the usefulness of that model.

This need not be done globally; we may only be able to carve out a small walled garden for ourselves, but make no mistake, even if only locally, the project of civilization is to capture Gnon.

Scott Alexander, Meditations on Moloch source

The insignificance of politics

Bauman shares Levitas’s analysis that the possibility of utopian change is located in politics and that political agency has been dissolved: ‘The most conspicuous feature of contemporary politics,’ Bauman writes, ‘is its insignificance’ (In Search 4). The dual translation between the public and the private that politics is supposed to facilitate has been turned inside out, as Bauman argues, and ‘politics has been effectively disarmed’ (Society 169). Flaws in policy cannot be grasped as unethical anymore, because they do not connect to the individual directly, whereas unethical private actions become political outrages (as witnessed in the Clinton– Lewinsky affair), when the individual life choices do not appeal to the public. Politicians have been replaced by idols, who share their private life politics, becoming examples not leaders.

Lars Schmeink, Dystopia, Science Fiction, Posthumanism, and Liquid Modernity

The hunter’s utopia

The hunter’s utopia is thus most importantly characterized not as a fixed goal to strive for, but as a process of living in constant movement. ‘Hunters search for fun, excitement, validation, consumer goods and identity’ (‘Liquid’ 76), Jacobsen argues, all the while sampling and surfing their way through an endless array of opportunities.

Lars Schmeink, Dystopia, Science Fiction, Posthumanism, and Liquid Modernity

Needs are socially constructed

Because of the socially constructed nature of both need and satisfaction, Levitas further argues, there cannot be an evaluation of utopia based on universal principles of what constitutes a ‘good society’ and what a ‘bad society’: ‘If needs are socially constructed, the project of trying to read off the good society from a definition of human nature and human needs is doomed to failure’ (184).

Lars Schmeink, Dystopia, Science Fiction, Posthumanism, and Liquid Modernity

Multipolar traps and the Science God

The modern research community knows they aren’t producing the best science they could be. There’s lots of publication bias, statistics are done in a confusing and misleading way out of sheer inertia, and replications often happen very late or not at all. And sometimes someone will say something like “I can’t believe people are too dumb to fix Science. All we would have to do is require early registration of studies to avoid publication bias, turn this new and powerful statistical technique into the new standard, and accord higher status to scientists who do replication experiments. It would be really simple and it would vastly increase scientific progress. I must just be smarter than all existing scientists, since I’m able to think of this and they aren’t.”

And yeah. That would work for the Science God. He could just make a Science Decree that everyone has to use the right statistics, and make another Science Decree that everyone must accord replications higher status.

But things that work from a god’s-eye view don’t work from within the system. No individual scientist has an incentive to unilaterally switch to the new statistical technique for her own research, since it would make her research less likely to produce earth-​shattering results and since it would just confuse all the other scientists. They just have an incentive to want everybody else to do it, at which point they would follow along. And no individual journal has an incentive to unilaterally switch to early registration and publishing negative results, since it would just mean their results are less interesting than that other journal who only publishes ground-​breaking discoveries. From within the system, everyone is following their own incentives and will continue to do so.

[…]

A basic principle unites all of the multipolar traps above. In some competition optimizing for X, the opportunity arises to throw some other value under the bus for improved X. Those who take it prosper. Those who don’t take it die out. Eventually, everyone’s relative status is about the same as before, but everyone’s absolute status is worse than before. The process continues until all other values that can be traded off have been – in other words, until human ingenuity cannot possibly figure out a way to make things any worse.

Any human with above room temperature IQ can design a utopia. The reason our current system isn’t a utopia is that it wasn’t designed by humans. Just as you can look at an arid terrain and determine what shape a river will one day take by assuming water will obey gravity, so you can look at a civilization and determine what shape its institutions will one day take by assuming people will obey incentives.

But that means that just as the shapes of rivers are not designed for beauty or navigation, but rather an artifact of randomly determined terrain, so institutions will not be designed for prosperity or justice, but rather an artifact of randomly determined initial conditions.

Scott Alexander, Meditations on Moloch source

From keeper to gardener

Upholding God’s design and keeping the boundaries of nature and culture under supervision were thus the ideals of pre-modern utopian endeavors.

With modernity came the urge to control and manipulate nature, resulting in the gardener’s approach as a concept of utopianism. (…) In gardening utopias, man took control over his garden, giving it ‘constant attention and effort,’ forcing ‘his preconceived design on the plot’ (Bauman, Liquid Times 99) by selection, cultivation, and exclusion of unwanted elements.

Lars Schmeink, Dystopia, Science Fiction, Posthumanism, and Liquid Modernity