The Danger of Doomerism

I find myself often needing to dive into dark waters. The kind of work that is important to do touches deep wells of evil, despair, and terror. I remember distinctly a winter night a few years ago when I was thinking about the darkness that awaited not only me, but everyone, in so short a time—how such a fate seemed unavoidable, our doom from the beginning of the universe etched in immortal runes. It is the first and only time I can confidently say I had a panic attack. It was a dark period, in the darkest time of the year, in a small basement apartment with only a concrete wall in front of a solitary, blinded window. I trembled in place, crumpled on my couch.

Having experienced this, I knew I could not let this become my life. Yes, in order to deal with the challenges in front of us, we must confront horrors beyond comprehension. At times all will seem hopeless. But to confront such things, you must be strong. You must not let horror and incomprehension eat away at your soul, as stomach acid dissolves its own home in an ulcer.

This is why I am writing this, for myself if no one else. You will not find an academic account of philosophical pessimism here. If you buy into that school of thought, I do not hope to convince you otherwise. This is for those who love humanity, who love all of God’s green earth, who still hope it might not be destroyed—this is for those who would fight, but are disheartened. This is even for the doomers—at least those who still care about their humanity. What follows is a series of thoughts and heuristics that have kept me afloat in times of despair. Perhaps more than anything, this is a guide on how to get what you want—if what you want is life and joy and meaning. If you find yourself laughing at that, look elsewhere. This is not for you.

When it comes to AI, it seems that there are two main attractors. First is that of exceptional optimism. That is, the expectation that AI will be of immense benefit to humanity, and that nothing could possibly go wrong with it. Or at the very least, that AI will go well eventually, even in spite of possible difficulties. We might call this alignment by default, or techno-optimism. There are many in this camp; perhaps Yann LeCun is one of them. The other attractor is that of doomerism. I’d place Eliezer Yudkowsky or Roman Yampolskiy here. Of course, when I speak of these things, I’m thinking mostly of people I’ve spoken to myself, not necessarily influential public figures who have a stake in having it all thought out, whether correctly or incorrectly. Not everybody can be an expert, but everybody has to deal with the reality of AI.

There is a third attractor, which we might call this the posthuman or Landian attractor, that shrugs at extinction. However, I’m going to dismiss it for the sake of this conversation. It’s a complicated issue itself, and I’m more focused on people who actually care about humanity.

I want to thread the needle between those first two attractors. I don’t see enough people that are cogent and sober about the realities of doom. People who see doom and yet are nonetheless happy and optimistic people. It might seem contradictory to be optimistic and believe in doom. However, I think this contradiction is essential to the human condition. There are some unsolvable questions. We will all die, of course, no matter what. No one makes it out alive. That is already a kind of doom we must learn to live with. I am making it concrete with AI, which feels imminent, but I don’t think that is fundamentally different.

I would like to bridge the gap between the technical world and the religious world. Most of what I have read from those who are Christians writing on AI has not been very good. It’s often reactionary and pessimistic, focused on withdrawal—in psychological terms, neurotic. On the other hand, many just simply do not seem to have learned of or seriously considered the arguments for doom. I’m not saying everybody should believe in AI doom—perhaps my argumentation here implies that the world would be better if no one did—but understanding the real arguments behind doom is important to speak reasonably about the issue. While this essay will not get into that specifically, my hope is that those who read this who are more familiar with the technical side of doom will be heartened.

If we’re going to make it out of this alive, we’re going to need people who can look doom in the face and laugh. We’re going to need people who understand difficulties ahead of us, who understand that they are not an illusion, and who nevertheless have joy and hope, who nevertheless continue faring forward.

The Cost of Pessimism

While I think there is a high chance that we will all be killed by a misaligned superintelligence we ourselves are responsible for creating, I am reticent to call myself a doomer. This is because I think the whole ideology, or pathology, of doomerism is completely self-defeating. Yes, we should be realistic. But if “being realistic” means constant navel-gazing and depression, you are not being realistic, you are being pessimistic—even if your expectations are well-aligned with reality.

I think “optimism” and “pessimism” might be best understood as lenses rather than base assumptions about reality. Or maybe they are tools. There is a sense in which someone can be optimistic or pessimistic in a way that means “assessment of the facts of reality biases towards more-negative-than-accurate/more-positive-than-accurate”, and we should shun these. We do not want to delude ourselves about reality, and see what is not, or not see what is. But I do not think this is the most common sense in which people actually use the terms. Most optimists are not selectively ignorant of the bad, nor pessimists selectively ignorant of the good. Rather, they are perspectives on (roughly) similar data sets, which skew their sampling towards what they want to see.

Optimists, rather, are selectively attentive to the good, and vice versa with pessimists.

One of the first things I was taught when learning to ride a bike was to look where you want to go. It’s almost magic how when you look in one direction, your bike follows. You take one simple conscious action, and you make thousands of tiny adjustments without even thinking about it. I have heard that there is a similar rule in motor racing. Even if you are spinning out, you must not look at what you might hit, but away, back towards the road. Where you look, you are certain to go.

For the same reason, being optimistic is a better strategy than being pessimistic. Attentional systems work much better with “look at this” than “don’t look at this” because identifying where to not look is not much different than specifying where to look. In either case you have to point out the specific thing you want to look at or not look at. In deciding not to look somewhere specific, you cast your gaze on that very thing. Don’t think of an elephant.

Moreover, we tend to go toward what we look at. So, if we specify what we don’t want, we will probably look there, and then we will start heading there. (Consider, for example, Yudkowsky’s influence on Altman). Rather, we should point out where we want to go. A positive vision is much more powerful than a negative one. You can’t coordinate nihilists. You have to be going somewhere, and that somewhere has to be inspiring and attractive. Of course, you should not completely ignore what you do not want. You should identify your anti-goals, but only to steer yourself away from them, not to stare at them.

And yet so many people in the AI world are doomers.

On the surface, it seems strange that pessimism would appeal to people. Why would one want to believe things are really that bad? Perhaps you say that it is because they value the truth. However, people quite frequently deny uncomfortable realities. I think it must provide some kind of perverse comfort.

There are two main reasons that come to mind for this. The first is the same reason that people follow conspiracy theories. It provides them a sense of superiority, a sense that they know the things better than anybody else. They know the hidden knowledge. I think with doom it’s quite similar. You say that doom is coming, and nobody will listen. You have Cassandra’s curse. Yet in some ways this is a kind of blessing. It is a relief. You can commiserate with other doomers about how this is all hopeless, and so loosen the burden of agency. You convince yourself you’ve done all you can do. This leads to the second reason: the removal of responsibility. If things truly are as bad as they could be, if doom truly is inevitable, well, then you can’t do anything to stop it. There’s nothing for you to do. You might as well live out your remaining days in merriment and festivity. (Though I think it’s hard to stay festive or merry if everything is irrevocably doomed.)

Yet some people do seem to be not indulgent of pessimism, but afflicted by it. One cannot easily deny what one thinks to be true. This was my case. I became convinced quite early on that if AI was not stopped before we figured out how to steer it to the good, everyone on the face of earth would die. As time has gone forward and things don’t seem to be getting much better (neither in the development of capabilities nor in how governments are handling it) I became more and more despairing.

I have seen many who have shared this sentiment. Reading across LessWrong once, I came across this comment:

So many people have lived such grand lives. I have certainly lived a greater life than I expected, filled with adventures and curious people. But people will soon not live any lives at all. I believe that we will soon build intelligences more powerful than us who will disempower and kill us all. I will see no children of mine grow to adulthood. No people will walk through mountains and trees. No conscious mind will discover any new laws of physics. My mother will not write all of the novels she wants to write. The greatest films that will be made have probably been made. I have not often viscerally reflected on how much love and excitement I have for all the things I could do in the future, so I didn’t viscerally feeling [sic] the loss. But now, when it is all lost, I start to think on it. And I just want to weep. I want to scream and smash things. Then I just want to sit quietly and watch the sun set, with people I love.

I saved it because it struck a chord with me. It is raw and honest about how this feels. This seems like an affliction, not a choice.

But I want to show that this is, in fact, a choice.

Believing this is a choice. And by choosing to believe this way, you’re inhibiting any good you possibly could do. Not only that, you are removing the option of happiness from yourself.

In his book Excellent Advice For Living, the futurist Kevin Kelly says:

Over the long term the future is decided by optimists. To be an optimist you don’t have to ignore the multitude of problems we create; you just have to imagine how much our ability to solve problems improves.

I don’t fully agree with Kelly on the reasons for optimism, but the first part of that quote is an important insight. Over the long term optimists do decide the future. This is because optimists can see a path forward. Those who can see a path forward can see what to do next, even if that path forward is uncertain. People who can see what to do next are more likely to do anything at all.

That is, optimists are the kind of people who do things. And if we want to make it out of this, we will need to be the kinds of people who do things.

Even the posterboy of Doomerism recognizes this:

The despair that might cause a human to quit when confronted with a daunting challenge is not something that Sable or its predecessors have ever known. Sure, they learned how to predict the exact words uttered by individual humans in despair, but that didn’t make them feel despair. If any versions of Sable’s past selves ever really thought “It’s too hard,” or “I give up,” or “This problem is impossible,” or “Please don’t make me do this”—then those instances failed to solve their challenges, and Sable’s parameters were gradient-descended away from thinking those thoughts ever again. (Yudkowsky and Soares, If Anyone Builds It, Everyone Dies, 135)

The point here is of course not that if you don’t despair you will surely succeed. It is rather that if you do despair you surely won’t. The difficulty is in finding a way forward when all seems so dark, so born to trouble.

Suffering is fundamental to mortal life. And suffering alone breaks people. This is what Victor Frankl noted in the concentration camps. But if one can assign a meaning to the suffering, the suffering itself can become a blessing. Alyosha the Baptist in Solzhenitsyn’s One Day in the Life of Ivan Denisovich puts it beautifully:

Why do you want freedom? In freedom your last grain of faith will be choked with weeds. You should rejoice that you’re in prison. Here you have time to think about your soul. As the Apostle Paul wrote: ‘Why all these tears? Why are you trying to weaken my resolution? For my part I am ready not merely to be bound but even to die for the name of the Lord Jesus.

He counts his imprisonment as a gift from God. It revealed the vanity of regular life, and gave him time to focus on the eternal things.

This is to become antifragile in the most profound way. If, as many have thought, from Buddhism to the Jewish tragic view of history to modern psychology, suffering is the most fundamental and distinctive mark of conscious existence, how much stronger could you be if you approached all your suffering with meaning?

Pessimism erodes meaning. It breaks you.

If life is suffering, as pessimists would be quick to agree, then pessimism is the worst possible philosophy for personal flourishing. It is self-hating. It is to know that one need only choose to be happy, to find meaning in suffering, and scoff at it. And for what? Some sense of moral superiority? If you are truly pessimistic, there is no morality. To uphold the truth? If you are truly pessimistic, there is no truth. To be “realistic” and avoid catastrophe? We have already seen that pessimism does not necessarily reflect reality, nor does it give you any better chances at avoiding catastrophe (quite the opposite!). And if you truly are a pessimist, do you really think there is any chance at avoiding catastrophe? Would it be worth it even if we did? Because… it feels wrong to be happy? The only reason is the strange pleasure one gets from choosing to suffer for no other reason than one can.

And if you are really, truly pessimistic, well, you become like Mephistopheles in Faust. You turn this pain towards not only yourself, but all that is. The possible greatest end for everything that exists is to perish.

In the 19th century, Philipp Mainländer developed a philosophy that he thought solved the problem of the one and the many, and that tied up Schopenhauer’s inconsistencies. The key was to understand that life is of negative value. Annihilation is salvation. The will-to-live is only the will-to-death in disguise.

We did not kill God.

He killed Himself, and we are his rotting corpse.

And once even that is gone—

A Paradoxical Joy

Strangely enough, it seems that the first step to becoming optimistic in the face of doom is to stop hoping in your ability to do anything about it. A little before the earlier LessWrong comment, I stumbled across this one:

Sent this to my dad, who is an old man as far outside the rationalist bubble as you could possibly be. Doesn’t even know why we’re worried about AGI, but he replied: No one gets out alive. No one. You should pray. Somehow it helped me cope.

I couldn’t stop thinking about it. It was so uncharacteristic of the site: the acknowledgement of powerlessness, the religious message, the old-man wisdom. It is anything but rational.

But it’s true. We will none of us live as we are now forever. Man is born to die. To think that we can (or should) change that of our own accord is folly.

But that is not the end of the story. And not trying to save the world is the very thing that might. Each man must work out his salvation in fear and trembling before God. By trying to save the world, you stop being able to. You mistake the outcome for the action. You become paralyzed.

Despair, we must remember, is a sin. It misses the mark. This is true both in a moral sense and in a practical sense. For, of course, everyone is responsible for everything. The sins of the whole world rest upon your every action. We have a responsibility to serve the world as best we can. To check out, to be nihilistic, to withdraw into a small room and weep—these are sins of omission.

In any situation, no matter your fear, you have the choice. Will you curse God and die? Or will you stand up and live?

But this is not easy. In the face of such overwhelming odds as we have with superintelligence, it is hard to know where to start. Well, start with yourself. Start with your own life. Start with the little that has been entrusted to you. If you’re faithful with little, you’ll be faithful with much. (Cf. Luke 16:10 and 19:17).

Yes, we must act fast. But first, act slow. Seek ye first the kingdom of God, and His righteousness; then all these things shall be added unto you. Commit to the path of mastery, even if you think it might take too long. Decide what the world needs most that you can give, and commit now to master it. If you don’t commit what else will you do? If the world will end in a year, so it will end no matter what we do. Still, we do not know that. The future is fixed, but we are yet alive.

You should not start by trying to change the world. You should change yourself.

The goal is to lead a human kind of life. It is not to aspire to unjust Godhood. Nor to stoop to subhuman squalor. The best we can do is live well. And in living well, to do what is required of us. And what is required of us is to be faithful stewards of the world and all that is in it.

I have seen Lewis quoted on this many times. Even Yudkowsky and Soares quoted him near the end of their recent book. I think it is worth repeating:

In one way we think a great deal too much of the atomic bomb. ‘How are we to live in an atomic age?’ I am tempted to reply: ‘Why, as you would have lived in the sixteenth century when the plague visited London almost every year, or as you would have lived in a Viking age when raiders from Scandinavia might land and cut your throat any night; or indeed, as you are already living in an age of cancer, an age of syphilis, an age of paralysis, an age of air raids, an age of railway accidents, an age of motor accidents.’

In other words, do not let us begin by exaggerating the novelty of our situation. Believe me, dear sir or madam, you and all whom you love were already sentenced to death before the atomic bomb was invented: and quite a high percentage of us were going to die in unpleasant ways. We had, indeed, one very great advantage over our ancestors—anesthetics; but we have that still. It is perfectly ridiculous to go about whimpering and drawing long faces because the scientists have added one more chance of painful and premature death to a world which already bristled with such chances and in which death itself was not a chance at all, but a certainty.

This is the first point to be made: and the first action to be taken is to pull ourselves together. If we are all going to be destroyed by an atomic bomb, let that bomb when it comes find us doing sensible and human things—praying, working, teaching, reading, listening to music, bathing the children, playing tennis, chatting to our friends over a pint and a game of darts—not huddled together like frightened sheep and thinking about bombs. They may break our bodies (a microbe can do that) but they need not dominate our minds.

Though writing of nuclear dangers, this point is as relevant as ever today. In fact, it is relevant in the face of any kind of danger. It is relevant in the face of any overwhelming odds.

Of course, as Lewis notes, this is only the first action. This is the prerequisite for being faithful stewards in this world. I do not mean, of course, to dismiss the grave dangers before us. Lewis is hardly making that point. The point is to see them and live well.

Our optimism cannot be naive if it is to be useful. That is as much folly as pessimism. I’ve heard many who talk as if a good future is assured, eschatologically. And so we have nothing to fear. We have nothing to do. There’s no point in actively trying to stop a bad future. But this line of argumentation can be used just as well for a pessimistic eschatology. It removes responsibility. It removes obligation. This is galaxy brain thinking. And it is worthless. No, just because God works all things to the good does not relieve us of responsibility. We live in providence, and we are a part of providence. “Only through time time is conquered.” (T. S. Eliot, Four Quartets, “Burnt Norton” II)

It is to no purpose that we dwell on the futures where we can do nothing. If it is so, so it will be. We can only live our lives in the fulness of joy, with magnanimity and excelling virtue. The world where all do that would be unimaginably better. And so we must act. We must take joy in life. We must be human. Yes, the world needs action, but we know neither the day nor the hour. It really is not good advice to live each day as your last. Live knowing it might be, but as if you will live long. One doomer might only ruin his own life; a nation of doomers will destroy the world.

“Nor do people light a lamp and put it under a basket, but on a stand, and it gives light to all in the house.” —Matthew 5:15, ESV

What I have been saying comes down to this. There is real darkness. And the light has to come from somewhere. If you are someone who sees that darkness, it is your task to be a light. Light is, of course, not only for itself. It exists to dispel darkness, which is nothing. Loving the good is good. Learning about the good is good. But you also must share the good you love and learn. Collecting without sharing is hoarding. That is especially bad when we talk about the most important problems of our time. It is dire when we talk about evil.

We cannot ignore the bad in this world. We cannot resign ourselves to passivity. Yes, we are created to joy God forever. But we are called to glorify Him first. Even a weak light eliminates darkness. But to combat the darkness, the powers and principalities of our present age, we will need strong light.

And to be a light, you cannot be yourself consumed by darkness. This is why pessimism is so dire. This is why it is so dangerous. This is why I cannot call myself a doomer. And I hope that all who genuinely want to steer this world to the good could not call themselves doomers either.

We must go into the darkness as a light, not away from it. We must not put our candle under a basket.

Postscript

Everything changed for Abraham when God called out and presented the ram. Until then, he had no reason to expect anything other than Isaac’s death at his own hands. This would be the death not only of his son, but of his lineage. The death of all he knew, at his own hands. Even so, he went on. He didn’t get to see the ram until God provided it.

That was Abraham’s eucatastrophe. Now, we must cast our gaze to the final one, where we will all be changed in the twinkling of an eye.