Singularitarianism: what singularitarians believe about AI

Singularitarianism is a movement built on two beliefs: that a technological singularity, meaning the creation of superintelligence, will likely happen in the medium future, and that people should act on purpose to make sure it benefits humans. Its members, singularitarians, do not just expect the singularity; they think it is desirable if guided well, and some work to bring it about safely. This post explains what singularitarians believe, where the movement came from and what its critics say, with no verdict. The claims come from Wikipedia's Singularitarianism article.

What singularitarianism means

Plenty of futurists talk about a coming singularity. What sets singularitarians apart, the article says, is that they think it is not only possible but desirable if it is guided prudently. Some of them dedicate their lives to helping it arrive quickly but safely.

So the movement rests on three beliefs, each one going a step further:

1. It is possible: superintelligence will likely be created in the medium future.
2. It is desirable: guided well, it would be good for humans.
3. Act on it: deliberate action should be taken to make sure it benefits humans.

Many people believe a singularity is possible without taking the second and third steps. If the first idea is new to you, start with our post on the technological singularity.

How the word changed

The term was first defined in 1991 by Mark Plus (Mark Potts), described as an Extropian thinker, to mean "one who believes the concept of a Singularity". Since then it has been redefined as "Singularity activist" or "friend of the Singularity": someone who acts to bring it about.

Some sources describe singularitarianism as a moral philosophy that argues for deliberately bringing about and steering the development of a superintelligence. Others describe it as an outlook that favors enhancing human intelligence in general, rather than focusing on one technology such as AI.

Ray Kurzweil, the author of The Singularity Is Near, has his own definition: a singularitarian is someone "who understands the Singularity and who has reflected on its implications for his or her own life".

Where the movement came from

The article traces an early version of the idea, that history is moving toward a point of superhuman intelligence, to the philosopher Hegel's The Phenomenology of Spirit.

The modern idea starts in 1993, when the mathematician, computer scientist and science fiction author Vernor Vinge described a moment when technology would allow the "creation of entities with greater than human intelligence". He called it "the Singularity" and suggested it might pose an existential risk to humanity. He gave four ways it could happen:

• Computers that are "awake" and superhumanly intelligent.
• Large computer networks, with their users, that "wake up" as one superhumanly intelligent entity.
• Links between computers and people so close that the users themselves count as superhumanly intelligent.
• Biology finding ways to improve the natural human mind.

Yudkowsky, MIRI and the Singularitarian Principles

The movement became a coherent ideology in 2000, the article says, when the AI researcher Eliezer Yudkowsky wrote The Singularitarian Principles. In it, a singularitarian believes the singularity is a secular, non-mystical event that is possible, good for the world, and worth working toward. Michael Anissimov argued for a stricter definition: only people who advocate building superintelligence.

In June 2000, Yudkowsky founded what is now the Machine Intelligence Research Institute (MIRI), to work toward self-improving Friendly AI. These singularitarians believed that reaching the singularity swiftly and safely was the best way to lower the overall risk to humanity's existence. Our post on Eliezer Yudkowsky covers his later work.

A small movement

The article calls it a small movement, which includes the philosopher Nick Bostrom. Kurzweil did much to make it popular with his 2005 book.

Singularitarians and the risks of AI

Singularitarians are not only optimists. In July 2009, many of them took part in a conference run by the Association for the Advancement of Artificial Intelligence (AAAI). It asked whether computers and robots could gain autonomy, make their own decisions, and become a threat.

They noted that some machines already had partial autonomy, such as finding their own power sources or choosing targets for weapons. They warned that some computer viruses can avoid being removed and had reached "cockroach intelligence". They said self-awareness as shown in science fiction is probably unlikely, but that there are other dangers.

What critics say

Objections come even from optimists in AI. The author Douglas Hofstadter argued that Kurzweil's human-level AI by 2045 is not viable. The namesake of Moore's Law, the trend the singularity idea was built on, maintained that the singularity will never happen.

Others see it as a kind of religion. Some observers say it works as a quasi-religious answer to the fear of death. The science journalist John Horgan wrote that "the singularity is a religious rather than a scientific vision", quoting the science fiction writer Ken MacLeod's phrase "the rapture for nerds". Horgan argued that engineers and scientists should help solve the world's problems instead of "indulging in escapist, pseudoscientific fantasies". Our post on TESCREAL covers a related critique.

Kurzweil rejects this. He says his views are driven by data showing that computing has long grown exponentially, and that his critics take an intuitive, straight-line view of progress.

A worked example: where do you stand on the three beliefs?

The three beliefs make a quick way to sort any opinion about the singularity, including your own.

1. Possible? Hofstadter and the namesake of Moore's Law doubt it, at least on Kurzweil's timetable. Vinge and Kurzweil think so.
2. Desirable? Singularitarians say yes, if guided well. Vinge, who gave the modern term its meaning, also suggested it might be an existential risk.
3. Should we act to steer it? Singularitarians say yes; Yudkowsky founded MIRI in 2000 to work toward self-improving Friendly AI.

Now place yourself. You might say yes to the first and no to the second, which makes you a futurist but not a singularitarian. Or yes to all three. The exercise shows that "do you believe in the singularity?" is really three questions.

Exploring the idea in Contain ASI

Contain ASI is a story strategy game for 1 to 4 players that runs in your browser and needs WebGL 2. Chapter 1, Before ASI, starts in January 2024 with four fictional AI labs in one race. As a researcher, research manager, CEO or government, you try for three years to keep every lab from crossing into recursive self-improvement before 2027.

It is a game, not a forecast: the page says all its labs, people and events are fictional, and it does not mention singularitarianism. It takes the opposite stance to the movement's: instead of steering toward superintelligence, your job is to hold it back.

Frequently asked questions

What is a singularitarian?

Someone who believes a technological singularity, the creation of superintelligence, is likely in the medium future, thinks it is desirable if guided well, and wants deliberate action so that it benefits humans.

Who started singularitarianism?

The word was first defined by Mark Plus in 1991. The article says it became a coherent ideology in 2000, when Eliezer Yudkowsky wrote The Singularitarian Principles.

Is Ray Kurzweil a singularitarian?

He has his own definition of the word, places the singularity around 2045, and did much to popularize the movement with his 2005 book.

Why is singularitarianism criticized?

Critics doubt the timetable or that a singularity will happen at all, and some, like John Horgan, call it a religious rather than scientific vision.

Get started

Want to see what it is like to hold superintelligence back instead? Open Contain ASI in your browser and start a campaign as a researcher, research manager, CEO or government.