Rationalist community: what it is and how it links to AI risk

The rationalist community is a 21st-century movement that formed around a group of internet blogs, mainly LessWrong and Astral Codex Ten (once called Slate Star Codex). People search for it because its name keeps turning up in talk about AI risk, and the name alone does not say much. This post sets out what the community is, where it came from, why AI risk is one of its main interests, and what critics say about it. It draws on Wikipedia's Rationalist community article, and it takes no side.

The rationalist community in one paragraph

The article says the movement first gained prominence in the San Francisco Bay Area. Its members try to use ideas from fields such as epistemology, probability theory and psychology, along with an awareness of cognitive biases, to think better. The article names two aims. One is epistemic rationality, which means coming to true beliefs about the world. The other is instrumental rationality, which means acting in a way that achieves your objectives.

The borders are blurry. The article says they are debated both inside the community and in nearby groups. People who drift from the usual beliefs often call themselves "rationalist-adjacent" or "post-rationalist".

Where it started: a few blogs

The article gives a short timeline. The community existed on blogs before LessWrong, including Overcoming Bias, which was founded in 2006. LessWrong was founded in 2009. Slate Star Codex was launched in 2013, and its successor Astral Codex Ten was launched on January 21, 2021.

Eliezer Yudkowsky created LessWrong and is regarded as a major figure in the movement. He wrote a series of blog posts between 2006 and 2009 called the Sequences, later collected as ebooks under the title Rationality: from AI to Zombies. From 2010 to 2015 he also published a Harry Potter fanfiction, Harry Potter and the Methods of Rationality. The article says it led people toward LessWrong, and that a 2013 LessWrong survey found a quarter of the site's users had arrived because of it. For more on the man himself, see our post on Eliezer Yudkowsky.

What rationalists mean by rationality

The article says rationalists care about applying science and probability to many topics, with special attention to Bayesian inference. It quotes the journalist Ellen Huet, who says the community aims "to keep their thinking unbiased, even when the conclusions are scary".

The early blogs drew a STEM-minded audience that cared about self-improvement. The article says that audience was wary of the academic humanities and of how emotion can get in the way of clear thought. It also says rationalists rejected the Spock-style picture of a person with no feelings. Yudkowsky argued that emotions can themselves be instrumentally rational responses to a situation.

Writing in The New Atlantis, Tara Isabella Burton describes the culture as having a "technocratic focus on ameliorating the human condition through hyper-utilitarian goals". She adds that a central idea was that nothing, not social niceties and not fear of political incorrectness, should get between people and their ability to see the world as it really is.

Why AI risk is a main interest

The article names AI safety as one of the community's main interests. It describes the aim as fighting the existential risk posed by the emergence of an artificial superintelligence. It also says many members believe theirs is one of the only communities with a chance of saving humanity from extinction.

Two Bay Area organizations tied to the community are named. The Center for Applied Rationality teaches the techniques rationalists favor. The Machine Intelligence Research Institute does research on AI safety, and we cover it in MIRI: what the Machine Intelligence Research Institute does. The article also says the stress of that sense of responsibility has contributed to mental health crises among several rationalists.

If you want the research side of the question rather than the community, our post on existential risk studies covers the academic field and its debates.

Overlapping movements

The article says the community has a large overlap with effective altruism and transhumanism, and that AI safety overlaps too. Critics such as the computer scientist Timnit Gebru and the philosopher Émile P. Torres link rationalism with other philosophies under the name TESCREAL. The letters stand for transhumanism, extropianism, singularitarianism, cosmism, rationalism, effective altruism and longtermism. That label is itself disputed, and we set out both sides in TESCREAL: what the term means and why it is debated.

The article also describes the postrationalists, a loose group of one-time rationalists who became disillusioned. It says they came to see the community as cult-like and dogmatic, and as having lost focus on the less quantifiable parts of a well-lived human life. The term is also used as a hedge by people tied to the community who have drifted from its usual views.

What critics say

The article reports several criticisms, each from a named writer, so read them as claims by those writers.

• Ellen Huet says the movement "valorizes extremes", and that its clarity can also "provide cover for destructive or despicable behavior".
• Gideon Lewis-Kraus, writing in The New Yorker, argues that rationalists "have given safe harbor to some genuinely egregious ideas". He ties this to a willingness to hold orderly exchanges on objectionable topics, which the article says is based on a view that vile ideas should be countenanced and refuted rather than left to gain the status of forbidden knowledge.
• The article says rationalists also hold that some ideas, called information hazards, are dangerous and should be suppressed. It names Roko's basilisk as a commonly cited example, and we explain it in Roko's basilisk: the thought experiment explained.
• Some current and former members have said aspects of the community can be cult-like. The religious scholar Greg Epstein is quoted in The New York Times asking how much more cult-like it would have to be for the label to apply, given the stakes its members describe.

The article gives no reply from the community to these points, so this post does not invent one. It also does not say how many members agree with any of them.

A worked example: four questions for any claim about the community

Claims about "the rationalists" often mix very different things. Here is a way to read one, using only what the article gives you. Take the sentence "Rationalists think AI will end humanity."

1. Who is "the rationalists"? The article says the borders are blurry and debated. Ask whether the claim means bloggers, members of an organization, or anyone who overlaps with effective altruism or AI safety.
2. Is it about belief or behavior? The article says AI extinction risk is a main interest. That is a claim about what many members worry about, not a claim that all of them hold one forecast.
3. Who is the source? A member, a former member, a journalist and a critic describe the community differently. Note the name and the outlet, as the article does.
4. What does it leave out? The article says some members call themselves "rationalist-adjacent" or "post-rationalist". A claim that treats the group as one block hides that.

Run the sentence through them. "The rationalists" is a loose group, the worry about extinction is one of its main interests and not a fixed belief of every member, and the source matters. A fairer version is: "Many people in the rationalist community treat AI extinction risk as a main concern, and critics debate how the community handles that concern."

Frequently asked questions

What is the rationalist community?

It is a 21st-century movement that formed around internet blogs, mainly LessWrong and Astral Codex Ten. Members try to use fields such as probability theory and psychology, and an awareness of cognitive biases, to hold true beliefs and to act on their goals.

Who founded LessWrong?

Eliezer Yudkowsky created LessWrong. The article says it was founded in 2009, and that he is regarded as a major figure in the movement.

Is the rationalist community the same as effective altruism?

No, but they overlap a lot. The article says the rationalist community has a large overlap with effective altruism and transhumanism.

Why is the community linked to AI risk?

The article says one of its main interests is combating existential risk from the emergence of an artificial superintelligence. Organizations tied to it include the Machine Intelligence Research Institute, which does AI safety research.

Is it a cult?

That is a claim some people make, not a settled fact. The article says some current and former members have said aspects of the community can be cult-like, and it quotes Greg Epstein making that case. It gives no reply, so this post gives no verdict.

Get started

If you want to try a story about an AI race for yourself, Contain ASI is a story strategy game for 1 to 4 players that runs in your browser and needs WebGL 2. Chapter 1, Before ASI, starts in January 2024 with four fictional AI labs in one race. As a researcher, research manager, CEO or government, you try for three years to keep every lab from crossing into recursive self-improvement before 2027. Its labs, people and events are all fictional, and it is a game, not a forecast.

Play Contain ASI: a story strategy game for 1 to 4 players that runs in your browser. Its labs, people and events are all fictional.