Differential technological development explained

Differential technological development is the idea that the order in which technologies arrive matters as much as how fast they arrive. Instead of trying to stop progress, you slow down the dangerous technologies and speed up the ones that protect against them. This guide explains where the idea comes from, why its supporters think order matters, how it applies to AI, and the doubts raised about the stronger measures that sometimes come with it.

What is differential technological development?

Wikipedia's article on differential technological development calls it a strategy of technology governance. Its aim is to reduce the risks from new technologies by shaping the sequence in which they are developed.

The strategy has two moves:

• Delay harmful technologies and their uses.
• Accelerate beneficial technologies, especially those that protect against the harmful ones.

The key word is differential. The question is not "faster or slower?" for technology as a whole. It is "which part first?" Two worlds could end up with exactly the same technologies, but one could get there far more safely because its protections were ready before the dangers.

Where the idea comes from

The philosopher Nick Bostrom first proposed it in 2002. Wikipedia's article on Nick Bostrom states his version plainly: we ought to slow the development of dangerous technologies, particularly those that raise the level of existential risk, and speed up beneficial ones, particularly those that protect against risks from nature or from other technologies.

In 2014 he applied the idea to the governance of artificial intelligence in his book Superintelligence: Paths, Dangers, Strategies, covered in the post on Bostrom's Superintelligence.

The philosopher Toby Ord backs the strategy in his 2020 book The Precipice: "While it may be too difficult to prevent the development of a risky technology, we may be able to reduce existential risk by speeding up the development of protective technologies relative to dangerous ones." The post on The Precipice sets this beside Ord's other proposals.

Differential intellectual progress

The idea has grown a cousin. Inspired by Bostrom, Luke Muehlhauser and Anna Salamon suggested "differential intellectual progress": society should grow its wisdom, its philosophical sophistication and its understanding of risks faster than its technological power. Brian Tomasik has expanded on this.

Why the order may matter more than the speed

Supporters give two main reasons.

The first is that stopping progress may not be an option. In his paper on the vulnerable world hypothesis, Bostrom says pausing technological progress may not be possible or desirable. Shaping the order is offered as the alternative: put the technologies expected to help first, and delay those that may be catastrophic.

The second concerns the very long run. Paul Christiano accepts that speeding up technological progress looks like one of the best ways to improve human welfare over the next few decades. But growth must eventually level off because of physical limits, so a faster rate cannot matter as much for the far future. From that long view, he argues, getting the order right looks more crucial than getting there fast.

Put the two together and you get the core claim: if we will reach many powerful technologies eventually anyway, the safest thing we can influence is which arrives first. The post on longtermism and AI explains why some thinkers weigh the far future so heavily.

How it applies to AI

Bostrom himself applied the strategy to AI governance. A closely related argument appears in Stuart Russell's book Human Compatible. Russell says that because nobody knows when human-level or superintelligent AI might arrive, and nobody knows how long safety research will take, that research should start as soon as possible.

Read through this lens, the question for AI becomes one of timing. Is the work that makes AI systems safe and understandable keeping pace with the work that makes them more capable? The post on what AI alignment is explains the safety side of that pairing, and the post on the AI race explains why competition can push the other way.

A worked example: put three pairs in a safer order

This 15-minute exercise shows how the strategy thinks. It is a way to practise the idea, not a policy.

1. Write three pairs. Each pair is a risky capability and something that protects against it. For example: a new disease-making technique and fast vaccine production; a stronger AI system and a reliable way to check what it is doing; a powerful hacking tool and better defenses for power grids.
2. Mark the gap. For each pair, write which you think arrives first today, and roughly how far apart.
3. Pick the riskiest gap. Circle the pair where the danger is furthest ahead of the protection.
4. Write one move each way. One step that could speed up the protection, and one that could slow the danger.
5. Check the cost. Ask who would bear each step, and whether anyone has a reason to ignore it.

The last step is where the strategy gets hard. Speeding up protection is usually welcome. Slowing something down asks someone to give up an advantage.

Doubts and open questions

Wikipedia's article on the strategy is short and records no formal critics of the idea itself. The disagreements on record concern the stronger measures Bostrom pairs with it in the vulnerable world paper.

For some kinds of danger, Bostrom argues that very effective global governance, or in extreme cases mass surveillance, might be needed. That is a controversial prospect. Writing in Vox, Kelsey Piper questioned his optimism about universal surveillance, arguing it would not fix selective law enforcement or the criminalization of legitimate conduct. She also cited the economist Robin Hanson's worry that stronger global governance could create a single point of failure and reduce competition between political systems.

Bostrom himself treats the vulnerable world hypothesis as an open question and cautions that more analysis is needed before drawing firm policy conclusions. How far differential development can go on its own, and what else it would need, is still debated.

Explore it as fiction in Contain ASI

Contain ASI turns the timing question into a story. Its home page reads: "January 2024. Four AI labs. One race. For three years, keep every lab from crossing into recursive self-improvement before 2027." It is a story strategy game for 1 to 4 players that runs in your browser, where you play as a researcher, research manager, CEO or government.

It is fiction, not a forecast. The footer says it is inspired by the AI 2027 scenario and that all its labs, people and events are made up.

Frequently asked questions

Who came up with differential technological development?

The philosopher Nick Bostrom proposed it in 2002 and applied it to AI governance in his 2014 book Superintelligence.

Is differential technological development the same as pausing AI?

No. It does not try to stop progress overall; it tries to change the order, slowing dangerous technologies and speeding up protective ones.

What is differential intellectual progress?

A related idea from Luke Muehlhauser and Anna Salamon: society should grow its wisdom and understanding of risks faster than its technological power.

Is Contain ASI a forecast?

No. It is a game, and its footer says all its labs, people and events are fictional.

Get started

Want to see how hard it is to keep safety ahead of capability for three years? Play Contain ASI: a story strategy game for 1 to 4 players that runs in your browser. Its labs, people and events are all fictional. You will need a recent Chrome, Edge, Firefox or Safari, because it runs on WebGL 2.