Walsh 2017 survey

Toby Walsh surveyed hundreds of experts and non-experts in 2016 and found their median estimates for ‘when a computer might be able to carry out most human professions at least as well as a typical human’ were as follows:

|  |  |  |  |
| --- | --- | --- | --- |
| Probability of HLMI |  | Group of survey respondents |  |
|  | AI experts | Robotics experts | Non-experts |
| 10% | 2035 | 2033 | 2026 |
| 50% | 2061 | 2065 | 2039 |
| 90% | 2109 | 2118 | 2060 |

Details
-------

Toby Walsh, professor of AI at the University of New South Wales and Technical University of Berlin, conducted a poll of AI experts, robotics experts, and non-experts from late January to early February 2017. The survey focused on the potential automation of various occupations and the arrival of high-level machine intelligence (HLMI).

### Survey respondents

There were 849 total survey respondents composing three separate groups: AI experts, robotics experts, and non-experts.

The AI experts consisted of 200 authors from two AI conferences: the 2015 meeting of the Association for the Advancement of AI (AAAI) and the 2011 International Joint Conference on AI (IJCAI).

The robotics experts consisted of 101 individuals who were either Fellows of the Institute for Electrical and Electronics Engineers (IEEE) Robotics & Automation Society or authors from the 2016 meeting of the IEEE Conference on Robotics & Automation (ICRA).

The non-experts consisted of 548 readers of [an article about AI on the website The Conversation](https://theconversation.com/know-when-to-fold-em-ai-beats-worlds-top-poker-players-71713). While it seems data on their possible expertise in AI or robotics was not collected, Walsh writes that “it is reasonable to suppose that most are not experts in AI & robotics, and that they are unlikely to be publishing in the top venues in AI and robotics like IJCAI, AAAI or ICRA” (p. 635). Some additional demographic data was collected and reported (for this survey group only):

* **Geographic distribution:** 36% Australia, 29% United States, 7% United Kingdom, 4% Canada, and 24% rest of the world
* **Education:** 85% have an undergraduate degree or higher
* **Age:** >33% are 34 or under, 59% are under 44, and 11% are 65 or older
* **Employment status:** >66% are employed and 25% are in or about to enter higher education
* **Income:** 40% reported an annual income of >$100,000

### Classifying occupations at risk of automation

The first seven survey questions (out of eight total) asked respondents to classify occupations as either at risk of automation in the next two decades or not (binary response). For each occupation, respondents were provided with information about the work involved and skills required. There were 70 total occupations, which came from a previous study that had used a machine learning (ML) classifier to rank them in terms of their risk for automation. These rankings were then used in the present survey: Each question had respondents classify 10 occupations, starting with the five most likely and five least likely at risk of automation according to the ML classifier. This continued through subsequent questions until respondents classified all 70 occupations.

### Arrival of high-level machine intelligence (HLMI)

The last survey question asked by what year there would be a 10%, 50%, and 90% chance of HLMI, which was defined as “when a computer might be able to carry out most human professions at least as well as a typical human” (p. 634). For each probability respondents chose from among eight options: 2025, 2030, 2040, 2050, 2075, 2100, After 2100, and Never. Median responses were calculated by interpolating the cumulative distribution function between the two nearest dates.

### Results

#### Probability of when (in years) HLMI will arrive

Table 1 below summarized the median responses and is reproduced here for convenience.

**Table 1**

|  |  |  |  |
| --- | --- | --- | --- |
| Probability of HLMI |  | Group of survey respondents |  |
|  | AI experts | Robotics experts | Non-experts |
| 10% | 2035 | 2033 | 2026 |
| 50% | 2061 | 2065 | 2039 |
| 90% | 2109 | 2118 | 2060 |

Figures 1-3 below show the cumulative distribution functions (CDFs) for 10%, 50%, and 90% probability of HLMI (respectively) at different years.

![](https://lh4.googleusercontent.com/-lqeKuWnnSoSjsZqHuZ2HACLIbo6VuUBDDn-4hg0Z7ej4KppPxuU0ITUf26wECMK4E_fxnVVQoPmuIwKH1BH5RhHHrW2EKxzwWt3o8PHuaJS-u_v4O7KOf-k7IsRwbafn2mciNyu)
**Figure 1**

![](https://lh6.googleusercontent.com/wXgJBebGuJiQ2sX52SOxghPxAyNZrqS9XEOBXQpBWSa1F8xXqKwe3yWA-CZ45YmVXDXTqds5mGCNIkVCYso4ZraSCDkbqg70qXsqpn0BVTWCjj6BYhD5y9FeXYUAC_hYT8v1EL0G)
**Figure 2**

![](https://lh3.googleusercontent.com/hTr9LcPT5_NO1LFovGuT9FYvEMN8Y1A8k5Qb93BZhKpDZqYaq-pOKon6YaS5DQ5G7N6AIdejpOViIRb87qo8tzLAnEsJCcVKv8G9KxxFB97xHx3NPlQfIlnGAEEp0vG4Xm-lQK9A)
**Figure 3**

#### Occupations at risk of automation

Table 2 below contains descriptive statistics about the number of occupations (out of 70 total) classified as being at risk of automation in the next two decades. Confidence intervals (last column) are at the 95% level. It is unclear why the sample size for Non-experts is listed as 473 when earlier in the article the number reported is 548.

**Table 2**

![](https://lh4.googleusercontent.com/guJlx3jpU_whEAQSGEhg1DriXYtdVmP-NdcqKWSELL8NtA7wTQSKQ43mKD-DNlnmztNjroH8EcoIAVJxByudx3q1vTIw6koTlpYx9uglLyN93Ht6y2MAFABHJ9S01kR7FlObalJn)
The difference in means between the Robotics (29.0) and AI experts (31.1) was not statistically significant (two-sided t-test, p = 0.096), while the differences in means between both expert groups and the non-expert group (36.5) separately were significant (two-sided t-test, both p’s < 0.0001).

Table 3 below lists some of the largest differences in the proportion of experts (AI and robotics combined) compared to non-experts who classified occupations as at risk for automation.

**Table 3**

|  |  |  |
| --- | --- | --- |
| Occupation | Proportion of respondents predicting risk for automation |  |
|  | Experts | Non-experts |
| Economist | 12% | 39% |
| Electrical engineer | 6% | 33% |
| Technical writer | 31% | 54% |
| Civil engineer | 6% | 30% |

Figure 4 below shows that respondents who predicted that HLMI would arrive earlier also classified more occupations as being at risk of automation (and vice versa).

![](https://lh5.googleusercontent.com/R8wieKFMBhSV95mZjqrDbS3UzQV4PDKXGmTAl1H_Ak9ZH7amjONnJaSUX22OCudxw73jRj9XcQi3YHUF23jEa0Wf8eDbcLB_qazUUlDeXA6BVUdPBisEZgerCwOt9I_bwlRsi6Jt)
**Figure 4**

Walsh 2017 survey

Summary: In this survey, AI experts, robotics experts, and the public estimated a 50% chance of high-level machine intelligence (HLMI) by 2061, 2065, and 2039 respectively. The post presents other similar data from the survey.


Conversation with Adam Gleave

AI Impacts talked to AI safety researcher Adam Gleave about his views on AI risk. With his permission, we have transcribed this interview.

**Participants**
----------------

* [Adam Gleave](https://gleave.me/) — PhD student at the Center for Human-Compatible AI, UC Berkeley
* Asya Bergal – AI Impacts
* [Robert Long](http://robertlong.online/) – AI Impacts

**Summary**
-----------

We spoke with Adam Gleave on August 27, 2019. Here is a brief summary of that conversation:

* Gleave gives a number of reasons why it’s worth working on AI safety:
	+ It seems like the AI research community currently isn’t paying enough attention to building safe, reliable systems.
	+ There are several unsolved technical problems that could plausibly occur in AI systems without much advance notice.
	+ A few additional people working on safety may be extremely high leverage, especially if they can push the rest of the AI research community to pay more attention to important problems.
* Gleave thinks there’s a ~10% chance that AI safety is very hard in the way that MIRI would argue, a ~20-30% chance that AI safety will almost certainly be solved by default, and a remaining ~60-70% chance that what we’re working on actually has some impact.
	+ Here are the reasons for Gleave’s beliefs, weighted by how much they factor into his holistic viewpoint:
		- 40%: The traditional arguments for risks from AI are unconvincing:
			* Traditional arguments often make an unexplained leap from having superintelligent AIs to superintelligent AIs being catastrophically bad.
			* It’s unlikely that AI systems not designed from mathematical principles are going to inherently be unsafe.
			* They’re long chains of heuristic reasoning, with little empirical validation.
			* Outside view: most fears about technology have been misplaced.
		- 20%: The AI research community will solve the AI safety problem naturally.
		- 20%: AI researchers will be more interested in AI safety when the problems are nearer.
		- 10%: The hard, MIRI version of the AI safety problem is not very compelling.
		- 10%: AI safety problems that seem hard now will be easier to solve once we have more sophisticated ML.
* Fast takeoff defined as “GDP will double in 6 months before it doubles in 24 months” is plausible, though Gleave still leans towards slow takeoff.
* Gleave thinks discontinuous progress in AI is extremely unlikely:
	+ There is unlikely to be a sudden important insight dropped into place, since AI has empirically progressed more by accumulation of lots of bags and tricks and compute.
	+ There isn’t going to be a sudden influx of compute in the near future, since well-funded organizations are currently already spending billions of dollars to optimize it.
	+ If we train impressive systems, we will likely train other systems beforehand that are almost as capable.
	+ Given discontinuous progress, the most likely story is that we combine many narrow AI systems in a way where the integrated whole is much better than half of them.
* Gleave guesses a ~10-20% chance that AGI technology will only be a small difference away from current techniques, and a ~50% chance that AGI technology will be easily comprehensible to current AI researchers:
	+ There are fairly serious roadblocks in current techniques right now, e.g. memory, transfer learning, Sim2Real, sample inefficiency.
	+ Deep learning is slowing down compared to 2012 – 2013:
		- Much of the new progress is going to different domains, e.g. deep RL instead of supervised deep learning.
		- Computationally expensive algorithms will likely hit limits without new insights.
			* Though it seems possible that in fact progress will come from more computationally efficient algorithms.
	+ Outside view, we’ve had lots of different techniques for AI over time, so it would be surprising is the current one is the right one for AGI.
	+ Pushing more towards current techniques getting to AGI, from an economic point of view, there is a lot of money going into companies whose current mission is to build AGI.
* Conditional on advanced AI technology being created, Gleave gives a 60-70% chance that it will pose a significant risk of harm without additional safety efforts.
	+ Gleave thinks that best case, we drive it down to 20 – 10%, median case, we drive it down to 40 – 30%. A lot of his uncertainty comes from how difficult the problem is.
* Gleave thinks he could see evidence that could push him in either direction in terms of how likely AI is to be safe:
	+ Evidence that would cause Gleave to think AI is less likely to be safe:
		- Evidence that thorny but speculative technical problems, like inner optimizers, exist.
		- Seeing more arms race dynamics, e.g. between U.S. and China.
		- Seeing major catastrophes involving AI, though they would also cause people to pay more attention to risks from AI.
		- Hearing more solid arguments for AI risk.
	+ Evidence that would cause Gleave to think AI is more likely to be safe:
		- Seeing AI researchers spontaneously focus on relevant problems would make Gleave think that AI is less risky.
		- Getting evidence that AGI was going to take longer to develop.
* Gleave is concerned that he doesn’t understand why members of the safety community come to widely different conclusions when it comes to AI safety.
* Gleave thinks a potentially important question is the extent to which we can successfully influence field building within AI safety.

This transcript has been lightly edited for concision and clarity.

Transcript
----------

**Asya Bergal:** We have a bunch of questions, sort of around the issue of– basically, we’ve been talking to people who are more optimistic than a lot of people in the community about AI. The proposition we’ve been asking people to explain their reasoning about is, ‘Is it valuable for people to be expending significant effort doing work that purports to reduce the risk from advanced artificial intelligence?’ To start with, I’d be curious for you to give a brief summary of what your take on that question is, and what your reasoning is.

**Adam Gleave:** Yeah, sure. The short answer is, yes, I think it’s worth people spending a lot of effort on this, at the margins, it’s still in absolute terms quite a small number. Obviously it depends a bit whether you’re talking about diverting resources of people who are already really dedicated to having a high impact, versus having your median AI researchers work more on safety related things. Maybe you think the median AI researcher isn’t trying to optimize for impact anyway, so the opportunity cost might be lower. The case I see from reducing the risk of AI is maybe weaker than some people in the community, but I think it’s still overall very strong.

The goal of AI as a field is still to build artificial general intelligence, or human-level AI. If we’re successful in that, it does seem like it’s going to be an extremely transformative technology. There doesn’t seem to be any roadblock that would prevent us from eventually reaching that goal. The path to that, the timeline is quite murky, but that alone seems like a pretty strong signal for ‘oh, there should be some people looking at this and being aware of what’s going on.’

And then, if I look at the state of the art in AI, there’s a number of somewhat worrying trends. We seem to be quite good at getting very powerful superhuman systems in narrow domains when we can specify the objective that we want quite precisely. So AlphaStar, AlphaGo, OpenAI Five, these systems are very much lacking in robustness, so you have some quite surprising failure modes. Mostly we see adversarial examples in image classifiers, but some of these RL systems also have somewhat surprising failure modes. This seems to me like an area the AI research community isn’t paying much attention to, and I feel like it’s almost gotten obsessed with producing flashy results rather than necessarily doing good rigorous science and engineering. That seems like quite a worrying trend if you extrapolate it out, because some other engineering disciplines are much more focused on building reliable systems, so I more trust them to get that right by default.

Even in something like aeronautical engineering where safety standards are very high, there are still accidents in initial systems. But because we don’t even have that focus, it doesn’t seem like the AI research community is going to put that much focus on building safe, reliable systems until they’re facing really strong external or commercial pressures to do so. Autonomous vehicles do have a reasonably good safety track record, but that’s somewhere where it’s very obvious what the risks are. So that’s kinda the sociological argument, I guess, for why I don’t think that the AI research community is going to solve all of the safety problems as far ahead of time as I would like.

And then, there’s also a lot of very thorny technical problems that do seem like they’re going to need to be solved at some point before AGI. How do we get some information about what humans actually want? I’m a bit hesitant to use this phrase ‘value learning’ because you could plausibly do this just by imitation learning as well. But there needs to be some way of getting information from humans into the system, you can’t just derive it from first principles, we still don’t have a good way of doing that.

There’s lots of more speculative problems, e.g. inner optimizers. I’m not sure if these problems are necessarily going to be real or cause issues, but it’s not something that we– we’ve not ruled it in or out. So there’s enough plausible technical problems that could occur and we’re not necessarily going to get that much advance notice of, that it seems worrying to just charge ahead without looking into this.

And then to caveat all this, I do think the AI community does care about producing useful technology. We’ve already seen some backlashes against autonomous weapons. People do want to do good science. And when the issues are obvious, there’s going to be a huge amount of focus on them. And it also seems like some of the problems might not actually be that hard to solve. So I am reasonably optimistic that in the default case of there’s no safety community really, things will still work out okay, but it also seems like the risk is large enough that just having a few people working on it can be extremely high leverage, especially if you can push the rest of the AI research community to pay a bit more attention to these problems.

Does that answer that question?

**Asya Bergal:** Yeah, it totally does.

**Robert Long:** Could you say a little bit more about why you think you might be more optimistic than other people in the safety community?

**Adam Gleave:** Yeah, I guess one big reason is that I’m still not fully convinced by a lot of the arguments for risks from AI. I think they are compelling heuristic arguments, meaning it’s worth me working on this, but it’s not compelling enough for me to think ‘oh, this is definitely a watertight case’.

I think the common area where I just don’t really follow the arguments is when you say, ‘oh, you have this superintelligent AI’. Let’s suppose we get to that, that’s already kind of a big leap of faith. And then if it’s not aligned, humans will die. It seems like there’s just a bit of a jump here that no one’s really filled in.

In particular it seems like sure, if you have something sufficiently capable, both in terms of intelligence and also access to other resources, it could destroy humanity. But it doesn’t just have to be smarter than an individual human, it has to be smarter than all of humanity potentially trying to work to combat this. And humanity will have a lot of inside knowledge about how this AI system works. And it’s also starting from a potentially weakened position in that it doesn’t already have legal protection, property ownership, all these other things.

I can certainly imagine there being scenarios unfolding where this is a problem, so maybe you actually give an AI system a lot of power, or it just becomes so, so much more capable than humans that it really is able to outsmart all of us, or it might just be quite easy to kill everyone. Maybe civilization is just much more fragile than we think. Maybe there are some quite easy bio ex-risks or nanotech that you could reason about from first principles. If it turned out that a malevolent but very smart human could kill all of humanity, then I would be more worried about the AI problem, but then maybe we should also be working on the human x-risk problem. So that’s one area that I’m a bit skeptical about, though maybe flushing that argument out more is bad for info-hazard reasons.

Then the other thing is I guess I feel like there’s a distribution of how difficult the AI safety problem is going to be. So there’s one world where anything that is not designed from mathematical principles is just going to be unsafe– there are going to be failure modes we haven’t considered, these failure modes are only going to arise when the system is smart enough to hurt you, and the system is going to be actively trying to deceive you. So this is I think, maybe a bit of a caricature, but I think this is roughly MIRI’s viewpoint. I think this is a productive viewpoint to inhabit when you’re trying to identify problems, but I think it’s probably not the world we actually live in. If you can solve that version, great, but it seems like a lot of the failure modes that are going to occur with advanced AI systems you’re going to see signs of earlier, especially if you’re actually looking out for them.

I don’t see much reason for AI progress to be discontinuous in particular. So there’s a lot of empirical records you could bring to bear on this, and it also seems like a lot of commercially valuable interesting research applications are going to require solving some of these problems. You’ve already seen this with value learning, that people are beginning to realize that there’s a limitation to what we can just write a reward function down for, and there’s been a lot more focus on imitation learning recently. Obviously people are solving much narrower versions of what the safety community cares about, but as AI progresses, they’re going to work on broader and broader versions of these problems.

I guess the general skepticism I have with the arguments, is, a lot of them take the form of ‘oh, there’s this problem that we need to solve and we have no idea how to solve it,’ but forget that we only need to solve that problem once we have all this other treasure trove of AI techniques that we can bring to bear on the problem. It seems plausible that this very strong unsupervised learning is going to do a lot of heavy lifting for us, maybe it’s going to give us a human ontology, it’s going to give us quite a good inductive bias for learning values, and so on. So there’s just a lot of things that might seem a lot stickier than they actually are in practice.

And then, I also have optimism that yes, the AI research community is going to try to solve these problems. It’s not like people are just completely disinterested in whether their systems cause harm, it’s just that right now, it seems to a lot of people very premature to work on this. There’s a sense of ‘how much good can we do now, where nearer to the time there’s going to just be naturally 100s of times more people working on the problem?’. I think there is still value you can do now, in laying the foundations of the field, but that maybe gives me a bit of a different perspective in terms of thinking, ‘What can we do that’s going to be useful to people in the future, who are going to be aware of this problem?’ versus ‘How can I solve all the problems now, and build a separate AI safety community?’.

I guess there’s also the outside view of just, people have been worried about a lot of new technology in the past, and most of the time it’s worked out fine. I’m not that compelled by this. I think there are real reasons to think that AI is going to be quite different. I guess there’s also just the outside view of, if you don’t know how hard a problem is, you should put a probability distribution over it and have quite a lot of uncertainty, and right now we don’t have that much information about how hard the AI safety problem is. Some problems seem to be pretty tractable, some problems seem to be intractable, but we don’t know if they actually need to be solved or not.

So, decent chance– I think I put a reasonable probability, like 10% probability, on the hard-mode MIRI version of the world being true. In which case, I think there’s probably nothing we can do. And I also put a significant probability, 20-30%, on AI safety basically not needing to be solved, we’ll just solve it by default unless we’re completely completely careless. And then there’s this big chunk of probability mass in the middle where maybe what we’re working on will actually have an impact, and obviously it’s hard to know whether at the margin, you’re going to be changing the outcome.

**Asya Bergal:** I’m curious– I think a lot of people we’ve talked to, some people have said somewhat similar things to what you said. And I think there’s two classic axes on which peoples’ opinions differ. One is this slow takeoff, fast takeoff proposition. The other is whether they think something that looks like current methods is likely to lead to AGI. I’m curious on your take on both those questions.

**Adam Gleave:** Yeah, sure. So, for slow vs. fast takeoff, I feel like I need to define the terms for people who use them in slightly different ways. I don’t expect there to be a discontinuity, in the sense of, we just see this sudden jump. But I wouldn’t be that surprised if there was exponential growth and quite a high growth rate. I think Paul defines fast takeoff as, GDP will double in six months before it doubles in 24 months. I’m probably mangling that but it was something like that. I think that scenario of fast takeoff seems plausible to me. I probably am still leaning slightly more towards the slow takeoff scenario, but it seems like fast takeoff will be plausible in terms of very fast exponential growth.

I think a lot of the case for the discontinuous progress argument falls on there being sudden insight that dropped into place, and it doesn’t seem to me like that’s what’s happening in AI, it’s more just a cumulation of lots of bags of tricks and a lot of compute. I also don’t see there being bags of compute falling out of the sky. Maybe if there was another AI winter, leading to a hardware overhang, then you might see sudden progress when AI gets funding again. But right now a lot of very well-funded organizations are spending billions of dollars on compute, including developing new application-specific integrated circuits for AI, so we’re going to be very close to the physical limits there anyway.

Probably the strongest case I see for discontinuities are the discontinuities you see when you’re training systems. But I just don’t think that’s going to be strong enough, because you’ll train other systems before that’ll be almost as capable. I guess we do see sometimes cases where one technique lets you solve a new class of problems.

Maybe you could see something where you get increasingly capable narrow systems, and there’s not a discontinuity overall, you already had very strong narrow AI. But eventually you just have so many narrow AI systems that they can basically do everything, and maybe you get to a stage where the integrated whole of those is much stronger than if you just had half of them, let’s say. I guess this is sort of the comprehensive AI services model. But again that seems a bit unlikely to me, because most of the time you can probably outsource some other chunks to humans if you really needed to. But yeah, I think it’s a bit more plausible than some of the other stories.

And then, in terms of whether I think current techniques are likely to get us to human-level AI– I guess I put significant probability mass on that depending on how narrowly you define it. One fuzzy definition is that a PhD thesis describing AGI being something that a typical AI researcher today could read and understand without too much work. Under this definition I’d assign 40 – 50%. And that could still include introducing quite a lot of new techniques, right, but just– I mean plausibly I think something based on deep learning, deep RL, you could describe to someone in the 1970s in a PhD thesis and they’d still understand it. But it’s just showing you, it’s not that much real theory that was developed, it was applying some pretty simple algorithms and a lot of compute in the right way. Which implies no huge new theoretical insights.

But if we’re defining it more narrowly, only allowing small variants of current techniques, I think that’s much less likely to lead to AGI: around 10-20%. I think that case is almost synonymous with the argument that you just need more compute, because it seems like there are so many things right now that we really cannot do: we still don’t have great solutions to memory, we still can’t really do transfer learning, Sim2Real just barely works sometimes. We’re still extremely sample inefficient. It just feels like all of those problems are going to require quite a lot of research in themselves. I can’t see there being one simple trick that would solve all of them. But maybe, current algorithms if you gave them 10000x compute would do a lot better on these, that is somewhat plausible.

And yeah, I do put fairly significant probability, 50%, on it being something that is kind of radically different. And I guess there’s a couple of reasons for that. One is, just trying to extrapolate progress forward, it does seem like there are some fairly serious roadblocks. Deep learning is slowing down in terms of, it’s not hitting as many big achievements as it was in the past. And also just AI has had many kinds of fads over time, right. We’ve had good old-fashioned AI, symbolic AI, we had expert systems, we had Bayesianism. It would be sort of surprising that the current method is the right one.

I don’t find that people are focusing on these techniques is necessarily particularly strong evidence that these systems are going to lead us to AGI. First, many researchers are not focused on AGI, and you can probably get useful applications out of current techniques. Second, AI research seems like it can be quite fashion driven. Obviously, there are organizations whose mission is to build AGI who are working within the current paradigm. And I think it is probably still the best bet, of the things that we know, but I still think it’s a bet that’s reasonably unlikely to pay off.

Does that answer your question?

**Asya Bergal:** Yeah.

**Robert Long:** Just on that last bit, you said– I might just be mixing up the different definitions you had and your different credences in those– but in the end there you said that’s a bet that you think is reasonably unlikely to pay off, but you’d also said 50% that it’s something radically different, so how– I think I was just confusing which ones you were on.

**Adam Gleave:** Right. So, I guess these definitions are all quite fuzzy, but I was saying 10-20% that something that is only a small difference away from current techniques would build AGI, and 50% that AGI was going to be comprehensible to us. I guess the distinction I’m trying to draw is the narrow one, which I give 10-20% credence, is we basically already have the right algorithms and we just need a few tricks and more compute. And the other more expansive definition, which I give 40-50% credence to, is allows for completely different algorithms, but excludes any deep theoretical insight akin to a whole new field of mathematics. So we might not be using back propagation any longer, we might not be using gradient descent, but it’ll be something similar — like the difference between gradient descent and evolutionary algorithms.

There’s a separate question of, if you’re trying to build AGI right now, where should you be investing your resources? Should you be trying to come up with a completely new novel theory, or should you be trying to scale up current techniques? And I think it’s plausible that you should just be trying to scale up techniques and figure out if we can push them forward, because trying to come up with a completely new way of doing AI is also very challenging, right. It’s not really a sort of insight you can force.

**Asya Bergal:** You kind of covered this earlier– and maybe you even said the exact number, so I’m sorry if this is a repeat. But one thing we’ve been asking people is the credence that without additional intervention– so imagining a world where EA wasn’t pushing for AI safety, and there wasn’t this separate AI safety movement outside of the AI research community, imagining that world. In that world, what is the chance that advanced artificial intelligence poses a significant risk of harm?

**Adam Gleave:** The chance it does cause a significant risk of harm?

**Asya Bergal:** Yeah, that’s right.

**Adam Gleave:** Conditional on advanced artificial intelligence being created, I think 60, 70%. I have a much harder time giving an unconditional probability, because there are other things that could cause humanity to stop developing AI. Is a conditional probability good enough, or do you want me to give an unconditional one?

**Asya Bergal:** No, I think the conditional one is what we’re looking for.

**Robert Long:** Do you have a hunch about how much we can expect dedicated efforts to drive down that probability? That is, the EA-focused AI safety efforts.

**Adam Gleave:** I think the best case is, you drive it down to 20 – 10%. I’m kind of picturing a lot of this uncertainty coming from just, how hard is the problem technically? And if we do inhabit this really hard version where you have to solve all of the problems perfectly and you have to have a formally verified AI system, I just don’t think we’re going to do that in time. You’d have to solve a very hard coordination problem to stop people developing AI without those safety checks. It seems like a very expensive process, developing safe AI.

I guess the median case, where the AI safety community just sort of grows at its current pace, I think maybe that gets it down to 40 – 30%? But I have a lot of uncertainty in these numbers.

**Asya Bergal:** Another question, going back to original statements for why you believe this– do you think there’s plausible concrete evidence that we could get or are likely to get that would change your views on this one direction or the other?

**Adam Gleave:** Yeah, so, seeing evidence of some of the more thorny but currently quite speculative technical problems, like inner optimizers, would make me update towards, ‘oh, this is just a really hard technical problem, and unless we really work hard on this, the default outcome is definitely going to be bad’. Right now, no one’s demonstrated an inner optimizer existing, it’s just a sort of theoretical problem. This is a bit of an unfair thing to ask in some sense, in that the whole reason that people are worried about this is that it’s only a problem with very advanced AI systems. Maybe I’m asking for evidence that can’t be provided. But relative to many other people, I am unconvinced by heuristic arguments appealing just to mathematical intuitions. I’m much more convinced either by very solid theoretical arguments that are proof-based, or by empirical evidence.

Another thing that would update me in a positive direction, as in AI seems less risky, would be seeing more AI researchers spontaneously focus on some relevant problems. There’s already, I guess this is a bit of a tangent, but I think maybe– people tend to conceive as the AI safety community as people who would identify as AI safety researchers. But I think the vast majority of AI safety research work is happening by people who have never heard of AI safety, but they have been working on related problems. This is useful to me all of the time. I think where we could plausibly end up having a lot more of this work happening without AI safety ever really becoming a thing is people realizing ‘oh, I want my robot to do this thing and I have a really hard time making it do that, let’s come up with a new imitation learning technique’.

But yeah, other things that could update me positively… I guess, AI seeming like a harder problem, as in, it seems like AI, general artificial intelligence is further away, that would probably update me in a positive direction. It’s not obvious but I think generally all else being equal, longer timelines is going to generally have more time to diagnose problems. And also it seems like the current set of AI techniques — deep learning and very data-driven approaches — are particularly difficult to analyze or prove anything about, so some other paradigm is probably going to be better, if possible.

Other things that would make me scared would be more arms race dynamics. It’s been very sad to me what we’re seeing with China – U.S. arms race dynamics around AI, especially since it doesn’t even seem like there is much direct competition, but that meme is still being pushed for political reasons.

Any actual major catastrophes involving AI would make me think it’s more risky, although it would also make people pay more attention to AI risk, so I guess it’s not obvious what direction it would push overall. But it certainly would make me think that there’s a bit more technical risk.

I’m trying to think if there’s anything else that would make more pessimistic. I guess just more solid arguments for AI safety, because a lot of my skepticism is coming from there’s just this very unlikely sounding set of ideas, and there are just heuristic arguments that I’m convinced enough by to work on the problem, but not convinced by enough to say, this is definitely going to happen. And if there was a way to patch some of the holes in those arguments, then I probably would be more convinced as well.

**Robert Long:** Can I ask you a little bit more about evidence for or against AGI being a certain distance away? You mentioned that as evidence that would change your mind. What sort of evidence do you have in mind?

**Adam Gleave:** Sure, so I guess a lot of the short timelines scenarios basically are coming from current ML techniques scaling to AGI, with just a bit more compute. So, watching for if those milestones are being achieved at the rate I was expecting, or slower.

This is a little bit hard to crystallize, but I would say right now it seems like the rate of progress is slowing down compared to something like 2012, 2013. And interestingly, I think a lot of the more interesting progress has come from, I guess, from going to different domains. So we’ve seen maybe a little bit more progress happening in deep RL compared to supervised deep learning. And the optimistic thing is to say, well, that’s because we’ve solved supervised learning, but we haven’t really. We’ve got superhuman performance on ImageNet, but not on real images that you just take on your mobile phone. And it’s still very sample inefficient, we can’t do few-shot learning well. Sometimes it seems like there’s a lack of interest on the part of the research community in solving some of these problems. I think it’s partly because no one has a solid angle of attack on solving these problems.

Similarly, while some of the recent progress in deep RL has been very exciting, it seems to have some limits. For example, AlphaStar and OpenAI Five both involved scaling up self-play and population based training. These were hugely computationally expensive, and that was where a lot of the scaling was coming from. So while there have been algorithmic improvements, I don’t see how you get this working in much more complicated environments without either huge additional compute or some major insights. These are things that are pushing me towards thinking deep learning will not continue to scale, and therefore very short timelines are unlikely.

Something that would update me towards shorter timelines would be if  something that I thought was impossible turns out to be very easy. So OpenAI Five did update me positively, because I just didn’t think PPO was going to work well in Dota and it turns out that it does if you have enough compute. I don’t think it updated me that strongly towards short timelines, because it did need a lot of compute, and if you scale it to a more complex game you’re going to have exponential scaling. But it did make me think, well, maybe there isn’t a deep insight required, maybe this is going to be much more about finding more computationally efficient algorithms rather than lots of novel insights.

I guess there’s also sort of economic factors– I mention mostly because I often see people neglecting them. One thing that makes me bullish on short timelines is that, there’s some very well-resourced companies whose mission is to build AGI. OpenAI just raised a billion, DeepMind is spending considerable resources. As long as this continues, it’s going to be a real accelerator. But that could go away: if AI doesn’t start making people money, I expect another AI winter.

**Robert Long:** One thing we’re asking people, and again I think you’ve actually already given us a pretty good sense of this, is just a relative weighting of different considerations. And as I say that, you actually have already been tagging this. But just to half review, from what I’ve scrawled down. A lot of different considerations in your relative optimism are: cases for AI as an x-risk being not as watertight as you’d like them, arguments for failure modes being the default and really hard, not being sold on those arguments, ideas that these problems might become easier to solve the closer we get to AGI when we have more powerful techniques, and then the general hope that people will try to solve them as we get closer to AI. Yeah, I think those were at least some of the main considerations I got. How strong relatively are those considerations in your reasoning?

**Adam Gleave:** I’m going to quote numbers that may not add up to 100, so we’ll have to normalize it at the end. I think the skepticism surrounding AI x-risk arguments is probably the strongest consideration, so I would put maybe 40% of my weight on that. This is because the outside view is quite strong to me, so if you talk about this very big problem that there’s not much concrete evidence for, then I’m going to be reasonably optimistic that actually we’re wrong and there isn’t a big problem.

The second most important thing to me is the AI research community solving this naturally. We’re already seeing signs of a set of people beginning to work on related problems, and I see this continuing. So I’m putting 20% of my weight on that.

And then, the hard version of AI safety not seeming very likely to me, I think that’s 10% of the weight. This seems reasonably important if I buy into the AI safety argument in general, because that makes a big difference in terms of how tractable these problems are. What were the other considerations you listed?

**Robert Long:** Two of them might be so related that you already covered them, but I had distinguished between the problems getting easier the closer we get, and people working more on them the closer we get.

**Adam Gleave:** Yeah, that makes sense. I think I don’t put that much weight on the problems getting easier. Or I don’t directly put weight on it, maybe it’s just rolled into my skepticism surrounding AI safety arguments, because I’m going to naturally find an argument a bit uncompelling if you say ‘we don’t know how to properly model human preferences’. I’m going to say, ‘Well, we don’t know how to properly do lots of things humans can do right now’. So everything needs to be relative to our capabilities. Whereas I find arguments of the form ‘we can solve problems that humans can’t solve, but only when we know how to specify what those problems are’, that seems more compelling, that’s talking about a relative strength between ability to optimize vs. ability to specify objectives. Obviously that’s not the only AI safety problem, but it’s a problem.

So yeah, I think I’m putting a lot of the weight on people paying more attention to these problems over time, so that’s probably actually 15 – 20% of my weight. And then I’ll put 5% on the problems getting easier and then some residual probability mass on things I haven’t thought about or haven’t mentioned in this conversation.

**Robert Long:** Is there anything you wish we had asked that you would like to talk about?

**Adam Gleave:** I guess, I don’t know if this is really useful, but I do wish I had a better sense of what other people in the safety community and outside of it actually thought and why they were working on it, so I really appreciate you guys doing these interviews because it’s useful to me as well. I am generally a bit concerned about lots of people coming to lots of different conclusions regarding how pessimistic we should be, regarding timelines, regarding the right research agenda.

I think disagreement can be healthy because it’s good to explore different areas. The ideal thing would be for us to all converge to some common probability distribution and we decide we’re going to work on different areas. But it’s very hard psychologically to do this, to say, ‘okay, I’m going to be the person working on this area that I think isn’t very promising because at the margin it’s good’– people don’t work like that. It’s better if people think, ‘oh, I am working on the best thing, under my beliefs’. So having some diversity of beliefs is good. But it bothers me that I don’t know why people have come to different conclusions to me. If I understood why they disagree, I’d be happier at least.

I’m trying to think if there’s anything else that’s relevant… yeah, so I guess another, this is merely just a question for you guys to maybe think about, is, I’m still unsure about how valuable field-building should be. And in particular, to what extent AI safety researchers should be working on this. It seems like a lot of reasons why I was optimistic assume the the AI research community is going to solve some of these problems naturally. A natural follow up to that is to ask whether we should be doing something to encourage this to happen, like writing more position papers, or just training up more grad students? Should we be trying to actively push for this rather than just relying on people to organically develop an interest in this research area? And I don’t know whether you can actually change research directions in this way, because it’s very far outside my area of expertise, but I’d love someone to study it.

Conversation with Adam Gleave

Summary:
Adam finds the traditional arguments for AI risk unconvincing. First, it isn't clear that we will build an AI system that is so capable that it can fight all of humanity from its initial position where it doesn't have any resources, legal protections, etc. While discontinuous progress in AI could cause this, Adam doesn't see much reason to expect such discontinuous progress: it seems like AI is progressing by using more computation rather than finding fundamental insights. Second, we don't know how difficult AI safety will turn out to be; he gives a probability of ~10% that the problem is as hard as (a caricature of) MIRI suggests, where any design not based on mathematical principles will be unsafe. This is especially true because as we get closer to AGI we'll have many more powerful AI techniques that we can leverage for safety. Thirdly, Adam does expect that AI researchers will eventually solve safety problems; they don't right now because it seems premature to work on those problems. Adam would be more worried if there were more arms race dynamics, or more empirical evidence or solid theoretical arguments in support of speculative concerns like inner optimizers. He would be less worried if AI researchers spontaneously started to work on relative problems (more than they already do).

Adam makes the case for AI safety work differently. At the highest level, it seems possible to build AGI, and some organizations are trying very hard to build AGI, and if they succeed it would be transformative. That alone is enough to justify some effort into making sure such a technology is used well. Then, looking at the field itself, it seems like the field is not currently focused on doing good science and engineering to build safe, reliable systems. So there is an opportunity to have an impact by pushing on safety and reliability. Finally, there are several technical problems that we do need to solve before AGI, such as how we get information about what humans actually want.

Adam also thinks that it's 40-50% likely that when we build AGI, a PhD thesis describing it would be understandable by researchers today without too much work, but ~50% that it's something radically different. However, it's only 10-20% likely that AGI comes only from small variations of current techniques (i.e. by vastly increasing data and compute). He would see this as more likely if we hit additional milestones by investing more compute and data (OpenAI Five was an example of such a milestone).


Historic trends in ship size

***This page may be out-of-date. Visit the [updated version of this page](https://wiki.aiimpacts.org/doku.php?id=takeoff_speed:continuity_of_progress:historic_trends_in_ship_size) on our [wiki](https://wiki.aiimpacts.org/doku.php?id=start).***

Trends for ship tonnage (builder’s old measurement) and ship displacement for Royal Navy first rate line-of-battle ships saw eleven and six discontinuities of between ten and one hundred years respectively during the period 1637-1876, if progress is treated as linear or exponential as usual. There is a hyperbolic extrapolation of progress such that neither measurement sees any discontinuities of more than ten years.

We do not have long term data for ship size in general, however the SS *Great Eastern* seems to have represented around 400 years of discontinuity in both tonnage (BOM) and displacement if we use Royal Navy ship of the line size as a proxy, and exponential progress is expected, or 11 or 13 in the hyperbolic trend. This discontinuity appears to have been the result of some combination of technological innovation and poor financial decisions.

Details
-------

This case study is part of AI Impacts’ [discontinuous progress investigation](https://aiimpacts.org/discontinuous-progress-investigation/).

### Background

According to Wikipedia, naval tactics in the age of sail rewarded larger ships, because larger ships were harder to sink and could carry more guns, and battles were usually lengthy affairs in which two lines of ships fired at each other until one side surrendered.[1](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-1-1385 "“One consequence of the line of battle was that a ship had to be strong enough to stand in it. In the old type of mêlée battle a small ship could seek out an opponent of her own size, or combine with others to attack a larger one. As the line of battle was adopted, navies began to distinguish between vessels that were fit to form parts of the line in action, and the smaller ships that were not. By the time the line of battle was firmly established as the standard tactical formation during the 1660s, merchant ships and lightly armed warships became less able to sustain their place in a pitched battle. In the line of battle, each ship had to stand and fight the opposing ship in the enemy line, however powerful she might be. The purpose-built ships powerful enough to stand in the line of battle came to be known as a ship of the line.”
“Sailing Ship Tactics.” Wikipedia. February 08, 2019. Accessed April 23, 2019. https://en.wikipedia.org/wiki/Sailing_ship_tactics.") [2](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-2-1385 "See these three pages for more on battle tactics during this period.") Our understanding is that when steamships and iron-clad ships appeared, financial constraints sometimes prevented navies from building ships as big as technically possible, but the incentives towards bigger ships remained, since the best way to punch through heavy armor was to carry heavy guns, which required a big ship.[3](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-3-1385 "“the only effective weapons against a big ship were big guns, which required a big ship to carry them.”

“Naval Gazing Main/A Brief History of the Destroyer.” Accessed October 26, 2019. https://www.navalgazing.net/A-Brief-History-of-the-Destroyer.   ")

![](https://lh5.googleusercontent.com/RmoWPU2DuNAWoapiemXUBC3EiKc_a682Ra8e77oYFF8cRkj7JEVZeimAFKb2xuGOFmOYCwYL5T1RvLKqYmz1T-OlqnT0kbEVXhA-idIdw_LtQc95gkYdRigvwCdYk1YqukAwfTd8)Figure 1: A Royal Navy First-Rate Ship of the Line[4](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-4-1385 "From Wikimedia Commons.
Creator: John Cantiloe Joy [Public domain]")
### Trends

#### Royal Navy first-rate line-of-battle ships ‘tonnage’ (BOM)

[‘Tonnage’ (BOM)](https://en.wikipedia.org/wiki/Builder%27s_Old_Measurement) is a pre-20th century measure of ship cargo capacity.[5](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-5-1385 "“Builder’s Old Measurement (BOM, bm, OM, and o.m.) is the method used in England from approximately 1650 to 1849 for calculating the cargo capacity of a ship.”
“Builder’s Old Measurement.” Wikipedia. December 27, 2018. Accessed April 22, 2019. https://en.wikipedia.org/wiki/Builder’s_Old_Measurement. ") It is calculated as:

![](https://lh6.googleusercontent.com/nkY3ZgjvejU0iKff1LcAmoZ1QuBA2NE3vfMpjgr_p4YG-nr_ZMm2QC0GZ5V-utGYDcE3UoBfAB4Xx-6n4Ry7hA7A72a_OC-w6i3OISEiPl5O6KQxAjz5G1V33knG1SkXNykF1EqA)
We use it because that’s what we have data on; [displacement](https://en.wikipedia.org/wiki/Displacement_(ship)), the modern way of calculating ship weight, was not widely recorded until towards the end of our dataset. Unfortunately, BOM seems to be less accurate for estimating the cargo capacity of ships after 1860, which could affect some of the findings in the report. However, our [Spot check](https://aiimpacts.org/feed/?paged=10#spot-check) section goes into more detail about this and offers some evidence that this choice of metric isn’t responsible for producing the largest discontinuity as an uninteresting artifact.

##### Data

Figure 2 shows ship [‘tonnage’ (BOM)](https://en.wikipedia.org/wiki/Builder%27s_Old_Measurement) over time for UK Royal Navy first-rate line-of-battle ships, according to Wikipedia contributors Toddy1 and Morn The Gorn.[6](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-6-1385 "“File: Weight Growth of RN First Rate Line-of-Battle Ships 1630-1875.Svg.” In Wikipedia. Accessed October 26, 2019. https://de.wikipedia.org/wiki/Datei:Weight_Growth_of_RN_First_Rate_Line-of-Battle_Ships_1630-1875.svg.") [This spreadsheet](https://docs.google.com/spreadsheets/d/1iUTwqlafTPd74MEG8w9rHyTYFKLwg2MsBI4-I8bC5Us/edit?usp=sharing) contains their data. We have not vetted it thoroughly, but have spot-checked it (see [Spot check](https://aiimpacts.org/feed/?paged=10#spot-check) section below). We extract the record-breaking subset of ships (see Figure 3).

![](https://upload.wikimedia.org/wikipedia/commons/thumb/b/b3/Weight_Growth_of_RN_First_Rate_Line-of-Battle_Ships_1630-1875.svg/2560px-Weight_Growth_of_RN_First_Rate_Line-of-Battle_Ships_1630-1875.svg.png)Figure 2: Tonnage (in BOM) of Royal Navy First-Rate Line-of-Battle Ships, from [Wikipedia](https://de.wikipedia.org/wiki/Datei:Weight_Growth_of_RN_First_Rate_Line-of-Battle_Ships_1630-1875.svg).[7](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-7-1385 "“File: Weight Growth of RN First Rate Line-of-Battle Ships 1630-1875.Svg.” In Wikipedia. Accessed October 26, 2019. https://de.wikipedia.org/wiki/Datei:Weight_Growth_of_RN_First_Rate_Line-of-Battle_Ships_1630-1875.svg.")
![](https://aiimpacts.org/wp-content/uploads/2019/10/Tonnage-1024x768.png)Figure 3: The subset of tonnages from Figure 2 that are the highest so far.
##### Discontinuity measurement

###### Exponential prior

If we have a strong prior on technological trends being linear or exponential, we might treat this data as a linear trend through 1804 followed by an exponential trend.[8](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-8-1385 "See our spreadsheet, sheet ‘Tonnage calculations’ to see the trends, and our methodology page for details on how we divide the data into trends and how to interpret the spreadsheet.") Extrapolated in this way, tonnage saw ten greater than ten year discontinuities in this data, shown in the table below.[9](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-9-1385 "See our methodology page for explanation of how we calculated these numbers. Also see our spreadsheet, sheet ‘Tonnage calculations’ for these calculations.")

|  |  |  |  |
| --- | --- | --- | --- |
| **Year** | **Tonnage (BOM)** | **Discontinuity** | **Name** |
| 1701 | 1,883 | 11 years | *[Royal Sovereign](https://en.wikipedia.org/wiki/HMS_Royal_Sovereign_(1786))* |
| 1756 | 2,047 | 13 years | *[Royal George](https://en.wikipedia.org/wiki/HMS_Royal_George_(1756))* |
| 1762 | 2,116 | 10 years | *[Brittania](https://en.wikipedia.org/wiki/HMS_Britannia_(1762))* |
| 1795 | 2,351 | 31 years | *[Ville de Paris](https://en.wikipedia.org/wiki/French_ship_Ville_de_Paris_(1764))* |
| 1804 | 2,530 | 25 years | *[Hibernia](https://en.wikipedia.org/wiki/HMS_Hibernia_(1804))* |
| 1839 | 3,104 | 41 years | *[Queen](https://en.wikipedia.org/wiki/HMS_Queen_(1839))* |
| 1852 | 3,759 | 41 years | *[Duke of Wellington](https://en.wikipedia.org/wiki/HMS_Duke_of_Wellington_(1852))* |
| 1859 | 4,116 | 12 years | *[Victoria](https://en.wikipedia.org/wiki/Victoria_(ship))* |
| 1860 | 6,039 | 77 years | *[Warrior](https://en.wikipedia.org/wiki/HMS_Warrior_(1860))* |
| 1863 | 6,643 | 13 years | *[Minotaur](https://en.wikipedia.org/wiki/HMS_Minotaur_(1863))* |
| 1867 | 8946 | 33 years | Inflexible |

In addition to the size of these discontinuity in years, we have tabulated a number of other potentially relevant metrics [here](https://docs.google.com/spreadsheets/d/1iMIZ57Ka9-ZYednnGeonC-NqwGC7dKiHN9S-TAxfVdQ/edit?usp=sharing).[10](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-10-1385 "See our methodology page for more details.")
###### Other curves

With a weaker prior on linear or exponential trends in technology progress, one might prefer to extrapolate this data as a more exotic curve, such as a hyperbola. For instance, *Tonnage = (1/(c\*year + d))^(1/3)* for some constants *c* and *d* appears to be a good model, since *1/(tonnage^3)* looks fairly linear (see Figure 4).

![](https://aiimpacts.org/wp-content/uploads/2019/10/1_tonnage^3-vs.-launched.png)Figure 4: 1/(tonnage^3) is roughly linear
Using this to extrapolate past progress, we get no discontinuities (see [**our spreadsheet, sheet ‘Tonnage calculations’**](https://docs.google.com/spreadsheets/d/1iUTwqlafTPd74MEG8w9rHyTYFKLwg2MsBI4-I8bC5Us/edit?usp=sharing) for this calculation). However this is unsurprising toward the end, since hyperbolas have asymptotes (potentially going to infinity in finite time), and this particular one reaches such a singularity in about 1869. So on that model, any size ship is expected by 1869, and discontinuities cannot be larger than the time remaining until that date. (The largest discontinuity is nine years, from *Warrior*, which is within a year of the implied ship-tonnage singularity.)

##### Discussion of causes

Given that modeling the data as hyperbolic means there are no discontinuities, a plausible cause for apparent discontinuities when modeling it as exponential is that the process of ship size increase is fundamentally closer to being hyperbolic (though it must have departed from this trend before long, since it would have implied arbitrarily large ships from 1869). We do not know why this trend in particular would be hyperbolic, given that we understand exponential curves to be much more common in technological progress.

On a brief investigation of possible causes of particular discontinuities in this trend, *Ville de Paris, Hibernia, and Queen* do not appear to use any dramatically different technology to previous ships.

The *[Duke of Wellington](https://en.wikipedia.org/wiki/HMS_Duke_of_Wellington_(1852))* was the first Royal Navy ship of the line to be steam powered, and it was apparently lengthened to fit the engines.[11](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-11-1385 "“Although the Royal Navy had been using steam power in smaller ships for three decades, it had not been adopted for ships of the line, partly because the enormous paddle-boxes required would have meant a severe reduction in the number of guns carried. This problem was solved by the adoption of the screw propeller in the 1840s. Under a crash programme announced in December 1851 to provide the navy with a steam-driven battlefleet, the design was further modified by the new Surveyor, Captain Baldwin Walker. The ship was cut apart in two places on the stocks in January 1852, lengthened by 30 feet (9.1 m) overall and given screw propulsion.”
“HMS Duke of Wellington (1852).” Wikipedia. February 08, 2019. Accessed April 22, 2019. https://en.wikipedia.org/wiki/HMS_Duke_of_Wellington_(1852). ")
The largest discontinuity was from *[Warrior](https://en.wikipedia.org/wiki/HMS_Warrior_(1860))*, which was one of the two first armor-plated, iron-hulled warships.[12](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-12-1385 "“Warrior and her sister ship HMS Black Prince were the first armour-plated, iron-hulled warships, and were built in response to France’s launching in 1859 of the first ocean-going ironclad warship, the wooden-hulled Gloire.” “HMS Warrior (1860).” Wikipedia. December 18, 2018. Accessed April 23, 2019. https://en.wikipedia.org/wiki/HMS_Warrior_(1860). ") It seems likely that iron hulls allowed larger ships. For example, the wooden steamship *[Mersey](https://en.wikipedia.org/wiki/HMS_Mersey_(1858))*, unusually large for a wooden ship yet smaller than the *Warrior,* is considered to have been beyond the limits of wood as a structural material.[13](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-13-1385 "

She and her sister ship the Orlando were the longest wooden warships built for the Royal Navy. …  The length, the unique aspect of the ship, was actually an Achilles’ heel of the Mersey and Orlando. The extreme length of the ship put enormous strains on her hull due to the unusual merging of heavy machinery, and a lengthy wooden hull, resulting in her seams opening up. They were pushing the limits of what was possible in wooden ship construction:

Even the biggest of the 5,000-6,000-ton wooden battleships of the mid-to-late 19th century and the 5,000-ton wooden motorships constructed in the United States during World War I did not exceed 340 feet in length or 60 feet in width. The longest of these ships, the Mersey-class frigates, were unsuccessful, and one, HMS Orlando, showed signs of structural failure after an 1863 voyage to the United States. The Orlando was scrapped in 1871 and the Mersey soon after. Both the Mersey-class frigates and the largest of the wooden battleships, the 121-gun Victoria class, required internal iron strapping to support the hull, as did many other ships of this kind. In short, the construction and use histories of these ships indicated that they were already pushing or had exceeded the practical limits for the size of wooden ships.[1]
Britain had built two long frigates in 1858 – HMS Mersey and HMS Orlando – the longest, largest and most powerful single-decked wooden fighting ships. Although only 335 feet long, they suffered from the strain of their length, proving too weak to face a ship of the line in close quarters.[2]

 “HMS Mersey (1858).” Wikipedia. August 29, 2017. Accessed April 24, 2019. https://en.wikipedia.org/wiki/HMS_Mersey_(1858). ") Moreover, the very large civilian ship [*Great Eastern*](https://en.wikipedia.org/wiki/SS_Great_Eastern), made extensive use of iron for structure and appears to have been regarded as innovative for its structure.[14](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-14-1385 "“Brunel’s Great Eastern represented the next great development in shipbuilding. Built in association with John Scott Russell, it used longitudinal stringers for strength, inner and outer hulls, and bulkheads to form multiple watertight compartments.”
“Shipbuilding.” Wikipedia. April 18, 2019. Accessed April 23, 2019. https://en.wikipedia.org/wiki/Shipbuilding#Industrial_Revolution.

“The hull was an all-iron construction, a double hull of 19 mm (0.75 in) wrought iron in 0.86 metres (2 feet 10 inches) plates with ribs every 1.8 m (5.9 ft). Internally, the hull was divided by two 107 m (351 ft) long, 18 m (59 ft) high, longitudinal bulkheads and further transverse bulkheads dividing the ship into nineteen compartments.”
“SS Great Eastern.” Wikipedia. April 22, 2019. Accessed April 23, 2019. https://en.wikipedia.org/wiki/SS_Great_Eastern. ") So plausibly this was an important enough innovation to produce an immediate jump in ship size.

##### Spot check

Our metric, BOM, doesn’t measure volume or weight directly; it is derived from a ship’s width and length. Thus it might often be a reasonable proxy for a more normal notion of size, but change arbitrarily between different ship designs. There is particular reason to suspect this here, since according to Wikipedia, “[s]teamships required a different method of estimating tonnage, because the ratio of length to beam was larger and a significant volume of internal space was used for boilers and machinery.”[15](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-15-1385 "Wikipedia, The Free Encyclopedia, s.v. “Builder’s Old Measurement,” (accessed April 5, 2019)
https://en.wikipedia.org/wiki/Builder%27s_Old_Measurement ")

To check whether the *Warrior* discontinuity was an artifact of this measurement scheme, we also searched for displacement figures for some of these ships. (We also made a brief attempt to find ships from other navies, like the French, that might destroy the discontinuity. We didn’t find any.) We did not collect many, but they cover the period of the largest discontinuity, 1850-1860, and confirm that it is probably robust to different ship size metrics, and thus not an artifact. See [this spreadsheet](https://docs.google.com/spreadsheets/d/1fkJ5pkLBt5dvciFkOb4S1ala7ZPmOhGLW8lI_CkpO9Y/edit?usp=sharing).

#### Royal Navy first-rate line-of-battle ships displacement (tons)

The [displacement](https://en.wikipedia.org/wiki/Displacement_(ship)) of a ship is its weight, measured by looking at the amount of water that a ship displaces when it’s floating.[16](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-16-1385 "“The displacement or displacement tonnage of a ship is its weight based on the amount of water its hull displaces at varying loads.” – “Displacement (Ship)”. 2019. En.Wikipedia.Org. Accessed July 8 2019. https://en.wikipedia.org/w/index.php?title=Displacement_(ship)&oldid=899439176.")
##### Data

We took displacement and ‘estimated displacement’ data from the same Wikipedia table[17](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-17-1385 "“File: Weight Growth of RN First Rate Line-of-Battle Ships 1630-1875.Svg.” In Wikipedia. Accessed October 26, 2019. https://de.wikipedia.org/wiki/Datei:Weight_Growth_of_RN_First_Rate_Line-of-Battle_Ships_1630-1875.svg.") for Royal Navy first-rate line-of-battle ships and put it in [this spreadsheet, sheet ‘Displacement calculations’](https://docs.google.com/spreadsheets/d/1iUTwqlafTPd74MEG8w9rHyTYFKLwg2MsBI4-I8bC5Us/edit?usp=sharing). Figure 5 below shows this data.

![](https://aiimpacts.org/wp-content/uploads/2019/10/Displacement-1024x768.png)Figure 5: Ship Weight (displacement) over time
##### Discontinuity Measurement

If we model this data as a linear trend through 1795 followed by an exponential trend,[18](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-18-1385 "See our spreadsheet, sheet ‘Displacement calculations’ to see the trends, and our methodology page for details on how we divide the data into trends and how to interpret the spreadsheet.") then compared to previous rates in these trends, tonnage contained six greater than ten year discontinuities, shown in the table below.[19](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-19-1385 "See our methodology page for explanation of how we calculated these numbers. Also see our spreadsheet, sheet ‘Displacement calculations’ for these calculations.")

|  |  |  |  |
| --- | --- | --- | --- |
| **Year** | **Displacement (tons)** | **Discontinuity** | **Ship Surname** |
| 1804 | 4,200 | 17 years | *[Hibernia](https://en.wikipedia.org/wiki/HMS_Hibernia_(1804))* |
| 1839 | 5,100 | 35 years | *[Queen](https://en.wikipedia.org/wiki/HMS_Queen_(1839))* |
| 1852 | 5,829 | 25 years | *[Duke of Wellington](https://en.wikipedia.org/wiki/HMS_Duke_of_Wellington_(1852))* |
| 1860 | 7,000 | 30 years | *[Victoria](https://en.wikipedia.org/wiki/Victoria_(ship))* |
| 1860 | 9,180 | 59 years | *[Warrior](https://en.wikipedia.org/wiki/HMS_Warrior_(1860))* |
| 1863 | 10,690 | 23 years | *[Minotaur](https://en.wikipedia.org/wiki/HMS_Minotaur_(1863))* |

In addition to the sizes of these discontinuities in years, we have tabulated a number of other potentially relevant metrics [here](https://docs.google.com/spreadsheets/d/1iMIZ57Ka9-ZYednnGeonC-NqwGC7dKiHN9S-TAxfVdQ/edit?usp=sharing).[20](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-20-1385 "See our methodology page for more details.")
Once again, if we model the data as hyperbolic, it contains no discontinuities of more than ten years, however this is unsurprising after about 1859 end given the proximity of the asymptote (see further explanation in previous section, and calculations in the [spreadsheet, sheet ‘Displacement calculations’](https://docs.google.com/spreadsheets/d/1iUTwqlafTPd74MEG8w9rHyTYFKLwg2MsBI4-I8bC5Us/edit?usp=sharing)).

### The SS *Great Eastern*

In the process of another investigation, we noted that a civilian ship, the [SS *Great Eastern*](https://en.wikipedia.org/wiki/SS_Great_Eastern), launched in 1858, was about six times larger by volume than any other ship at the time. It apparently took more than forty years for its length, gross tonnage, and passenger capacity to be surpassed.[21](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-21-1385 "” She was by far the largest ship ever built at the time of her 1858 launch, and had the capacity to carry 4,000 passengers from England to Australia without refueling. Her length of 692 feet (211 m) was only surpassed in 1899 by the 705-foot (215 m) 17,274-gross-ton RMS Oceanic, her gross tonnage of 18,915 was only surpassed in 1901 by the 701-foot (214 m) 21,035-gross-ton RMS Celtic, and her 4,000-passenger capacity was surpassed in 1913 by the 4,935-passenger SS Imperator. The ship’s five funnels were rare. These were later reduced to four.” “These measurements were six times larger by volume than any ship afloat…”
“SS Great Eastern.” Wikipedia. April 22, 2019. Accessed April 23, 2019. https://en.wikipedia.org/wiki/SS_Great_Eastern.") We calculate its tonnage (BOM) to be 22,990 tons (see [spreadsheet, tab Tonnage calculations](https://docs.google.com/spreadsheets/d/1iUTwqlafTPd74MEG8w9rHyTYFKLwg2MsBI4-I8bC5Us/edit#gid=2046407726)). Supposing our Royal Navy dataset is a good proxy for overall ship size records during this time, and treating past progress as exponential, the SS *Great Eastern* represents a 416-year discontinuity in tonnage (BOM) over previous Royal Navy ships. (See [spreadsheet, tab Tonnage calculations](https://docs.google.com/spreadsheets/d/1iUTwqlafTPd74MEG8w9rHyTYFKLwg2MsBI4-I8bC5Us/edit#gid=2046407726)) If the trend is modeled as a hyperbola instead, the SS *Great Eastern* still represents an 11 year discontinuity, which is as big as a discontinuity can be in the context of the theoretical expectation of arbitrarily large ships when the hyperbola reaches its asymptote in 11 years.[22](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-22-1385 "See discussion above, and spreadsheet for calculation.")

![](https://lh6.googleusercontent.com/scZLJrtcEF0YRC_vdElum3DNzSvSpkq0u2YZo7EVMzcsvo5dL1EY_Up4UMBXDiU9394a7UGSagOIdNJmzbwmt7QV2OIE2HUPWAIlcD2a2wiNpRKyUSjcCtVIDuujTi4-PkGpBF3L)Figure 6: The SS *Great Eastern*[23](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-23-1385 "From Wikimedia Commons:
Charles Parsons (1821-1910) [Public domain] ")
It is possible that there were other civilian ships prior to the *Great Eastern* that were also big, making the *Great Eastern* not a discontinuity. However we think this is unlikely. The [RMS Persia](https://en.wikipedia.org/wiki/RMS_Persia), launched only three years prior to the *Great Eastern*, was the ‘largest’ ship in the world at that time[24](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-24-1385 "“She was the first Atlantic record breaker constructed of iron and was the largest ship in the world at the time of her launch.”

“RMS Persia.” Wikipedia. February 25, 2018. Accessed April 30, 2019. https://en.wikipedia.org/wiki/RMS_Persia. "), and yet was only slightly bigger than the biggest military ships, in terms of tonnage (BOM). ([See spreadsheet](https://docs.google.com/spreadsheets/d/1iUTwqlafTPd74MEG8w9rHyTYFKLwg2MsBI4-I8bC5Us/edit?usp=sharing)). If in the intervening two or three years a larger ship appeared, we do not know of it.

Using displacement, recorded on Wikipedia as 32,160 tons, and again assuming that our Royal Navy dataset is a good proxy for all ships, we also get a large discontinuity– 407 years when compared to our previous exponential trend for Royal Navy ships. ([See spreadsheet](https://docs.google.com/spreadsheets/d/1iUTwqlafTPd74MEG8w9rHyTYFKLwg2MsBI4-I8bC5Us/edit?usp=sharing)).

![](https://aiimpacts.org/wp-content/uploads/2019/10/DisplacementGE-1024x768.png)Figure 7: Ship weight (displacement) over time, now with the *Great Eastern*.
We do not know why the *Great Eastern* was so exceptional. It seems that itwas innovative in several ways,[25](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-25-1385 " The Great Eastern was the first ship to use a double-skinned hull:
“Great Eastern was the first ship to incorporate the double-skinned hull, a feature which would not be seen again in a ship for several decades, but which is now compulsory for reasons of safety. ”  We understand that it was also one of the first European ships to be divided into watertight compartments using internal bulkheads, one of the first iron-hulled ships, and one of the first ships to use screw propulsion (although it also had sails and paddlewheels) “SS Great Eastern.” Wikipedia. April 22, 2019. Accessed April 23, 2019. https://en.wikipedia.org/wiki/SS_Great_Eastern. ") and that it was designed by a pair of exceptional engineer-scientists, one of whom may have been influential to the design of the *Warrior*.[26](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-26-1385 "“During the 1850s he argued within the Navy for the construction of iron warships and the first design, HMS Warrior, is said by some to be a “Russell ship”.[25][26] ”
“John Scott Russell.” Wikipedia. March 29, 2019. Accessed May 01, 2019. https://en.wikipedia.org/wiki/John_Scott_Russell.

“Isambard Kingdom Brunel FRS (/ˈɪzəmbɑːrd bruːˈnɛl/; 9 April 1806 – 15 September 1859[1]), was an English mechanical and civil engineer who is considered “one of the most ingenious and prolific figures in engineering history”,[2] “one of the 19th-century engineering giants”,[3] and “one of the greatest figures of the Industrial Revolution, [who] changed the face of the English landscape with his groundbreaking designs and ingenious constructions”.[4]“
“Isambard Kingdom Brunel.” Wikipedia. April 11, 2019. Accessed May 01, 2019. https://en.wikipedia.org/wiki/Isambard_Kingdom_Brunel. ") However, the ship’s size might have also been the result of poor business sense, as it appears to have been a financial failure. [27](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-27-1385 "In what appears to be the beginning of a pattern of financial problems, the ship’s maiden voyage brought in considerably less revenue than expected:

“From a financial perspective, the American venture had been a disaster; the ship had taken in only $120,000 against a $72,000 overhead, whereas the company had expected to take in $700,000. In addition, the company was facing a daily interest payment of $5,000, which ate into any profits the ship made.[15]”
“SS Great Eastern” Wikipedia. September 19, 2022. Accessed September 26, 2022.")
![](https://aiimpacts.org/wp-content/uploads/2019/07/GreatEastern-1-1024x930.jpg)
Figure 8: Cutaway of one of the SS *Great Eastern*‘s engine rooms.[28](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-28-1385 "This 3D reconstruction image was made by Simon Edwards at History Rebuilt. Reproduced with permission.")
Nuño Sempere has also investigated the *Great Eastern* as a potential discontinuity to passenger and sailing vessel length trends.[29](https://aiimpacts.org/historic-trends-in-ship-size/#easy-footnote-bottom-29-1385 "Sempere, Nuño. “Discontinuous Progress in Technological Trends.” In brightest day, in blackest night. Accessed May 10, 2020. https://nunosempere.github.io/rat/Discontinuous-Progress.html.") We learned of this after our own investigation, so have not measured these discontinuities by the same methods as those noted above, nor checked the data. Sempere notes that it took 41 years for the length trend excluding the *Great Eastern* to surpass it. Figures 9-11 show some of this data.

![](https://aiimpacts.org/wp-content/uploads/2020/05/sempere-SS-Great-Eastern-2.png)Figure 9: Nuño Sempere’s data on passenger ship lengths over history.
![](https://aiimpacts.org/wp-content/uploads/2020/05/sempere-SS-Great-Eastern-3-1024x512.png)Figure 10: Nuño Sempere’s data on passenger ship lengths over recent history (note that points prior to the *Great Eastern* are not all-time records).
![](https://aiimpacts.org/wp-content/uploads/2020/05/sempere-SS-Great-Eastern-Dataset2-with-beam.png)Figure 11: Nuño Sempere’s data on sailing ship lengths over history, beams included. *SS Great Eastern* is the highest point.
Acknowledgements
----------------

Thanks to bean for his help with this.  He blogs about naval history at [navalgazing.net](http://navalgazing.net/).

Notes
-----

Historic trends in ship size


Effects of breech loading rifles on historic trends in firearm progress

*Published Feb 7 2020*

We do not know if breech loading rifles represented a discontinuity in military strength. They probably did not represent a discontinuity in fire rate.

Details
-------

This case study is part of AI Impacts’ [discontinuous progress investigation](https://aiimpacts.org/discontinuous-progress-investigation/).

We have not investigated this topic in depth. What follows are our initial impressions.

### Background

From [Wikipedia](https://en.wikipedia.org/wiki/Breechloader)[1](https://aiimpacts.org/effects-of-breech-loading-rifles-on-historic-trends-in-firearm-progress/#easy-footnote-bottom-1-1370 "

“Breechloader.” In Wikipedia, May 14, 2019. https://en.wikipedia.org/w/index.php?title=Breechloader&oldid=897060135.    "):

> A **breechloader**[[1]](https://en.wikipedia.org/wiki/Breechloader#cite_note-1)[[2]](https://en.wikipedia.org/wiki/Breechloader#cite_note-2) is a [firearm](https://en.wikipedia.org/wiki/Firearm) in which the [cartridge](https://en.wikipedia.org/wiki/Cartridge_(firearms)) or [shell](https://en.wikipedia.org/wiki/Shell_(projectile)) is inserted or loaded into a chamber integral to the rear portion of a [barrel](https://en.wikipedia.org/wiki/Gun_barrel).
>
> Modern [mass production](https://en.wikipedia.org/wiki/Mass_production) firearms are breech-loading (though [mortars](https://en.wikipedia.org/wiki/Mortar_(weapon)) are generally muzzle-loaded), except those which are intended specifically by design to be [muzzle-loaders](https://en.wikipedia.org/wiki/Muzzleloader), in order to be legal for certain types of hunting. Early firearms, on the other hand, were almost entirely muzzle-loading. The main advantage of breech-loading is a reduction in reloading time – it is much quicker to load the projectile and the charge into the breech of a gun or cannon than to try to force them down a long tube, especially when the bullet fit is tight and the tube has spiral ridges from [rifling](https://en.wikipedia.org/wiki/Rifling). In field artillery, the advantages were similar: the crew no longer had to force powder and shot down a long barrel with rammers, and the shot could now tightly fit the bore (increasing accuracy greatly), without being impossible to ram home with a fouled barrel.
>
>

### Trends

Breech loading rifles were suggested to us as a potential discontinuity in some measure of army strength, due to high fire rate and ability to be used while lying down. We did not have time to investigate this extensively, and have not looked for evidence for or against discontinuities in military strength overall. That said, the reading we have done does not suggest any such discontinuities.

We briefly looked for evidence of discontinuity in firing rate, since firing rate seemed to be a key factor of any advantage in military strength.

#### Firing rate

Upon brief review it seems unlikely to us that breech loading rifles represented a discontinuity in firing rate alone. [Revolvers](https://en.wikipedia.org/wiki/Revolver) developed in parallel with breech-loading rifles, and appear to have had similar or higher rates of fire. This includes revolver rifles, which (being rifles) appear to be long-ranged enough to be comparable to muskets and breech-loading rifles.[2](https://aiimpacts.org/effects-of-breech-loading-rifles-on-historic-trends-in-firearm-progress/#easy-footnote-bottom-2-1370 "For an example of a revolver rifle in use at roughly the same time as the Dreyse needle gun, the first breech-loading rifle to get widespread uptake, see the Colt New Model Revolving Rifle. Quote: “Revolving rifles were an attempt to increase the rate of fire of rifles by combining them with the revolving firing mechanism that had been developed earlier for revolving pistols. Colt began experimenting with revolving rifles in the early 19th century, making them in a variety of calibers and barrel lengths.” “Colt’s New Model Revolving Rifle.” Wikipedia. April 16, 2019. Accessed April 19, 2019. https://en.wikipedia.org/wiki/Colt’s_New_Model_Revolving_rifle.")

The best candidate we found for a breech loading rifle constituting a discontinuity in firing rate is the Ferguson Rifle, first used in 1777 in the American Revolutionary War.[3](https://aiimpacts.org/effects-of-breech-loading-rifles-on-historic-trends-in-firearm-progress/#easy-footnote-bottom-3-1370 "“The Ferguson rifle was one of the first breech-loading rifles to be put into service by the British military. It fired a standard British carbine ball of .615″ calibre and was used by the British Army in the American War of Independence at the Battle of Saratoga in 1777, and possibly at the Siege of Charleston in 1780.[1]” – “Ferguson Rifle.” Wikipedia. March 09, 2019. Accessed April 29, 2019. https://en.m.wikipedia.org/wiki/Ferguson_rifle.") It was expensive and fragile, so it did not see widespread use;[4](https://aiimpacts.org/effects-of-breech-loading-rifles-on-historic-trends-in-firearm-progress/#easy-footnote-bottom-4-1370 "“The two main reasons that Ferguson rifles were not used by the rest of the army: The gun was difficult and expensive to produce using the small, decentralized gunsmith and subcontractor system in use to supply the Ordnance in early Industrial Revolution Britain. The guns broke down easily in combat, especially in the wood of the stock around the lock mortise. The lock mechanism and breech were larger than the stock could withstand with rough use. All surviving military Fergusons feature a horseshoe-shaped iron repair under the lock to hold the stock together where it repeatedly broke around the weak, over-drilled out mortise.” – “Ferguson Rifle.” Wikipedia. March 09, 2019. Accessed April 29, 2019. https://en.m.wikipedia.org/wiki/Ferguson_rifle.") breech-loading rifles did not become standard in any army until the Prussian “Needle gun” in 1841 and the Norwegian “Kammerlader” in 1842.[5](https://aiimpacts.org/effects-of-breech-loading-rifles-on-historic-trends-in-firearm-progress/#easy-footnote-bottom-5-1370 "“The Kammerlader, or “chamber loader”, was the first Norwegian breech-loadingrifle, and among the very first breech loaders adopted for use by an armed force anywhere in the world.” “Kammerlader.” Wikipedia. January 07, 2019. Accessed May 01, 2019. https://en.wikipedia.org/wiki/Kammerlader.
“Dreyse Needle Gun.” Wikipedia. March 09, 2019. Accessed May 01, 2019. https://en.wikipedia.org/wiki/Dreyse_needle_gun. ") Both the Ferguson and the Dreyse needle gun could fire about six rounds a minute (sources vary),[6](https://aiimpacts.org/effects-of-breech-loading-rifles-on-historic-trends-in-firearm-progress/#easy-footnote-bottom-6-1370 "“In the British trials, the Dreyse was shown to be capable of six rounds per minute” “Dreyse Needle Gun.” Wikipedia. March 09, 2019. Accessed May 01, 2019. https://en.wikipedia.org/wiki/Dreyse_needle_gun.  “Since the weapon was loaded from the breech, rather than from the muzzle, it had an amazingly high rate of fire for its day, and in capable hands, it fired six to ten rounds per minute.” – “Ferguson Rifle.” Wikipedia. March 09, 2019. Accessed April 19, 2019. https://en.m.wikipedia.org/wiki/Ferguson_rifle.") but by the time of the Ferguson well-trained British soldiers could fire muskets at about four rounds a minute.[7](https://aiimpacts.org/effects-of-breech-loading-rifles-on-historic-trends-in-firearm-progress/#easy-footnote-bottom-7-1370 "“The main advantage of the British Army was that the infantry soldier trained at this procedure almost every day. A properly trained group of regular infantry soldiers was able to load and fire four rounds per minute. A crack infantry company could load and fire five rounds in a minute. ” – “Muskets.” Wikipedia. June 08, 2017. Accessed April 19, 2019. https://en.wikipedia.org/wiki/Muskets. ") Moreover, apparently there are some expensive and fragile revolvers that predate the Ferguson, again suggesting that breech-loading rifles did not lead to a discontinuity in rate of fire.[8](https://aiimpacts.org/effects-of-breech-loading-rifles-on-historic-trends-in-firearm-progress/#easy-footnote-bottom-8-1370 " “During the late 16th century in China, Zhao Shi-zhen invented the Xun Lei Chong, a five-barreled musket revolver spear. Around the same time, the earliest examples of what today is called a revolver were made in Germany. These weapons featured a single barrel with a revolving cylinder holding the powder and ball. They would soon be made by many European gun-makers, in numerous designs and configurations.[4] However, these weapons were difficult to use, complicated and prohibitively expensive to make, and as such they were not widely distributed. In 1836, an American, Samuel Colt, patented the mechanism which led to the widespread use of the revolver,[5] the mechanically indexing cylinder.” – “Revolver.” Wikipedia. April 07, 2019. Accessed April 19, 2019. https://en.wikipedia.org/wiki/Revolver#History.") All in all, while we don’t have enough data to plot a trend, everything we’ve seen is consistent with continuous growth in firing rate.

![](https://lh4.googleusercontent.com/uq8v6G7KQD-dRRrfE32UeUYqbP9IdH3YOsjFdphqTQbRgKQEc6ZjdqYkFGVCI9Wf6ckcKn4rHzKDepV2mrxE02a28_zgO4U81swtKQxqniW29Y9BJ8uSKSv5Pa6cAvIJjjQjAXDL)Figure 1: Diagram of how to load the Ferguson rifle[9](https://aiimpacts.org/effects-of-breech-loading-rifles-on-historic-trends-in-firearm-progress/#easy-footnote-bottom-9-1370 "From Wikimedia Commons: See page for author [Public domain]")
#### Other metrics

It is still possible that a combination of factors including fire rate contributed to a discontinuity in a military strength metric, or that a narrower metric including fire rate saw some discontinuity.

*Thanks to Jesko Zimmerman for suggesting breech-loading rifles as a potential area of discontinuity.*

Notes
-----


Historic trends in transatlantic passenger travel

The speed of human travel across the Atlantic Ocean has seen at least seven discontinuities of more than ten years’ progress at past rates, two of which represented more than one hundred years’ progress at past rates: Columbus’ second journey, and the first non-stop transatlantic flight.

Details
-------

This case study is part of AI Impacts’ [discontinuous progress investigation](https://aiimpacts.org/discontinuous-progress-investigation/).

### Trends

#### Transatlantic passenger crossing speed

We investigated fastest recorded passenger trips across the Atlantic Ocean over time. By ‘passenger’ we mean that any human made the crossing, or could have done.

We look for fastest speeds of real historic systems that could have with high probability delivered a live person across the Atlantic Ocean. We do not require that a person was actually sent by the method in question, though in fact all of our records did involve a passenger traveling.

We generally use whatever route was actually taken (or supposed in an estimate), and do not attempt to infer faster speeds possible had an optimal route been taken (though note that because we are measuring speed rather than time to cross the Ocean, route length is adjusted for to a first approximation).

##### Data

We collated records of historic speeds to cross the Atlantic Ocean from online sources.[1](https://aiimpacts.org/historic-trends-in-transatlantic-passenger-travel/#easy-footnote-bottom-1-1860 "See the ‘Passenger’ tab of this spreadsheet for our sources.") These are available at the ‘Passenger’ tab of [this spreadsheet](https://docs.google.com/spreadsheets/d/11WV8JUIZeVNWKfggCmHaCEJrtG_YMiOdq18zjfwFUVk/edit?usp=sharing), and are shown in Figure 1 below. We have not verified this data.

###### Detailed overview of data

We collected some data on speeds of drifting across the Atlantic Ocean and Viking ship speeds as evidence about the previous trend, but do not look for discontinuities until Columbus’ trips, for which relatively detailed descriptions are available.

Between then and 1841 the fastest records of transatlantic crossings we know of come from [Slavevoyages.org](http://Slavevoyages.org)‘s [database](https://slavevoyages.org/voyage/database#results) of over thirty six thousand voyages made by slave ships. We combined this data with distances between recorded ports for trips that might plausibly be fastest, to find speed records. This produced only three record trips. These were substantial outliers in speed, which suggests to us that those records may have been driven by error, or may have involved different types of ship or circumstances to the others. The latter explanation would suggest that faster trips were likely made for purposes other than slave transport, meaning that these slave trips were unlikely to represent discontinuities in crossing speed across all types of ship. Given this, and that we do not have data for other types of ship at that time, we do not measure discontinuities during this period. We do include these ships to estimate the longer term trend, for measuring later discontinuities. The existence of later discontinuities does not appear to be sensitive to whether we include outlier slave ships in the historic trend, or replace them with more credible slower slave ships.

From 1841 to 1909 all of our records are from Wikipedia’s page, [Blue Riband](https://en.wikipedia.org/wiki/Blue_Riband). That page describes the Blue Riband was ‘an unofficial accolade given to the [passenger liner](https://en.wikipedia.org/wiki/Passenger_ship) crossing the Atlantic Ocean in regular service with the record highest speed’. It appears that this title was sought after, and the records during that time are dense, so this part of the dataset is probably relatively accurate and complete for passenger steam ships. The main potential gap in this data is that we cannot be sure there are not other types of boat at the time that traveled faster than passenger steam ships.

From the first flight in 1919, speed records were held by planes. We found these in a variety of places, and we judged the data to be relatively complete when we ceased to find new records with moderate searching.

We are particularly interested in avoiding missing data just before apparent discontinuities, since continuous progress may look discontinuous if data is missing. There is a fifteen year gap before the Concorde discontinuity in 1973 where we didn’t find any records. However we note that a record for fastest subsonic Atlantic crossing set in 1979 was substantially slower than the Concorde. This means that if there were no other supersonic transatlantic crossings prior to the Concorde, the Concorde must have been substantially faster than the previous record even if we were missing some data. For instance if we were missing a 1965 record as fast as the 1979 record (which might make sense, since the 1979 record was set by a 1965 aircraft), then the Concorde would still be a discontinuity of around twenty years. We could not find other supersonic transatlantic crossings, but cannot rule them out.

![](http://aiimpacts.org/wp-content/uploads/2019/12/Passenger-1024x791.png)Figure 1: Historical progress in passenger travel across the Atlantic
![](https://aiimpacts.org/wp-content/uploads/2019/12/PassengerZoom-1024x791.png) Figure 2: Historical progress in passenger travel across the Atlantic, since 1730
##### Discontinuity measurement

We measure discontinuities by comparing progress made at a particular time to the past trend. For this purpose, we treat the past trend at any given point as exponential or linear depending on apparent fit, and judge a new trend to have begun when the recent trend has diverged sufficiently from the longer term trend. See [our spreadsheet](https://docs.google.com/spreadsheets/d/11WV8JUIZeVNWKfggCmHaCEJrtG_YMiOdq18zjfwFUVk/edit?usp=sharing), tab ‘Passenger’ to view the trends we break this data into, and [our methodology page](https://aiimpacts.org/methodology-for-discontinuity-investigation/#time-period-selection) for details on how to interpret our sheets and how we divide data into trends.

Given these judgments about past progress, there were seven greater than 10-year discontinuities during the periods that we looked at, summarized in the following table.[2](https://aiimpacts.org/historic-trends-in-transatlantic-passenger-travel/#easy-footnote-bottom-2-1860 "See our methodology page for more details, and our spreadsheet, tab ‘Passenger’ for our calculation.") Two of them were large (more than one hundred years of progress at previous rate).

|  |  |  |  |
| --- | --- | --- | --- |
| **Date** | **Mode of transport** | **Knots** | **Discontinuity size (years progress at past rate)** |
| 1493 | Columbus’ second voyage | 5.8 | 1465 |
| 1884 | Oregon (steamship) | 18.6 | 10 |
| 1919 | WWI Bomber (first non-stop transatlantic flight) | 106 | 351 |
| 1938 | Focke-Wulf Fw 200 Condor | 174 | 19 |
| 1945 | Lockheed Constellation | 288 | 25 |
| 1973 | Concorde | 1035 | 19 |
| 1974 | SR-71 | 1569 | 21 |

In addition to the sizes of these discontinuity in years, we have tabulated a number of other potentially relevant metrics [here](https://docs.google.com/spreadsheets/d/1iMIZ57Ka9-ZYednnGeonC-NqwGC7dKiHN9S-TAxfVdQ/edit?usp=sharing).[3](https://aiimpacts.org/historic-trends-in-transatlantic-passenger-travel/#easy-footnote-bottom-3-1860 "See our methodology page for more details.")
##### Discussion of causes

The first measured discontinuity comes from Columbus’ second voyage being much quicker than his first. We expect this is for non-technological reasons, such as noise in crossing times (such that if there had been a longer history of crossing, Columbus’ first voyage would not have been record-setting), Columbus’ crew benefiting from experience, and the second voyage being intended to reach its destination rather than doing so accidentally.

The largest discontinuity we noted (352 years at previous rates) came from [the first non-stop transatlantic flight](https://en.wikipedia.org/wiki/Transatlantic_flight_of_Alcock_and_Brown), in 1919.[4](https://aiimpacts.org/historic-trends-in-transatlantic-passenger-travel/#easy-footnote-bottom-4-1860 "“British aviators John Alcock and Arthur Brown made the first non-stop transatlantic flight in June 1919. They flew a modified First World War Vickers Vimy bomber from St. John’s, Newfoundland, to Clifden, Connemara, County Galway, Ireland.” – “Transatlantic Flight Of Alcock And Brown”. 2019. En.Wikipedia.Org. Accessed June 25 2019. https://en.wikipedia.org/w/index.php?title=Transatlantic_flight_of_Alcock_and_Brown&oldid=902818541.") This represented a relatively fundamental change in the means of crossing the Atlantic, supporting the hypothesis that discontinuities tend to be associated with more fundamental technological progress.

We have not investigated the significance of the developments underlying the other smaller discontinuities.

During the Blue Riband period, attention appears to have been given to Atlantic crossing speed in particular, suggesting that more effort may have been directed to this metric then. During the later era of flight, record Atlantic crossing time appears to have been less of a goal.[5](https://aiimpacts.org/historic-trends-in-transatlantic-passenger-travel/#easy-footnote-bottom-5-1860 "For instance, the fastest crossing for a subsonic airliner achieved in 1979 was done in a 1965 plane, suggesting that such a record could plausibly have been set earlier (and before 1973 it would have been the record for any kind of flight, not just subsonic). The captain’s description also makes the attempt sound like it was unplanned and motivated by thinking it sounded feasible not long before, rather than a major effort. (See sheet for sources.) ") This, in combination with the much more incremental progress in the earlier era, weakly supports the hypothesis that discontinuities are associated with metrics that receive less attention.

Notes
-----


Robin Hanson on the futurist focus on AI

*By Asya Bergal, 13 November 2019*

[![Robin Hanson](http://aiimpacts.org/wp-content/uploads/2019/11/robin_hanson.jpg)](http://aiimpacts.org/wp-content/uploads/2019/11/robin_hanson.jpg)Robin Hanson
Robert Long and I recently talked to Robin Hanson—GMU economist, prolific [blogger](http://www.overcomingbias.com/), and longtime thinker on the future of AI—about the amount of futurist effort going into thinking about AI risk.

It was noteworthy to me that Robin thinks human-level AI is a century, perhaps multiple centuries away— much longer than the 50-year number given by AI researchers. I think these longer timelines are the source of a lot of his disagreement with the AI risk community about how much of futurist thought should be put into AI.

Robin is particularly interested in the notion of ‘lumpiness’– how much AI is likely to be furthered by a few big improvements as opposed to a slow and steady trickle of progress. If, as Robin believes, most academic progress and AI in particular are not likely to be ‘lumpy’, he thinks we shouldn’t think things will happen without a lot of warning.

The full recording and transcript of our conversation can be found [here](https://aiimpacts.org/conversation-with-robin-hanson/).