Evidence against current methods leading to human level artificial intelligence

This is a list of published arguments that we know of that current methods in artificial intelligence will not lead to human-level AI.

Details
-------

### Clarifications

We take ‘current methods’ to mean techniques for engineering artificial intelligence that are already known, involving no “qualitatively new ideas”.[1](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-1-1938 "“It now seems possible that we could build ‘prosaic’ AGI, which can replicate human behavior but doesn’t involve qualitatively new ideas about ‘how intelligence works’”. — Christiano, Paul. “Prosaic AI Alignment”. 2017. Medium. Accessed August 13 2019. https://ai-alignment.com/prosaic-ai-control-b959644d79c2.") We have not precisely defined ‘current methods’. Many of the works we cite refer to currently *dominant* methods such as machine learning (especially deep learning) and reinforcement learning.

By human-level AI, we mean AI with a level of *performance* comparable to humans. We have in mind the operationalization of ‘high-level machine intelligence’ from our [2016 expert survey on progress in AI](https://aiimpacts.org/2016-expert-survey-on-progress-in-ai/): “Say we have ‘high-level machine intelligence’ when unaided machines can accomplish every task better and more cheaply than human workers.”[2](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-2-1938 "Grace, Katja, John Salvatier, Allan Dafoe, Baobao Zhang, and Owain Evans. “When will AI exceed human performance? Evidence from AI experts.” Journal of Artificial Intelligence Research 62 (2018): 729-754.")
Because we are considering intelligent performance, we have deliberately excluded arguments that AI might lack certain ‘internal’ features, even if it manifests human-level performance.[3](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-3-1938 "Block, Ned. 1981. “Psychologism and behaviorism.” The Philosophical Review 90, no. 1 (1981): 5-43.") [4](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-4-1938 "Searle, J. 1980. Minds, brains, and programs. Behavioral and Brain Sciences 3:417-57.
") We assume, concurring with Chalmers (2010), that “If there are systems that produce apparently [human-level intelligent] outputs, then whether or not these systems are truly conscious or intelligent, they will have a transformative impact.”[5](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-5-1938 "Chalmers, David. “The Singularity: A Philosophical Analysis”. 2010. David Chalmers. Accessed August 12 2019. http://consc.net/papers/singularity.pdf.")
### Methods

We read well-known criticisms of current AI approaches of which we were already aware. Using these as a starting point, we searched for further sources and solicited recommendations from colleagues familiar with artificial intelligence.

We include arguments that sound plausible to us, or that we believe other researchers take seriously. Beyond that, we take no stance on the relative strengths and weaknesses of these arguments.

We cite works that plausibly support pessimism about current methods, regardless of whether the works in question (or their authors) actually claim that current methods will not lead to human-level artificial intelligence.

We do not include arguments that serve primarily as undercutting defeaters of *positive* arguments that current methods *will* lead to human-level intelligence. For example, we do not include arguments that recent progress in machine learning has been overstated.

These arguments might overlap in various ways, depending on how one understands them. For example, some of the challenges for current methods might be special instances of more general challenges.

### List of arguments

#### Inside view arguments

These arguments are ‘inside view’ in that they look at the specifics of current methods.

* **Innate knowledge:** Intelligence relies on prior knowledge which it is currently not feasible to embed via learning techniques, recapitulate via artificial evolution, or hand-specify. — Marcus (2018)[6](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-6-1938 "Section 3.1, “Deep learning thus far is data hungry” — Marcus, Gary. 2018. “Deep Learning: A Critical Appraisal”. arXiv. Accessed August 12 2019. https://arxiv.org/abs/1801.00631.")
* **Data hunger:** Training a system to human level using current methods will require more data than we will be able to generate or acquire. — Marcus (2018)[7](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-7-1938 "Section 3.1, “Deep learning thus far is data hungry” — Marcus, Gary. 2018. “Deep Learning: A Critical Appraisal”. arXiv. Accessed August 12 2019. https://arxiv.org/abs/1801.00631.")

##### Capacities

Some researchers claim that there are capacities which are required for human-level intelligence, but difficult or impossible to engineer with current methods.[8](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-8-1938 "One could disagree with the claim that a given capacity is in fact required, or with the claim that current methods cannot engineer it.") Some commonly-cited capacities are:

* **Causal models:** Building causal models of the world that are rich, flexible, and explanatory — Lake et al. (2016)[9](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-9-1938 "Section 4.2.2, “Causality” — Lake, Brenden M., Tomer D. Ullman, Joshua B. Tenenbaum, and Samuel J. Gershman. 2016. “Building Machines That Learn And Think Like People”. arXiv. Accessed August 12 2019. https://arxiv.org/abs/1604.00289."), Marcus (2018)[10](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-10-1938 "Section 3.7, “Deep learning thus far cannot inherently distinguish causation from correlation ” — Marcus, Gary. 2018. “Deep Learning: A Critical Appraisal”. arXiv. Accessed August 12 2019. https://arxiv.org/abs/1801.00631."), Pearl (2018)[11](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-11-1938 "Pearl, Judea. 2018. “Theoretical Impediments To Machine Learning With Seven Sparks From The Causal Revolution”. arXiv. Accessed August 12 2019. https://arxiv.org/abs/1801.04016.
")
* **Compositionality:** Exploiting systematic, compositional relations between entities of meaning, both linguistic and conceptual — Fodor and Pylyshyn (1988)[12](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-12-1938 "Part III: The need for Symbol Systems: Productivity, Systematicity, Compositionality and Inferential Coherence, in particular Sections “Systematicity of cognitive representation” and “Compositionality of representations” — Fodor, Jerry A. and Pylyshyn, “Connectionism and Cognitive Architecture: A Critical Analysis.” Zenon W. 1988. Rutgers Center for Cognitive Science. Accessed August 12 2019.  http://ruccs.rutgers.edu/images/personal-zenon-pylyshyn/proseminars/Proseminar13/ConnectionistArchitecture.pdf."), Marcus (2001)[13](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-13-1938 "Marcus, G.F., 2001. The algebraic mind: Integrating connectionism and cognitive science. MIT press."), Lake and Baroni (2017)[14](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-14-1938 "Lake, Brenden M., and Marco Baroni. 2017. “Generalization Without Systematicity: On The Compositional Skills Of Sequence-To-Sequence Recurrent Networks”. arXiv. Accessed August 13 2019. https://arxiv.org/abs/1711.00350.")
* **Symbolic rules:** Learning abstract rules rather than extracting statistical patterns — Marcus (2018)[15](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-15-1938 "Section 5.2, “Symbol-manipulation, and the need for hybrid models” — Marcus, Gary. 2018. “Deep Learning: A Critical Appraisal”. arXiv. Accessed August 12 2019. https://arxiv.org/abs/1801.00631.")
* **Hierarchical structure:** Dealing with hierarchical structure, e.g. that of language — Marcus (2018)[16](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-16-1938 "Section 3.3, “Deep learning thus far has no natural way to deal with hierarchical structure” — Marcus, Gary. 2018. “Deep Learning: A Critical Appraisal”. arXiv. Accessed August 12 2019. https://arxiv.org/abs/1801.00631.")
* **Transfer learning:** Learning lessons from one task that transfer to other tasks that are similar, or that differ in systematic ways — Marcus (2018)[17](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-17-1938 "Section 3.2, “Deep learning thus far is shallow and has limited capacity for transfer” — Marcus, Gary. 2018. “Deep Learning: A Critical Appraisal”. arXiv. Accessed August 12 2019. https://arxiv.org/abs/1801.00631."), Lake et al. (2016)[18](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-18-1938 "Section 4.2.3, “Learning-to-learn” — Lake, Brenden M., Tomer D. Ullman, Joshua B. Tenenbaum, and Samuel J. Gershman. 2016. “Building Machines That Learn And Think Like People”. arXiv. Accessed August 12 2019. https://arxiv.org/abs/1604.00289.")
* **Common sense understanding:** Using common sense to understand language and reason about new situations — Brooks (2019)[19](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-19-1938 "Section 3, “Read a Book” — Brooks, Rodney. “[For&AI] Steps Toward Super Intelligence III, Hard Things Today – Rodney Brooks”. 2019. Rodney Brooks. Accessed August 12 2019. http://rodneybrooks.com/forai-steps-toward-super-intelligence-iii-hard-things-today/."), Marcus and Davis (2015)[20](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-20-1938 "Marcus, Gary and Davis, Ernest. “Commonsense reasoning and commonsense knowledge in artificial intelligence”. Commun. ACM 58, no. 9 (2015): 92-103.")

#### Outside view arguments

These arguments are ‘outside view’ in that they look at “a class of cases chosen to be similar in relevant respects”[21](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-21-1938 "Kahneman, Daniel and Lovallo, Dan. “Timid Choices and Bold Forecasts: A Cognitive Perspective on Risk Taking”. 1993. Warrington College of Business. Accessed August 13 2019. http://bear.warrington.ufl.edu/brenner/mar7588/Papers/kahneman-lovallo-mansci1993.pdf.") to current artificial intelligence research, without looking at the specifics of current methods.

* **Lack of progress:** There are many tasks specified several decades ago that have not been solved, e.g. effectively manipulating a robot arm, open-ended question-answering. — Brooks (2018)[22](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-22-1938 "Section 2, “Real Manipulation” — Brooks, Rodney. “[For&AI] Steps Toward Super Intelligence III, Hard Things Today – Rodney Brooks”. 2019. Rodney Brooks. Accessed August 12 2019. http://rodneybrooks.com/forai-steps-toward-super-intelligence-iii-hard-things-today/.
"), Jordan (2018)[23](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-23-1938 "Jordan, Michael. “Artificial Intelligence — The Revolution Hasn’t Happened Yet”. 2018. Medium. Accessed August 12 2019.")
* **Past predictions:** Past researchers have incorrectly predicted that we would get to human-level AI with then-current methods. — Chalmers (2010)[24](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-24-1938 "“It must be acknowledged that every path to AI has proved surprisingly difficult to date. The history of AI involves a long series of optimistic predictions by those who pioneer a method, followed by a periods of disappointment and reassessment. This is true for a variety of methods involving direct programming, machine learning, and artificial evolution, for example. Many of the optimistic predictions were not obviously unreasonable at the time, so their failure should lead us to reassess our prior beliefs in significant ways. It is not obvious just what moral should be drawn: Alan Perlis has suggested ‘A year spent in artificial intelligence is enough to make one believe in God’. So optimism here should be leavened with caution.” — Chalmers, David. “The Singularity: A Philosophical Analysis”. 2010. David Chalmers. Accessed August 12 2019. http://consc.net/papers/singularity.pdf.")
* **Other fields:** Several fields have taken centuries or more to crack; AI could well be one of them. — Brooks (2018)[25](https://aiimpacts.org/evidence-against-current-methods-leading-to-human-level-artificial-intelligence/#easy-footnote-bottom-25-1938 "“Einstein predicted gravitational waves in 1916. It took ninety nine years of people looking before we first saw them in 2015. Rainer Weiss, who won the Nobel prize for it, sketched out the successful method after fifty one years in 1967. And by then the key technologies needed, laser and computers, were in widespread commercial use. It just took a long time. Controlled nuclear fusion has been forty years away for well over sixty years now. Chemistry took millennia, despite the economic incentive of turning lead into gold (and it turns out we still can’t do that in any meaningful way). P=NP? has been around in its current form for forty seven years and its solution would guarantee whoever did it to be feted as the greatest computer scientist in a generation, at least. No one in theoretical computer science is willing to guess when we might figure that one out. And it doesn’t require any engineering or production. Just thinking. Some things just take a long time, and require lots of new technology, lots of time for ideas to ferment, and lots of Einstein and Weiss level contributors along the way. I suspect that human level AI falls into this class. But that it is much more complex than detecting gravity waves, controlled fusion, or even chemistry, and that it will take hundreds of years.“ — Brooks, Rodney. “[For&AI] Steps Toward Super Intelligence I, How We Got Here – Rodney Brooks”. 2018. Rodney Brooks. Accessed August 13 2019. https://rodneybrooks.com/forai-steps-toward-super-intelligence-i-how-we-got-here/.")

Contributions
-------------

Robert Long and Asya Bergal contributed research and writing.

Notes
-----

Featured image from [www.extremetech.com](https://www.extremetech.com/extreme/215170-artificial-neural-networks-are-changing-the-world-what-are-they).

Summary: This post briefly lists arguments that current AI techniques will not lead to high-level machine intelligence (HLMI), without taking a stance on how strong these arguments are.


Historic trends in land speed records

Land speed records did not see any greater-than-10-year discontinuities relative to linear progress across all records. Considered as several distinct linear trends it saw discontinuities of 12, 13, 25, and 13 years, the first two corresponding to early (but not first) jet-propelled vehicles.

The first jet-propelled vehicle just predated a marked change in the rate of progress of land speed records, from a recent 1.8 mph / year to 164 mph / year.

Details
-------

This case study is part of AI Impacts’ [discontinuous progress investigation](https://aiimpacts.org/discontinuous-progress-investigation/).

### Background

According to Wikipedia, the land speed record is “the highest speed achieved by a person using a vehicle on land.”[1](https://aiimpacts.org/historic-trends-in-land-speed-records/#easy-footnote-bottom-1-1621 "“Land Speed Record”. 2019. En.Wikipedia.Org. Accessed May 25 2019. https://en.wikipedia.org/wiki/Land_speed_record.") Wheel-driven cars, which supply power to their axles, held the records for land speed record through 1963, when the first turbojet powered vehicles arrived on the scene. No wheel-driven car has held the record since 1964.[2](https://aiimpacts.org/historic-trends-in-land-speed-records/#easy-footnote-bottom-2-1621 "“Craig Breedlove’s mark of 407.447 miles per hour (655.722 km/h), set in Spirit of America in September 1963, was initially considered unofficial. The vehicle breached the FIA regulations on two grounds: it had only three wheels, and it was not wheel-driven, since its jet engine did not supply power to its axles. […] The confusion of having three different LSRs lasted until December 11, 1964, when the FIA and FIM met in Paris and agreed to recognize as an absolute LSR the higher speed recorded by either body, by any vehicles running on wheels, whether wheel-driven or not. […] No wheel-driven car has since held the absolute record.” – “Land Speed Record”. 2019. En.Wikipedia.Org. Accessed May 25 2019. https://en.wikipedia.org/wiki/Land_speed_record.")
![](https://lh4.googleusercontent.com/nJlqOk9tZJI4inBz01-Hcq9mwCDWUXOBGzDGwrPRY4hOa9TLQNIMXtP7CAChmU0fVcub5HPR7yKa4Puljh0bON4M85aHZKsXXe7tP7ctQccRX5H7qg-GoJYieih7w9RdIeJ0cvDo)Figure 1: Three record-setting vehicles: Sunbeam, Sunbeam Blue Bird, and Blue Bird[3](https://aiimpacts.org/historic-trends-in-land-speed-records/#easy-footnote-bottom-3-1621 "From Wikimedia Commons: sv1ambo [CC BY 2.0 (https://creativecommons.org/licenses/by/2.0)]")
### Trends

#### Land speed records

##### Data

We took data from Wikipedia’s list of land speed records,[4](https://aiimpacts.org/historic-trends-in-land-speed-records/#easy-footnote-bottom-4-1621 "“Land Speed Record”. 2019. En.Wikipedia.Org. Accessed May 25 2019. https://en.wikipedia.org/wiki/Land_speed_record.") which we have not verified, and added it to [this spreadsheet](https://docs.google.com/spreadsheets/d/1sezw2CCJ3WxcrAcqsw7ZWK1rJoxJkVYxW9HljWVe-vg/edit?usp=sharing). See Figure 2 below.

![](https://lh5.googleusercontent.com/eCOr_JdyKmcr8otgQ7ts2YzG5ZaY0iashNCOlPDbIEh5BsKevQvJQfqAlKuvi-rcTlw8uhCielPs80qxKpwWz5l6If8mVpuQnSfWh83sFnlw_XFwYIlmzAFjBNvk4eAIvMKcVzH3)Figure 2: Historic land speed records in mph over time. Speeds on the left are an average of the record set in mph over 1 km and over 1 mile. The red dot represents the first record in a cluster that was from a jet propelled vehicle. The discontinuities of more than ten years are the third and fourth turbojet points, and the last two points.
##### Discontinuity measurement

If we treat the data as a linear trend across all time,[5](https://aiimpacts.org/historic-trends-in-land-speed-records/#easy-footnote-bottom-5-1621 "See our our methodology page for more details.") then the land speed record did not contain any greater than 10-year discontinuities.

However we divide the data into several linear trends.[6](https://aiimpacts.org/historic-trends-in-land-speed-records/#easy-footnote-bottom-6-1621 "See our spreadsheet  to view the trends, and our methodology page for details on how to interpet our sheets and when and how we divide data into trends.") Extrapolating based on these trends, there were four discontinuities of sizes 12, 13, 25, and 13 years, produced by different turbojet-powered vehicles.[7](https://aiimpacts.org/historic-trends-in-land-speed-records/#easy-footnote-bottom-7-1621 "See our methodology page for more details, and our spreadsheet for our calculation.")In addition to the size of these discontinuities in years, we have tabulated a number of other potentially relevant metrics **[here](https://docs.google.com/spreadsheets/d/1iMIZ57Ka9-ZYednnGeonC-NqwGC7dKiHN9S-TAxfVdQ/edit?usp=sharing)**.[8](https://aiimpacts.org/historic-trends-in-land-speed-records/#easy-footnote-bottom-8-1621 "See our methodology page for more details.")
##### Changes in the rate of progress

There are several marked changes in the rate of progress in this history. The first two discontinuities are near the start of a sharp change, that seemed to come from the introduction of jet-propulsion (though note that the first jet-propelled vehicle in the trend is neither discontinuous with the previous trend, nor seemingly within the period of faster growth).

If we look at the rates of progress in the stretches directly before the second jet propelled vehicle in 1964, and the stretch directly after that through 1965, the rate of progress increases from 1.8 mph / year to 164 mph / year.[9](https://aiimpacts.org/historic-trends-in-land-speed-records/#easy-footnote-bottom-9-1621 "See our methodology page for more details, and our spreadsheet for our calculations.")
Notes
-----

Historic trends in land speed records


Methodology for discontinuous progress investigation

AI Impacts’ [discontinuous progress investigation](https://aiimpacts.org/discontinuous-progress-investigation/) was conducted according to methodology outlined on this page.

Details
-------

Contributions to the [discontinuous progress investigation](https://aiimpacts.org/discontinuous-progress-investigation/) were made over at least 2015-2019, by a number of different people, and methods have varied somewhat. In 2019 we attempted to make methods across the full collection of case studies more consistent. The following is a description of methodology as of December 2019.

### Overview

To learn about the prevalence and nature of discontinuities in technological progress[1](https://aiimpacts.org/methodology-for-discontinuity-investigation/#easy-footnote-bottom-1-1656 "See the project page for more on the motivations and goals of the project."), we:

1. **Searched for potential examples of discontinuous progress (e.g. ‘Eli Whitney’s cotton gin’)** We collected around ninety suggestions of technological change which might have been discontinuous, from various people. Some of these pointed to particular technologies, and others to trends.[2](https://aiimpacts.org/methodology-for-discontinuity-investigation/#easy-footnote-bottom-2-1656 "For instance, a person might suggest to us that ‘building heights’ has been discontinuous, or that the Burj Khalifa was some sort of discontinuity.") We added further examples as they arose when we were already working on later steps. A list of suggested examples not ultimately included is available [here](https://aiimpacts.org/incomplete-case-studies-of-discontinuous-progress/).
2. **Chose specific metrics related to some of these potential examples (e.g. ‘cotton ginned per person per day’, ‘value of cotton ginned per cost’) and found historic data on progress on those metrics (usually in conjunction).** We took cases one by one and searched for data on relevant metrics in their vicinity. For instance, if we were told that fishing hooks became radically stronger in 1997, we might look for data on the strength of fishing hooks over time, but also for the cost of fishing hooks per year, or how many fish could be caught by a single fishing hook, because these are measures of natural interest that we might expect to be affected by a change in fishing hook strength. Often we ended up collecting data on several related trends. This was generally fairly dependent on what data could be found. Many suggestions are not included in the investigation so far because we have not found relevant data. Though sometimes we proceeded with quite minimal data, if it was possible to at least assess a single development’s likelihood of having been discontinuous.
3. **Defined a ‘rate of past progress’ throughout each historic dataset**  At each datapoint in a trend after the first one, we defined a ‘previous rate of progress’. This was generally either linear or exponential, and was the average rate of progress between the previous datapoint and some earlier datapoint, though not necessarily the first. For instance, if a trend was basically flat from 1900 until 1967, then became steep, then in defining the previous rate of progress for the 1992 datapoint, we may decide to call this linear progress since 1967, rather than say exponential progress since 1900.
4. **Measured the discontinuity at each datapoint** We did this by comparing the progress at the point to the expected progress at that date based on the last datapoint and the rate of past progress. For instance, if the last datapoint five years ago was 600 units, and progress had been going at two units per year, and now a development took it to 800 units, we would calculate 800 units – 600 units = 200 units of progress = 100 years of progress in 5 years, so a 95 year discontinuity.
5. **Noted any discontinuities** of more than ten years (‘moderate discontinuities’), and more than one hundred years (‘large discontinuities’)
6. **Noted anything interesting about the circumstances of each discontinuity** (e.g. the type of metric it was in, the events that appeared to lead to the discontinuity, the patterns of progress around it.)

### Choosing areas

We collected around ninety suggestions of technological change which might have been discontinuous. Many of these were offered to us in response to a Facebook question, a [Quora question](http://www.quora.com/When-has-any-technological-capability-increased-abruptly-rather-than-incrementally), personal communications, and a bounty posted on this website. We obtained some by searching for abrupt graphs in google images, and noting their subject matter. We found further contenders in the process of investigating others. Some of these are particular technologies, and others are trends.[3](https://aiimpacts.org/methodology-for-discontinuity-investigation/#easy-footnote-bottom-3-1656 "For instance, a person might suggest to us that ‘building heights’ has been discontinuous, or that the Burj Khalifa was some sort of discontinuity.")
We still have around fifty suggestions for trends that may have been discontinuities that we have not looked into, or have [not finished looking into](http://aiimpacts.org/incomplete-case-studies-of-discontinuous-progress/).

### Choosing metrics

For any area of technological activity, there are many specific metrics one could measure progress on. For instance consider ginning cotton (that is, taking the seeds out of it so that the fibers may be used for fabric). The development of new cotton gins might be expected to produce progress in all of the following metrics:

* Cotton ginnable per minute under perfect laboratory conditions
* Cotton ginned per day by users
* Cotton ginned per worker per day by users
* Quality-adjusted cotton ginned per quality-adjusted worker per day
* Cost to produce $1 of ginned cotton
* Number of worker injuries stemming from cotton ginning
* Prevalence of cotton gins
* Value of cotton

(These are still not entirely specific—in order to actually measure one, you would need to also for instance specify how the information would reach you. For instance, “cotton ginned per day by users, as claimed in a source findable by us within one day of searching online”.)

We choose both general areas to investigate, and particular metrics according to:

* Apparent likelihood of containing discontinuous progress (e.g. because it was suggested to us in a bounty submission[4](https://aiimpacts.org/methodology-for-discontinuity-investigation/#easy-footnote-bottom-4-1656 "We initially posted a bounty on discontinuous technologies. We have not investigated all of the submissions yet, and are yet to pay out bounties on suggestions we recently found to be discontinuous."), suggestions from readers and friends, and our own understanding.[/note], or by readers.)
* Ease of collecting clear data (e.g. because someone pointed us to a dataset, or because we could find one easily). We often began investigating a metric and then set it aside to potentially finish later, or gave up.
* Not seeming trivially likely to contain discontinuities for uninteresting reasons. For instance we expect the following to have a high number of discontinuities, which do not seem profitable to individually investigate:
	+ obscure metrics constructed to contain a discontinuity (e.g. Average weekly rate of seltzer delivery to Katja’s street from a particular grocery store over the period during which Katja’s household discovered that that grocery store had the cheapest seltzer)
	+ metrics very far from anyone’s concern (e.g. number of live fish in Times Square)
	+ metrics that are very close to metrics we already know contain discontinuities (e.g. if explosive power per gram of material sees a large discontinuity, then probably explosive power per gram of material divided by people needed to detonate bomb would also see a large discontinuity.)

Our goal with the project was to understand roughly how easy it is to find large discontinuities, and to learn about the situations in which they tend to arise, rather than to clearly assess the frequency of discontinuities within a well-specified reference class of metrics (which would have been hard, for instance because good data is rarely available). Thus we did not follow a formal procedure for selecting case studies. One important feature of the set of case studies and metrics we have is that they are likely to be heavily skewed in favor of having more large discontinuities, since we were explicitly trying to select discontinuous technologies and metrics.

### Data collection

Most data was either from a particular dataset that we found in one place, or was gathered by AI Impacts researchers.

When we gathered data ourselves, we generally searched for sources online until we felt that we had found most of what was readily available, or had at least investigated thoroughly the periods relevant to whether there were discontinuities. For instance, it is important to know about the trend just prior to an apparent discontinuity, than it is to know about the trend between two known records, where it is clear that little total progress has taken place.

In general, we report the maximal figures that we are confident of. i.e. we report the best known thing at each date, not the best possible thing at that date. So if in 1909 a thing was 10-12, we report 10, though we may note if we think 12 is likely and it makes a difference to the point just after. If all we know is that progress was made between 2010 and 2015, we report it in 2015.

### Discontinuity calculation

We measure discontinuities in terms of how many years it would have taken to see the same amount of progress, if the previous trend had continued.

To do this, we:

* Decide which points will be considered as potential discontinuities
* Decide what we think the previous trend was for each of those points
	+ Determine the shape of the previous curve
	+ Estimate the growth rate of that curve
* Calculate how many years the previous trend would need to have continued to see as much progress as the new point represents
* Report as ‘discontinuities’ all points that represented more than ten years of progress at previous rates

#### Requirements for measuring discontinuities

Sometimes we exclude points from being considered as potential discontinuities, though include them to help establish the trend. This is usually because:

* We have fewer than two earlier points, so no prior trend to compare them to
* We expect that we are missing prior data, so even if they were to look discontinuous, this would be uninformative.
* The value of the metric at the point is too ambiguous

Sometimes when we lack information we still reason about whether a point is a discontinuity. For instance, we think the Great Eastern very likely represents a discontinuity, even though we don’t have an extensive trend for ship size, because we know that a recent Royal Navy ship was the largest ship in the world, and we know the trend for Royal Navy ship size, which the trend for overall ship size cannot ever go below. So we can reason that the recent trend for ship size cannot be any steeper than that of Royal Navy ship size, and we know that at that rate, the Great Eastern represented a discontinuity.

#### Calculating previous rates of progress

##### Time period selection and trend fitting

As history progresses, a best guess about what the trend so far is can change. The best guess trend might change apparent shape (e.g. go from seeming linear to seeming exponential) or change apparent slope (e.g. what seemed like a steeper slope looks after a few slow years like noise in a flatter slope) or change its apparent relevant period (e.g. after multiple years of surprisingly fast progress, you may decide to treat this as a new faster growth mode, and expect future progress accordingly).

We generally reassess the best guess trend so far for each datapoint, though this usually only changes occasionally within a dataset.

We have based this on researcher judgments of fit, which have generally had the following characteristics:

* Trends are expected to be linear or exponential unless they are very clearly something else. We don’t tend search for better fitting curves.
* If curves are not upward curving, we tend to treat them as linear
* In ambiguous cases, we lean toward treating curves as exponential
* When there appears to be a new a newer faster growth mode, we generally recognize this and start a new trend at the third point (i.e. if there has been one discontinuity we don’t immediately treat the best guess for future progress as much faster, but after two in a row, we do).

We color the growth rate column in the spreadsheets according to periods where the growth rate is calculated as having the same overall shape and same starting year (though within those periods, the calculated growth rate changes as new data points are added to the trend).

##### Trend calculation

We calculate the rate of past progress as the average progress between the first and last datapoints in a subset of data, rather than taking a line of best fit. (This being a reasonable proxy for expected annual progress is established via trend selection described in the last section.)

#### Discontinuity measurement

For each point, we calculate how much progress it represents since the last point, and how many years of progress that is according to the past trend, then subtract the number of years that actually passed, for the discontinuity size.

This means that if no progress is seen for a hundred years, and then all of the progress expected in that time occurs at once, this does not count as a discontinuity.

#### Reporting discontinuities

We report discontinuities as ‘substantial’ if they are at least ten years of progress at once, and ‘large’ if they are at least one hundred years of progress at once.

### ‘Robust’ discontinuities

Many developments classified as discontinuities by the above methods are ahead of a best guess trend, but unsurprising because the data should have left much uncertainty about the best trend. For instance, if the data does not fit a consistent curve well, or is very sparse, one should be less surprised if a new point fails to line up with any particular line through it.

In this project we are more interested in clear departures from established trends than in noisy or difficult to extrapolate trends, so a researcher judged each discontinuity as a clear divergence from an established trend or not. We call discontinuities judged to clearly involve a departure from an established trend ‘robust discontinuities’.

*See the project’s [main page](https://aiimpacts.org/discontinuous-progress-investigation/) for authorship and acknowledgements.*


Historic trends in particle accelerator performance

*Published Feb 7 2020*

None of particle energy, center-of-mass energy nor Lorentz factor achievable by particle accelerators appears to have undergone a discontinuity of more than ten years of progress at previous rates.

Details
-------

This case study is part of AI Impacts’ [discontinuous progress investigation](https://aiimpacts.org/discontinuous-progress-investigation/).

### Background

[Particle accelerators](https://en.wikipedia.org/wiki/Particle_accelerator) propel charged particles at high speeds, typically so that experiments can be conducted on them.[1](https://aiimpacts.org/particle-accelerator-performance-progress/#easy-footnote-bottom-1-1350 "“A particle accelerator is a machine that uses electromagnetic fields to propel charged particles to very high speeds and energies, and to contain them in well-defined beams. Large accelerators are used for basic research in particle physics.” – “Particle Accelerator”. 2019. En.Wikipedia.Org. Accessed June 30 2019. https://en.wikipedia.org/w/index.php?title=Particle_accelerator&oldid=903597299.")
![](https://lh5.googleusercontent.com/QUDe8-AID5ih4W5Sry27HEP7i9QRiMtxwqVyieEIiDKQPjDARIwL3Ud6XNGRCioHfQXDkqtXpQg3_Xun8BQGwfaZVTlKMGRh9sjhWcyV1nv-oG3a76n0jx_pQ1ZJYLb9aoDIvGv7)Fermi National Laboratory particle accelerator[2](https://aiimpacts.org/particle-accelerator-performance-progress/#easy-footnote-bottom-2-1350 "From Wikimedia Commons:
Fermilab, Reidar Hahn [Public domain] ")
### Trends

Our understanding is that key performance metrics for particle accelerators include how much kinetic energy they can generate in particles, how much center-of-mass energy they can create in collisions between particles, and the [Lorentz factor](https://en.wikipedia.org/wiki/Lorentz_factor) they can achieve.

‘Livingston charts’ show progress in particle accelerator efficacy over time, and seem to be common. We took data from a relatively recent and populated one in a slide deck from a Cornell accelerator physics course (see slide 45),[3](https://aiimpacts.org/particle-accelerator-performance-progress/#easy-footnote-bottom-3-1350 "Hoffstaetter 2019. Classe.Cornell.Edu. Accessed June 30 2019. https://www.classe.cornell.edu/~hoff/LECTURES/10USPAS/notes01.pdf., Slide 45") and extracted data from it, shown in [this spreadsheet](https://docs.google.com/spreadsheets/d/1fsEV5vtdArk6Q0RqAacmrErDif3x7O1JcPOn3BqIBY0/edit?usp=sharing) (see columns ‘year’ and ‘eV’ in tabs ‘Hoffstaetter Hadrons’ and ‘Hoffstaetter Leptons’ for original data).

The standard performance metric in a Livingston chart is ‘energy needed for a particle to hit a stationary proton with the same center of mass energy as the actual collisions in the accelerator’. We are uncertain why this metric is used, though it does allow for comparisons to earlier technology in a way that CM energy does not. We used a Lorentz transform to obtain particle energy, center-of-mass energy, and Lorentz factors from the Livingston chart data.[4](https://aiimpacts.org/particle-accelerator-performance-progress/#easy-footnote-bottom-4-1350 "A Lorentz transform allows us to recalculate velocities with a changed frame of reference, taking into account special relativity, which is a material consideration for such fast-moving objects.

See Rick Korzekwa’s explanation of his calculation of this here: https://docs.google.com/document/d/1Nv-0Jg6lMNobcDbfuruLwA8hYCXiPvq32f3NHz0BCLs/edit?usp=sharing")
#### Particle energy

##### Data

 Figure 1 shows our data on particle energy over time, also available in [our spreadsheet](https://docs.google.com/spreadsheets/d/1fsEV5vtdArk6Q0RqAacmrErDif3x7O1JcPOn3BqIBY0/edit?usp=sharing), tab ‘Particle energy’.

![](https://aiimpacts.org/wp-content/uploads/2019/11/ParticleEnergy-1024x768.png)Figure 1: Particle energy in eV over time
##### Discontinuity measurement

We chose to model the data as a single exponential trend.[5](https://aiimpacts.org/particle-accelerator-performance-progress/#easy-footnote-bottom-5-1350 "See our methodology page for more details.") There are no greater than 10-year discontinuities in particle energy at previous rates within this trend.[6](https://aiimpacts.org/particle-accelerator-performance-progress/#easy-footnote-bottom-6-1350 "See our methodology page for more details, and our spreadsheet, tab ‘Particle energy’ for our calculation of discontinuities.")
#### Center-of-mass energy

##### Data

Figure 2 shows our data on center-of-mass energy over time, also available in [this spreadsheet](https://docs.google.com/spreadsheets/d/1fsEV5vtdArk6Q0RqAacmrErDif3x7O1JcPOn3BqIBY0/edit?usp=sharing), tab ‘CM energy’.

![](https://aiimpacts.org/wp-content/uploads/2019/11/CMEnergy-1024x768.png)Figure 2: Center-of-mass energy in eV over time
##### Discontinuity measurement

We treated the data as exponential.[7](https://aiimpacts.org/particle-accelerator-performance-progress/#easy-footnote-bottom-7-1350 "See our methodology page for more details.") There are no greater than 10-year discontinuities in center-of-mass energy at previous rates within this trend.[8](https://aiimpacts.org/particle-accelerator-performance-progress/#easy-footnote-bottom-8-1350 "See our methodology page for more details, and this spreadsheet, tab ‘CM energy’ for our calculation of discontinuities.")
#### Lorentz factor

According to Wikipedia, ‘[The Lorentz factor](https://en.wikipedia.org/wiki/Lorentz_factor) or Lorentz term is the factor by which time, length, and relativistic mass change for an object while that object is moving.’

##### Data

[This spreadsheet](https://docs.google.com/spreadsheets/d/1fsEV5vtdArk6Q0RqAacmrErDif3x7O1JcPOn3BqIBY0/edit#gid=654930714), tab ‘Lorentz factor’, shows our calculated data for progress on Lorentz factors attained over time.

![](https://aiimpacts.org/wp-content/uploads/2020/02/LorentzFactor-1024x768.png) Figure 3: Lorentz factor (gamma) over time.
##### Discontinuity measurement

We treated the data as one exponential trend.[9](https://aiimpacts.org/particle-accelerator-performance-progress/#easy-footnote-bottom-9-1350 "See our methodology page for more details.") There were no greater than 10-year discontinuities at previous rates within this trend.[10](https://aiimpacts.org/particle-accelerator-performance-progress/#easy-footnote-bottom-10-1350 "See our methodology page for more details, and our spreadsheet, tab ‘Lorentz factor’ for our calculation of discontinuities.")
*Primary author: Rick Korzekwa*

Notes
-----


AI conference attendance

Six of the largest seven AI conferences hosted a total of 27,396 attendees in 2018. Attendance at these conferences has grown by an average of 21% per year over 2011-2018. These six conferences host around six times as many attendees as six smaller AI conferences.

Details
-------

[![](http://aiimpacts.org/wp-content/uploads/2019/03/Attendance-at-large-conferences-1984-2018-3.png)](http://aiimpacts.org/wp-content/uploads/2019/03/Attendance-at-large-conferences-1984-2018-3.png)Data from AI Index. The  conference IROS is excluded because AI Index did not have data for them in 2018. IROS had 2,678 attendees in 2017, so this does not dramatically change the graph.
Artificial Intelligence Index reports on this, from data they collected from conferences directly.[1](https://aiimpacts.org/ai-conference-attendance/#easy-footnote-bottom-1-1334 "Artificial Intelligence Index 2018 Report, p26-27, Our copy of their spreadsheet, with some changes is here (http://cdn.aiindex.org/2018/AI%20Index%202018%20Annual%20Report.pdf)") We extended their spreadsheet to measure precise growth rates, and visualize the data differently (data [here](https://docs.google.com/spreadsheets/d/15PnVTrBLTk5IxFKaJ6Dtgr7nOWZTxiBdXb1N4Byfh3c/edit?usp=sharing)). They are missing 2018 data for IROS, so while we include it in Figure 1, we excluded it from the growth rate calculations.

From their spreadsheet we calculate:

| Large conferences (>2000 2018 participants) total participants | 27,396 |
| Small conferences (<2000 2018 participants) total participants | 4,754 |

This means large conferences have 5.9 times as many participants as smaller conferences.

According to this data, total large conference participation has grown by a factor 3.76 between 2011 and 2019, which is equivalent to a factor of 1.21 per year during that period.

References
----------

AI conference attendance

Summary: This post presents data on attendance numbers at AI conferences. The main result: "total large conference participation has grown by a factor 3.76 between 2011 and 2019, which is equivalent to a factor of 1.21 per year during that period". Looking at the graph, it seems to me that the exponential growth started in 2013, which would mean a slightly higher factor of around 1.3 per year. This would also make sense given that the current boom is often attributed to the publication of AlexNet in 2012.


Historical economic growth trends

An analysis of historical growth supports the possibility of radical increases in growth rate. Naive extrapolation of long-term trends would suggest massive increases in growth rate over the coming century, although growth over the last half-century has lagged very significantly behind these long-term trends.

Support
-------

Bradford DeLong [has published](http://holtz.org/Library/Social%20Science/Economics/Estimating%20World%20GDP%20by%20DeLong/Estimating%20World%20GDP.htm) estimates for historical world GDP, piecing together data on recent GDP, historical population estimates, and crude estimates for historical per capita GDP. We have not analyzed these estimates in depth, but they appear to be plausible. (Robin Hanson [has expressed](http://hanson.gmu.edu/longgrow.html) complaints with the population estimates from before 10,000 BC, but our overall conclusions do not seem to be sensitive to these estimates.)

The raw data produced by DeLong, together with log-scale graphs of that data, are available [here](https://docs.google.com/spreadsheets/d/1vyKWVp_RImFzru_4X3KPEW6kD86TllkxRB2fgWwhnxA/edit?usp=sharing) (augmented with one data point for 2013 found in the [CIA world factbook](https://www.cia.gov/library/publications/the-world-factbook/geos/xx.html), population data from the US census bureau via [Wikipedia](http://en.wikipedia.org/wiki/World_population#cite_note-USCBcite-1), and the website [usinflationcalculator](http://www.usinflationcalculator.com/)). Note that brief periods of negative growth have not been indicated, and that we have used what DeLong refers to as “ex-nordhaus” data, neglecting quality-of-life adjustments arising from improvements in the diversity of goods.

[![](https://sites.google.com/site/aiimpactslibrary/historical-growth-trends/HistoricalGrowth.png)](https://sites.google.com/site/aiimpactslibrary/historical-growth-trends/HistoricalGrowth.png?attredirects=0)Figure 1: The relationship of GWP and doubling time, historically. Note that the x-axis is log(GWP), not time—date lines mark GWP at those dates.
The data suggest that (proportional) rates of economic and population growth increase roughly linearly with the size of the world economy and population. Certainly, a constant rate of growth is a poor model for the data, as growth rates range over 5 orders of magnitude; rather, the data appear to be consistent with substantially superlinear returns to scale, such that doubling the size of the world multiplies the absolute rate of growth by 21.5 – 21.75(as opposed to 2, which would be expected by exponential growth).

Extrapolating this model implies that at a time when the economy is growing 1% per year, growth will diverge to infinity after about 200 years. This outcome of course seems impossible, but this does suggest that the historical record is consistent with relatively large changes in growth rate, and in fact rates of economic growth experienced today are radically larger (even proportionally) than those experienced prior to the industrial revolution.

From around 0 to 500 CE, the predicted divergence occurs between 1700 and 2000, from 500 to 1000 CE it occurs around 2100, and from 1300 to 1950 it occurred in the later part of the 20th century.

In fact growth has fallen substantially behind this trend over the course of the 20th century; growth has continued but the acceleration of growth has slowed substantially (indeed reversing itself over the last 50 years). Moreover, it is unclear to us whether historically increasing returns to scale reflect returns to *economic* scale, or *population* scale, and if the latter then a profound slowdown seems likely–population growth rates seem to robustly fall at very high levels of development, and at any rate doubling times much shorter than 10-20 years would require radical changes in fertility patterns.[1](https://aiimpacts.org/historical-growth-trends/#easy-footnote-bottom-1-102 "“Most developed countries have completed the demographic transition and have low birth rates; most developing countries are in the process of this transition.[2][3] The major (relative) exceptions are some poor countries, mainly in sub-Saharan Africa and some Middle Eastern countries, which are poor or affected by government policy or civil strife, notably, Pakistan, Palestinian territories, Yemen, and Afghanistan.” – Demographic Transition, Wikipedia, 6 March 2019, https://en.wikipedia.org/wiki/Demographic_transition") That said, any such biologically contingent dynamics might be modified in a world where machine intelligence can substitute for human labor. Our impression is that this slowdown has been the subject of extensive inquiry by economists, but we have not reviewed this literature.

Implications
------------

Overall, it seems unclear how much weight one should place on historical trends in predicting the future, and it seems unclear whether we should focus on very long-term trends of accelerating growth or short-term trends of stagnant growth (at least as measured by GDP). However, at a minimum it seems that extrapolation from history is consistent with extreme increases in the growth rate.

Historical economic growth trends

Summary: Data on historical economic growth "suggest that (proportional) rates of economic and population growth increase roughly linearly with the size of the world economy and population", at least from around 0 CE to 1950. However, this trend has not held since 1950 - in fact, growth rates have fallen since then.


Primates vs birds: Is one brain architecture better than the other?

*By Tegan McCaslin, 28 February 2019*

The boring answer to that question is, “Yes, birds.” But that’s only because birds can pack more neurons into a walnut-sized brain than a monkey with a brain four times that size. So let’s forget about brain volume for a second and ask the really interesting question: neuron per neuron, who’s coming out ahead?

You might wonder why I picked birds and primates instead of, say, dogs and cats, or mice and elephants, or any other pair of distinct animals. But check out this mouse brain:

[![](http://aiimpacts.org/wp-content/uploads/2019/02/mouse-brain-image-287x300.png)](http://aiimpacts.org/wp-content/uploads/2019/02/mouse-brain-image.png)[By [Mamunur Rashid – Own work, CC BY 4.0](https://commons.wikimedia.org/w/index.php?curid=64106364)]
See how, on the outside of the lobe (the part closer to the upper righthand corner), you can pick out a series of stripes in a neat little row? Those stripes are the six layers of the neocortex, a specifically mammalian invention—all mammals have it, and no one else does. People have been pointing to this structure to explain why we’re so much better than fish since the [*scala naturae*](https://en.wikipedia.org/wiki/Great_chain_of_being) fell out of favor.

And that would be a pretty convenient story if birds hadn’t come along and messed the whole picture up. If you look at a similar cross section of a bird’s brain, it kind of just looks like a structureless blob. For a long time, comparative neuroanatomists thought birds must be at a more primitive stage of brain evolution, with no cortex but *huge* basal ganglia (the bit that we have sitting under our own fancy cortex). But we’ve since realized that this “lower” structure is actually a totally different, independently-evolved form of cortex, which seems to control all the same areas of behavior that mammalian cortex does. In fact, birds have substantially more of their brain neurons concentrated in their cortices than we mammals have in ours.

Alright, so it’s not that surprising that another form of cortical tissue exists in nature. But could it really work as well as *ours*? Surprisingly, no one has really tried to figure this out before.

If, for instance, primates were head and shoulders above birds, that might mean that intelligent brains aren’t *just* energetically expensive (in terms of the energy required for developing and operating neurons), they’re also exceptionally tricky to get right from a design standpoint. Of course, if bird and primate architectures worked equally well, that doesn’t mean brains are easy to get right–it would just mean that evolution happened to stumble into two independent solutions around 100 million years ago. Still, that would imply substantially more flexibility in neural tissue architectures than the world in which one tissue architecture outstripped all others.

Answering the question of birds vs. primates conclusively would be an enormous undertaking (and to be honest, answering it inconclusively was a pretty big pain already), so instead I focused on a very small sample of species in a narrow range of brain sizes and tried to get a really good sense of how smart those animals in particular were, relative to one another. I also got 80+ other people (non-experts) to look at the behavioral repertoire of these animals and rank how cognitively demanding they sounded.

With my methodology of just digging through all of the behavioral literature I could find on these species, full and representative coverage of their entire behavioral repertoire was a major challenge, and I think it fell well short of adequate in some categories. This can be a big problem if an animal only displays its full cognitive capacities in one or a few domains, and worse, you might not even know which those are. I think this wasn’t as big an issue with the species I studied as it could have been, since we have pretty good priors with respect to what selective pressures drove cognitive development in the smartest animals (like primates and parrots). Plus, scientists are much more likely to study the most complex and interesting behaviors, and those are very often the ones that display the most intelligence.

One of the behaviors scientists are really keen on is tool use. Our survey participants seemed to like it too, because they rated its importance higher than any other category, and it ended up being the most discriminatory behavior, too–neither the small-brained monkey nor the small-brained parrot had recorded examples of tool use in the wild, while both of the larger-brained animals did.

In the end, people didn’t seem to think the two primate species I included acted smarter than the two bird species or vice versa, but did think the larger-brained animals acted smarter than the smaller-brained animals. The fact that this surveying method both confirmed my intuitions and didn’t seem *totally* overwhelmed by noise kind of impressed me, because who knew you could just ask a bunch of random people to look at some animal behaviors and have them kind of agree on what the smartest were? That said, we didn’t validate it against anything, and even though we have reasons to suspect this method works as intended (see the full article), how well and whether this was a good implementation aren’t clear.

So this is all pretty cool, but even if we could prove definitively that macaws and squirrel monkeys are smarter than grey parrots and owl monkeys, it’s not a knock-down argument for architecture space being chock full of feasible designs, or even for birds and primates having identical per-neuron cognitive capacity. It’s mostly just a demonstration that the old self-flattering dogma of primate exceptionalism doesn’t really hold water. But it also points to an interesting trend: instead of trying to tweak a bunch of parameters in brains to squeeze the best possible performance out of a given size, evolution seems to have gotten a lot of mileage out of just throwing more neurons at the problem.

There’s a lot more dirt [here](https://aiimpacts.org/investigation-into-the-relationship-between-neuron-count-and-intelligence-across-differing-cortical-architectures/), in the full analysis.

Summary: Progress in AI can be driven by both larger models as well as architectural improvements (given sufficient data and compute), but which of these is more important? One source of evidence comes from animals: different species that are closely related will have similar neural architectures, but potentially quite different brain sizes. This post compares intelligence across birds and primates: while primates (and mammals more generally) have a neocortex (often used to explain human intelligence), birds have a different, independently-evolved type of cortex. Using a survey over non-expert participants about how intelligent different bird and primate behavior is, it finds that there is not much difference in intelligence ratings between birds and primates, but that species with larger brains are rated as more intelligent than those with smaller brains. This only suggests that there are at least two neural architectures that work -- it could still be a hard problem to find them in the vast space of possible architectures. Still, it is some evidence that at least in the case of evolution, you get more intelligence through more neurons, and architectural improvements are relatively less important.