Historic trends in structure heights

Trends for tallest ever structure heights, tallest ever freestanding structure heights, tallest existing freestanding structure heights, and tallest ever building heights have each seen 5-8 discontinuities of more than ten years. These are:

* **Djoser and Meidum pyramids** (~2600BC, >1000 year discontinuities in all structure trends)
* Three cathedrals that were shorter than the all-time record (**Beauvais** **Cathedral** in 1569, **St Nikolai** in 1874, and **Rouen** **Cathedral** in 1876, all >100 year discontinuities in current freestanding structure trend)
* **Washington Monument** (1884, >100 year discontinuity in both tallest ever structure trends, but not a notable discontinuity in existing structure trend)
* **Eiffel Tower** (1889, ~10,000 year discontinuity in both tallest ever structure trends, 54 year discontinuity in existing structure trend)
* Two early skyscrapers: the **Singer Building** and the **Metropolitan Life Tower** (1908 and 1909, each >300 year discontinuities in building height only)
* **Empire State Building** (1931, 19 years in all structure trends, 10 years in buildings trend)
* **KVLY-TV mast** (1963, 20 year discontinuity in tallest ever structure trend)
* **Taipei 101** (2004, 13 year discontinuity in building height only)
* **Burj Khalifa** (2009, ~30 year discontinuity in both freestanding structure trends, 90 year discontinuity in building height trend)

Details
-------

### Background

Over human history, the tallest man-made structures have included mounds, pyramids, churches, towers, a monument, skyscrapers, and radio and TV masts.

Height records often distinguish between structures and buildings, where a building is ‘regularly inhabited or occupied’ according to Wikipedia, or is ‘designed for residential, business or manufacturing purposes’ and ‘has floors’ according to the Council on Tall Buildings and Urban Habitat.[1](https://aiimpacts.org/discontinuity-from-the-burj-khalifa/#easy-footnote-bottom-1-1178 "“However, though all of these are structures, some are not buildings in the sense of being regularly inhabited or occupied. It is in this sense of being regularly inhabited or occupied that the term “building” is generally understood to mean when determining what is the world’s tallest building. The non-profit international organization Council on Tall Buildings and Urban Habitat (CTBUH), which maintains a set of criteria for determining the height of tall buildings, defines a “building” as “(A) structure that is designed for residential, business or manufacturing purposes” and “has floors”.” – “History Of The World’s Tallest Buildings”. 2011. En.Wikipedia.Org. Accessed July 3 2019. https://en.wikipedia.org/w/index.php?title=History_of_the_world%27s_tallest_buildings&oldid=903623843") Figure 1a is an illustration from Wikipedia showing the historic relationship between the heights of buildings and structures.[2](https://aiimpacts.org/discontinuity-from-the-burj-khalifa/#easy-footnote-bottom-2-1178 "“History Of The World’s Tallest Buildings”. 2011. En.Wikipedia.Org. Accessed July 3 2019. https://en.wikipedia.org/w/index.php?title=History_of_the_world%27s_tallest_buildings&oldid=903623843")
[![](http://aiimpacts.org/wp-content/uploads/2018/07/History_of_tallest_buildings_chart.jpg)](http://aiimpacts.org/wp-content/uploads/2018/07/History_of_tallest_buildings_chart.jpg)
Figure 1a: Recent history of tall structures by type.[3](https://aiimpacts.org/discontinuity-from-the-burj-khalifa/#easy-footnote-bottom-3-1178 "From Wikimedia Commons:Herostratus [CC BY-SA 3.0 (https://creativecommons.org/licenses/by-sa/3.0)]")
Height records also distinguish ‘freestanding’ structures from other structures. According to Wikipedia, “To be freestanding a structure must not be supported by guy wires, the sea or other types of support. It therefore does not include guyed masts, partially guyed towers and drilling platforms but does include towers, skyscrapers (pinnacle height) and chimneys.”[4](https://aiimpacts.org/discontinuity-from-the-burj-khalifa/#easy-footnote-bottom-4-1178 "“List of Tallest Freestanding Structures.” In Wikipedia, January 22, 2020. https://en.wikipedia.org/w/index.php?title=List_of_tallest_freestanding_structures&oldid=937089557.
") Definitions vary, for instance Guinness World Records apparently treats underwater structures as ‘freestanding’.[5](https://aiimpacts.org/discontinuity-from-the-burj-khalifa/#easy-footnote-bottom-5-1178 "“The Petronius Platform stands 610 m (2,000 ft) off the sea floor leading some, including Guinness World Records 2007, to claim it as the tallest freestanding structure in the world. However, it is debated whether underwater height should be counted, in the same manner as height below ground is ignored on buildings.”

“List of Tallest Buildings and Structures.” In Wikipedia, February 2, 2020. https://en.wikipedia.org/w/index.php?title=List_of_tallest_buildings_and_structures&oldid=938797794.") We ignore underwater height in general, excluding underwater structures from ‘freestanding’ records and and ‘all structures’ records.

The heights of buildings in particular are commonly measured in terms of ‘architectural height’ or ‘height to tip’, which both start at the lowest, significant, open-air, pedestrian entrance, but differ in that ‘to tip’ includes ‘functional-technical equipment’ like antennae, signage or flag poles, while architectural height does not[6](https://aiimpacts.org/discontinuity-from-the-burj-khalifa/#easy-footnote-bottom-6-1178 "Definitions from CTBUH’s Skyscraper Center:

Height: Architectural Height is measured from the level of the lowest, significant, open-air, pedestrian entrance to the architectural top of the building, including spires, but not including antennae, signage, flag poles or other functional-technical equipment. This measurement is the most widely utilized and is employed to define the Council on Tall Buildings and Urban Habitat (CTBUH) rankings of the “World’s Tallest Buildings.”

Height: To Tip Height is measured from the level of the lowest, significant, open-air, pedestrian entrance to the highest point of the building, irrespective of material or function of the highest element (i.e., including antennae, flagpoles, signage and other functional-technical equipment).

“The Skyscraper Center.” Accessed February 3, 2020. https://www.skyscrapercenter.com/definitions/Building. ") Our understanding is that ‘pinnacle height’ is the same as ‘height to tip’. There are also several less common measures in use.

Height records must also distinguish between the tallest structure standing at a given time, and the tallest structure to have ever existed, at that time. The tallest building or structure at a particular time is sometimes not the tallest ever, when the tallest is damaged without anything taller being built. For instance, the tallest structures in the 1700s were shorter than earlier records, because those were church spires which became damaged without replacement (see Figure 1b).

[![](http://aiimpacts.org/wp-content/uploads/2018/07/Wikipedia-structure-heights-1024x544.png)](http://aiimpacts.org/wp-content/uploads/2018/07/Wikipedia-structure-heights.png)
Figure 1b: An illustration of structure heights over time by location from Wikipedia. [7](https://aiimpacts.org/discontinuity-from-the-burj-khalifa/#easy-footnote-bottom-7-1178 "“History of the World’s Tallest Buildings,” in Wikipedia, July 19, 2019, https://en.wikipedia.org/w/index.php?title=History_of_the_world%27s_tallest_buildings&oldid=906924179.") (Click to enlarge)

### Trends

We collected data for several combinations of measurement possibilities mentioned:

* Tallest ever structures on land (i.e. freestanding or not, but not underwater), measured to tip
* Tallest ever *freestanding* structures, measured to tip
* Tallest *existing* freestanding structures, measured to tip
* Tallest ever *buildings*, measured to architectural height

#### Tallest ever structure heights

##### Data

We collected height records from numerous Wikipedia lists of tall buildings and structures.[8](https://aiimpacts.org/discontinuity-from-the-burj-khalifa/#easy-footnote-bottom-8-1178 "For instance, https://en.wikipedia.org/wiki/List_of_tallest_structures_built_before_the_20th_century, https://en.wikipedia.orug/wiki/List_of_tallest_buildings_and_structures#Tallest_freestanding_structures_on_land, https://en.wikipedia.org/wiki/List_of_tallest_freestanding_structures#Timeline_of_world’s_tallest_freestanding_structures, https://en.wikipedia.org/wiki/History_of_the_world%27s_tallest_buildings") We have not extensively verified these sources, though we made minor adjustments and additions from elsewhere online where sources were inconsistent or records incomplete. Our data is in [this spreadsheet](https://docs.google.com/spreadsheets/d/1cnvlMpQsLye0Z0m98vMXN7k4nryxvoN3L1_aH9xho4I/edit?usp=sharing), sheet ‘Structures collection’. Figure 2 shows this data.

![](https://aiimpacts.org/wp-content/uploads/2020/02/Recorded-heights-of-land-structures-all-types-1-1.png)Figure 2: Our collection of height records for man-made above ground structures, from a variety of online sources (excluding two earlier records). Note that some records are repeated in slightly different versions or are for the same structure being extended, or becoming a record again after the destruction of another structure. The collection is constructed to contain the tallest structures, but the subset of non-tallest structures included is arbitrary.
We constructed a timeline of tallest ever structures by pinnacle height from the tallest ever records in this dataset (see sheet ‘Structures (all time, pinnacle)’ in [the spreadsheet](https://docs.google.com/spreadsheets/d/1cnvlMpQsLye0Z0m98vMXN7k4nryxvoN3L1_aH9xho4I/edit?usp=sharing)). This is shown in figures 3a and 3b below.

![](http://aiimpacts.org/wp-content/uploads/2020/01/StructureRecordZoom-1024x768.png)**Figure 3a:** Recent history of tallest structures ever built on land (not necessarily freestanding). The record may be taller than any structure standing at a given time.
![](http://aiimpacts.org/wp-content/uploads/2020/01/StructureRecord-1024x768.png)**Figure 3b:** Longer term history of tallest structures ever built on land (not necessarily freestanding), on a log scale. The record may be taller than any structure standing at a given time.
#####

##### Discontinuity measurement

We treat this data as exponential initially followed by three linear trends. Using these trends as the previous rate to compare to, we calculated for each record how many years ahead of the trend it was.[9](https://aiimpacts.org/discontinuity-from-the-burj-khalifa/#easy-footnote-bottom-9-1178 "See our methodology page for details on how we choose what to treat as the ‘previous trend’ at a given point, and how we calculate discontinuities. See our spreadsheet, sheet ‘Structures (all time, pinnacle)’ for the division of our data into different trends and the discontinuity calculations.") The series contained six unambiguous greater-than-ten-year discontinuities, shown in Table 1 below.

The Bent Pyramid appears to represent a 12 year discontinuity, but we ignore this because its date of construction seems uncertain relative to the small discontinuity.[10](https://aiimpacts.org/discontinuity-from-the-burj-khalifa/#easy-footnote-bottom-10-1178 "The pyramid is around 4600 years old, and for instance, the source we used had 2605 BC as its record date, whereas the pyramid’s own Wikipedia page gives ‘c 2600 BC’ as its date of construction.

“Bent Pyramid.” In Wikipedia, December 12, 2019. https://en.wikipedia.org/w/index.php?title=Bent_Pyramid&oldid=930419772. ")
While our early records are presumably incomplete, we do not avoid measuring early discontinuities for this reason, because the large discontinuities that we find before the 19th Century seem unlikely to depend substantially on the exact set of earlier records.

**Table 1:** discontinuities in tallest ever structure heights

|  |  |  |  |
| --- | --- | --- | --- |
| **Year** | **Height (m)** | **Discontinuity (years)** | **Structure** |
| 2650 BC | 62.5 | ~9000 | Pyramid of Djoser |
| 2610 BC | 91.65 | ~1000 | Meidum Pyramid |
| 1884 | 169.3 | 106 | Washington Monument |
| 1889 | 300 | ~10,000 | Eiffel Tower |
| 1931 | 381 | 19 | Empire State Building |
| 1963 | 628.8 | 20 | KVLY-TV mast |

A number of other potentially relevant metrics are tabulated [here](https://docs.google.com/spreadsheets/d/1iMIZ57Ka9-ZYednnGeonC-NqwGC7dKiHN9S-TAxfVdQ/edit?usp=sharing).

#### Tallest ever freestanding structure heights

##### Data

This data is another subset of the ‘structures collection’ described above, this time including only records for ‘freestanding’ structures. This excludes structures supported by guy ropes, such as radio masts. Guyed masts were the tallest structures on land overall between 1954 and 2008, so this dataset differs from the ‘tallest ever structure heights’ dataset above between those years.

This dataset can be found in [this spreadsheet](https://docs.google.com/spreadsheets/d/1cnvlMpQsLye0Z0m98vMXN7k4nryxvoN3L1_aH9xho4I/edit?usp=sharing), sheet ‘Freestanding structures (all time, pinnacle)’. Figures 4-5 below illustrate it.

![](http://aiimpacts.org/wp-content/uploads/2020/01/RecordFreeZoom-1024x768.png)Figure 4: Recent history of tallest freestanding structures ever built.
![](http://aiimpacts.org/wp-content/uploads/2020/01/RecordFreestandingStructure-1024x768.png)Figure 5: Longer term history of tallest freestanding structures ever built, on a log scale.
##### Discontinuity measurement

We treat this data as exponential initially followed by three linear trends. Using these trends as the previous rate to compare to, we calculated for each record how many years ahead of the trend it was.[11](https://aiimpacts.org/discontinuity-from-the-burj-khalifa/#easy-footnote-bottom-11-1178 "See our methodology page for details on how we choose what to treat as the ‘previous trend’ at a given point, and how we calculate discontinuities. See our spreadsheet, sheet ‘Freestanding structures (all time, pinnacle)‘ for the division of our data into different trends and the discontinuity calculations.") The series contained six unambiguous greater-than-ten-year discontinuities. The first five are the same as those in the previous dataset, since the series do not diverge until later (see *Tallest ever structure heights* section above for further details). The last discontinuity is a 32 year jump in 2009 from the Burj Khalifa.

We tabulated a number of other potentially relevant metrics [here](https://docs.google.com/spreadsheets/d/1iMIZ57Ka9-ZYednnGeonC-NqwGC7dKiHN9S-TAxfVdQ/edit?usp=sharing).

#### Tallest existing freestanding structure heights

We constructed a dataset of tallest freestanding structures over time largely from Wikipedia’s Timeline of world’s tallest freestanding structures[12](https://aiimpacts.org/discontinuity-from-the-burj-khalifa/#easy-footnote-bottom-12-1178 " “List of Tallest Freestanding Structures.” In Wikipedia, January 22, 2020. https://en.wikipedia.org/w/index.php?title=List_of_tallest_freestanding_structures&oldid=937089557. "), with some modifications. This is available in [our spreadsheet](https://docs.google.com/spreadsheets/d/1cnvlMpQsLye0Z0m98vMXN7k4nryxvoN3L1_aH9xho4I/edit?usp=sharing), sheet ‘Freestanding structures (current, pinnacle)’, and is shown in Figures 6-7 below.

![](http://aiimpacts.org/wp-content/uploads/2020/01/CurrentFreeZoom-1024x768.png)**Figure 6:** Recent history of tallest freestanding structures standing. New records are sometimes shorter than old records.
![](http://aiimpacts.org/wp-content/uploads/2020/01/CurrentFreestandingStructure-1024x768.png)Figure 7: Longer term history of tallest freestanding structures standing, on a log scale. New records are sometimes shorter than old records.
##### Discontinuity measurement

We treat this data as exponential initially, followed by four linear trends. Using these trends as the ‘previous rate’ to compare to,[13](https://aiimpacts.org/discontinuity-from-the-burj-khalifa/#easy-footnote-bottom-13-1178 "See our methodology page for details on how we choose what to treat as the ‘previous trend’ at a given point. See our spreadsheet, sheet ‘Freestanding structures (current, pinnacle)’ for the trends.") the data contained eight unambiguous greater than ten year discontinuities, shown in Table 2 below.[14](https://aiimpacts.org/discontinuity-from-the-burj-khalifa/#easy-footnote-bottom-14-1178 "We again ignore the Bent Pyramid, because the apparent discontinuity is small relative to uncertainty about its date of construction. See our methodology page for explanation of how we calculated discontinuities. Also see our spreadsheet, sheet ‘Freestanding structures (current, pinnacle)’ for these calculations.")
This series differs from that of all-time tallest freestanding structures above by the insertion of a series of records between Lincoln Cathedral in 1311 and the Washington Monument in 1889. This change made the Washington Monument unexceptional rather than a 100 year discontinuity, and the Eiffel Tower a fifty-year discontinuity rather than a ten-thousand year one. Later discontinuities from the Empire State Building and Burj Khalifa are very similar.

Table 2: discontinuities in tallest existing freestanding structures

|  |  |  |  |
| --- | --- | --- | --- |
| **Year** | **Height (m)** | **Discontinuity (years)** | **Structure** |
| 2650 BC | 62.5 | ~9000 | Pyramid of Djoser |
| 2610 BC | 91.65 | ~1000 | Meidum Pyramid |
| 1569 | 153 | 138 | Beauvais Cathedral |
| 1874 | 147.3 | 224 | St Nikolai |
| 1876 | 151 | 307 | Rouen Cathedral |
| 1889 | 300 | 54 | Eiffel Tower |
| 1931 | 381 | 19 | Empire State Building |
| 2009 | 829.8 | 35 | Burj Khalifa |

We have tabulated a number of other potentially relevant metrics [here](https://docs.google.com/spreadsheets/d/1iMIZ57Ka9-ZYednnGeonC-NqwGC7dKiHN9S-TAxfVdQ/edit?usp=sharing).[15](https://aiimpacts.org/discontinuity-from-the-burj-khalifa/#easy-footnote-bottom-15-1178 "See our our methodology page for more details.")
#### Tallest ever building heights

##### Data

We collected data on the tallest ever buildings from Wikipedia’s *History of the world’s tallest buildings*,[16](https://aiimpacts.org/discontinuity-from-the-burj-khalifa/#easy-footnote-bottom-16-1178 "“History Of The World’s Tallest Buildings”. 2011. En.Wikipedia.Org. Accessed May 26 2019. https://en.wikipedia.org/wiki/History_of_the_world%27s_tallest_buildings.") and added it to [this spreadsheet](https://docs.google.com/spreadsheets/d/1cnvlMpQsLye0Z0m98vMXN7k4nryxvoN3L1_aH9xho4I/edit?usp=sharing) (sheet ‘Buildings (all time, architectural)’). We have not thoroughly verified it, but have made minor modifications (noted in the spreadsheet). Figure 8 shows this data.

![](http://aiimpacts.org/wp-content/uploads/2020/01/TallestBuilding-1024x768.png)Figure 8: Height of tallest buildings ever built, measured using ‘architectural height’, which excludes some additions such as antennae.
![](http://aiimpacts.org/wp-content/uploads/2020/01/TallestBuildingZoom-1024x768.png)Figure 9: Close up of Figure 8
##### Discontinuity measurement

We treated this data as an exponential trend followed by a linear trend.[17](https://aiimpacts.org/discontinuity-from-the-burj-khalifa/#easy-footnote-bottom-17-1178 "See our methodology page for details on how we decide what to treat as the past trend for each point.") Compared to previous rates within these trends, tallest buildings over time contained five greater than ten year discontinuities, shown in Table 3 below.[18](https://aiimpacts.org/discontinuity-from-the-burj-khalifa/#easy-footnote-bottom-18-1178 "See our methodology page for explanation of how we calculated these numbers. See our spreadsheet, sheet ‘Buildings (all time, architectural)’ for these calculations.")
**Table 3:** discontinuities in tallest ever building heights

|  |  |  |  |
| --- | --- | --- | --- |
| **Year** | **Height (m)** | **Discontinuity (years)** | **Building** |
| 1908 | 186.57 | 383 | Singer Building |
| 1909 | 213.36 | 320 | [Metropolitan Life Tower](https://en.wikipedia.org/wiki/Metropolitan_Life_Insurance_Company_Tower) |
| 1931 | 381 | 10 | [Empire State Building](https://en.wikipedia.org/wiki/Empire_State_Building) |
| 2004 | 509.2 | 13 | [Taipei 101](https://en.wikipedia.org/wiki/Taipei_101) |
| 2010 | 828 | 90 | [Burj Khalifa](https://en.wikipedia.org/wiki/Burj_Khalifa) |

We have tabulated a number of other potentially relevant metrics [here](https://docs.google.com/spreadsheets/d/1iMIZ57Ka9-ZYednnGeonC-NqwGC7dKiHN9S-TAxfVdQ/edit?usp=sharing).[19](https://aiimpacts.org/discontinuity-from-the-burj-khalifa/#easy-footnote-bottom-19-1178 "See our methodology page for more details.")
![](https://aiimpacts.org/wp-content/uploads/2020/02/alex-azabache-9ZChTzDtCww-unsplash-787x1024.jpg)Figure 10: Burj Khalifa, current record holder for every listed metric, and discontinuously tall freestanding structure and building.
Notes
-----

Historic trends in structure heights


Interpreting AI compute trends

*This is a guest post by Ryan Carey, 10 July 2018.*

Over the last few years, we know that AI experiments have used much more computation than previously. But just last month, an [investigation by OpenAI](https://blog.openai.com/ai-and-compute/) made some initial estimates of just how fast this growth has been. Comparing AlphaGo Zero to AlexNet, they found that the largest experiment now is 300,000-fold larger than the largest experiment six years ago. In the intervening time, the largest experiment in each year has been growing exponentially, with a doubling time of 3.5 months.

The rate of growth of experiments according to this *AI-Compute trend* is astoundingly fast, and this deserves some analysis. In this piece, I explore two issues. The first is that if experiments keep growing so fast, they will quickly become unaffordable, and so the trend will have to draw to a close. Unless the economy is drastically reshaped, this trend can be sustained for at most 3.5-10 years, depending on spending levels and how the cost of compute evolves over time. The second issue is that if this trend is sustained for even 3.5 more years, the amount of compute used in an AI experiment will have passed some interesting milestones. Specifically, the compute used by an experiment will have passed the amount required to simulate, using spiking neurons, a human mind thinking for eighteen years. Very roughly speaking, we could say that the trend would surpass the level required to reach the level of intelligence of an adult human, given an equally efficient algorithm. In sections (1) and (2), I will explore these issues in turn, and then in section (3), I will discuss the limitations of this analysis and weigh how this work might bear on AGI forecasts.

1. How long can the AI-Compute trend be sustained?
--------------------------------------------------

To figure out how long the AI-Compute trend can be economically sustained, we need to know three things: the rate of growth of the cost of experiments, the cost of current experiments, and the maximum amount that can be spent on an experiment in the future.

The size of the largest experiments is increasing with a doubling time of 3.5 months, (about an order of magnitude per year)[1](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-1-1170 "3.5/(log10(2))/12 = 0.9689, the number of years over which each 10x increase in compute occurs."), while the cost per unit of computation is decreasing by an order of magnitude every 4-12 years (the long-run trend has improved costs by 10x every 4 years, whereas recent trends have improved costs by 10x every 12 years)[2](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-2-1170 "AI Impacts gives a recent-hardware-prices figure, and past quarter-century FLOPS/$ figure, and I use these as a range. “The cheapest hardware prices (for single precision FLOPS/$) appear to be falling by around an order of magnitude every 10-16 years. This rate is slower than the trend of FLOPS/$ observed over the past quarter century, which was an order of magnitude every 4 years. There is no particular sign of slowing between 2011 and 2017.” “Figure 4 shows the 95th percentile fits an exponential trendline quite well, with a doubling time of 3.7 years, for an order of magnitude every 12 years. This has been fairly consistent, and shows no sign of slowing by early 2017. This supports the 10-16 year time we estimated from the Wikipedia theoretical performance above.”"). So the cost of the largest experiments is increasing by an order of magnitude every 1.1 – 1.4 years.[3](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-3-1170 "12/(12×.96889-1) = 1.129 years. 4/(4×.96889-1) = 1.391 years.")

The largest current experiment, AlphaGo Zero, probably cost about $10M.[4](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-4-1170 "$10M is a rough estimate of the costs of computation including pre-run experiments, based on two lines of evidence. First, the stated hardware costs. “During the development of Zero, Hassabis says the system was trained on hardware that cost the company as much as $35 million. The hardware is also used for other DeepMind projects.” If we allow for some non-AlphaGo usage of this hardware, and some other (e.g. electricity) costs for running AlphaGo, then the total compute cost for AlphaGo would be something like $10-30M. Second, the costs of the final training run, based on costs of public compute. If the TPU compute cost Google the $6.50/hr rate offered to the general public, then it would have cost $35M. Cloud compute is probably 10x more expensive than Google’s internal compute costs, so the cost of the final training run is probably around $3M, and the cost of the whole experiment something like $3-10M.")

The largest that experiments can get depends who is performing them. The richest actor is probably the US government. Previously, the US spent 1% of annual GDP[5](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-5-1170 "Peak spending for the Manhattan project itself was around 1%, although total spending on nuclear weaponry in general was a little higher at 4.5% for several years following. “After getting fully underway in 1942, the Manhattan Project’s three-year cost of $2 billion (in 1940’s dollars) comprised nearly 1% of 1945 U.S. GDP. Extraordinary levels of spending and commitment of national resources to nuclear technology continued for many decades afterward. From 1947-1952, spending on nuclear weapons averaged 30% of total defense spending, which in 1952 was 15% of U.S. GDP.”

Allen, Greg, and Taniel Chan. Artificial Intelligence and National Security. Belfer Center for Science and International Affairs, 2017.") on the Manhattan Project, and ~0.5% of annual GDP on NASA during the Apollo program.[6](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-6-1170 "NASA spending peaked at $4.5B/yr in 1966 (during the Apollo program), while US GDP was $815B https://history.nasa.gov/SP-4029/Apollo_18-16_Apollo_Program_Budget_Appropriations.htm

https://data.worldbank.org/indicator/NY.GDP.MKTP.CD?end=1990&locations=US-CN-KR&start=1966") So let’s suppose they could similarly spend at most 1% of GDP, or $200B, on one AI experiment. Given the growth of one order of magnitude per 1.1-1.4 years, and the initial experiment size of $10M, the AI-Compute trend predicts that we would see a $200B experiment in 5-6 years.[7](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-7-1170 "1.129×log10(200B/10M) = 4.86 years. log10(200B/10M)×1.391 = 5.98 years.") So given a broadly similar economic situation to the present one, that would have to mark an end to the AI-Compute trend.

We can also consider how long the trend can last if government is not involved. Due to their smaller size, economic barriers hit a little sooner for private actors. The largest among these are tech companies: Amazon and Google have current research and development budgets of about ~20B/yr each[8](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-8-1170 "Amazon and Google, the two largest players, have research and development budgets of $22.6B and $16.6B respectively. (Confirmed also from the primary source, “Research and development expenses 16,625 [in 2017]” on p36 of GOOG 10k)."), so we can suppose that the largest individual experiment outside of government is $20B. Then the private sector can keep pace with the AI-Compute trend for around ¾ as long as government, or ~3.5-4.5 years.[9](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-9-1170 "log10(200B/10M)/log10(200B/10M) = 0.767 times as long. 1.129×log10(2000)=3.727 years. 1.391×log10(2000) = 4.591 years.")[10](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-10-1170 "Greg Brockman of OpenAI recently said of the AI-Compute trend at a House Committee Hearing: “We expect this to continue for the next 5 years, using only today’s proven hardware technologies and not assuming any breakthroughs like quantum or optical.” This is a bold prediction according to my models. After five years, private organizations have already become unable to keep pace with the trend, and government would be unable to keep pace for more than another year.")

On the other hand, the development of specialized hardware could cheapen computation, and thereby cause the trend to be sustainable for a longer period. If some new hardware cheapened compute by 1000x over and above price-performance Moore’s Law, then the economic barriers bite a little later– after an extra 3-4 years.[11](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-11-1170 "1.129 years×log10(1000) = 3.387 years. 1.391×log10(1000) = 4.173 years.")

In order for the AI-Compute trend to be maintained for a really long time (more than about a decade), economic output would have to start growing by an order of magnitude or more per year. This is a really extreme scenario, but the main thing that would make it possible would presumably be some massive economic gains from some extremely powerful AI technology, that would also serve to justify the massive ongoing AI investment.

Of course, it’s important to be clear that these figures are upper bounds, and they do not preclude the possibility that the AI-Compute trend may halt sooner (e.g. if AI research proves less economically useful than expected) either in a sudden or more gradual fashion.

So we have shown one kind of conclusion from a rapid trend — that it cannot continue for very long, specifically, beyond 3.5-10 years.

2. When will the AI-Compute trend pass potentially AGI-relevant milestones?
---------------------------------------------------------------------------

The second conclusion that we can draw is that if the AI-Compute trend continues at its current rapid pace, it will pass some interesting milestones. If the AI-Compute trend continues for 3.5-10 more years, then the size of the largest experiment is projected to reach 107-5×1013 Petaflop/s-days, and so the question is which milestones arrive below that level.[12](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-12-1170 "2^((3.727×12)/3.5) = 7025-fold growth of experiment.

2^(((5.98 + 4.173)×12)/3.5) = 3.01×1010-fold growth of experiment.

Alphago Zero size is 1860 Petaflop/s-days so projected experiment size is 1.3×107-5.6×1013 Petaflop/s-days.") Which milestones might allow the development of AGI is a controversial topic, but three candidates are:

1. The amount of compute required to simulate a human brain for the duration of a human childhood
2. The amount of compute required to simulate a human brain to play the number of Go games Alphago Zero required to become superhuman
3. The amount of compute required to simulate the evolution of the human brain

### Human-childhood milestone

One natural guess for the amount of computation required to create artificial intelligence is the amount of computation used by the human brain. Suppose an AI had (compared to a human):

* a similarly efficient algorithm for learning to perform diverse tasks (with respect to both both compute and data),
* similar knowledge built in to its architecture,
* similar data, and
* enough computation to simulate a human brain running for eighteen years, at sufficient resolution to capture the intellectual performance of that brain.

Then, this AI should be able to learn to solve a similarly wide range of problems as an eighteen year-old can solve.[13](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-13-1170 "An implicit premise is that the amount of computation used by the human brain is less than the amount used to simulate the human brain. The idea here is that some fraction of the resources used to simulate a human brain are actually used for thinking.")

There is a range of estimates for how many floating point operations per second are required to simulate a human brain for one second. Those [collected by AI Impacts](https://aiimpacts.org/brain-performance-in-flops/) have a median of 1018 FLOPS (corresponding roughly to a whole-brain simulation using Hodgkin-Huxley neurons[14](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-14-1170 "Table 9 of the WBE roadmap shows that a spiking neural network would use 1×1011 neurons. Table 7 shows that a Hodgkin-Huxley model would use about 1.2 million floating point operations to simulate a neuron for one second. Their product yields the 1×1018 figure.")), and ranging from 3×1013FLOPS (Moravec’s estimate) to 1×1025FLOPS (simulating the metabolome). Running such simulations for eighteen years would correspond to a median of 7 million Petaflop/s-days (range 200 – 7×1013 Petaflop/s-days).[15](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-15-1170 "3×1013×365×18 = 1.97×1017. 1×1018×365×18 = 6.57×1021. 1×1025×365×18 = 6.57×1028.")

So for the shortest estimates, such as the Moravec estimate, we have already reached enough compute to pass the human-childhood milestone. For the median estimate, and the Hodgkin-Huxley estimates, we will have reached the milestone within 3.5 years. For the metabolome estimates, the required amount of compute cannot be reached within the coming ten year window before the AI-Compute trend is halted by economic barriers. After the AI-Compute trend is halted, it’s worth noting that Moore’s Law could come back to the fore, and cause the size of experiments to continue to slowly grow. But on Moore’s Law, milestones like the metabolome estimate are still likely decades away.

### AlphaGo Zero-games milestone

One objection to the human-childhood milestone is that AI systems presently are “slower-learners” than humans. AlphaGo Zero used 2.5 million Go games to become superhuman[16](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-16-1170 "From figure 3a, in the Alphago Zero paper, reinforcement learning surpassed AlphaGo Lee (which in turn defeated Lee Sedol) around halfway through a series of 4.9 million games of self-play. This analysis is from a guesstimate model by Median Group."), which if each game took at hour, would correspond to 300 years of Go games[17](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-17-1170 "2.5×106 / 24 / 365 = 285 years"). We might ask how long it would take to run something as complex as the human brain, for 300 years, rather than just eighteen. In order for this milestone to be reached, the trend would have to continue for another 14 months longer than the human-childhood milestone[18](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-18-1170 "0.9689×log10(285/18) = 1.16").

### Brain-evolution milestone

A more conservative milestone is the amount of compute required to simulate all neural evolution. One approach, described by [Shulman and Bostrom 2012](http://datascienceassn.org/sites/default/files/How%20Hard%20is%20Artificial%20Intelligence%20-%20Evolutionary%20Arguments%20and%20Selection%20Effects.pdf), is to look at the cost of simulating the evolution of nervous systems. This entails simulating 1025 neurons for one billion years.[19](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-19-1170 "From How Hard is Artificial Intelligence? “Erring on the side of conservatively high, if we assigned all 1019 insects fruit-fly numbers of neurons the total would be 1024 insect neurons in the world. This could be augmented with an additional order of magnitude, to reflect aquatic copepods, birds, reptiles, mammals, etc., to reach 1025”") Shulman and Bostrom estimate the cost simulating a neuron for one second at 1-1010 floating point operations,[20](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-20-1170 "“The computational cost of simulating one neuron depends on the level of detail… Extremely simple neuron models use about 1,000 floating-point operations per second (FLOPS) to simulate one neuron (for one second of simulated time); an electrophysiologically realistic Hodgkin-Huxley model uses 1,200,000 FLOPS; a more detailed multicompartmental model would add another 3-4 orders of magnitude, while higher-level models that abstract systems of neurons could subtract 2-3 orders of magnitude from the simple models.” This range is 2-3 orders of magnitude lower than the per-neuron costs implied by the range of collated AI Impacts estimates for brain simulation.") and so the total cost for simulating evolution is 3×1021-3×1031 Petaflop/s-days[21](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-21-1170 "1025×365×109×1 = 3.65×1036 FLOP/s-days. 1025×365×109×1010 = 3.65×1046 FLOP/s-days"). This figure would not be reached until far beyond the time when the current AI-Compute trend must end. So the AI-Compute trend does not change the conclusion of Shulman and Bostrom that simulation of brain evolution on Earth is far away — even with a rapid increase in spending, this compute milestone would take many decades of advancement of Moore’s Law to be reached[22](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-22-1170 "This only slightly shortens the timelines compared to Shulman and Bostrom’s remarks: “The computing resources to match historical numbers of neurons in straightforward simulation of biological evolution on Earth are severely out of reach, even if Moore’s law continues for a century. The argument from evolutionary algorithms depends crucially on the magnitude of efficiency gains from clever search, with perhaps as many as thirty orders of magnitude required.”").

Overall, we can see that although the brain-evolution milestone is well beyond the AI-Compute trend, the others are not necessarily. For some estimates — especially metabolome estimates — the human-childhood and AlphaGo Zero-games milestones cannot be reached either. But some of the human-childhood and AlphaGo Zero-games milestones will be reached if the AI Compute trend continues for the next few years.

3. Discussion and Limitations
-----------------------------

In light of this analysis, a reasonable question to ask is: for the purpose of predicting AGI, which milestone should we care most about? This is very uncertain, but I would guess that building AGI is easier than the brain-evolution milestone would suggest, but that AGI could arrive either before, or after the AlphaGo Zero-games milestone is reached.

The first claim is because the brain-evolution milestone assumes that the process of algorithm discovery must be performed by the AI itself. It seems more likely to me that the appropriate algorithm is provided (or mostly provided) by the human designers at no computational cost (or at hardly any cost compared to simulating evolution).

The second matter — evaluating the difficulty of AGI relative to the AlphaGo Zero-games milestone — is more complex. One reason for thinking that the AlphaGo Zero-games milestone makes AGI look too easy is that more training examples ought to be required to teach general intelligence, than are required to learn the game of Go.[23](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-23-1170 "Analogously to the “Difficulty ratio” in the guesstimate model by Median Group.") In order to perform a wider range of tasks, it will be necessary to consider a larger range of dependencies and to learn a more intricate mapping from actions to utilities. This matter could be explored further by comparing the sample efficiency of various solved AI problems and extrapolating the sample efficiency of AGI based on how much more complicated general intelligence seems. However, there are also reasons the AlphaGo Zero-games milestone might make things look too hard. Firstly, AlphaGo Zero does not use any pre-existing knowledge, whereas AGI systems might. If we had looked instead at the original AlphaGo, this would have required an order of magnitude fewer games relative to AlphaGo Zero[24](https://aiimpacts.org/interpreting-ai-compute-trends/#easy-footnote-bottom-24-1170 "AlphaGo used hundreds of thousands, rather than single-digit millions of games. From Mastering the game of Go with deep neural networks and tree search:  “We trained the policy network to classify positions according to expert moves played in the KGS data set. This data set contains 29.4 million positions from 160,000 games played by KGS 6 to 9 dan human players; 35.4% of the games are handicap games… The policy network was trained in this way for 10,000 minibatches of 128 games, using 50 GPUs, for one day”"), and further efficiency gains might be possible for more general learning tasks. Secondly, there might be one or more orders of magnitude of conservatism built-in to the approach of using simulations of the human brain. Simulating the human brain on current hardware may be a rather inefficient way to capture its computing function: that is, the human brain might only be using some fraction of the computation that is needed to simulate it. So it’s hard to judge whether the AlphaGo Zero-games milestone is too late or too soon for AGI.

There is another reason for some more assurance that AGI is more than six years away. We can simply look at the AI-Compute trend and ask ourselves: is AGI as close to AlphaGo Zero as AlphaGo Zero is to AlexNet? If we think that the difference (in terms of some combination of capabilities, compute, or AI research) between the first pair is larger than the second, then we should think that AGI is more than six years away.

In conclusion, we can see that the AI-Compute trend is an extraordinarily fast trend that economic forces (absent large increases in GDP) cannot sustain beyond 3.5-10 more years. Yet the trend is also fast enough that if it is sustained for even a few years from now, it will sweep past some compute milestones that could plausibly correspond to the requirements for AGI, including the amount of compute required to simulate a human brain thinking for eighteen years, using Hodgkin Huxley neurons. However, other milestones will not be reached before economic factors halt the AI-Compute trend. For example, this analysis shows that we will not have enough compute to simulate the evolution of the human brain for (at least) decades.

*Thanks Jack Gallagher, Danny Hernandez, Jan Leike, and Carl Shulman for discussions that helped with this post.*

---

Interpreting AI compute trends

Summary: A previous [OpenAI post](https://blog.openai.com/ai-and-compute/) showed that the amount of compute used in the most expensive AI experiments has been growing exponentially for six years, with a doubling time of 3.5 _months_. This is extraordinarily fast, and can be thought of as a combination of growth in the amount spent on an experiment, and a decrease in the cost of computation. Such a trend can only continue for a few more years, before the cost of the experiment exceeds the budget of even the richest actors (such as the US government). However, this might still be enough to reach some important milestones for compute, such as "enough compute to simulate a human brain for 18 years", which is plausibly enough to get to AGI. (This would not happen for some of the larger estimates of the amount of computation in the human brain, but would happen for some of the smaller estimates.) It is still an open question which milestone we should care about.


Occasional update July 5 2018

*By Katja Grace, 5 July 2018*

Before I get to substantive points, there has been some confusion over the distinction between blog posts and pages on AI Impacts. To make it clearer, this **blog post** shall proceed in a way that is silly, to distinguish it from the very serious and authoritative reference pages that comprise the bulk of AI Impacts.

Now for a picture of a duck, to remind you that this is silly, and also that we are all fragile biological organisms that evolved because apparently that’s what happens if you just leave a bunch of wet mud on a space rock for long enough, alone and vulnerable in a hostile and uncharted world.

[![](http://aiimpacts.org/wp-content/uploads/2018/07/animal-bird-cute-162142-300x161.jpg)](http://aiimpacts.org/wp-content/uploads/2018/07/animal-bird-cute-162142.jpg)

And now to the exciting facts on the ground, as we try to marginally rectify that situation.

### People

Tegan McCaslin is now working at AI Impacts, as far as I can tell about five hundred hours a week. It’s going well, except that the rate at which she sends me extensive, carefully researched articles about neuroanatomy and genetics and such to review is in slight tension with my preferred lifestyle.

We also welcome Carl Shulman as occasional consultant on everything, and reviewer of things (especially articles about neuroanatomy and genetics and such…)

Justis Mills joined us last year, to work on miscellany. He usually does one of those software related things, but in his spare time he has been [making illustrative timelines of near-term AI predictions](https://cdn.knightlab.com/libs/timeline3/latest/embed/index.html?source=1NbsZ5kiaRxTW8Jo6jkJgOkHatVQHtMqKu22WwdMrwZc&font=Default&lang=en&initial_zoom=2&height=650) and checking whether everything on AI Impacts isn’t obviously false, and fixing bits of it, and such.

We mostly-farewell Michael Wulfsohn—an Australian economist, [called to us](https://aiimpacts.org/the-tyranny-of-the-god-scenario/) from the Central Bank of Lesotho by [WaitButWhy](https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html)—who is winding up his assessment of how great avoiding human extinction might be (having already estimated how [how much of a bother it might be](https://aiimpacts.org/costs-of-extinction-risk-mitigation/)). He has gone to get a PhD, the better to save the world.

### Places

Our implicit office has moved from a spare room in my house to Tegan’s house. This is good, because she has an absurdly nice rug, and an excellent snack drawer, and it is a minor ambition of mine to head an organization which has Bay Area start-up quality snack areas.

We have also been trying out co-working with other save-the-world-something-something-AI related folks around Berkeley, which seems promising. We have also been trying out co-working with Oxford, which seems promising, but not as Bay Area convenient as we would like.

### Things we want

We [want to hire](https://aiimpacts.org/jobs/) more people. Relatedly, we would like [money](https://aiimpacts.org/donate/). We think these things would nicely complement our many brilliant and tractable [research ideas](https://aiimpacts.org/promising-research-projects/) and our ambition. We also want to have our own T-shirts, but that is on the back-burner.

### Things we got

[$100,000 from The Open Philanthropy Project](https://www.openphilanthropy.org/focus/global-catastrophic-risks/potential-risks-advanced-artificial-intelligence/ai-impacts-general-support-2018), for the next two years.

$39,000 from another donor to support several specific research projects from our [list of promising research projects](https://aiimpacts.org/promising-research-projects/).

### Projects

You can mostly see what we are up to by watching various parts of [our front page](https://aiimpacts.org/), so I shan’t go into it all, except to say that I for one am especially enjoying my investigation into reasons to (or not to) expect [AI discontinuities](https://aiimpacts.org/likelihood-of-discontinuous-progress-around-the-development-of-agi/). If you too are fascinated by this topic, and want to give especially many pointed comments on it, you can do so [on this doc version](https://docs.google.com/document/d/1Gf_G_5AYy11Xgu1O_SbpXHQ5dtpWeY8L3gIurHDfUk0/edit?usp=sharing).

Our [survey](https://aiimpacts.org/2016-expert-survey-on-progress-in-ai/) became the [16th most discussed journal article](https://www.altmetric.com/top100/2017/#list&article=20475804) of 2017, so that was neat. If I recall, I was at least relatively in favor of not writing a paper about it, so I was probably wrong there. (Probably good job, everyone else!) I suspect this success is related to the journalists who have been writing to me endlessly, and me being invited to give talks, and [go to Chile](https://worldlypositions.tumblr.com/post/170134252744/chile-vii-futures-congress) and be on the radio and that kind of thing. Which has all been an unusual experience.

### How you can get involved

If you want to do this kind of work, consider applying for a [job](https://aiimpacts.org/jobs/) with us, or just doing one of [these projects](https://aiimpacts.org/promising-research-projects/) anyway, and sending it to us. If you want to chat about this kind of research, or spy on it, or help a tiny bit noncommittally, ask us nicely and we might add you to our fairly open Slack. If you want to help in some other way, we especially welcome money and any good researchers you have hanging around, but are open to other ideas.

Occasional update July 5 2018


Trend in compute used in training for headline AI results

Compute used in the largest AI training runs appears to have roughly doubled every 3.5 months between 2012 and 2018.

Details
-------

According to [Amodei and Hernandez, on the OpenAI Blog](https://blog.openai.com/ai-and-compute/):

> …since 2012, the amount of compute used in the largest AI training runs has been increasing exponentially with a 3.5 month-doubling time (by comparison, Moore’s Law [had](https://www.nature.com/articles/s41928-017-0005-9) an 18-month doubling period). Since 2012, this metric has grown by more than 300,000x (an 18-month doubling period would yield only a 12x increase)…
>
>

They give the following figure, and some of their calculations. We have not verified their calculations, or looked for other reports on this issue.

[![](http://aiimpacts.org/wp-content/uploads/2018/05/OpenAI-AI-Compute-Trend.jpeg)](http://aiimpacts.org/wp-content/uploads/2018/05/OpenAI-AI-Compute-Trend.jpeg)**Figure 1:** Originally captioned: The chart shows the total amount of compute, in petaflop/s-days, that was used to train selected results that are relatively well known, used a lot of compute for their time, and gave enough information to estimate the compute used. A petaflop/s-day (pfs-day) consists of performing 1015 neural net operations per second for one day, or a total of about 1020 operations. The compute-time product serves as a mental convenience, similar to kW-hr for energy. We don’t measure peak theoretical FLOPS of the hardware but instead try to estimate the number of actual operations performed. We count adds and multiplies as separate operations, we count any add or multiply as a single operation regardless of numerical precision (making “FLOP” a slight misnomer), and we ignore [ensemble models](http://web.engr.oregonstate.edu/~tgd/publications/mcs-ensembles.pdf). Example calculations that went into this graph are provided in this [appendix](https://blog.openai.com/ai-and-compute/#appendixmethods). Doubling time for line of best fit shown is 3.43 months.


The tyranny of the god scenario

*By Michael Wulfsohn, 6 April 2018*

I was convinced. An intelligence explosion would result in the sudden arrival of a superintelligent machine. Its abilities would far exceed those of humans in ways we can’t imagine or counter. It would likely arrive within a few decades, and would wield complete power over humanity. Our species’ most important challenge would be to solve the [value alignment problem](https://www.fhi.ox.ac.uk/edge-article/622578843254063104/). The impending singularity would lead either to our salvation, our extinction, or worse.

Intellectually, I knew that it was not certain that this “god scenario” would come to pass. If asked, I would even have assigned it a relatively low probability, certainly much less than 50%. Nevertheless, it dominated my thinking. Other possibilities felt much less real: that humans might achieve direct control over their superintelligent invention, that reaching human-level intelligence might take hundreds of years, that there might be a slow progression from human-level intelligence to superintelligence, and many others. I paid lip service to these alternatives, but I didn’t want them to be valid, and I didn’t think about them much. My mind would always drift back to the god scenario.

I don’t know how likely the god scenario really is. With currently available information, nobody can know for sure. But whether or not it’s likely, the idea definitely has powerful intuitive appeal. For example, it led me to change my beliefs about the world more quickly and radically than I ever had before. I doubt that I’m the only one.

Why did I find the god scenario so captivating? I like science fiction, and the idea of an intelligence explosion certainly has science-fictional appeal. I was able to relate to the scenario easily, and perhaps better think through the implications. But the transition from science fiction to reality in my mind wasn’t immediate. I remember repeatedly thinking “nahhh, surely this can’t be right!” My mind was trying to put the scenario in its science-fictional place. But each time the thought occurred, I remember being surprised at the scenario’s plausibility, and at my inability to rule out any of its key components.

I also tend to place high value on intelligence itself. I don’t mean that I’ve assessed various qualities against some measure of value and concluded that intelligence ranks highly. I mean it in a personal-values sense. For example, the level of intelligence I have is a big factor in my level of self-esteem. This is probably more emotional than logical.

This emotional effect was an important part of the god scenario’s impact on me. At first, it terrified me. I felt like my whole view of the world had been upset, and almost everything people do day to day seemed to no longer matter. I would see a funny video of a dog barking at its reflection, and instead of enjoying it, I’d notice the grim analogy of the intellectual powerlessness humanity might one day experience. But apart from the fear, I was also tremendously excited by the thought of something so sublimely intelligent. Having not previously thought much about the limits of intelligence itself, the concept was both consuming and eye-opening, and the possibilities were inspiring. The notion of a superintelligent being appealed to me similarly to the way Superman’s abilities have enthralled audiences.

Other factors included that I was influenced by highly engaging prose, since I first learned about superintelligence by reading this excellent [waitbutwhy.com blog post](http://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html). Another was my professional background; I was accustomed to worrying about improbable but significant threats, and to arguments based on expected value. The concern of prominent people—Bill Gates, Elon Musk, and Stephen Hawking—helped. Also, I get a lot of satisfaction from working on whatever I think is humanity’s most important problem, so I really couldn’t ignore the idea.

But there were also countervailing effects in my mind, leading away from the god scenario. The strongest was the outlandishness of it all. I had always been dismissive of ideas that seem like doomsday theories, so I wasn’t automatically comfortable giving the god scenario credence in my mind. I was hesitant to introduce the idea to people who I thought might draw negative conclusions about my judgement.

I still believe the god scenario is a real possibility. We should assiduously prepare for it and proceed with caution. However, I believe I have gradually escaped its intuitive capture. I can now consider other possibilities without my mind constantly drifting back to the god scenario.

I believe a major factor behind my shift in mindset was my research interest in analyzing [AI safety as a global public good](https://aiimpacts.org/friendly-ai-as-a-global-public-good/). Such research led me to think concretely about other scenarios, which increased their prominence in my mind. Relatedly, I began to think I might be better equipped to contribute to outcomes in those other scenarios. This led me to want to believe that the other scenarios were more likely, a desire compounded by the danger of the god scenario. My personal desires may or may not have influenced my objective opinion of the probabilities. But they definitely helped counteract the emotional and intuitive appeal of the god scenario.

Exposure to mainstream views on the subject also moderated my thinking. In one instance, reading an Economist [special report](http://www.economist.com/news/special-report/21700762-techies-do-not-believe-artificial-intelligence-will-run-out-control-there-are) on artificial intelligence helped counteract the effects I’ve described, despite that I actually disagreed with most of their arguments against the importance of existential risk from AI.

Exposure to work done by the Effective Altruism community on different future possibilities also helped, as did my discussions with Katja Grace, Robin Hanson, and others during my work for AI Impacts. The exposure and discussions increased my knowledge and the sophistication of my views such that I could better imagine the range of AI scenarios. Similarly, listening to Elon Musk’s views of the importance of developing brain-computer interfaces, and seeing OpenAI pursue goals that may not squarely confront the god scenario, also helped. They gave me a choice: decide without further ado that Elon Musk and OpenAI are misguided, or think more carefully about other potential scenarios.

**Relevance to the cause of AI safety**

I believe the AI safety community probably includes many people who experience the god scenario’s strong intuitive appeal, or have previously experienced it. This tendency may be having some effects on the field.

Starting with the obvious, such a systemic effect could cause pervasive errors in decision-making. However, I want to make clear that I have no basis to conclude that it has done so among the Effective Altruism community. For me, the influence of the god scenario was subtle, and driven by its emotional facet. I could override it when asked for a rational assessment of probabilities. But its influence was pervasive, affecting the thoughts to which my mind would gravitate, the topics on which I would tend to generate ideas, and what I would feel like doing with my time. It shaped my thought processes when I wasn’t looking.

Preoccupation with the god scenario may also entail a public relations risk. Since the god scenario’s strong appeal is not universal, it may polarize public opinion, as it can seem bizarre or off-putting to many. At worst, a rift may develop between the AI safety community and the rest of society. This matters. For example, policymakers throughout the world have the ability to promote the cause of AI safety through funding and regulation. Their involvement is probably an essential component of efforts to prevent an AI arms race through international coordination. But it is easier for them to support a cause that resonates with the public.

Conversely, the enthusiasm created by the intuitive appeal of the god scenario can be quite positive, since it attracts attention to related issues in AI safety and existential risk. For example, others’ enthusiasm and work in these areas led me to get involved.

I hope readers will share their own experience of the intuitive appeal of the god scenario, or lack thereof, in the comments. A few more data points and insights might help to shed light.

The tyranny of the god scenario


Promising research projects

This is an incomplete list of concrete projects that we think are tractable and important. We may do any of them ourselves, but many also seem feasible to work on independently. Those we consider especially well suited to this are marked Ψ. More potential projects are listed [here](http://aiimpacts.org/possible-investigations/).

### Project

**Review the literature on forecasting (in progress) Ψ**

Summarize what is known about procedures that produce good forecasts, and measures that are relatively easier to forecast. This may involve reading secondary sources, or collecting past forecasts and investigating what made some of them successful.

This would be an input to improving our own forecasting practices, and to knowing which other forecasting efforts to trust.

We have [reviewed](http://aiimpacts.org/evidence-on-good-forecasting-practices-from-the-good-judgment-project/) some literature associated with the Good Judgment Project in particular.

**Review considerations regarding the chance of local, fast takeoff Ψ**

We have a list of considerations [here](http://aiimpacts.org/likelihood-of-discontinuous-progress-around-the-development-of-agi/). If you find local, fast take-off likely, check if the considerations that lead you to this view are represented. Alternately, interview someone else with a strong position about the considerations they find important. If there are any arguments or counterarguments that you think are missing, write a short page explaining the case.

Collecting arguments on this topic is helpful because opinion among well-informed thinkers on the topic seems to diverge from what would be expected given the arguments that we know about. This suggests that we are missing some important considerations, that we would need to well assess the chance of local, fast takeoff.

**Quantitatively model an intelligence explosion Ψ**

An intelligence explosion (or ‘recursive self-improvement’) consists of a feedback loop where researcher efforts produce scientific progress, which produces improved AI performance, which produces more efficient researcher efforts. This forms a loop, because the researchers involved are artificial themselves.

Though this loop does not yet exist, relatively close analogues to all of the parts of it already occur: for instance, researcher efforts do lead to scientific progress; scientific progress does lead to better AI; better AI does lead to more capacity at the kinds of tasks that AI can do.

Collect empirical measurements of proxies like these, for different parts of the hypothesized loop (each part of this could be a stand-alone project). Model the speed of the resulting loop if they were put together, under different background conditions.

This would give us a very rough estimate of the contribution of intelligence explosion dynamics to the speed of intelligence growth in a transition to an AI-based economy. Also, a more detailed model may inform our understanding of available strategies to improve outcomes.

**Interview AI researchers on topics of interest Ψ**

Find an AI researcher with views on matters of interest (e.g. AI risk, timelines, the relevance of neuroscience to AI progress) and interview them. Write a summary, or transcript (with their permission). Some examples [here](http://aiimpacts.org/conversation-with-steve-potter/), [here](http://aiimpacts.org/conversation-with-tom-griffiths/), [here](http://aiimpacts.org/joscha-bach-on-the-unfinished-steps-to-human-level-ai/). (If you do not expect to run an interview well enough to make a good impression on the interviewee, consider practicing elsewhere first, so as not to discourage interacting with similar researchers in the future.)

Talking to AI researchers about their views can be informative about the nature of AI research (e.g. What problems are people trying to solve? [How much does it seem like hardware matters?](http://aiimpacts.org/2016-expert-survey-on-progress-in-ai/)), and provide an empirically informed take on questions and considerations of interest to us (e.g. [Current techniques seem really far from general](http://aiimpacts.org/what-do-ml-researchers-think-you-are-wrong-about/)). They also tell us about state of opinion within the AI research community, which may be relevant in itself.

**Review what is known about the relative intelligence of humans, chimps, and other animals (in progress)**

Review efforts to measure animal and human intelligence on a single scale, and efforts to quantify narrower cognitive skills across a range of animals.

Humans are radically more successful than other animals, in some sense. This is [taken as reason](http://aiimpacts.org/likelihood-of-discontinuous-progress-around-the-development-of-agi/) to expect that small modifications to brain design (for instance whatever evolution did between the similar brains of chimps and humans) can produce outsized gains in some form of mental performance, and thus that AI researchers may see similar astonishing progress near human-level AI.

However without defining or quantifying the mental skills of any relevant animals, it is unclear a) whether *individual intelligence* in particular accounts for humans’ success (rather than e.g. ability to accrue culture and technology), b) whether the gap in capabilities between chimps and humans is larger than expected (maybe chimps are also astonishingly smarter than smaller mammals), or c) whether the success stems from something that evolution was ‘intentionally’ progressing on. These things are all relevant to the strength of an argument for AI ‘fast take-off’ based on human success over chimps (see [here](http://aiimpacts.org/likelihood-of-discontinuous-progress-around-the-development-of-agi/)).

**Review explanations for humans’ radical success over apes (in progress)**

Investigate what is known about the likely causes of human success, relative to that of other similar animals. In particular, we are interested in how likely improvement in individual cognitive ability is to account for this (as opposed to say communication and group memory abilities).

This would help resolve the same issues described in the last section (‘Review what is known about the relative intelligence of humans, chimps, and other animals’).

**Collect data on time to cross the human range on intellectual skills where machines have surpassed us (in progress) Ψ**

For intellectual skills where machines have surpassed humans, find out how long it took to go from the worst performance to average human skill, and from average human skill to superhuman skill.

This would contribute to [this project](http://aiimpacts.org/is-the-range-of-human-intelligence-small/).

**Measure the importance of hardware progress in a specific narrow AI trajectory Ψ**

Take an area of AI progress, and assess how much of annual improvement can be attributed to hardware improvements vs. software improvements, or what the more detailed relationship between the two is.

Understanding the overall importance of hardware progress and software progress (and other factors) in overall AI progress lets us know to what extent our future expectations should be a function of expected hardware developments, versus software developments. This both alters what our timelines look like (e.g. see [here](http://aiimpacts.org/human-level-hardware-timeline/)), and tells us what we should be researching to better understand AI timelines.

Promising research projects


Brain wiring: The long and short of it

*By Tegan McCaslin, 30 March 2018*

When I took on the task of counting up all the brain’s fibers and figuratively laying them end-to-end, I had a sense that it would be relatively easy–do a bit of strategic Googling, check out a few neuroscience references, and you’ve got yourself a relaxing Sunday afternoon project. By that afternoon project’s 40th hour, I had begun to question my faith in Paul Christiano’s project length estimates.

It was actually pretty surprising how thin on the ground numbers and quantities about the metrics I was after seemed to be. Even somewhat simple questions, like “how many of these neuron things does the brain even have”, proved not to have the most straightforward answer. According to [one author](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2776484/), the widely-cited, rarely-caveated figure in textbooks of 100 billion neurons couldn’t be sourced in any primary literature published before the late 2000s, and this echo chamber-derived estimate was subsequently denounced for being off by tens of billions in either direction (depending on who you ask). But hey, what’s a few tens of billions between friends?

The question of why these numbers are so hard to find is an interesting one. One answer is that it’s genuinely difficult to study populations of cells at the required level of detail. Another is that perhaps neuroscientists are too busy studying silly topics like “how the brain works” or “clinically relevant things” to get down to the real meat of science, which is anal-retentively cataloging every quantity that could plausibly be measured. Perhaps the simplest explanation is just that questions like “how long is the entire dendritic arbor of a Purkinje cell” didn’t have a great argument for why they might be useful, prior to now.

[![](http://aiimpacts.org/wp-content/uploads/2018/03/Purkinje_cell_by_Cajal.png)](http://aiimpacts.org/wp-content/uploads/2018/03/Purkinje_cell_by_Cajal.png)*Or maybe its “fuck it”-quotient was too high.*
Which brings us rather neatly to the point of why an AI forecasting organization might care about the length of all the wires in the brain, even when the field of neuroscience seems not to. At a broad level, it’s probably the case that neuroscientists care about very different aspects of the brain than AI folks do, because neuroscientists mostly aren’t trying to solve an engineering problem (at least, not the engineering problem of “build a brain out of bits of metal and plastic”). The particular facet of that engineering problem we were interested in here was: how much of a hurdle is hauling information around going to be, once computation is taken care of?

Our length estimates don’t provide an exhaustive answer to that question, and to be honest they can’t really tell you anything on their own. But, as is the case with AI Impacts’ 2015 article on [brain performance in TEPS](https://aiimpacts.org/brain-performance-in-teps/), learning these facts about the brain moves us incrementally closer to understanding how promising our current models of hardware architectures are, and where we should expect to encounter trouble.

Some interesting takeaways: long-range fibers–that is, myelinated ones–probably account for about 20% of the total length of brain wires. Also, the neocortex is *huge*, but not because it has lots of neurons.

Scroll to the bottom of [our article](https://aiimpacts.org/transmitting-fibers-in-the-brain-total-length-and-distribution-of-lengths/) if you’re a cheater who just wants to see a table full of summary statistics, but read the whole thing if you want those numbers to have some context. And please contact me if you spot anything wrong, or think I missed something, or if you’re Wen or Chklovskii of [Wen and Chklovskii 2004](http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.0010078#pcbi-0010078-g002) and you want to explain your use of tildes in full detail.

Brain wiring: The long and short of it


Transmitting fibers in the brain: Total length and distribution of lengths

The human brain’s approximately 86 billion neurons are probably connected by something like 850,000 km of axons and dendrites. Of this total, roughly 80% is short-range, local connections (averaging 680 microns in length), and approximately 20% is long-range, global connections in the form of myelinated fibers (likely averaging several centimeters in length).

Background
----------

The brain’s precisely coordinated action relies on a dense network of fibers capable of rapidly transmitting information, both locally (to adjacent neurons whose separation can be measured in microns) and to distant locations removed by many centimeters from one another. And while manipulation of that information–”computation”–is an important component of what the brain does, it would be hard-pressed to make any use of that computational power without the ability to communicate within itself.

So how much of a problem does the need for moving around information pose for brains and, by extension, brain-like computers? It’s clear from a cursory physical examination of the brain that evolution has prioritized information transfer, since the vast majority of brain tissue is taken up by the tendrils of axons and dendrites snaking through a convoluted maze of cables. Some proportion of these form short-range connections with neurons that are both nearby in physical space, and are probably also close in “functional space” as well. The rest are long-range fibers, which move information from these local, functionally similar regions to areas separated by significant physical, and likely also functional, distance. Whether we can expect one type of connection or the other to impose a larger cost on hardware, as well as the transferability of total brain fiber length to communication requirements in hardware, depends largely on the kind of hardware in question. One can imagine both types of brain-mimicking computer architecture that might make long-distance communication the main limiting factor, as well as architectures where long-distance communication was trivial compared to short-distance communication.

AI Impacts [previously](https://aiimpacts.org/brain-performance-in-teps/) estimated brain communication costs in terms of the benchmark TEPS, or “traversed edges per second”, where “edges” corresponded roughly to synaptic connections between neurons. However, this benchmark measures performance in a certain family of graphs that may not be very representative of connectivity patterns in the brain. Characterizing the actual topology of connections in the brain, especially the proportions contributed by long and short fibers, may give us a more informative picture of the capacities hardware will need in order to mimic wetware.

Short fibers
------------

Our estimates for length and length distribution of short fibers were found by comparing the results of what might be called “top-down” and “bottom-up” approaches. Directly measuring any cell-level metric for the entire brain is challenging, but two substantially different methodologies converging on similar answers is probably a reasonable substitute for direct measurement. The first of these relied on observations of fiber density in the neocortex of rats, which there is reason to believe translates fairly well to the human brain as a whole. The second required gathering morphology data on various types of human neurons, then adjusting for the proportion of each cell type in the brain.

#### Some important notes on brain structure and animal models

In this section, our estimates were drawn from only two brain regions: the cerebral cortex alone in the first case, and the cerebral cortex and cerebellar cortex in the second. However, taken together, these regions account for roughly 85% of total brain volume and as many as 99% of all brain neurons in humans, making this a safe approximation for all gray matter (which represents short connections–see [here](https://aiimpacts.org/transmitting-fibers-in-the-brain-total-length-and-distribution-of-lengths/#Myelination_as_indicative_of_fiber_length)) in the brain.

Since the first case considers the brains of rats rather than humans, it may seem to have little utility, but in fact the composition of tissue in rats’ neocortex differs from ours in only [a few predictable ways](https://link.springer.com/book/10.1007/978-3-662-03733-1). There are more neurons per cubic millimeter in the cortex of small animals [(3-10x)](http://www.scholarpedia.org/article/Brain), meaning that somewhat more of the brain’s volume is taken up by cell bodies, slightly decreasing the density of fibers compared to larger brains. However, cell bodies are measured in the tens of microns, so this is unlikely to bear on our conclusions.

### Total length from neocortical fiber density

While the cerebral cortex comprises [82%](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2776484/) of the volume of the human brain, only [19%](https://en.wikipedia.org/wiki/Human_brain#Microanatomy) of the brain’s 86 billion neurons reside here, cushioned in the dense web of axons and dendrites known as [“neuropil”](http://www.scholarpedia.org/article/Neuroanatomy#The_Neuropil). The amount of neuropil packed into any given tissue sample can give us a sense of the lengths of these fibers per unit volume, as long as we also know their diameters.

After determining rat neocortical fiber density, [Braitenberg and Schüz (1998)](https://link.springer.com/book/10.1007/978-3-662-03733-1) concluded that the total length of the average neuron’s axonal tree was between 10 and 40 mm, and that the average dendritic tree came out to 4 mm. These numbers were derived from examining electron micrographs of tissue samples to find the proportion of area taken up by axons and dendrites, then measuring the average diameter of these fibers to find an axonal density of 4 km per mm^3, and a dendritic density of 456 m per mm^3. It’s not quite clear to us how the authors got from these numbers to average fiber length per neuron, but since their average values agreed with values we obtained by other methods (see below), we were inclined to assume their process was reasonable.

Assuming mouse neocortical neurons are comparable to human neurons, their average fiber length suggests that the neocortex alone contains at least **220,000 km** of short range connections between dendrites and axons.[1](https://aiimpacts.org/transmitting-fibers-in-the-brain-total-length-and-distribution-of-lengths/#easy-footnote-bottom-1-1118 "16 billion neocortical neurons x (minimum total axonal length per neuron (10 mm) + total dendritic length per neuron (4mm))")

### Total length and distribution of lengths from morphological data

In principle, obtaining estimates of average fiber lengths from a representative sample of different varieties of neuron should yield something close to the sum total fiber length for all brain neurons, when combined with information about the neuronal composition of the brain.

#### Granule cells

The most numerous neuron type in the brain is the cerebellar granule cell, at around [50 billion](http://www.scholarpedia.org/article/Cerebellum) (58% of the brain’s total neurons). These small cells have three to five unbranched dendrites, each around 15 microns long and appended by a “claw”. They’re primarily distinguished by their unusual axonal morphology, which extends from the lowest of the three cerebellar cortex layers to the outermost layer, then splits perpendicularly into two fibers, forming a “T”. The fibers forming the top of this “T” run an average of 6 mm total, and while it was difficult to find a direct measurement for the other axonal component, the number is bounded by the thickness of the cerebellar cortex at [1176 microns](https://www.researchgate.net/figure/Comparative-analysis-of-cerebellar-cortex-thickness-m_fig1_6417166), and is probably much shorter on average. Overall, the fiber length of the average cerebellar granule cell is probably in the neighborhood of 6.6 mm, giving us around **330,000 km in total**.[2](https://aiimpacts.org/transmitting-fibers-in-the-brain-total-length-and-distribution-of-lengths/#easy-footnote-bottom-2-1118 "50 billion cerebellar granule cells x (total parallel fiber length per neuron (6 mm) + other axonal component length per neuron (~0.6 mm) + total dendritic length per neuron (0.045 mm))")

#### Pyramidal cells

The next most numerous neuron type, at around [2/3rds](http://www.cell.com/current-biology/pdf/S0960-9822(11)01198-5.pdf) to [85%](https://link.springer.com/book/10.1007/978-3-662-03733-1) of the cerebral cortex (or 10.5-13.6 billion cells in humans), is the well-studied pyramidal cell. Pyramidal cells are in close contact with their neighbors in the vertical direction, forming tiny “[columns](https://en.wikipedia.org/wiki/Cortical_column)” along the cerebral cortex that are thought to have functional relevance, with relatively less connectivity between columns. This is reflected in the structure of the pyramidal cell’s dendritic tree, with a long fiber extending vertically from the cell body (the apical dendrite) and several relatively short fibers branching laterally (the basal dendrites). Some pyramidal cells have long, myelinated axons that connect the two hemispheres or different functional areas of the same hemisphere, and these axons will be considered in the next section on long fibers, but for now we will focus exclusively on more local connections.

Quantitative descriptions of pyramidal cell morphology were lacking, so we collected data on 2130 human pyramidal neurons from the [NeuroMorpho.org](http://neuromorpho.org/) database, computing various metrics for each neuron using [L-Measure](http://cng.gmu.edu:8080/Lm/), and then performed our analysis with R ([data here](https://drive.google.com/file/d/1WCmXkUr0cBqV6aRIKryODQ5SLHFzWq9d/view?usp=sharing)).

Dendritic trees had an average total length of 3.4 mm per cell, with a standard deviation of 1.8 mm. We also analyzed path distance, or the length between the terminal point of one branch and the soma. The average path distance of pyramidal dendrites’ longest branch was 340 microns, which likely corresponds to the apical dendrites, while a typical branch was 180 microns. Axons were vastly less well represented in our dataset–only 243 had nonzero values, and while the mean length for these axons was in the same ballpark as the estimate found by Braitenberg and Schüz, at 11.5 mm, it’s probable that not all axons in the dataset were complete.[3](https://aiimpacts.org/transmitting-fibers-in-the-brain-total-length-and-distribution-of-lengths/#easy-footnote-bottom-3-1118 "This data set was coded to specify the integrity of cell compartments (dendrites, soma, axon), and a very high proportion of axons were coded as incomplete. However, this coding was not reliable enough to filter the data effectively, so all non-zero axons were included in the first pass analysis.") In particular, the length distribution was bimodal, with maxima around 100 microns and 20 mm, and this latter number may be the more accurate. This bimodal distribution was also reflected in the path distance of axonal branches, with (what we suspect to be) the more realistic values around 2.2 mm average for each neuron, and 4 mm for the longest branches. In all, pyramidal cells as measured here probably contribute roughly 23 mm each to the brain’s fiber network, or ~**240,000 to ~310,000 km**, in basic agreement with the numbers obtained from neocortical fiber density.[4](https://aiimpacts.org/transmitting-fibers-in-the-brain-total-length-and-distribution-of-lengths/#easy-footnote-bottom-4-1118 "Number of pyramidal cells (10.5 to 13.6 billion) x (total dendritic length per neuron (~3 mm) + total axonal length per neuron (~20 mm))")

#### Other cell types

The remaining cell types made up a much smaller proportion of total fiber length. Besides pyramidal neurons, stellate cells are the other primary residents of the cerebrum, and are known to be substantially smaller than their cortical comrades, with axonal projections no longer than their dendrites. They could therefore add no more than 9,600-22,000 km to the total.[5](https://aiimpacts.org/transmitting-fibers-in-the-brain-total-length-and-distribution-of-lengths/#easy-footnote-bottom-5-1118 "Number of stellate cells (2.4 to 5.5 billion) x (estimated total dendritic and axonal length per neuron (4mm))") After the 50 billion granule cells, the cerebellum still has 13-20 billion neurons to account for, [over half of which are small stellate cells, a quarter basket cells, and the remainder split evenly between Purkinje cells and Golgi cells](http://www.scholarpedia.org/article/Cerebellum). Between them, these cerebral and cerebellar cells probably contribute **65,000 to 110,000 km**.[6](https://aiimpacts.org/transmitting-fibers-in-the-brain-total-length-and-distribution-of-lengths/#easy-footnote-bottom-6-1118 "Total length of cerebellar stellate cells (8.5 to 13 billion cells x 4 mm) + total length of basket cells (3.25 billion to 5 billion cells x ~4 mm) + total length of Purkinje cells (0.6 to 1 billion cells x ~10 mm) + total length of Golgi cells (0.6 to 1 billion cells x ~4 mm) + total length of cerebral stellate cells (see previous footnote); Note that length estimations preceded by a tilde are a rough guess based on the size of the neuron. Because these cells were so few in number, a high degree of precision was not expected to improve our overall estimate very much.")

Long fibers
-----------

#### Myelination as indicative of fiber length

The most natural point of transition from “short range” to “long range” is the length of fiber for which conduction velocity of action potentials in a bare axon becomes unacceptably slow. Rather conveniently, this demarcation is evident from a glance at a cross section of the brain, where the white of myelinated fibers stands in stark contrast to the gray matter.

Fatty insulating sheaths of myelin are used by the brain’s longest fibers to increase conduction velocity at the cost of taking up more volume in the brain, as well as rendering myelinated segments of axons unable to synapse onto nearby neurons. Frequently, axons running through white matter tracts bundle with others inside a single myelin sheath, a frugal move for a brain with space and energy constraints. It’s unlikely that in these circumstances a brain would expend resources to myelinate short connections with no great need for it, so it’s reasonable to assume that all myelinated fibers are long. Furthermore, gray and white matter are [highly segregated](https://onlinelibrary.wiley.com/doi/abs/10.1002/cne.10714), and myelin is rarely found in cortical tissue.

### Length of myelinated fibers from white matter volume

This protective myelin coating not only insulates axons from lost transmission, but also, unfortunately, from the prying eyes of scientists. This means that long distance connections are difficult to study, and there have been few attempts to characterize white matter fibers at an appropriate level of detail for our purposes.

One frequently cited figure comes from [Marner et al 2003](http://onlinelibrary.wiley.com/doi/10.1002/cne.10714/abstract), where the method called for divvying up preserved brains into slabs and take needle biopsies from random points on the slabs, then slicing these biopsies into fine sections and staining them. These could then be inspected for dark colored rings corresponding to myelin sheaths, and the total length of fibers could be approximated by multiplying “length density”, or total length of fibers per volume of white matter, with white matter volume. This method yielded a total of **149,000 km of myelinated fibers in female brains**, and **176,000 km in males**.

As for the distribution of these fiber lengths in the brain, we’re left somewhat in the dark. A very imprecise estimate for a portion of them can be gotten from a few key facts about the cerebrum. The largest and most famous white matter tract in the human brain is the corpus callosum, which connects the two hemispheres and contains 200-250 million fibers, [about as many](https://en.wikipedia.org/wiki/White_matter#White_matter) as one can find in tracts connecting areas within hemispheres. Given the width of the corpus callosum (~100 mm, or two thirds of the brain’s total width), a reasonable value for average fiber length in this tract is 10 cm, suggesting that perhaps **50,000 km** or less of long-range fiber connects the cerebrum with itself.[7](https://aiimpacts.org/transmitting-fibers-in-the-brain-total-length-and-distribution-of-lengths/#easy-footnote-bottom-7-1118 "(250 million callosal fibers + 250 million intrahemispheric fibers) x axon length (~10 cm)") Clearly, this leaves much white matter to be accounted for, which can presumably be attributed to connections within and between the cerebellum and subcortical structures, as well as the occasional cerebral white matter found outside the white matter tracts.

This vague picture can be supplemented by the relationship between long-range connection length and brain volume alluded to in [Wen and Chklovskii 2004](http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.0010078#pcbi-0010078-g002). These authors estimate that average global connection length should be roughly similar to the cube root of [brain volume](https://hypertextbook.com/facts/2001/ViktoriyaShchupak.shtml), or 10.6 cm – 11.4 cm, much like the figure we approximated above for intracortical connections.

Discussion
----------

### Summary of conclusions

Our estimates are aggregated in the table below:

| Connection type | Total length (km) | Average length per neuron (mm) | Contributing neuron types | Sources of evidence |
| --- | --- | --- | --- | --- |
| Cerebral, short-range | 220,000 - 320,000[9](https://aiimpacts.org/transmitting-fibers-in-the-brain-total-length-and-distribution-of-lengths/#easy-footnote-bottom-9-1118 "Lower bound is from neocortical fiber density estimate; Upper bound is from pyramidal cells + cerebral stellate cells") | 14 - 20 | Pyramidal (2/3rds to 85%), stellate | Fiber density in rats, morphometry |
| Cerebellar, short-range | 390,000 - 420,000[8](https://aiimpacts.org/transmitting-fibers-in-the-brain-total-length-and-distribution-of-lengths/#easy-footnote-bottom-8-1118 "Estimate is from cerebellar granule cells + the cerebellar portion of “other cell types”") | 5.7 - 6.1 | Granule (~70%), stellate, basket, Purkinje, Golgi | Morphometry |
| Total, short-range | 610,000 - 740,000 | - | - | - |
| Cerebral, long-range | ~50,000 | 100 | Pyramidal | Width of corpus callosum, relationship between brain volume and global connection length |
| Total, long-range | 150,000 - 180,000 | ? | ? | Length density per white matter volume |
| Total, all fibers | 760,000 - 920,000 | - | - | - |

Overall, we’re somewhat less confident in our total for long-range fiber length than our other estimates, since this was obtained using a methodology whose reliability we’re not able to judge, and its findings couldn’t be directly corroborated with other methods. However, there is indirect evidence that these numbers will hold up reasonably well: the proportion of total cerebral wiring that cerebral long-distance connections account for (14%) is quite similar to the proportion that long-distance connections purportedly account for overall (20%), despite the former number coming from independent lines of evidence.

### Implications and future directions

The brain is the most metabolically expensive organ in the human body by volume, and has pushed the limits of natural birth by increasing the pelvic width of human females, via enlarged infant head sizes, to the edge of feasibility for walking. The massive resource requirements of the brain are clear, but the proportion demanded by communication (versus computation) is less clear.

Costs to the brain can be expressed in terms of space, energy (for development, maintenance and operation), and the difficulty or error-proneness of orchestrating complex activities. Space may be the cost most strongly influenced by brain wiring, and can easily be predicted to translate to computers, but the amount brain wiring also contributes to energy costs. In computers, this will take the form of operation energy, or the power needed to send “action potentials” along connections.

By themselves, our estimates of fiber lengths in the brain won’t answer any questions about the difficulty of communication in computers broadly. However, they can be informative when considering a specific hardware architecture, and are likely to be especially so in the case of massively parallel architectures. Combining our estimates with other estimates relating to information transfer in the brain, like information density, may also yield insights relevant to AI hardware.

Contributions
-------------

*Research, analysis and writing were done by Tegan McCaslin. Katja Grace contributed feedback and editing. Paul Christiano proposed the question and provided guidance on hardware-related matters.*

Footnotes
---------


Will AI see sudden progress?

*By Katja Grace, 24 February 2018*

Will advanced AI let some small group of people or AI systems take over the world?

AI X-risk folks and others have accrued lots of arguments about this over the years, but I think this debate has been disappointing in terms of anyone changing anyone else’s mind, or much being resolved. I still have hopes for sorting this out though, and I thought a written summary of the evidence we have so far (which often seems to live in personal conversations) would be a good start, for me at least.

To that end, I started a [collection of reasons to expect discontinuous progress near the development of AGI](http://aiimpacts.org/likelihood-of-discontinuous-progress-around-the-development-of-agi/).

I do think the world could be taken over without a step change in anything, but it seems less likely, and we can talk about the arguments around that another time.

Paul Christiano had basically the same idea at the same time, so for a slightly different take, [here](https://sideways-view.com/2018/02/24/takeoff-speeds/) is his account of reasons to expect slow or fast take-off.

Please tell us in the comments or [feedback box](https://aiimpacts.org/feedback/) if your favorite argument for AI Foom is missing, or isn’t represented well. Or if you want to represent it well yourself in the form of a short essay, and send it to me [here](mailto:katja@intelligence.org), and we will gladly consider posting it as a guest blog post.

I’m also pretty curious to hear which arguments people actually find compelling, even if they are already listed. I don’t actually find any of the ones I have that compelling yet, and I think a lot of people who have thought about it do expect ‘local takeoff’ with at least substantial probability, so I am probably missing things.

Will AI see sudden progress?