Transmitting fibers in the brain: Total length and distribution of lengths

The human brain's approximately 86 billion neurons are probably connected by something like 850,000 km of axons and dendrites. Of this total, roughly 80% is short-range, local connections (averaging 680 microns in length), and approximately 20% is long-range, global connections in the form of myelinated fibers (likely averaging several centimeters in length).
Background
The brain's precisely coordinated action relies on a dense network of fibers capable of rapidly transmitting information, both locally (to adjacent neurons whose separation can be measured in microns) and to distant locations removed by many centimeters from one another. And while manipulation of that information–"computation"–is an important component of what the brain does, it would be hard-pressed to make any use of that computational power without the ability to communicate within itself.
So how much of a problem does the need for moving around information pose for brains and, by extension, brain-like computers? It's clear from a cursory physical examination of the brain that evolution has prioritized information transfer, since the vast majority of brain tissue is taken up by the tendrils of axons and dendrites snaking through a convoluted maze of cables. Some proportion of these form short-range connections with neurons that are both nearby in physical space, and are probably also close in "functional space" as well. The rest are long-range fibers, which move information from these local, functionally similar regions to areas separated by significant physical, and likely also functional, distance. Whether we can expect one type of connection or the other to impose a larger cost on hardware, as well as the transferability of total brain fiber length to communication requirements in hardware, depends largely on the kind of hardware in question. One can imagine both types of brain-mimicking computer architecture that might make long-distance communication the main limiting factor, as well as architectures where long-distance communication was trivial compared to short-distance communication.
AI Impacts previously estimated brain communication costs in terms of the benchmark TEPS, or "traversed edges per second", where "edges" corresponded roughly to synaptic connections between neurons. However, this benchmark measures performance in a certain family of graphs that may not be very representative of connectivity patterns in the brain. Characterizing the actual topology of connections in the brain, especially the proportions contributed by long and short fibers, may give us a more informative picture of the capacities hardware will need in order to mimic wetware.
Short fibers
Our estimates for length and length distribution of short fibers were found by comparing the results of what might be called "top-down" and "bottom-up" approaches. Directly measuring any cell-level metric for the entire brain is challenging, but two substantially different methodologies converging on similar answers is probably a reasonable substitute for direct measurement. The first of these relied on observations of fiber density in the neocortex of rats, which there is reason to believe translates fairly well to the human brain as a whole. The second required gathering morphology data on various types of human neurons, then adjusting for the proportion of each cell type in the brain.
Some important notes on brain structure and animal models
In this section, our estimates were drawn from only two brain regions: the cerebral cortex alone in the first case, and the cerebral cortex and cerebellar cortex in the second. However, taken together, these regions account for roughly 85% of total brain volume and as many as 99% of all brain neurons in humans, making this a safe approximation for all gray matter (which represents short connections–see here) in the brain.
Since the first case considers the brains of rats rather than humans, it may seem to have little utility, but in fact the composition of tissue in rats' neocortex differs from ours in only a few predictable ways. There are more neurons per cubic millimeter in the cortex of small animals (3-10x), meaning that somewhat more of the brain's volume is taken up by cell bodies, slightly decreasing the density of fibers compared to larger brains. However, cell bodies are measured in the tens of microns, so this is unlikely to bear on our conclusions.
Total length from neocortical fiber density
While the cerebral cortex comprises 82% of the volume of the human brain, only 19% of the brain's 86 billion neurons reside here, cushioned in the dense web of axons and dendrites known as "neuropil". The amount of neuropil packed into any given tissue sample can give us a sense of the lengths of these fibers per unit volume, as long as we also know their diameters.
After determining rat neocortical fiber density, Braitenberg and Schüz (1998) concluded that the total length of the average neuron's axonal tree was between 10 and 40 mm, and that the average dendritic tree came out to 4 mm. These numbers were derived from examining electron micrographs of tissue samples to find the proportion of area taken up by axons and dendrites, then measuring the average diameter of these fibers to find an axonal density of 4 km per mm^3, and a dendritic density of 456 m per mm^3. It's not quite clear to us how the authors got from these numbers to average fiber length per neuron, but since their average values agreed with values we obtained by other methods (see below), we were inclined to assume their process was reasonable.
Assuming mouse neocortical neurons are comparable to human neurons, their average fiber length suggests that the neocortex alone contains at least 220,000 km of short range connections between dendrites and axons.1
Total length and distribution of lengths from morphological data
In principle, obtaining estimates of average fiber lengths from a representative sample of different varieties of neuron should yield something close to the sum total fiber length for all brain neurons, when combined with information about the neuronal composition of the brain.
Granule cells
The most numerous neuron type in the brain is the cerebellar granule cell, at around 50 billion (58% of the brain's total neurons). These small cells have three to five unbranched dendrites, each around 15 microns long and appended by a "claw". They're primarily distinguished by their unusual axonal morphology, which extends from the lowest of the three cerebellar cortex layers to the outermost layer, then splits perpendicularly into two fibers, forming a "T". The fibers forming the top of this "T" run an average of 6 mm total, and while it was difficult to find a direct measurement for the other axonal component, the number is bounded by the thickness of the cerebellar cortex at 1176 microns, and is probably much shorter on average. Overall, the fiber length of the average cerebellar granule cell is probably in the neighborhood of 6.6 mm, giving us around 330,000 km in total.2
Pyramidal cells
The next most numerous neuron type, at around 2/3rds to 85% of the cerebral cortex (or 10.5-13.6 billion cells in humans), is the well-studied pyramidal cell. Pyramidal cells are in close contact with their neighbors in the vertical direction, forming tiny "columns" along the cerebral cortex that are thought to have functional relevance, with relatively less connectivity between columns. This is reflected in the structure of the pyramidal cell's dendritic tree, with a long fiber extending vertically from the cell body (the apical dendrite) and several relatively short fibers branching laterally (the basal dendrites). Some pyramidal cells have long, myelinated axons that connect the two hemispheres or different functional areas of the same hemisphere, and these axons will be considered in the next section on long fibers, but for now we will focus exclusively on more local connections.
Quantitative descriptions of pyramidal cell morphology were lacking, so we collected data on 2130 human pyramidal neurons from the NeuroMorpho.org database, computing various metrics for each neuron using L-Measure, and then performed our analysis with R (data here).
Dendritic trees had an average total length of 3.4 mm per cell, with a standard deviation of 1.8 mm. We also analyzed path distance, or the length between the terminal point of one branch and the soma. The average path distance of pyramidal dendrites' longest branch was 340 microns, which likely corresponds to the apical dendrites, while a typical branch was 180 microns. Axons were vastly less well represented in our dataset–only 243 had nonzero values, and while the mean length for these axons was in the same ballpark as the estimate found by Braitenberg and Schüz, at 11.5 mm, it's probable that not all axons in the dataset were complete.3 In particular, the length distribution was bimodal, with maxima around 100 microns and 20 mm, and this latter number may be the more accurate. This bimodal distribution was also reflected in the path distance of axonal branches, with (what we suspect to be) the more realistic values around 2.2 mm average for each neuron, and 4 mm for the longest branches. In all, pyramidal cells as measured here probably contribute roughly 23 mm each to the brain's fiber network, or ~240,000 to ~310,000 km, in basic agreement with the numbers obtained from neocortical fiber density.4
Other cell types
The remaining cell types made up a much smaller proportion of total fiber length. Besides pyramidal neurons, stellate cells are the other primary residents of the cerebrum, and are known to be substantially smaller than their cortical comrades, with axonal projections no longer than their dendrites. They could therefore add no more than 9,600-22,000 km to the total.5 After the 50 billion granule cells, the cerebellum still has 13-20 billion neurons to account for, over half of which are small stellate cells, a quarter basket cells, and the remainder split evenly between Purkinje cells and Golgi cells. Between them, these cerebral and cerebellar cells probably contribute 65,000 to 110,000 km.6
Long fibers
Myelination as indicative of fiber length
The most natural point of transition from "short range" to "long range" is the length of fiber for which conduction velocity of action potentials in a bare axon becomes unacceptably slow. Rather conveniently, this demarcation is evident from a glance at a cross section of the brain, where the white of myelinated fibers stands in stark contrast to the gray matter.
Fatty insulating sheaths of myelin are used by the brain's longest fibers to increase conduction velocity at the cost of taking up more volume in the brain, as well as rendering myelinated segments of axons unable to synapse onto nearby neurons. Frequently, axons running through white matter tracts bundle with others inside a single myelin sheath, a frugal move for a brain with space and energy constraints. It's unlikely that in these circumstances a brain would expend resources to myelinate short connections with no great need for it, so it's reasonable to assume that all myelinated fibers are long. Furthermore, gray and white matter are highly segregated, and myelin is rarely found in cortical tissue.
Length of myelinated fibers from white matter volume
This protective myelin coating not only insulates axons from lost transmission, but also, unfortunately, from the prying eyes of scientists. This means that long distance connections are difficult to study, and there have been few attempts to characterize white matter fibers at an appropriate level of detail for our purposes.
One frequently cited figure comes from Marner et al 2003, where the method called for divvying up preserved brains into slabs and take needle biopsies from random points on the slabs, then slicing these biopsies into fine sections and staining them. These could then be inspected for dark colored rings corresponding to myelin sheaths, and the total length of fibers could be approximated by multiplying "length density", or total length of fibers per volume of white matter, with white matter volume. This method yielded a total of 149,000 km of myelinated fibers in female brains, and 176,000 km in males.
As for the distribution of these fiber lengths in the brain, we're left somewhat in the dark. A very imprecise estimate for a portion of them can be gotten from a few key facts about the cerebrum. The largest and most famous white matter tract in the human brain is the corpus callosum, which connects the two hemispheres and contains 200-250 million fibers, about as many as one can find in tracts connecting areas within hemispheres. Given the width of the corpus callosum (~100 mm, or two thirds of the brain's total width), a reasonable value for average fiber length in this tract is 10 cm, suggesting that perhaps 50,000 km or less of long-range fiber connects the cerebrum with itself.7 Clearly, this leaves much white matter to be accounted for, which can presumably be attributed to connections within and between the cerebellum and subcortical structures, as well as the occasional cerebral white matter found outside the white matter tracts.
This vague picture can be supplemented by the relationship between long-range connection length and brain volume alluded to in Wen and Chklovskii 2004. These authors estimate that average global connection length should be roughly similar to the cube root of brain volume, or 10.6 cm – 11.4 cm, much like the figure we approximated above for intracortical connections.
Discussion
Summary of conclusions
Our estimates are aggregated in the table below:

Connection typeTotal length (km)Average length per neuron (mm)Contributing neuron typesSources of evidence

Cerebral, short-range220,000 - 320,000914 - 20Pyramidal (2/3rds to 85%), stellateFiber density in rats, morphometry

Cerebellar, short-range390,000 - 420,00085.7 - 6.1Granule (~70%), stellate, basket, Purkinje, GolgiMorphometry

Total, short-range610,000 - 740,000---

Cerebral, long-range~50,000100PyramidalWidth of corpus callosum, relationship between brain volume and global connection length

Total, long-range150,000 - 180,000??Length density per white matter volume

Total, all fibers760,000 - 920,000---

Overall, we're somewhat less confident in our total for long-range fiber length than our other estimates, since this was obtained using a methodology whose reliability we're not able to judge, and its findings couldn't be directly corroborated with other methods. However, there is indirect evidence that these numbers will hold up reasonably well: the proportion of total cerebral wiring that cerebral long-distance connections account for (14%) is quite similar to the proportion that long-distance connections purportedly account for overall (20%), despite the former number coming from independent lines of evidence.
Implications and future directions
The brain is the most metabolically expensive organ in the human body by volume, and has pushed the limits of natural birth by increasing the pelvic width of human females, via enlarged infant head sizes, to the edge of feasibility for walking. The massive resource requirements of the brain are clear, but the proportion demanded by communication (versus computation) is less clear.
Costs to the brain can be expressed in terms of space, energy (for development, maintenance and operation), and the difficulty or error-proneness of orchestrating complex activities. Space may be the cost most strongly influenced by brain wiring, and can easily be predicted to translate to computers, but the amount brain wiring also contributes to energy costs. In computers, this will take the form of operation energy, or the power needed to send "action potentials" along connections.
By themselves, our estimates of fiber lengths in the brain won't answer any questions about the difficulty of communication in computers broadly. However, they can be informative when considering a specific hardware architecture, and are likely to be especially so in the case of massively parallel architectures. Combining our estimates with other estimates relating to information transfer in the brain, like information density, may also yield insights relevant to AI hardware.
Contributions
Research, analysis and writing were done by Tegan McCaslin. Katja Grace contributed feedback and editing. Paul Christiano proposed the question and provided guidance on hardware-related matters.
Footnotes


Will AI see sudden progress?

Will advanced AI let some small group of people or AI systems take over the world?
AI X-risk folks and others have accrued lots of arguments about this over the years, but I think this debate has been disappointing in terms of anyone changing anyone else's mind, or much being resolved. I still have hopes for sorting this out though, and I thought a written summary of the evidence we have so far (which often seems to live in personal conversations) would be a good start, for me at least.
To that end, I started a collection of reasons to expect discontinuous progress near the development of AGI.
I do think the world could be taken over without a step change in anything, but it seems less likely, and we can talk about the arguments around that another time.
Paul Christiano had basically the same idea at the same time, so for a slightly different take, here is his account of reasons to expect slow or fast take-off.
Please tell us in the comments or feedback box if your favorite argument for AI Foom is missing, or isn't represented well. Or if you want to represent it well yourself in the form of a short essay, and send it to me here, and we will gladly consider posting it as a guest blog post.
I'm also pretty curious to hear which arguments people actually find compelling, even if they are already listed. I don't actually find any of the ones I have that compelling yet, and I think a lot of people who have thought about it do expect 'local takeoff' with at least substantial probability, so I am probably missing things.
By Katja Grace

Will AI see sudden progress?


Likelihood of discontinuous progress around the development of AGI

We aren't convinced by any of the arguments we've seen to expect large discontinuity in AI progress above the extremely low base rate for all technologies. However this topic is controversial, and many thinkers on the topic disagree with us, so we consider this an open question.
Details
Definitions
We say a technological discontinuity has occurred when a particular technological advance pushes some progress metric substantially above what would be expected based on extrapolating past progress. We measure the size of a discontinuity in terms of how many years of past progress would have been needed to produce the same improvement. We use judgment to decide how to extrapolate past progress.
For instance, in the following trend of progress in chess AI performance, we would say that there was a discontinuity in 2007, and it represented a bit over five years of progress at previous rates.
Figure 1: Machine chess progress, in particular SSDF records.
Relevance
Discontinuity by some measure, on the path to AGI, lends itself to:

A party gaining decisive strategic advantage
A single important 'deployment' event
Other very sudden and surprising events

Arguably, the first two require some large discontinuity. Thus the importance of planning for those outcomes rests on the likelihood of a discontinuity.
Outline
We investigate this topic in two parts. First, with no particular knowledge of AGI as a technology, how likely should we expect a particular discontinuity to be? We take the answer to be quite low. Second, we review arguments that AGI is different from other technologies, and lends itself to discontinuity. We currently find these arguments uncompelling, but not decisively so.
Default chance of large technological discontinuity
Discontinuities larger than around ten years of past progress in one advance seem to be rare in technological progress on natural and desirable metrics.1 We have verified around five examples, and know of several other likely cases, though have not completed this investigation.
This does not include the discontinuities when metrics initially go from zero to a positive number. For instance, the metric 'number of Xootr scooters in the world' presumably went from zero to one on the first day of production, though this metric had seen no progress before. So on our measure of discontinuity size, this was infinitely many years of progress in one step. It is rarer for a broader metric (e.g. 'scooters') to go from zero to one, but it must still happen occasionally. We do not mean to ignore this type of phenomenon, but just to deal with it separately, since discontinuity in an ongoing progress curve seems to be quite rare, whereas the beginning of positive progress on a metric presumably occurs once for every metric that begins at zero.
Proposed reasons to expect a discontinuity near AGI
We have said that the base rate of large technological discontinuities appears to be low. We now review and examine arguments that AGI is unusually likely to either produce a discontinuity, or follow one.
Hominid variation
Argument
Humans are vastly more successful in certain ways than other hominids, yet in evolutionary time, the distance between them is small. This suggests that evolution induced discontinuous progress in returns, if not in intelligence itself, somewhere approaching human-level intelligence. If evolution experienced this, this suggests that artificial intelligence research may do also.
Counterarguments
Evolution does not appear to have been optimizing for the activities that humans are good at. In terms of what evolution was optimizing for—short term ability to physically and socially prosper—it is quite unclear that humans are disproportionately better than earlier hominids, compared to differences between earlier hominids and less optimized animals.
This counterargument implies that if one were to optimize for human-like success that it would be possible (and likely at some point) to make agents with brains the size of chimps, that were humanlike but somewhat worse in their capabilities.
It is also unclear to us that humans succeed so well on something like 'intelligence' from vastly superior individual intelligence, rather than vastly superior group intelligence, via innovations such as language.
Status
See status section of 'brain scaling' argument—they are sufficiently related to be combined.
Brain scaling
Argument
If we speculate that the main relevant difference between us and our hominid relatives is brain size, then we can argue that brain size in particular appears to produce very fast increase in impressiveness. So if we develop AI systems with certain performance, we should expect to make disproportionately better systems by using more hardware.
Variation in brain size among humans also appears to be correlated with intelligence, and differences in intelligence in the human range anecdotally correspond to large differences in some important capabilities, such as ability to develop new physics theories. So if the relatively minor differences in human brain size cause the differences in intelligence, this would suggest that larger differences in 'brain size' possible with computing hardware could lead to quite large differences in some important skills, such as ability to successfully theorize about the world.
Counterarguments
This is closely related to the 'hominids saw discontinuous progress from intelligence' argument (see above), so shares the counterargument that evolution does not appear to have been optimizing for those things that humans excel at.
Furthermore, if using more hardware generally gets outsized gains, one might expect that by the time we have any particular level of AGI performance, we are already using extremely large amounts of hardware. If this is a cheap axis on which to do better, it would be surprising if we left that value on the table until the end. So the only reason to expect a discontinuity here would seem to be if the opportunity to use more hardware and get outsized results started existing suddenly. We know of no particular reason to expect this, unless there was a discontinuity in something else, for instance if there was indeed one algorithm. That type of discontinuity is discussed elsewhere.
Status
This argument seems weak to us currently, but further research could resolve these questions in directions that would make it compelling:

Are individual humans radically superior to apes on particular measures of cognitive ability? What are those measures, and how plausible is it that evolution was (perhaps indirectly) optimizing for them?
How likely is improvement in individual cognitive ability to account for humans' radical success over apes? (For instance, compared to new ability to share innovations across the population)
How does human brain size relate to intelligence or particular important skills, quantitatively?

Intelligence explosion
Argument
Intelligence works to improve many things, but the main source of intelligence—human brains—is arguably not currently improved substantially itself. The development of artificial intelligence will change that, by allowing the most intelligent entities to directly contribute to their increased intelligence. This will create a feedback loop where each step of progress in artificial intelligence makes the next steps easier and so faster. Such a feedback loop could potentially go fast. This would not strictly be a discontinuity, in that each step would only be a little faster than the last, however if the growth rate became sufficiently fast sufficiently quickly, it would be much like a discontinuity from our perspective.
This feedback loop is usually imagined to kick in after AI was 'human-level' on at least some tasks (such as AI development), but that may be well before it is recognized as human-level in general.
Counterarguments
Positive feedback loops are common in the world, and very rarely move fast enough and far enough to become a dominant dynamic in the world. So we need strong reason to expect an intelligence-of-AGI feedback loop to be exceptional, beyond the observation of a potential feedback effect. We do not know of such arguments. However, this is a widely discussed topic, so contenders probably exist.
Status
The counterargument currently seems to carry to us. However, we think the argument could be strong if:

Strong reason is found to expect an unusually fast and persistent feedback loop
Quantitative models of current relationships between the variables hypothesized to form a feedback loop suggest very fast intelligence growth.

One algorithm
Argument
If a technology is simple, we might expect to discover it in a small number of steps, perhaps one. If it is also valuable, we might expect each of those those steps to represent huge progress in some interesting metric. The argument here is that intelligence—or something close enough to what we mean by intelligence—is like this.
Some reasons to suspect that intelligence is conceptually simple:

Evolution discovered it relatively quickly (though see earlier sections on hominid evolution and brain size)
It seems conceivable to some that the mental moves required to play Go are basically all the ones you need for a large class of intellectual activities, at least with the addition of some way to navigate the large number of different contexts that are about that hard to think about.
It seems plausible that the deep learning techniques we have are already such a thing, and we just need more hardware and adjustment to make it work.

We expect that this list is missing some motivations for this view.
This argument can go in two ways. If we do not have the one algorithm yet, then we might expect that when we find it there will be a discontinuity. If we already have it, yet don't have AGI, then what remains to be done is probably accruing sufficient hardware to run it. This might traditionally be expected to lead to continuous progress, but it is also sometimes argued to lead to discontinuous progress. Discontinuity in hardware is discussed in a forthcoming section, so here we focus on discontinuity from acquiring the insight.
Counterarguments
Society has invented many simple things before. So even if they are unusually likely to produce discontinuities, the likelihood would still seem to be low, since discontinuities are rare. To the extent that you believe that society has not invented such simple things before, you need a stronger reason to expect that intelligence is such a thing, and we do not know of strong reasons.
Status
We currently find this argument weak, but would find it compelling if:

Strong reason emerged to expect intelligence to be a particularly simple innovation, among all innovations, and
We learned that very simple innovations do tend to cause discontinuities, and we have merely failed to properly explore possible past discontinuities well.

Deployment scaling
Argument
Having put in the research effort to develop one advanced AI system, it is very little further effort to make a large number of them. Thus at the point that a single AGI system is developed, there could shortly thereafter be a huge number of AGI systems. This would seem to represent a discontinuity in something like global intelligence, and also in 'global intelligence outside of human brains'.
Counterarguments
It is the nature of almost every product that large quantities of upfront effort are required to build a single unit, at which point many units can be made more cheaply. This is especially true of other software products, where a single unit can be copied without the need to build production facilities. So this is far from a unique feature of AGI, and if it produced discontinuities, we would see them everywhere.
The reason that the ability to quickly scale the number of instances of a product does not generally lead to a discontinuity seems to be that before the item's development, there was usually another item that was slightly worse, which there were also many copies of. For instance, if we write a new audio player, and it works well, it can be scaled up across many machines. However on each machine it is only replacing a slightly inferior audio player. Furthermore, because everyone already has a slightly inferior audio player, adoption is gradual.
If we developed human-level AGI software right now, it would be much better than what computing hardware is already being used for, and because of that and other considerations, it may spread to a lot of hardware fast. So we would expect a discontinuity. However if we developed human-level AGI software now, this would already be a discontinuity in other metrics.  In general, it seems to us that this scaling ability may amplify the effects of existing discontinuities, but it is hard to see how it could introduce one.
Status
This argument seems to us to rest on a confusion. We do not think it holds unless there is already some other large discontinuity.
Train vs. test
Argument
There are at least three different processes you can think of computation being used for in the development of AGI:

Searching the space of possible software for AGI designs
Training specific AGI systems
Running a specific AGI system

We tend to think of the first requiring much more computation than the second, which requires much more computation than the third. This suggests that when we succeed at the first, hardware will have to be so readily available that we can quickly do a huge amount of training, and having done the training, we can run vast numbers of AI systems. This seems to suggest a discontinuity in number of well-trained instances of AGI systems.
Counterarguments
This seems to be a more specific version of the deployment scaling argument above, and so seems to fall prey to the same counterargument: that whenever you find a good AGI design and scale up the number of systems running it, you are only replacing somewhat worse software, so this is not a discontinuity in any metric of high level capability. For there to be a discontinuity, it seems we would need a case where step 1 returns an AGI design that is much better than usual, or returns a first AGI design that is already very powerful compared to other ways of solving the high level problems that it is directed at. These sources of discontinuity are both discussed elsewhere.
Status
This seems to be a variant of the deployment scaling argument, and so similarly weak. However it is more plausible to us here that we have failed to understand some stronger form of the argument, that others find compelling.
Starting high
Argument
As discussed in Default chance of technological discontinuity above, the first item in a trend always represents a 'large' discontinuity in the number of such items—the number has been zero forever, and now it is one, which is more progress at once than in all of history. When we said that discontinuities were rare, we explicitly excluded discontinuities such as these, which appear to be common but uninteresting. So perhaps a natural place to find an AGI-related discontinuity is here.
Discontinuity from zero to one in the number of AGI systems would not be interesting on its own. However perhaps there are other metrics where AGI will represent the first step of progress. For instance, the first roller coaster caused 'number of roller coasters' to go from zero to one, but also 'most joy caused by a roller coaster' to go from zero to some positive number. And once we admit that inventing some new technology T could cause some initial discontinuity in more interesting trends than 'number of units of T', we do not know very much about where or how large such first steps may be (since we were ignoring such discontinuities). So the suggestion is that AGI might see something like this: important metrics going from zero to high numbers, where 'high' is relative to social impact (our previous measure of discontinuities being unhelpful in the case where progress was previously flat). Call this 'starting high'.
Having escaped the presumption of a generally low base rate of interesting discontinuities then, there remain the questions of what the base rate should be for new technologies 'starting high' on interesting metrics, and whether we should expect AGI in particular to see this even above the new base rate.
One argument that the base rate should be high is that new technologies have a plethora of characteristics, and empirically some of them seem to start 'high', in an intuitive sense. For instance, even if the first plane did not go very far, or fast, or high, it did start out 40 foot wide—we didn't go through flying machines that were only an atom's breadth, or only slightly larger than albatrosses. 'Flying machine breadth' is not a very socially impactful metric, but if non-socially impactful metrics can start high, then why not think that socially impactful metrics can?
Either way, we might then argue that AGI is especially likely to produce high starting points on socially impactful metrics:

AGI will be the first entry in an especially novel class, so it is unusually likely to begin progress on metrics we haven't seen progress on before.
Fully developed AGI would represent progress far above our current capabilities on a variety of metrics. If fully developed AGI is unusually impactful, this suggests that the minimal functional AGI is also unusually impactful.
AGI is unusually non-amenable to coming in functional yet low quality varieties. (We do not understand the basis for this argument well enough to relay it. One possibility is that the insights involved in AGI are sufficiently few or simple that having a partial version naturally gets you a fully functional version quickly. This is discussed in another section. Another is that AGI development will see continuous progress in some sense, but go through non-useful precursors—much like 'half of a word processor' is not a useful precursor to a word processor. This is also discussed separately elsewhere.)

Counterarguments
While it is true that new technologies sometimes represent discontinuities in metrics other than 'number of X', these seem very unlikely to be important metrics. If they were important, there would generally be some broader metric that had previously been measured that would also be discontinuously improved. For instance, it is hard to imagine AGI suddenly possessing some new trait Z to a degree that would revolutionize the economy, without this producing discontinuous change in previously measured things like 'ability to turn resources into financial profit'.
Which is to say, it is not a coincidence that the examples of high starting points we can think of tend to be unimportant. If they were important, it would be strange if we had not previously made any progress on achieving similar good outcomes in other ways, and if the new technology didn't produce a discontinuity in any such broader goal metrics, then it would not be interesting. (However it could still be the case that we have so far systematically failed to look properly at very broad metrics as they are affected by fairly unusual new technologies, in our search for discontinuities.)
We know of no general trend where technologies that are very important when well developed cause larger discontinuities when they are first begun. We have also not searched for this specifically, but isn't an immediately apparent trend in the discontinuities we know of.2
Supposing that an ideal AGI is much smarter than a human, then humans clearly demonstrate the possibility of functional lesser AGIs. (Which is not to say that 'via human-like deviations from perfection' is a particularly likely trajectory.) Humans also seem to demonstrate the possibility of developing human-level intelligence, then not immediately tending to have far superior intelligence. This is usually explained by hardware limitations, but seems worth noting nonetheless.
Status
We currently think it is unlikely that any technology 'starts high' on any interesting metric, and either way AGI does not seem especially likely to do so. We would find this argument more compelling if:

We were shown to be confused about the inference that 'starting high' on an interesting metric should necessarily produce discontinuity in broader, previously improved metrics. For instance, if a counterexample were found.
Historical data was found to suggest that ultimately valuable technologies tended to 'start high' or produce discontinuities.
Further arguments emerged to expect AGI to be especially likely to 'start high'.

Awesome AlphaZero
Argument
We have heard the claim that AlphaZero represented a discontinuity, at least in the ability to play a variety of games using the same system, and maybe on other metrics. If so, this would suggest that similar technologies were unusually likely to produce discontinuities.
Counterarguments
Supposing that AlphaZero did represent discontinuity on playing multiple games using the same system, there remains a question of whether that is a metric of sufficient interest to anyone that effort has been put into it. We have not investigated this.
Whether or not this case represents a large discontinuity, if it is the only one among recent progress on a large number of fronts, it is not clear that this raises the expectation of discontinuities in AI very much, and in particular does not seem to suggest discontinuity should be expected in any other specific place.
Status
We have not investigated the claims this argument is premised on, or examined other AI progress especially closely for discontinuities. If it turned out that:—

Recent AI progress represented at least one substantial discontinuity
The relevant metric had been a focus of some prior effort

—then we would consider more discontinuities around AGI more likely. Beyond raising the observed base rate of such phenomena, this would also suggest to us that the popular expectation of AI discontinuity is well founded, even if we have failed to find good explicit arguments for it. So this may raise our expectation of discontinuous change more than the change in base rate would suggest.
Uneven skills
Argument
There are many axes of ability, and to replace a human at a particular task, a system needs to have a minimal ability on all of them. At which point, it should be substantially superhuman on some axes. For instance, by the time that self-driving cars are safe enough in all circumstances to be allowed to drive, they might be very much better than humans at noticing bicycles. So there might be a discontinuity in bicycle safety at the point that self-driving cars become common.
If this were right, it would also suggest a similar effect near overall human-level AGI: that the first time we can automate a variety of tasks as well as a human, we will be able to automate them much better than a human on some axes. So those axes may see discontinuity.
Figure 2: Illustration of the phenomenon in which the first entirely 'human-level' system is substantially superhuman on most axes. (Image from Superintelligence Reading Group)
Counterarguments
This phenomenon appears to be weakly related to AGI in particular, and should show up whenever one type of process replaces another one. Replacement of one type of system with another type does not seem very rare. So if it is true that large discontinuities are rare, then this type of dynamic does not produce large discontinuities often.
This argument benefits from the assumption that you have to 'replace' a human in one go, which seems usually false. New technology can usually be used to complement humans, admittedly with inefficiency relative to a more integrated system. For instance, one reason there will not be a discontinuity in navigation ability of cars on the road is that humans already use computers for their navigation. However this counterargument doesn't apply to all axes. For instance, if computers operate quite fast but cannot drive safely, you can't lend their speed to humans and somehow still use the human steering faculties.
Status
This argument seems successfully countered, unless:

We have missed some trend of discontinuities of this type elsewhere. If there were such a trend, it seems possible that we would have systematically failed to notice them because they look somewhat like 'initial entry goes from zero to one', or are in hard to measure metrics.
There are further reasons to expect this phenomenon to apply in the AGI case specifically, especially without being substantially mitigated by human-complementary technologies

Payoff thresholds
Argument
Even if we expect progress on technical axes to change continuously, there seems no reason this shouldn't translate to discontinuous change in a related axis such as 'economic value' or 'ability to take over the world'. For instance, continuous improvement in boat technology might still lead to a threshold in whether or not you can reach the next island.
Counterarguments
There actually seems to be a stronger theoretical reason to expect continuous progress in metrics that we care about directly, such as economic value, than on more technical metrics. This is that if making particular progress is expected to bring outsized gains, more effort should be directed toward the project until that effort has offset a large part of the gains. For instance, suppose that for some reason guns were a hundred times more useful if they could shoot bullets at just over 500m/s than just under, and otherwise increasing speed was slightly useful. Then if you can make a 500m/s gun, you can earn a hundred times more for it than a lesser gun. So if guns can currently shoot 100m/s, gun manufacturers will be willing to put effort into developing the 500m/s one up to the point that it is costing nearly a hundred times more to produce it. Then if they manage to produce it, they do not see a huge jump in value of making guns, and the buyer doesn't see a huge jump in value of gun – their gun is much better but also much more expensive. However at the same time, there may have been a large jump in 'speed of gun'.
This is a theoretical argument, and we do not know how well supported it is. (It appears to be at least supported by the case of nuclear weapons, which were discontinuous in the technical metric 'explosive power per unit mass', but not obviously so in terms of cost-effectiveness).
In practice, we don't know of many large discontinuities in metrics of either technical performance or things that we care about more directly.
Even if discontinuities in value metrics were common, this argument would need to be paired with another for why AGI in particular should be expected to bring about a large discontinuity.
Status
This argument currently seems weak. In the surprising event that that discontinuities are actually common in metrics more closely related to value (rather than generally rare, and especially so that case), the argument would only be mildly stronger, since it doesn't explain why AGI is especially susceptible to this.
Human-competition threshold
Argument
There need not be any threshold in nature that makes a particular level of intelligence much better. In a world dominated by humans, going from being not quite competitive with humans to slightly better could represent a step change in value generated.
This is a variant of the above argument from payoff thresholds that explains why AGI is especially likely to see discontinuity, even if it is otherwise rare: it is not often that something as ubiquitous and important as humans is overtaken by another technology.
Counterarguments
Humans have a variety of levels of skill, so even 'competing with humans' is not a threshold. You might think that humans are in some real sense very close to one another, but this seems unlikely to us.
Even competing with a particular human is unlikely to have threshold behavior—without for instance a requirement that someone hire one or the other—because there are a variety of tasks, and the human-vs.-machine differential will vary by task.
Status
This seems weak. We would reconsider if further evidence suggested that the human range is very narrow, in some ubiquitously important dimension of cognitive ability.
 Footnotes


Electrical efficiency of computing

Computer performance per watt has probably doubled every 1.5 years between 1945 and 2000. Since then the trend slowed. By 2015, performance per watt appeared to be doubling every 2.5 years.
Details
In 2011 Jon Koomey reported that computation per kWh had doubled every roughly 1.5 years since around 1950, as shown in figure 1 (taken from him).1 Wikipedia calls this trend 'Koomey's Law'. In 2015 Koomey and Naffziger reported in IEEE Spectrum that Koomey's law began to slow down in around 2000 and by 2015, electrical efficiency was taking 2.5 years to double.2
We have not investigated beyond this, except to note that there is not obvious controversy on the topic. We do not know the details of the methods involved in this research, for instance how 'computations' are measured.
Figure 1. Computations per kWh over recent history. Taken from Dr Jon Koomey, http://www.koomey.com/post/<PHONE>, CC BY-SA 3.0

Electrical efficiency of computing


Nordhaus hardware price performance dataset

This page contains the data from Appendix 2 of William Nordhaus' The progress of computing in usable formats.
Notes
This data was collected from Appendix 2 of The progress of computing, using Tabula (a program for turning tables in pdfs into other table formats). We have not checked its accuracy beyond a graph of the resulting data looking visually similar to a graph of the original.
Important: we previously noted that this data appears to be orders of magnitude different from other sources, and haven't had time to look into this discrepancy.
Data
Here is a Google sheet of the data. See 'Nordhaus via Tabula' page.


2018 price of performance by Tensor Processing Units

Tensor Processing Units (TPUs) perform around 1 GFLOPS/$, when purchased as cloud computing.
Details
In February 2018, Google Cloud Platform blog says their TPUs can perform up to 180 TFLOPS, and currently cost $6.50/hour.1 This gives us $171,000 to rent one TPU continually for a roughly three year lifecycle2 Which is 1.05 GFLOPS/$.
This service apparently began on February 12 2018.3 So this does not appear to be competitive with the cheapest GPUs, in terms of FLOPS/$, or even the cheapest cloud computing.


Examples of AI systems producing unconventional solutions

This page lists examples of AI systems producing solutions of an unexpected nature, whether due to goal misspecification or successful optimization.  This list is highly incomplete.
List

CoastRunners' burning boat
Incomprehensible evolved logic gates
AlphaGo's inhuman moves
Waze direction into fires


Historic trends in altitude

Published 7 Feb 2020
Altitude of objects attained by man-made means has seen six discontinuities of more than ten years of progress at previous rates since 1783, shown below.
YearHeight (m)Discontinuity (years)Entity178440001032Balloon180372801693Balloon191842,300227Paris gun194285,000120V-2 Rocket1944174,60011V-2 Rocket1957864,000,00035Pellets (after one day)

Details
This case study is part of AI Impacts' discontinuous progress investigation.
Trends
Altitude of objects attained by manmade means
We looked for records in height from the ground reached by any object via man-made technology.
'Man-made technology' is ambiguous, but we exclude for instance objects tied to birds and debris carried up by hurricanes. We include debris launched unintentionally via gunpowder explosion, and rocks launched via human arms.
We measure 'altitude' from the ground at the launch site. This excludes mountain climbing, but also early flight attempts that involve jumping from towers and traveling downward slowly.1 It also excludes early parachutes, which were mentioned in fiction thousands of years ago.2
Measured finely enough, there are never discontinuities in altitude, since objects travel continuously.3 This prohibits finding discontinuities in continuously measured altitude, but doesn't interfere with the dataset being relevant evidence to us. We are interested in discontinuities because they tell us about how much surprising progress can happen in a short time, and how much progress can come from a single innovation. So to make use of this data, we need to find alternate ways of measuring it that fulfill these purposes.
For the purpose of knowing about progress in short periods, we can choose a short period of interest, and measure jumps in progress made at that scale. For the purpose of knowing about progress made by single innovations, we can assign the maximum altitude reached to the time that the relevant innovation was made, for instance.4
We could measure both of these trends, but currently only measure a version of the former. For short periods of travel, we assign the maximum altitude reached to the date given (our understanding is that most of the entries took place over less than one day). For travel that appears to have taken more than a day, we record any altitudes we have particular information about, and otherwise estimate records on roughly an annual basis, including a record for the peak altitude (and possibly more than a year apart to allow for the final record to have the maximum altitude). This is ad hoc, but for the current purpose, converting what we have to a more consistent standard does not seem worth it. Instead, we consider these the effects of these choices when measuring discontinuities. They do not appear to matter, except to make modest differences to the size of the pellet discontinuity, discussed below (section, 'Discontinuity measurement').
Data
We collected data from various sources, and added them to this spreadsheet, tab 'Manned and unmanned'. This data is shown in Figures 1-3 below. We have not thoroughly verified this data.
Record altitudes might plausibly be reached by a diversity of objects for a diversity of purposes, so collecting such data is especially dependent on imagination for the landscape of these.5 For this reason, this data is especially likely to be incomplete.
We also intentionally left the data less complete than usual in places where completeness seemed costly and unlikely to affect conclusions about discontinuities. The following section discusses our collection of data for different periods in history and details of our reasoning about it.
Detailed overview of data
Here we describe the history of progress in altitude reached and the nature of the data we collected during different times. See the spreadsheet for all uncited sources.
Chimps throw rocks, so we infer that humans have probably also done this from the beginning.6 A good rock throw can apparently reach around 25m. Between then and the late 1700s, humanity developed archery, sky lanterns, kites, gunpowder, other projectile weapons, rockets, and primitive wings7, among probably other things. However records before the late 1700s are hard or impossible to find, so we do not begin the search for discontinuities until a slew of hot air balloon records beginning in 1783s. We collected some earlier records in order to have a rough trend to compare later advances to, but we are likely missing many entries, and the entries we have are quite uncertain. (It is more important to have relatively complete data for measuring discontinuities than it is for estimating a trend.)
The highest altitude probably attained before the late 1700s that we know of was reached by debris in a large gunpowder building explosion in 1280, which we estimate traveled around 2.5km into the air. Whether to treat this as a 'man-made technology' is ambiguous, given that it was not intentional, but we choose to ignore intention.8
Kites may also have traveled quite high, quite early. It appears that they have been around for at least two thousand years.9 and were used in ancient warfare and even occasionally for lifting people. We find it hard to rule out the possibility that early kites could travel one or two thousand meters into the air: modern kites frequently fly at 2km altitudes, silk has been available for thousands of years, and modern silk at least appears to be about as strong as nylon.10 Thus if we are wrong about the gunpowder factory explosion, it is still plausible that two thousand meter altitudes were achieved by kites.
Over a period of three and a half months from August 1783, manned hot air balloons were invented,11 and taken from an initial maximum altitude of 24m up to a maximum altitude of 2700m. While this was important progress in manned travel12, most of these hot air balloons were still lower than the gunpowder explosion and perhaps kites. Nonetheless, there are enough records from around this time, that we begin our search for discontinuities here.
The first time that humanity sent any object clearly higher than ancient kites or explosion debris was December 1783, when the first hydrogen balloon flight ascended to 2,700m. This was not much more than we (very roughly) estimate that those earlier objects traveled. However the hot air balloon trend continued its steep incline, and in 1784 a balloon reached 4000m, which is over a thousand years of discontinuity given our estimates (if we estimated the rate of progress as an order of magnitude higher or lower, the discontinuity would remain large, so the uncertainties involved are not critical.)
The next hot air balloon that we have records for ascended nearly twice as high—7280m—in 1803, representing another over a thousand years of discontinuity. We did not thoroughly search for  records between these times. However if that progress actually accrued incrementally over the twenty years between these records, then still every year would have seen an extra 85 years of progress at the previous rate, so there must have been at least one year that saw at least that much progress, and it seems likely that in fact at least one year saw over a hundred years of progress. Thus there was very likely a large discontinuity at that time, regardless of the trend between 1784 and 1803.
We collected all entries from Wikipedia's Flight altitude record page, which claims to cover 'highest aeronautical flights conducted in the atmosphere, set since the age of ballooning'.13 It is not entirely clear to us what 'aeronautical flights' covers, but seemingly at least hot air balloons and planes. The list includes some unmanned balloons, but it isn't clear whether they are claiming to cover all of them. They also include two cannon projectiles, but not 38 cm SK L/45 "Max", which appears to be a record relative to anything they have, and cannon projectiles are probably not 'flights', so we think they are not claiming to have exhaustively covered those. Thus between the late 1700s, and the first flights beyond the atmosphere, the main things this data seems likely to be missing is military projectiles, and any other non-flight atmospheric-level objects.
We searched separately for military projectiles during this period. Wikipedia claims, without citation, that the 1918 Paris gun represented the greatest height reached by a human-made projectile until the first successful V-2 flight test in October 194214, which matches what we could find. We searched for military records prior to the Paris gun, and found only one other, "Max" mentioned above, a 38cm German naval gun from 1914.
We expect there are no much higher military records we are missing during this time but that we could easily have missed some similar ones. As shown in Figure 1, the trend of military records we are aware of is fairly linear, and that line is substantially below the balloon record trend until around 1900. So it would be surprising if there were earlier military records that beat balloon records, and less surprising if we were missing something between 1900 and 1918. It seems unlikely however that we could have missed enough data that the Paris Gun did not represent at least a moderate discontinuity.15
We could not think of other types of objects that might have gone higher than aeronautical flights and military projectiles between the record 1803 balloon and V-2 rockets reaching 'the edge of space' from 1942. Thus the data in this period seems likely to be relatively complete, or primarily missing less important military projectiles.
The German V-2 rockets are considered the first man-made objects to travel to space (though the modern definition of space is higher)16 so they are presumably the highest thing at that time (1942). They are also  considered the first projectile record since the Paris gun, supporting this. Wikipedia has an extensive list of V-2 test launches and their outcomes, from which we infer than three of them represent altitude records.17
The two gun records we know of were both German WWI guns, and the V2 rockets that followed were German WWII weapons, apparently developed in an attempt to replace the Paris Gun when it was banned under the Versailles Treaty.18 So all altitude records between the balloons of the 1800s and the space rockets of the 50s appear to be German military efforts.
Between the last record V-2 rocket in 1946 and 1957, we found a series of rockets that traveled to increasing altitudes. We are not confident that there were no other record rocket altitudes in this time. However the rockets we know of appear to have been important ones, so it seems unlikely that other rockets at the time were radically more powerful, and there does not appear to have been surprising progress over that entire period considered together, so there could not have been much surprising progress in any particular year of it, unless the final record should be substantially higher than we think. We are quite unsure about the final record (the R-7 Semyorka), however it doesn't seem as though it could have gone higher than 3000km, which would only add a further four years of surprising progress to be distributed in the period.
In October 1957, at least one centimeter-sized pellet was apparently launched into solar orbit, using shaped charges and a rocket. As far as we know, this was the first time an object escaped Earth's gravity to orbit the sun.19 This episode does not appear to be mentioned often, but we haven't found anyone disputing its being the first time a man-made object entered solar orbit, or offering an alternate object.
Because the pellets launched were just pellets, with no sophisticated monitoring equipment, it is harder to know what orbit they ended up in, and therefore exactly how long it took to reach their furthest distance from Earth, or what it was. Based on their speed and direction, we estimate they should still have been moving at around 10km/s as they escaped Earth's gravity. Within a day we estimate that they should have traveled more than six hundred times further away than anything earlier that we know of. Then conservatively they should have reached the other side of the sun, at a distance from it comparable to that of Earth, in around 1.5 years. However this is all quite uncertain.
At around this time, reaching maximum altitudes goes from taking on the order of days to on the order of years. As discussed at the start of section 'Altitude of objects attained by manmade means' above, from here on we record new altitudes every year or so for objects traveling at increasing altitudes over more than a year.
In the years between 1959 and 1973, various objects entered heliocentric orbit.20 It is possible that some of them reached greater altitudes than the pellets, via being in different orbits around the sun. Calculating records here is difficult, because reaching maximal distance from Earth takes years,21 and how far an object is from Earth at any time depends on how their (eccentric) orbits relate to Earth's, in 3D space. Often, the relevant information isn't available.
Among artificial objects in heliocentric orbit listed by Wikipedia22 none are listed as having orbits where they travel more than 1.6 times further from the Sun than Earth does23, though many are missing such data. This is probably less far than the pellets, though further away than our conservative estimate for the pellets. For an object to reach this maximal distance from the Earth, it would need to be at this furthest part of its orbit, while being on the opposite side of the Sun from Earth, on the same plane as Earth.
Given all of this, it seems implausible that anything went ten times as far from the Sun as Earth by 1960, but even this would not have represented a discontinuity of even ten years. Given this and the difficulty of calculating records, we haven't investigated this period of solar orbiters thoroughly.
In 1973 Pioneer 10 became the first of five space probes to begin a journey outside the solar system. In 1998 it was overtaken by Voyager 1. We know that no other probes were the furthest object during that time, however have not checked whether various other objects exiting the solar system (largely stages of multi-stage rockets that launched the aforementioned probes) might have gone further.
Figure 1 shows all of the altitude data we collected, including entries that turned out not to be records. Figures 2 and 3 show the best current altitude record over time.
 Figure 1: Post-1750 altitudes of various objects, including many  non-records. Whether we collected data for non-records is inconsistent,  so this is not a complete picture of progress within object types. It  should however contain most aircraft and balloon records since 1783. See  image in detail here.
To see detail:Download

Figure 2: record altitudes known to us since 1750
Figure 3: Record altitudes known to us since 6000 BC (early ones estimated imprecisely)
Discontinuity measurement
For measuring discontinuities, we treat the past trend at a given point as linear or exponential and as starting from earlier or later dates depending on what fits well at that time.24 Relative to these previous rates, this altitude trend contains six discontinuities of greater than ten years, with four of them being greater than 100 years:25
YearHeight (m)Discontinuity (years)Entity178440001032Balloon180372801693Balloon191842,300227Paris gun194285,000120V-2 Rocket1944174,60011V-2 Rocket1957864,000,00035Pellets (after one day)
The 1957 pellets would be a 66 year discontinuity if we counted all of their ultimate estimated altitude as one jump on the day after their launch, so exactly how one decides to treat altitudes that grow over years is unlikely to prevent these pellets representing a discontinuity of between ten and a hundred years.
In addition to the size of these discontinuities in years, we have tabulated a number of other potentially relevant metrics here.26

Primary authors: Katja Grace, Rick Korzekwa
Thanks to Stephen Jordan and others for suggesting a potential discontinuity in altitude records.
Notes

Historic trends in altitude


2015 FLOPS prices

In April 2015, the lowest GFLOPS prices we could find were approximately $3/GFLOPS. However recent records of hardware performance from 2015 and earlier imply substantially lower prices, suggesting that something confusing has happened with these sources of data. We have not resolved this.
Recent data
We have not finished exploring the apparent discrepancies between 2015 prices for performance and current records of 2015 prices for performance. However in the data described in our 2017 assessment of recent price trends (key figure here), prices appear to have been below $1 since 2008.1 The measurements are not entirely comparable, but we would not expect the differences to produce such a large price difference.
2015 research
The rest of this page is largely taken from our page written in 2015.
In April 2015, the lowest recorded GFLOPS prices we knew of were approximately $3/GFLOPS, for various CPU and GPU combinations. Amortized over three years, this was $1.1E-13/FLOPShour. Prices in the $3-5/GFLOPS range seemed to be common, for GPU and CPU combinations and sometimes for supercomputers. Using CPUs, prices were at least $11/GFLOPS, and computing as a service cost more like $160/GFLOPS.
Background
We have written about long term trends in the costs of computing hardware. We were interested in evaluating the current prices more thoroughly, both to validate the long term trend data, and because current hardware prices are particularly important to know about.
Details
We separately investigated CPUs, GPUs, computing as a service, and supercomputers. In all categories, we collected some contemporary instances which we judged heuristically as especially likely to be cost-effective. We did not find any definitive source on the most cost-effective in any category, or in general, so our examples are probably not the very cheapest.  Nevertheless, these figures give a crude sense for the cost of computation in the contemporary market. Our full dataset of CPUs, GPUs and supercomputers is here, and contains data on twenty two machines. Our data on computing as a service is all included in this page.
Included costs
For CPUs and GPUs, we list the price of the CPU and/or GPU (GPUs were always used with a CPU, so we include the cost for both), but not other computer components. We compared prices between one complete rack server and the set of four processors inside it, and found the complete server was around 36% more expensive ($30,000 vs. $22,000). We expect this is representative at this scale, but diminishes with scale.
For computing services, we list the cheapest price for renting the instance for a long period, with no additional features. We do not include spot prices.
For supercomputers, we list costs cited, which don't tend to come with elaboration. We expect that they only include upfront costs, and that most of the costs are for hardware.
We have not included the costs of energy or other ongoing expenses in any prices. Non-energy costs are hard to find, and we suspect a relatively small and consistent fraction of costs. Energy costs appear to be around 10% of hardware costs. For instance, the Intel Xeon E5-2699 uses 527.8 watts and costs $5,190.2 Over three years, with $0.05/kWh this is $694, or 13% of the hardware cost. Titan also uses 13% of its hardware costs in energy over three years.3 We might add these costs later for a more precise estimate.
FLOPS measurements
To our knowledge we report only empirical performance figures from benchmark tests, rather than theoretical maximums. We sometimes use figures for LINPACK and sometimes for DGEMM benchmarks, depending on which are available. Geekbench in particular does not use the common LINPACK, but LINPACK relies heavily on DGEMM, suggesting DGEMM is fairly comparable. We guess they differ by around 10%.4
Prices
Central processing units (CPUs)
We found prices and performance data for five contemporary CPUs, including three different instances of one of them. They ranged from $11-354/GFLOPS with most prices below $100/GFLOPS.5 The cheapest of these CPUs still looks several times more expensive than some GPUs and supercomputers, so we did not investigate these numbers in great depth, or search far for cheaper CPUs.
Graphics processing units (GPUs)
We found performance data for six recent combinations of CPUs and GPUs (with much overlap between CPUs and GPUs between combinations. They ranged from $3.22/GFLOPS to $4.17/GFLOPS.
Note that graphics cards are typically significantly restricted in the kinds of applications they can run efficiently; this performance is achieved for highly regular computations that can be carried out in parallel throughout a GPU (of the sort that are required for rendering scenes, but which have also proved useful in scientific computing).
Computing as service
Another way to purchase FLOPS is via virtual computers.
Amazon Elastic Cloud Compute (EC2) is a major seller of virtual computing. Based on their current pricing, renting a c4.8xlarge instance costs about $1.17 / hour.6 This is their largest instance optimized for computing performance (rather than e.g. memory). A c4.8xlarge instance delivers around 97.5 GFLOPS.7 This implies that a GFLOPShour costs $0.012. If we suppose this is an alternative to buying computer hardware, then the relevant time horizon is about three years. Over three years, renting this hardware will cost $316/GFLOPS, i.e. around two orders of magnitude more than buying GFLOPS in the form of GPUs.
Other sources of virtual computing seem to be similarly priced. An informal comparison of computing providers suggests that on a set of "real-world java benchmarks" three providers are quite closely comparable, with all between just above Amazon's price and just under half Amazon's price for completing the benchmarks, across different instance sizes. This analysis also suggests Amazon is a relatively costly provider, and suggests a cheap price for virtual computing is closer to $0.006/GFLOPShour or $160/GFLOPS over three years.
Even with this optimistic estimate, virtual computing appears to cost something like fifty times more than GPUs. This high price is presumably partly because there are non-hardware costs which we have not accounted for in the prices of buying hardware, but are naturally included in the cost of renting it. However it is unlikely that these additional costs make up a factor of fifty.
Supercomputing
The Titan supercomputer purportedly cost about $97M to produce, or about $4,000 dollars per hour amortized over 3 years. It performs 17,590,000 GFLOPS which comes to $5.51/GFLOPS. This makes it around the same price as the cheapest GPUs. It is made of a combination of GPUs and CPUs, so this similarity is unsurprising.
The other six built supercomputers we looked at were more expensive, ranging up to $95/GFLOPS. Another cost-effective supercomputer, the L-CSC, was being built at the time it was most recently reported on, and while it should be completed now we could not find more data on it. Extrapolating from the figures before it was finished, when completed it should cost $2.39/GFLOPS, and thus be the cheapest source of FLOPS we are aware of.
Summary
The lowest recorded GFLOPS prices we know of are approximately $3/GFLOPS, for various CPU and GPU combinations. Amortized over three years, this is $1.1E-13/FLOPShour. Prices in the $3-5/GFLOPS range seem to be common, for GPU and CPU combinations and sometimes for supercomputers. Using CPUs, prices are at least $11/GFLOPS, and computing as a service costs more like $160/GFLOPS.

2015 FLOPS prices


Effect of marginal hardware on artificial general intelligence

We do not know how AGI will scale with marginal hardware. Several sources of evidence may shed light on this question.
Details
Background
Suppose that at some point in the future, general artificial intelligence can be run on some quantity of hardware, h, producing some measurable performance p. We would like to know how running approximately the same algorithms using additional hardware (increasing h) affects p.
This is important, because if performance scales superlinearly, then at the time we can run a human-level intelligence on hardware that costs as much as a human, we can run an intelligence that performs more than twice as well as a human on twice as much hardware, and so already have superhuman efficiency at converting hardware into performance. And perhaps go on, for instance producing an entity which is 64 times as costly as a human yet almost incomparably better at thinking.
This might mean that the first 'human-level' effectiveness at converting dollars of hardware into performance would be earlier, when perhaps a mass of hardware costing one thousand times as much as a human can be used to produce something which performs a thousand times as well as a human. However it might be that the first time we have software to produce something roughly human-like, it does so at human-level with much smaller amounts of hardware. In which case, immediately scaling up the hardware might produce a substantially superhuman intelligence. This is one reason some people expect fast progress from sub-human to superhuman intelligence.
Whether gains are superexponential or sublinear depends on the metrics of performance. For instance, imagine hypothetically that doubling hardware generally produces a twenty point IQ increase (logarithmic gains in IQ), but twenty IQ points above the smartest human is enough to conquer areas of science that thousands of scientists have puzzled over ineffectually forever (much better than exponential gains in some metric of discovery or economic value). So the question must be how marginal hardware affects metrics that matter to us.
Considerations
We have not investigated this question, but the following sources of evidence seem promising.
Evidence from existing algorithms
We do not yet have any kind of artificial general intelligence. However we can look at how performance scales with hardware in other kinds of software applications, especially narrow AI.
Evidence from human brain scaling
Among humans, brain size and intelligence are related. The exact relationship has been studied, but we have not reviewed the literature.
Evidence from between-animal brain scaling
Between animals, brain size relative to body size is related to intelligence. The exact relationship has probably been studied, but we have not reviewed the literature.
Evidence from new types of computation being possible with additional hardware
Some gains with hardware could come not from better performance on a particular task, but from being able to perform new tasks that were previously infeasible with so little hardware. We do not know how big an issue this is, but examining past experience with increasing hardware availability (e.g. by talking to researchers) seems promising.


Human-level hardware timeline

We estimate that 'human-level hardware'— hardware able to perform as many computations per second as a human brain, at a similar cost to a human brain—has a 30% chance of having already occurred, a 45% third chance of occurring by 2040, and a 25% chance of occurring later. We are not confident about these estimates.
Support
Background
Say that computing hardware has reached 'human-level' performance when machines can perform as many computations as a human brain performs (under some natural interpretation of the brain as performing computations), at no greater cost than that of running a human brain.
We are interested in when hardware reaches this level of performance, because then if we have software that is also at least 'human-level',1 we will have 'human-level' AI overall: AI that can perform the tasks that a human brain performs, as efficiently as a human brain.
We may have human-level AI before having hardware and software that are both at least as good as the human brain—intuitively, if one of them is better than that, then the other may be worse. So the implications of human-level hardware alone are not straightforward. (We may also have disruptive change before we have human-level AI—the event of human-level AI is an upper bound for when some large changes in society can be expected to occur.)
It is unclear to us how important the contributions from hardware and software progress are, respectively, to overall AI progress. If hardware progress is much more important than software progress, then human-level hardware should approximately co-occur with human-level AI.
Calculation
To roughly forecast when computing hardware will reach 'human-level', we can combine estimates of how much computing the human brain performs per dollar, current hardware performance per dollar, and the rate of improvement in hardware performance.
Let:
T = time until human-level hardware performance per dollar.
H = human-level hardware performance per dollar = 0.4-13*1010 FLOPS/$.2
C = current hardware performance per dollar = 0.3-30 *109 FLOPS/$3
R = 1 + growth rate of hardware performance per dollar = 1.16-1.784
Then we have:
T =  logR(H/C)
=log1.16(0.4 x 1010/(30 x 109)) to log1.16(13 x 1010/(.3 x 109))
= -14 to 41 years

These are rough calculations, and the breadth of the intervals don't necessarily mean a lot—the intervals were non-specific to begin with, and then we combined several of them.
If we do something similar (shown here), using more realistic distributions for each variable and calculating using the entire distributions rather than end points, we get -14 to 22 years using the narrower estimates for human-level hardware that we used above, or -31 to 99 years for a very wide set of estimates for human-level hardware. The chance that human-level hardware has already occurred is around 20-40%, according to these calculations.
Based on these calculations, we estimate a 30% chance we are already past human-level hardware (at human cost), a 45% chance it occurs by 2040, and a 25% chance it occurs later.5
Implications
These figures suggest that the period when we most expect human-level hardware has already begun, and we are a substantial part of the way through it. In the case that hardware progress matters a lot more than software progress, this means that we should expect to see human-level AI in the next several decades, or possibly in the past. This is some evidence against hardware progress being so important, but still overall makes human-level AI likely to be sooner than one might have thought without the evidence considered here.

Human-level hardware timeline


Chance date bias

There is modest evidence that people consistently forecast events later when asked the probability that the event occurs by a certain year, rather than the year in which a certain probability of the event will have accrued.
Details
In the 2016 ESPAI and its preparation, AI experts and Mechanical Turk workers both consistently gave later probability distributions for events relating to AI when asked to give the probability that the event would occur by a given year, rather than the year by which there was a certain probability. See more on the ESPAI page.
We do not know which framing produces more reliable answers. We have not seen this bias elsewhere.
The following figure shows an example for some key figures: the distributions with stars are consistently a little flatter than those with circles.
Figure 1. Median answers to questions about probabilities by dates ('fixed year') and dates for probabilities ('fixed probability'), for different occupations, all current occupations, and all tasks (HLMI).

Chance date bias


GoCAS talk on AI Impacts findings

Here is a video summary of some highlights from AI Impacts research over the past years, from the GoCAS Existential Risk workshop in Göteborg in September. Thanks to the folks there for recording it.

GoCAS talk on AI Impacts findings


Price performance Moore's Law seems slow

When people make predictions about AI, they often assume that computing hardware will carry on getting cheaper for the foreseeable future, at about the same rate that it usually does. Since this is such a common premise, and whether reality has yet proved it false is checkable, it seems good to check sometimes. So we did.
Looking up the price and performance of some hardware turned out to be a real mess, with conflicting numbers everywhere and the resolution of each error or confusion mostly just leading to several more errors and confusions.
I suppose the way people usually make meaningful figures depicting computer performance changing over time is that they are doing it over long enough periods of time that even if each point is only accurate to within three orders of magnitude, it is fine because the whole trend is traversing ten or fifteen orders of magnitude. But since I wanted to know what was happening in the last few years, this wouldn't do—half an order of magnitude of progress could be entirely lost in that much noise.
In the end, the two best looking sources of data we could find are the theoretical performance of GPUs (via Wikipedia), and Passmark's collection of performance records for their own benchmark. Neither is perfect, but both make it look like prices for computing are falling substantially slower than they were. Over the last couple of decades it had been taking about four years for computing to get ten times cheaper, and now (on these measures) it's taking more like twelve years. Which could in principle be to do with these measures being different from usual, but I think probably not.
There are quite a few confusions still to resolve here. For instance, in spite of showing slower progress, these numbers look a lot cheaper than what would have been predicted by extrapolating past trends (or sometimes more expensive). Which might be because we are comparing performance using different metrics, and converting between them badly. Different records of past trends seem to disagree with one another too, which is perhaps a hint. Or it could be that there was faster growth somewhere in between that we didn't see. Or we might not have caught all of the miscellaneous errors in this cursed investigation.
But before we get too bogged down trying to work these things out, I just wanted to say that price performance Moore's Law tentatively looks slower than usual.
See full investigation at: Recent Trend in the Cost of Computing

Price performance Moore’s Law seems slow


2017 trend in the cost of computing

The cheapest hardware prices (for single precision FLOPS/$) appear to be falling by around an order of magnitude every 10-16 years. This rate is slower than the trend of FLOPS/$ observed over the past quarter century, which was an order of magnitude every 4 years. There is no particular sign of slowing between 2011 and 2017.
Support
Background
Computing power available per dollar has increased fairly evenly by a factor of ten roughly every four years in the last quarter of a century (a phenomenon sometimes called 'price-performance Moore's Law'). Because this trend is important and regular, it is useful in predictions. For instance, it is often used to determine when the hardware for an AI mind might become cheap. This means that a primary way such predictions might err is if this trend in computing prices were to leave its long run trajectory. This must presumably happen eventually, and has purportedly happened with other exponential trends in information technology recently.1
This page outlines our assessment of whether the long run trend is on track very recently, as of late 2017. This differs from assessing the long run trend (as we do here) in that it requires recent and relatively precise data. Data that may be off by one order of magnitude is still useful when assessing a long run trend that grows by many orders of magnitude. But if we are judging whether the last five years of that trend are on track, it is important to have more accurate figures.
Sources of evidence
We sought public data on computing performance, initial price, and date of release for different pieces of computing hardware. We tried to cover different types of computing hardware, and to prioritize finding large, consistent datasets using comparable metrics, rather than one-off measurements. We searched for computing performance measured using the Linpack benchmark, or something similar.
We ran into many difficulties finding consistently measured performance in FLOPS for different machines, as well as prices for those same machines. What data we could find used a variety of different benchmarks. Sometimes performance was reported as 'FLOPS' without explanation. Twice the 'same' benchmarks turned out to give substantially different answers at different times, at least for some machines, apparently due to the benchmarks being updated. Performance figures cited often refer to 'theoretical peak performance', which is calculated from the computer's specifications, rather than measured, and is higher than actual performance.
Prices are also complicated, because each machine can have many sellers, and each price fluctuates over time. We tried to use the release price, the manufacturer's 'recommended customer price', or similar where possible. However, many machines don't seem to have readily available release prices.
These difficulties led to many errors and confusions, such that progress required running calculations, getting unbelievable results, and searching for an error that could have made them unbelievable. This process is likely to leave remaining errors at the end, and those errors are likely to be biased toward giving results that we find believable. We do not know of a good remedy for this, aside from welcoming further error-checking, and giving this warning.
Evidence
GPUs appear to be substantially cheaper than CPUs, cloud computing (including TPUs), or supercomputers.2 Since GPUs alone are at the frontier of price performance, we focus on them. We have two useful datasets: one of theoretical peak performance, gathered from Wikipedia, and one of empirical performance, from Passmark.
GPU theoretical peak performance
We collected data from several Wikipedia pages, supplemented with other sources for some dates and prices.3 We think all of the performance numbers are theoretical peak performance, generally calculated from specifications given by the developer, but we have not checked Wikipedia's sources or calculations thoroughly. Our impression is that the prices given are recommended prices at launch, by the developers of the hardware, though again we have only checked a few of them.
We look at Nvidia and AMD GPUs and Xeon Phi processors here because they are the machines for which we could find data on Wikipedia easily. However, Nvidia and AMD are the leading producers of GPUs, so this should cover the popular machines. We excluded many machines because they did not have prices listed.
Figure 1 shows performance (single precision) over time for processors for which we could find all of the requisite data.
The recent rate of progress in this figure looks like somewhere between half an order of magnitude in the past eight years and an order of magnitude in the past ten, for an order of magnitude about every 10-16 years. We don't think the figure shows particular slowing down—the most cost-effective hardware has not improved in almost a year, but that is usual in the rest of the figure.
Figure 1
We also collected double precision performance figures for these machines, but the machines do not appear to be optimized for double precision performance,4 so we focus on single precision.
Peak theoretical performance is generally higher than actual performance, but our impression is that this should be by a roughly constant factor across time, so not make a difference to the trend.
GPU Passmark value
Passmark maintains a collection of benchmark results online, for both CPUs and GPUs. They also collect prices, and calculate price for performance (though it was not clear to us on brief inspection where their prices come from). Their performance measure is from their own benchmark, which we do not know a lot about. This makes their absolute prices hard to compare to others using more common measures, but the trend in progress should be more comparable.
We used archive.org to collect old versions of Passmark's page of the most cost-effective GPUs available, to get a history of price for passmark performance. The prices are from the time of the archive, not necessarily from when the hardware was new. That is, if we collected all of the results on the page on January 1, 2013, it might contain hardware that was built in 2010 and has maybe come down in price due to being old. You might wonder whether this means we are just getting a lot of really cheap old hardware with hardly any performance, which might be bad in other ways and so not represent a realistic price of hardware. This is possible, however given that people show interest in this (for instance, Passmark keep these records) it would be surprising to us if this metric mostly caught useless hardware.
Figure 2: Top GPU passmark performance per dollar scores over time.

Figure 3: The same data in Figure 1, showing progress for different percentiles.

Figure 4: Figure 3, with a log y axis.
We are broadly interested in the cheapest hardware available, but we probably don't want to look at the very cheapest in data like this, because it seems likely to be due to error or other meaningless exploitation of the particular metric.5 The 95th percentile machines (out of the top 50) appear to be relatively stable, so are probably close to the cheapest hardware without catching too many outliers. For this reason, we take them as a proxy for the cheapest hardware.
Figure 4 shows the 95th percentile fits an exponential trendline quite well, with a doubling time of 3.7 years, for an order of magnitude every 12 years. This has been fairly consistent, and shows no sign of slowing by early 2017. This supports the 10-16 year time we estimated from the Wikipedia theoretical performance above.

Other sources we investigated, but did not find relevant

The Wikipedia page on FLOPS contains a history of GFLOPS over time. The recent datapoints appear to overlap with the theoretical performance figures we have already.
Google has developed Tensor Processing Units (TPUs) that specialize in computation for machine learning. Based on information from Google, we estimate that they perform around 1.05 GFLOPS/$.
In 2015, cloud computing appeared to be around a hundred times more expensive than other forms of computing.6 Since then the price appears to have roughly halved.7 So cloud computing is not a competitive way to buy FLOPS all else equal, and the price of FLOPS may be a small influence on the cloud-computing price trend, making the trend less relevant to this investigation.
Top supercomputers perform at around $3/GFLOPS, so they do not appear to be on the forefront of cheap performance. See Price performance trend in top supercomputers for more details.
Geekbench has empirical performance numbers for many systems, but their latest version does not seem to have anything for GPUs. We looked at a small number of popular CPUs on Geekbench from the past five years, and found the cheapest to be around $0.71/GFLOPS. However there appear to be 5x disparities between different versions of Geekbench, which makes it less useful for fine-grained estimates.

Conclusions
We have seen that the theoretical peak single-precision performance of GPUs is improving at about an order of magnitude every 10-16 years. And that the Passmark performance/$ trend is improving by an order of magnitude every 12 years. These are slower than the long run price-performance trends of an order of magnitude every eight years (75 year trend) or four years (25 year trend).
The longer run trends are based on a slightly different set of measures, which might explain a difference in rates of progress.
Within these datasets the pace of progress does not appear to be slower in recent years relative to earlier ones.

2017 trend in the cost of computing


Price-performance trend in top supercomputers

A top supercomputer can perform a GFLOP for around $3, in 2017.
The price of performance in top supercomputers continues to fall, as of 2016.
Details
TOP500.org maintains a list of top supercomputers and their performance on the Linpack benchmark. The figure below is based on empirical performance figures ('Rmax') from Top500 and price figures collected from a variety of less credible sources, for nine of the ten highest performing supercomputers (we couldn't find a price for the tenth). Our data and sources are here.
Sunway Teihu Light performs the cheapest GFLOPS, at $2.94/GFLOPS. This is around one hundred times more expensive than peak theoretical performance of certain GPUs, but we do not know why there is such a difference (peak performance is generally higher than actual performance, but by closer to a factor of two).
There appears to be a downward trend in price, but it is not consistent, and with so few data points its slope is ambiguous. The best price for performance roughly halved in the last 4-5 years, for a 10x drop in 13-17 years. The K computer in 2011 was much more expensive, but appears to have been substantially more expensive than earlier computers.


Computing hardware performance data collections

This is a list of public datasets that we know of containing either measured or theoretical performance numbers for computer processors.
List

Top 500 maintains a list of the top 500 supercomputers, updated every six months. It includes measured performance.
List of Nvidia Graphics Processing Units contains GFLOPS figures for a large number of GPUs. Probably they are all theoretical peak performance numbers. It also contains release dates and release prices.
List of AMD Graphics Processing Units is much like the list of Nvidia GPUs, but for the other leading GPU brand.
Wikipedia's FLOPS page contains a small amount of data, seemingly empirical, from a variety of sources.
Wikipedia has other small collections of theoretical performance data. For instance on the Intel Xeon Phi page.
Moravec has perhaps the oldest and best known dataset. We link to an article discussing it, but its actual page was down last we checked.
Nordhaus expands on Moravec's data.
Koh and Magee expand on Moravec's data.
Rieber and Muehlhauser did have a dataset (discussed here) but links to it appear to be broken.
John McCallum's dataset (doesn't load at time of writing, but is discussed in Sandberg and Bostrom 2008 and on our page on trends in the cost of computing)
Passmark has a huge quantity of empirical performance data, for CPUs and GPUs. However it is all in terms of their own benchmark, so hard to compare to other things. They also list current prices. Looking at it over time (via archive.org) can let you also see past prices. Doing so suggests that they change their benchmarks on occasion, which makes it even harder to interpret what they mean.
Geekbench Browser collects empirical performance data from people testing their computers with Geekbench's service. They list many benchmark numbers for many computers. However identically named benchmark figures from 'Geekbench v4' vs. 'Geekbench v3' for the same hardware differ a lot (one of us recollects about a factor of five), apparently because they changed what the benchmark actually was then. This suggests care should be taken to use numbers from the same version of Geekbench, and also that any version is not necessarily comparable to other apparently identical measures from elsewhere. We are also not sure whether differences in benchmark meaning only occur between saliently labeled versions.
Export compliance metrics for Intel Processors is a collection of PDFs listing processors alongside a number for 'FLOP', which we suppose is related to FLOPS. It does not contain much explanation, and has some worrying characteristics.1
Karl Rupp has collected some data and made it available. He has also blogged about it here and here. However he says he got it from a combination of the Intel compliance metrics (listed above), and the list of Intel Xeon Microprocessors (below), and a) the export compliance metrics data seems strange, and b) we couldn't actually track down his data in those sources. Possibly we are misunderstanding the export compliance metrics, and he is interpreting them correctly, resolving both problems.
Asteroids@home lists Whetstone benchmark GFLOPS per core by CPU model for computers participating in their project.
The Microway knowledge center has a lot of pages containing at least some theoretical peak performance numbers (see any called 'detailed specifications of —', but most of the numbers on each page are inside figures, and so hard to export or read in detail.

Other useful hardware data

List of Intel Xeon Microprocessors does not include figures for FLOPS, but has price and release date data.


2016 ESPAI Narrow AI task forecast timeline

This is an interactive timeline we made, illustrating the median dates when respondents said they expected a 10%, 50% and 90% chance of different tasks being automatable, in the 2016 Expert Survey on progress in AI (further details on that page).
Timeline