Conversation with Steve Potter

*Posted 13 July 2015*

### Participants

[![](http://aiimpacts.org/wp-content/uploads/2015/07/SteveholdingMEA.jpg)](http://aiimpacts.org/wp-content/uploads/2015/07/SteveholdingMEA.jpg)**Figure 1:** Professor Steve Potter

* **[Professor Steve Potter](https://neurolab.gatech.edu/labs/potter/steve-potter)** – Associate Professor, Laboratory of NeuroEngineering, Coulter Department of Biomedical Engineering, Georgia Institute of Technology
* **[Katja Grace](http://katjagrace.com)** – Machine Intelligence Research Institute (MIRI)

**Note**: These notes were compiled by MIRI and give an overview of the major points made by Professor Steve Potter.

### Summary

Katja Grace spoke with Professor Steve Potter of Georgia Institute of Technology as part of AI Impacts’ investigation into the implications of neuroscience for artificial intelligence (AI). Conversation topics included how neuroscience now contributes to AI and how it might contribute in the future.

How has neuroscience helped AI in the past?
-------------------------------------------

Professor Potter found it difficult to think of examples where neuroscience has helped with higher level ideas in AI. Some elements of cognitive science have been implemented in AI, but these may not be biologically based. He described two broad instances of neuroscience-inspired projects.

### Subsumption architecture

Past work in AI has focused on disembodied computers with little work in robotics. Researchers now understand that AI does not need to be centralized; it can also take on physical form. Subsumption architecture is one way that robotics has advanced. This involves the coupling of sensory information to action selection. For example, Professor Rodney Brooks at MIT has developed robotic legs that respond to certain sensory signals. These legs also send messages to one another to control their movement. Professor Potter believes that this work could have been based on neuroscience, but it is not clear how much Professor Brooks was inspired by neuroscience while working on this project; the idea may have come to him independently.

### Neuromorphic engineering

This type of engineering employs properties of biological nervous systems in neural system AI, such as perception and motor control. One aspect of brain function can be imitated with silicon chips through pulse-coding, where analog signals are sent and received in tiny pulses. An application for this is in camera development by mimicking pulse-coded signals between the brain and the retina.

How is neuroscience contributing to AI today?
---------------------------------------------

Although neuroscience has not assisted AI development much in the past, Professor Potter has confidence that this intersection has considerable potential. This is because the brain works well in areas where AI falls short. For example, AI needs to improve how it works in real time in the real world. Self-driving cars may be improved through examining how a model organism, such as a bee, would respond to an analogous situation. Professor Potter believes it would be worthwhile research to record how humans use their brains while driving. Brain algorithms developed from this could be implemented into car design.

Current work at the intersection of neuroscience and AI include the following:

**Artificial neural networks**

Most researchers at the intersection of AI and neuroscience are examining artificial neural networks, and might describe their work as ‘neural simulations’. These networks are a family of statistical learning models that are inspired by biological neural networks. Hardware in this discipline includes neuromorphic chips, while software includes work in pattern recognition. This includes handwriting recognition and finding military tanks in aerial photographs. The translation of these networks into useful products for both hardware and software applications has been slow.

**Hybrots**

Professor Potter has helped develop hybrots, which are hybrid living tissue interfaced with robotic machines: robots controlled by neurons. Silent Barrage was an early hybrot that drew on paper attached to pillars. Video was taken of people viewing the Silent Barrage hybrots. This data was transmitted back to Prof. Potter’s lab, where it was used to trigger electrical stimulation in the living brain of the system. This was a petri dish interfaced to a culture of rat cortical neurons. This work is currently being expanded to include more types of hybrots. In one the control will be by living neurons, while the other will be controlled by a simulated neural network.

Meart (MultiElectrode Array Art) was an earlier hybrot. Controlled by a brain composed of rat neuron cells, it used robotic arms to draw on paper. It never progressed past the toddler stage of scribbling.

How is neuroscience likely to help AI in the future?
----------------------------------------------------

A particular line of research in neuroscience that is likely to help with AI is the concept of delays. Computer design is often optimized to reduce the amount of time between command and execution. The brain though may take milliseconds longer to respond. However delays in the brain were evolved to respond to the timing of the real world and are a useful part of the brain’s learning process.

Neuroscience probably also has potential to help AI in searching databases. It appears that the brain has methods for this that are completely unlike those used in computers, though we do not yet know what the brain’s methods are. One example given of the brain’s impressive abilities here is that Professor Potter can meet a new person and instantly be confident that he has never seen that person before.

How long will it take to duplicate human intelligence?
------------------------------------------------------

It will be hard to say when this has been achieved; success is happening at different rates for different applications. The future of neuroscience in AI will most likely involve taking elements of neuroscience and applying them to AI; it is unlikely that there will be a wait until we have a good understanding of the brain, then an export of that knowledge complete to AI.

Professor Potter greatly respects Ray Kurzweil, but does not think that he has an in depth knowledge of neuroscience. Professor Potter thinks the brain is much more complex than Kurzweil appears to believe, and that ‘duplicating’ human intelligence will take far longer than Kurzweil predicts. In Professor Potter’s consideration, it will take over a hundred years to develop a robot butler that can convince you that it is human.

### **Challenges to progress**

#### Lack of collaboration

Neuroscience-inspired AI progress has been hampered because researchers across neuroscience and AI seldom collaborate with one another. This may be from disinterest or limited understanding of each other’s fields. Neuroscientists are not generally interested in the goal of creating human-level artificial intelligence. Professor Potter believes that of the roughly 30,000 people who attend the Society for Neuroscience, approximately 20 people want this. Most neuroscientists, for example, want to learn how something works instead of learning how it can be applied (e.g. learning how the auditory system works instead of developing a new hearing aid). If more people saw benefits in applying neuroscience to AI and in particular human-level AI, there would be greater progress. However, the scale is hard to predict. There is the potential for very much more rapid progress. For researchers to move their projects in this direction, the priorities of funding agencies would first have to move; these as these effectively dictate which projects move forward.

#### Funding

Funding for work at the intersection of neuroscience and AI may be hard to find. The National Institute of Health (NIH) funds only health-related work and has not funded AI projects. The National Science Foundation (NSF) may not think the work fits its requirement of being basic science research; it may be too applied. NSF though, is more open-minded to funding research on AI than NIH is. The military is also interested in AI research. Outside (of )the U.S., the European Union (EU) funds cross-disciplinary work in neuroscience and AI.

##### National Science Foundation (NSF) funding

NSF had a call for radical proposals, from which Professor Potter received a four-year-long grant to apply neuroscience to electrical grid systems. Collaborators included a power engineer and people studying neural networks. The group was interested in addressing the U.S.’s large and uneven power supply and usage. The electrical grid has become increasingly difficult to control because of geographically varying differences in input and output.

Professor Potter believes that if people in neuroscience, AI, neural networks, and computer design talked more, this would bring progress. However, there were some challenges with this collaborative electrical grid systems project that need to be addressed. For example, the researchers needed to spend considerable time educating one another about their respective fields. It was also difficult to communicate with collaborators across the country; NSF paid for only one meeting per year, and the nuances of in-person interaction seem important for bringing together such diverse groups of people and reaping the benefits of their creative communication.

Other people working in this field
----------------------------------

* **Henry Markram** – Professor, École Polytechnique Fédérale de Lausanne, Laboratory of Neural Microcircuitry. Using EU funding, he creates realistic computer models of the brain, one piece at a time.
* **Rodney Douglas** – Professor Emeritus, University of Zurich, Institute of Neuroinformatics. He is a neuromorphic engineer who worked on emulated brain function.
* **Carver Mead –** Gordon and Betty Moore Professor of Engineering and Applied Science Emeritus, California Institute of Technology. He was a founding father of neuromorphic engineering.
* **Rodney Brooks** – Panasonic Professor of Robotics Emeritus, Massachusetts Institute of Technology (MIT). He was a pioneer in studying distributed intelligence and developed subsumption architecture.
* **Andy Clark** – Professor of Logic and Metaphysics, University of Edinburgh. He does work on embodiment, artificial intelligence, and philosophy.
* **Jose Carmena** – Associate Professor of Electrical Engineering and Neuroscience, University of California-Berkeley. Co-Director of the Center of Neural Engineering and Prostheses, University of California-Berkeley, University of California-San Francisco. He has researched the impact of electrical stimulation on sensorimotor learning and control in rats.
* **Guy Ben-Ary** – Manager, University of Western Australia, CELLCentral in the School of Anatomy and Human Biology. He is an artist and researcher who uses biologically related technology in his work. He worked in collaboration with Professor Potter on Silent Barrage.
* **Wolfgang Maass** – Professor of Computer Science, Graz University of Technology. He is doing research on artificial neural networks.
* **Thad Starner** – Assistant Professor, Georgia Institute of Technology, College of Computing. He applies biological concepts into developing wearable computing devices.
* **Jennifer Hasler** – Professor, Georgia Institute of Technology, Bioengineering and Electronic Design and Applications. She has studied neuromorphic hardware.

Conversation with Steve Potter


New funding for AI Impacts

*By Katja Grace, 4 July 2015*

AI Impacts has received two grants! We are grateful to the [Future of Humanity Institute](http://fhi.ox.ac.uk) (FHI) for $8,700 to support work on the project until September 2015, and the [Future of Life Institute](http://futureoflife.org) (FLI) for $49,310 for another year of work after that. Together this is enough to have a part time researcher until September 2016, plus a little extra for things like workshops and running the website.

We are big fans of FHI and FLI, and are excited to be working alongside them.

The FLI grant was part of the [recent contest](http://futureoflife.org/misc/2015selection) which distributed around $7M funding from Elon Musk and the [Open Philanthropy Project](http://www.openphilanthropy.org/) to projects designed to keep AI robust and beneficial. The full list of projects to be funded is [here](http://futureoflife.org/misc/2015awardees). You can see part of our proposal [here](http://aiimpacts.org/wp-content/uploads/2015/07/AI-Impacts-narrative-for-FLI-grant.pdf).

This funding means that AI Impacts is no longer in urgent [need of support](http://aiimpacts.org/supporting-ai-impacts/). Further [donations](http://aiimpacts.org/donate/) will likely go to additional research through contract work, guest research, short term collaborations, and outsourceable data collection.

Many thanks to those whose support—in the form of both funding and other feedback—has brought AI Impacts this far.

New funding for AI Impacts


Update on all the AI predictions

*By Katja Grace, 5 June 2015*

For the last little while, we’ve been looking into [a dataset of individual AI predictions](http://aiimpacts.org/miri-ai-predictions-dataset/), collected by MIRI a couple of years ago. We also previously gathered all the [surveys](http://aiimpacts.org/ai-timeline-surveys/) about AI predictions that we could find. Together, these are all the public predictions of AI that we know of. So we just wrote up [a quick summary](http://aiimpacts.org/predictions-of-human-level-ai-timelines/) of what we have so far.

Here’s a picture of most of the predictions, from our summary:

[![xxx](http://aiimpacts.org/wp-content/uploads/2014/12/AI-predictions-summary-3-1024x737.jpg)](http://aiimpacts.org/wp-content/uploads/2014/12/AI-predictions-summary-3.jpg)**Figure 1:** Predictions from the [MIRI dataset](http://aiimpacts.org/miri-ai-predictions-dataset/) (red = [maxIY](http://aiimpacts.org/miri-ai-predictions-dataset/) ≈ ‘AI more likely than not after …’, and green = [minPY](http://aiimpacts.org/miri-ai-predictions-dataset/) ≈ ‘AI less likely than not before …’) and [surveys](http://aiimpacts.org/ai-timeline-surveys/). This figure excludes one prediction of 3012 made in 2012, and the [Hanson survey](http://aiimpacts.org/hanson-ai-expert-survey/), which doesn’t ask directly about prediction dates.
Recent [surveys](http://aiimpacts.org/ai-timeline-surveys/) seem to pretty reliably predict AI between 2040 and 2050, as you can see. The [earlier](http://aiimpacts.org/michie-survey/) [surveys](http://aiimpacts.org/bainbridge-survey/) [which](http://aiimpacts.org/klein-agi-survey/) [don’t](http://aiimpacts.org/ai50-survey/) fit this trend also had less uniform questions, whereas the last six surveys ask about the year in which there is a 50% chance that (something like) human-level AI will exist. The entire set of individual predictions has a median somewhere in the 2030s, depending on how you count. However for predictions made since 2000, the median is 2042 ([minPY](http://aiimpacts.org/miri-ai-predictions-dataset/)), in line with the surveys. [The surveys that ask](http://aiimpacts.org/ai-timeline-surveys/) also consistently get median dates for a 10% chance of AI in the 2020s.

This consistency seems interesting, and these dates seem fairly soon. If we took these estimates seriously, and people really meant at least ‘AI that could replace most humans in their jobs’, the predictions of ordinary AI researchers seem pretty concerning. 2040 is not far off, and the 2020s seem too close for us to be prepared to deal with moderate chances of AI, at the current pace.

We are not sure what to make of these predictions. Predictions about AI are frequently distrusted, though often alongside complaints that seem weak to us. For instance that [people are biased to predict AI twenty years in the future](https://intelligence.org/files/PredictingAI.pdf), or [just before their own deaths](http://aiimpacts.org/the-maes-garreau-law/); that [AI researchers have always been very optimistic and continually proven wrong](http://aiimpacts.org/michie-and-overoptimism/); [that experts and novices make the same predictions](https://intelligence.org/files/PredictingAI.pdf) (**Edit (6/28/2016):** now found to be [based on an error](http://aiimpacts.org/error-in-armstrong-and-sotala-2012/)); or that [failed predictions of the past look like current predictions](http://aiimpacts.org/similarity-between-historical-and-contemporary-ai-predictions/). There really do seem to be selection biases, from [people who are optimistic about AGI](http://aiimpacts.org/group-differences-in-ai-predictions/) [working in the field](http://aiimpacts.org/why-do-agi-researchers-expect-ai-so-soon/) [for instance](http://aiimpacts.org/bias-from-optimistic-predictors/), and from [shorter predictions being more published](http://aiimpacts.org/short-prediction-publication-biases/). However there are ways to avoid these.

There seem to be a few good [reasons to distrust these predictions](http://aiimpacts.org/accuracy-of-ai-predictions/) however. First, it’s not clear that people can predict these kinds of events well in any field, at least without the help of tools. Relatedly, it’s not clear what tools and other resources people used in the creation of these predictions. Did they model the situation carefully, or just report their gut reactions? My guess is near the ‘gut reaction’ end of the spectrum, based on [looking for reasoning](http://aiimpacts.org/list-of-analyses-of-time-to-human-level-ai/) and finding only a little. Often gut reactions are reliable, but I don’t expect them to be so, on their own, in an area such as forecasting novel and revolutionary technologies.

Thirdly, phrases like, ‘human-level AI arrives’ appear to stand for [different events](http://aiimpacts.org/human-level-ai/) for different people. Sometimes people are talking about almost perfect human replicas, sometimes software entities that can undercut a human at work without resembling them much at all, sometimes human-like thinking styles which are far from being able to replace us. Sometimes they are talking about human-level abilities at human cost, sometimes at any cost. Sometimes consciousness is required, sometimes poetry is, sometimes calculating ability suffices. Our impressions from talking to people are that ‘AI predictions’ mean a wide variety of things. So the collection of predictions is probably about different events, which we might reasonably expect to happen at fairly different times. Before trusting experts here, it seems key to check we know what they are talking about.

Given all of these things, I don’t trust these predictions a huge amount. However I expect they are somewhat informative, and there are not a lot of good sources to trust at present.

The next things I’d like to know in this area:

* What do experts actually believe about human-level AI timelines, if you check fairly thoroughly that they are talking about what you think they are talking about, and aren’t making obviously different assumptions about other matters?
* How reliable are similar predictions? For instance, predictions of novel technologies, predictions of economic upheaval, predictions of disaster?
* [Why do](http://aiimpacts.org/are-ai-surveys-seeing-the-inside-view/) the results of the [Hanson survey](http://aiimpacts.org/hanson-ai-expert-survey/) conflict with the other surveys?
* How do people make the predictions they make? (e.g. How often are they thinking of hardware trends? Using intuition? Following the consensus of others?)
* [Why](http://aiimpacts.org/why-do-agi-researchers-expect-ai-so-soon/) are AGI researchers so much more optimistic than AI researchers, and are AI researchers [so much more optimistic](http://aiimpacts.org/group-differences-in-ai-predictions/) than others?
* What disagreements between AI researchers produce their different predictions?
* What do AI researchers know that informs their predictions that people outside the field (like me) do not know? (What do they know that doesn’t inform their predictions, but should?)

Hopefully we’ll be looking more into some of these things soon.

Update on all the AI predictions


Predictions of Human-Level AI Timelines

***Note: This page is out of date. See an [up-to-date version of this page](https://wiki.aiimpacts.org/doku.php?id=ai_timelines:predictions_of_human-level_ai_timelines) on our wiki.**

Updated 5 June 2015*

We know of around 1,300 public predictions of when human-level AI will arrive, of varying levels of quality. These include predictions from individual statements and larger surveys. Median predictions tend to be between 2030 and 2055 for predictions made since 2000, across different subgroups of predictors.

Details
-------

### The landscape of AI predictions

Predictions of when [human-level AI](http://aiimpacts.org/human-level-ai/) will be achieved exist in the form of [surveys](http://aiimpacts.org/ai-timeline-surveys/) and public statements (e.g. in articles, books or interviews). Some statements backed by analysis are discussed [here](http://aiimpacts.org/list-of-analyses-of-time-to-human-level-ai/). Many more statements have been [collected](http://aiimpacts.org/miri-ai-predictions-dataset/) by [MIRI](http://intelligence.org/). Figure 1 illustrates almost all of the predictions we know about, though most are aggregated there into survey medians. Altogether, we know of around 1,300 public predictions of when human-level AI will arrive, though 888 are from a single informal online poll. We know of ten surveys that address this question directly (plus [a set of interviews](http://aiimpacts.org/kruel-ai-survey/) which we sometimes treat as a survey but here count here as individual statements, and [a survey](http://aiimpacts.org/hanson-ai-expert-survey/) which asks about progress so far as a fraction of what is required for human-level AI). Only 65 predictions that we know of are not part of surveys.

### Summary of findings

[![](http://aiimpacts.org/wp-content/uploads/2014/12/AI-predictions-summary-3-1024x737.jpg)](http://aiimpacts.org/wp-content/uploads/2014/12/AI-predictions-summary-3.jpg)**Figure 1:** Predictions from the [MIRI dataset](http://aiimpacts.org/miri-ai-predictions-dataset/) (red = [maxIY](http://aiimpacts.org/miri-ai-predictions-dataset/) ≈ ‘AI more likely than not after …’, and green = [minPY](http://aiimpacts.org/miri-ai-predictions-dataset/) ≈ ‘AI less likely than not before …’) and [surveys](http://aiimpacts.org/ai-timeline-surveys/). This figure excludes one prediction of 3012 made in 2012, and the [Hanson survey](http://aiimpacts.org/hanson-ai-expert-survey/), which doesn’t ask directly about prediction dates.

Recent [surveys](http://aiimpacts.org/ai-timeline-surveys/) tend to have median dates between 2040 and 2050. All six of the surveys which ask for the year in which human-level AI will have arrived with 50% probability produce medians in this range (not including [Kruel’s interviews](http://aiimpacts.org/kruel-ai-survey/), which have a median of 2035, and are counted in the statements here). The median prediction in [statements](http://aiimpacts.org/miri-ai-predictions-dataset/) is 2042, though predictions of AGI researchers and futurists have medians in the early 2030s. [Surveys](http://aiimpacts.org/ai-timeline-surveys/) give median estimates for a 10% chance of human-level AI in the 2020s. We have not attempted to adjust these figures for [biases](http://aiimpacts.org/accuracy-of-ai-predictions/).

### Implications

Expert predictions about AI timelines are often considered uninformative. Evidence that predictions are less informative than in other messy fields [appears](http://aiimpacts.org/accuracy-of-ai-predictions/) to be weak. We have not evaluated baseline prediction accuracy in such fields however. We expect survey results and predictions from those further from AGI are more accurate than other sources, due to [selection biases](http://aiimpacts.org/accuracy-of-ai-predictions/). The differences between these sources appear to be a small number of decades.

Predictions of Human-Level AI Timelines


Accuracy of AI Predictions

*Updated 4 June 2015*

It is unclear how informative we should expect expert predictions about AI timelines to be. Individual predictions are undoubtedly often off by many decades, since they disagree with each other. However their aggregate may still be quite informative. The main potential reason we know of to doubt the accuracy of expert predictions is that experts are generally poor predictors in many areas, and AI looks likely to be one of them. However we have not investigated how accurate ‘poor’ is, or whether AI really is such a case.

Predictions of AI timelines are likely to be biased toward optimism by roughly decades, especially if they are [voluntary statements](http://aiimpacts.org/bias-from-optimistic-predictors/) rather than surveys, and especially if they are [from populations selected for optimism](http://aiimpacts.org/group-differences-in-ai-predictions/). We expect these factors account for less than a decade and around two decades’ difference in median predictions respectively.

Support
-------

### Considerations regarding accuracy

A number of reasons have been suggested for distrusting predictions about AI timelines:

* **Models of areas where people predict well**Research has produced a characterization of situations where experts predict well and where they do not. See table 1 [here](http://intelligence.org/files/PredictingAI.pdf). AI appears to fall into several classes that go with worse predictions. However we have not investigated this evidence in depth, or the extent to which these factors purportedly influence prediction quality.
* **Expert predictions are generally poor**Experts are notoriously poor predictors. However our impression is that this is because of their disappointing inability to predict some things well, rather than across the board failure. For instance, experts can predict the Higgs boson’s existence, outcomes of chemical reactions, and astronomical phenomena. So the question falls back to where AI falls in the spectrum of expert predictability, discussed in the last point.
* **Disparate predictions**One sign that AI predictions are not very accurate is that they differ over a range of a century or so. This strongly suggests that many individual predictions are inaccurate, though not that the aggregate distribution is uninformative.
* **[Similarity of old and new predictions](http://aiimpacts.org/similarity-between-historical-and-contemporary-ai-predictions/)**Older predictions seem to form a fairly similar distribution to more recent predictions, except for very old predictions. This is weak evidence that new predictions are not strongly affected by evidence, and are therefore more likely to be inaccurate.
* **Similarity of expert and lay opinions**[Armstrong and Sotala](https://intelligence.org/files/PredictingAI.pdf) found that expert and non-expert predictions look very similar.[1](https://aiimpacts.org/accuracy-of-ai-predictions/#easy-footnote-bottom-1-166 "‘Using a database of 95 AI timeline predictions, it will show that these expectations are borne out in practice: expert predictions contradict each other considerably, and are indistinguishable from non-expert predictions and past failed predictions.’ – Armstrong and Sotala 2012, p1") This finding is in doubt at the time of writing, due to errors in the analysis. If it were true, this would be weak evidence against experts having relevant expertise, since if they did, this might cause a difference with the opinions of lay-people. Note that it may also not, if the laypeople go to experts for information.
* **Predictions are about different things and often misinterpreted**Comments made around predictions of human-level AI suggest that predictors are sometimes thinking about different events as ‘AI arriving’.[2](https://aiimpacts.org/accuracy-of-ai-predictions/#easy-footnote-bottom-2-166 "For instance, in an interview with Alexander Kruel, Pei Wang says ‘Here by “roughly as good as humans” I mean the AI will follow roughly the same principles as human in information processing, though it does not mean that the system will have the same behavior or capability as human, due to the difference in body, experience, motivation, etc.’Nils Nilson interprets the question differently: ‘Because human intelligence is so multi-faceted, your question really should be divided into each of the many components of intelligence…A while back I wrote an essay about a replacement for the Turing test. It was called the “Employment Test.”  (See: http://ai.stanford.edu/~nilsson/OnlinePubs-Nils/General_Essays/AIMag26-04-HLAI.pdf)  How many of the many, many jobs that humans do can be done by machines?  I’ll rephrase your question to be: When will AI be able to perform around 80% of these jobs as well or better than humans perform?These researchers were asked for their predictions in a context conducive to elaboration. Had they been surveyed more briefly (as in most surveys), or chosen not to elaborate, at least one would have been misunderstood. It is an open question whether 80% of jobs being automated will roughly coincide with artificial minds using similar information processing principles to humans.") Even when they are predictions about the same event, ‘prediction’ can mean different things. One person might ‘predict’ the year when they think human-level AI is more likely than not, while another ‘predicts’ the year that AI seems almost certain.

This list is not necessarily complete.

### Purported biases

A number of biases have been posited to affect predictions of human-level AI:

* **[Selection biases from optimistic experts](http://aiimpacts.org/bias-from-optimistic-predictors/)**

Becoming an expert is probably correlated with independent optimism about the field, and experts make most of the credible predictions. We expect this to push median estimates earlier by less than a few decades.
* **[Biases from short-term predictions being recorded](http://aiimpacts.org/short-prediction-publication-biases/)**There are a few reasons to expect recorded public predictions to be biased toward shorter timescales. Overall these probably make public statements less than a decade more optimistic.
* **[Maes-Garreau law](http://aiimpacts.org/the-maes-garreau-law/)**The Maes-Garreau law is a posited tendency for people to predict important technologies not long before their own likely death. It probably doesn’t afflict predictions of human-level AI substantially.
* **Fixed period bias**

There is a stereotype that people tend to predict AI in 20-30 years. There is weak evidence of such a tendency around 20 years, though little evidence that this is due to a bias (that we know of).

### Conclusions

AI appears to exhibit several qualities characteristic of areas that people are not good at predicting. Individual AI predictions appear to be inaccurate by many decades in virtue of their disagreement. Other grounds for particularly distrusting AI predictions seem to offer weak evidence against them, if any. Our current guess is that AI predictions are less reliable than many kinds of prediction, though still potentially fairly informative.

Biases toward early estimates appear to exist, as a result of optimistic people becoming experts, and optimistic predictions being more likely to be published for various reasons. These are the only plausible substantial biases we know of.

Accuracy of AI Predictions


Publication biases toward shorter predictions

We expect predictions that [human-level AI](http://aiimpacts.org/human-level-ai/ "Human-Level AI") will come sooner to be recorded publicly more often, for a few reasons. Public statements are probably more optimistic than surveys because of such effects. The difference appears to be less than a decade, for median predictions.

Support
-------

### Plausible biases

Below we outline five reasons for expecting earlier predictions to be stated and publicized more than later ones. We do not know of compelling reasons to expect longer term predictions to be publicized more, unless they are so distant as to also fit under the first bias discussed below.

#### Bias from not stating the obvious

In many circumstances, people are disproportionately likely to state beliefs that they think others do not hold. For example, [“homeopathy works”](https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=%22homeopathy+works%22) gets more Google hits than [“homeopathy doesn’t work”](https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=%22homeopathy+doesn%27t+work%22), though this probably doesn’t reflect popular beliefs on the matter. Making public predictions seems likely to be a circumstance with this character. Predictions are often made in books and articles which are intended to be interesting and surprising, rather than by people whose job it is to report on AI forecasts regardless of how far away they are. Thus we expect people with unusual positions on AI timelines to be more likely to state them. This should produce a bias toward both very short and very long predictions being published.

#### Bias from the near future being more concerning

Artificial intelligence will arguably be hugely important, whether as a positive or negative influence on the world. Consequently, people are motivated to talk about its social implications. The degree of concern motivated by impending events tends to increase sharply with proximity to the event. Thus people who expect human-level AI in a decade will tend to be more concerned about it than people who expect human-level AI to take a century, and so will talk about it more. Similarly, publishers are probably more interested in producing books and articles making more concerning claims.

#### Bias from ignoring reverse predictions

If you search for people predicting AI by a given date, you can get downwardly biased estimates by taking predictions from sources where people are asked about certain specific dates, and respond that AI will or will not have arrived by that date. If people respond ‘AI will arrive by X’ and ‘AI will not arrive by X’ as appropriate, the former can look like ‘predictions’ while the latter do not.

This bias affected some data in the [MIRI dataset](https://sites.google.com/site/aiimpactslibrary/ai-timelines/predictions-of-human-level-ai-dates/miri-ai-predictions-dataset), though we have tried to minimize it now. For example, [this bet](http://longbets.org/1/) (“By 2029 no computer – or “machine intelligence” – will have passed the Turing Test.”) is interpreted in the above collection as Kurzweil making a prediction, but not as Kapor making a prediction. It also contained several estimates of 70 years, taken from a group who appear to have been asked whether AI would come within 70 years, much later, or never. The ‘within 70 years’ estimates are recorded as predictions, while the others ignored, producing ’70 years’ estimates, almost regardless of the overall opinions of the group surveyed. In a population of people with a range of beliefs, this method of recording predictions would produce ‘predictions’ largely determined by which year was asked about.

#### Bias from unavoidably ignoring reverse predictions

The aforementioned bias arises from an error that can be avoided in recording data, where predictions and reverse predictions are available. However similar types of bias may exist more subtly. Such bias could arise where people informally volunteer opinions in a discussion about some period in the future. People with shorter estimates who can make a positive statement might feel more as though they have something to say, while those who believe there will not be AI at that time do not. For instance, suppose ten people write books about the year 2050, and each predicts AI in a different decade in the 21st Century. Those who predict it prior to 2050 will mention it, and be registered as a prediction of before 2050. Those who predict it after 2050 will not mention it, and not be registered as making a prediction. This could also be hard to avoid if predictions reach you through a filter of others registering them as predictions.

#### Selection bias from optimistic experts

*Main article: **[Selection bias from optimistic experts](http://aiimpacts.org/bias-from-optimistic-predictors/)***

Some factors that cause people to make predictions about AI are likely to correlate with expectations of human-level AI arriving sooner. Experts are better positioned to make credible predictions about their field of expertise than more distant observers are. However since people are more likely to join a field if they are more optimistic about progress there, we might expect their testimony to be biased toward optimism.

### Measuring these biases

These forms of bias (except the last) seem to us as if they should be much weaker in survey data than voluntary statements, for the following reasons:

* Surveys come with a default of answering questions, so one does not need a strong reason or social justification for doing so (e.g. having a surprising claim, or wanting to elicit concern).
* One can assess whether a survey ignores reverse predictions, and there appears to be little risk of invisible reverse predictions.
* Participation in surveys is mostly determined before the questions are viewed, for a large number of questions at once. This allows less opportunity for views on the question to affect participation.
* Participation in surveys is relatively cheap, so people who care little about expressing any particular view are likely to participate for reasons of orthogonal incentives, whereas costly communications (such as writing a book) are likely to be sensible only for those with a strong interest in promoting a specific message.
* Participation in surveys is usually anonymous, so relatively unsatisfactory for people who particularly want to associate with a specific view, further aligning the incentives of those who want to communicate with those who don’t care.
* Much larger fractions of people participate in surveys when requested than volunteer predictions in highly publicized arenas, which lessens the possibility for selection bias.

We think publication biases such as those described here are reasonably likely on theoretical grounds. We are also not aware of other reasons to expect surveys and statements to differ in their optimism about AI timelines. Thus we can compare the predictions of statements and surveys to estimate the size of these biases. Survey data [appears to](http://aiimpacts.org/miri-ai-predictions-dataset/) produce median predictions of human-level AI somewhat later than similar public statements do: less than a decade, at a very rough estimate. Thus we think some combination of these biases probably exist, and introduce less than a decade of error to median estimates.

Implications
------------

**[Accuracy of AI predictions](https://sites.google.com/site/aiimpactslibrary/ai-timelines/predictions-of-human-level-ai-dates/interpretation-of-ai-predictions/accuracy-of-ai-predictions):** AI predictions made in statements are probably biased toward being early, by less than a decade. This suggests both that predictions overall are probably slightly earlier than they would be otherwise, and surveys should be trusted more relative to statements (though there may be other considerations there).

**Collecting data**: When collecting data about AI predictions, it is important to avoid introducing bias by recording opinions that AI is before some date while ignoring opinions that it is after that date.

**[MIRI dataset](https://sites.google.com/site/aiimpactslibrary/ai-timelines/predictions-of-human-level-ai-dates/miri-ai-predictions-dataset)**: The earlier version of the MIRI dataset is somewhat biased due to ignoring reverse predictions, however this has been at least partially resolved.


Selection bias from optimistic experts

Experts on AI probably systematically underestimate time to human-level AI, due to a selection bias. The same is more strongly true of AGI experts. The scale of such biases appears to be decades. Most public AI predictions are from AI and AGI researchers, so this bias is relevant to interpreting these predictions.

Details
-------

### Why we expect bias

We can model a person’s views on AI timelines as being influenced both by their knowledge of AI and other somewhat independent factors, such as their general optimism and their understanding of technological history. People who are initially more optimistic about progress in AI seem more likely to enter the field of AI than those who are less so. Thus we might expect experts in AI to be selected for being optimistic, for reasons independent of their expertise. Similarly, AI researchers presumably enter the subfield of AGI more if they are optimistic about human-level intelligence being feasible soon.

This means expert predictions should tend to be more optimistic than they would if they were made by random people who became well informed, and thus are probably overall too optimistic (setting aside any other biases we haven’t considered).

This reason to expect bias only applies to the extent that predictions are made based on personal judgments, rather than explicit procedures that can be verified to avoid such biases. However predictions in AI appear to be very dependent on such judgments. Thus we expect some bias toward earlier predictions from AI experts, and more so from AGI experts. How large such biases might be is unclear however.

### Empirical evidence for bias

Analysis of the [MIRI dataset](http://aiimpacts.org/miri-ai-predictions-dataset/ "MIRI AI Predictions Dataset") supports a selection bias existing. Median people working in AGI are around two decades more optimistic than median AI researchers from outside AGI. Those in AI [are more optimistic](http://aiimpacts.org/group-differences-in-ai-predictions/ "Group Differences in AI Predictions") again than ‘others’, and futurists are slightly more optimistic than even AGI researchers, though these are less clear due to small and ambiguous samples. In sum, the groups do make different predictions in the directions that we would expect as a result of such bias.

However it is hard to exclude expertise as an explanation for these differences, so this does not strongly imply that there are biases. There could also be biases that are not caused by selection effects, such as [wishful thinking](http://en.wikipedia.org/wiki/Wishful_thinking), [planning fallacy](http://en.wikipedia.org/wiki/Planning_fallacy), or [self-serving bias](http://en.wikipedia.org/wiki/Self-serving_bias). There may also be other plausible explanations we haven’t considered.

Since there are several plausible reasons for the differences we see here, and few salient reasons to expect effects in the opposite direction (expertise could go either way), the size of the selection biases in question are probably at most as large as the gaps between the predictions of the groups. That is, roughly two decades between AI and AGI researchers, and another several decades between AI researchers and others. Part of this span should be a bias of the remaining group toward being too pessimistic, but in both cases the remaining groups are much larger than the selected group, so most of the bias should be in the selected group.

### Effects of group biases on predictions

People being selected into groups such as ‘AGI researchers’ based on their optimism does not in itself introduce a bias. The problem arises when people from different groups start making different numbers of predictions. In practice, they do. Among the predictions we know of, most are from AI researchers, and a large fraction of those are from AGI researchers. Of surveys [we have recorded](http://aiimpacts.org/ai-timeline-surveys/ "AI Timeline Surveys"), 80% target AI or AGI researchers, and around half of them target AGI researchers in particular. Statements in the [MIRI dataset](http://aiimpacts.org/miri-ai-predictions-dataset/ "MIRI AI Predictions Dataset") since 2000 include 13 from AGI researchers, 16 from AI researchers, 6 from futurists, and 6 from others. This suggests we should expect aggregated predictions from surveys and statements to be optimistic, by roughly decades.

### Conclusions

It seems likely that AI and AGI researchers’ predictions exhibit a selection bias toward being early, based on reason to expect such a bias, the large disparity between AI and AGI researchers’ predictions (while AI researchers seem likely to be optimistic if anything), and the consistency between the distributions we see and those we would expect under the selection bias explanation for disagreement. Since AI and AGI researchers are heavily represented in prediction data, predictions are likely to be biased toward optimism, by roughly decades.

Relevance
---------

**Accuracy of AI predictions**: many AI timeline predictions come from AI researchers and AGI researchers, and people interested in futurism. If we want to use these predictions to estimate AI timelines, it is valuable to know how biased they are, so we can correct for such biases.

**Detecting relevant expertise**: if the difference between AI and AGI researcher predictions is not due to bias, then it suggests one group had additional information. Such information would be worth investigating.

Selection bias from optimistic experts


Why do AGI researchers expect AI so soon?

*By Katja Grace, 24 May 2015*

People have been predicting when human-level AI will appear for many decades. A few years ago, MIRI [made](http://lesswrong.com/lw/e79/ai_timeline_prediction_data/) a big, organized collection of such predictions, along with helpful metadata. We are grateful, and just put up a [page about this dataset](http://aiimpacts.org/miri-ai-predictions-dataset/ "MIRI AI Predictions Dataset"), including some analysis. Some of you saw an earlier version of on an earlier version of our site.

There are lots of interesting things to say about the collected predictions. One interesting thing you might say is ‘wow, the median predictor thinks human-level AI will arrive in the 2030s—that’s kind of alarmingly soon’. While this is true, another interesting thing is that different groups have fairly different predictions. This means the overall median date is especially sensitive to who is in the sample.

In this particular dataset, who is in the sample depends a lot on who bothers to make public predictions. And [another interesting fact](http://aiimpacts.org/ai-timeline-predictions-in-surveys-and-statements/) is that people who bother to make public predictions have shorter AI timelines than people who are surveyed more randomly. This means the predictions you see here are probably biased in the somewhat early direction. We’ll talk about that another time. For now, I’d like to show you some of the interesting [differences between groups of people](http://aiimpacts.org/group-differences-in-ai-predictions/ "Group Differences in AI Predictions").

We divided the people who made predictions into those in AI, those in [AGI](http://en.wikipedia.org/wiki/Artificial_general_intelligence), futurists and others. This was a quick and imprecise procedure mostly based on Paul’s knowledge of the fields and the people, and some Googling. Paul doesn’t think he looked at the prediction dates before categorizing, though he probably basically knew some already. For each person in the dataset, we also interpreted their statement as a loose claim about when human-level AI was less likely than not to have arrived and when it was more likely than not to have arrived.

Below is what some of the different groups’ predictions look like, for predictions made since 2000. At each date, the line shows what fraction of predictors in that group think AI will already have happened by then, more likely than not. Note that they may also think AI will have happened before then: statements were not necessarily about the first year on which AI would arrive.

[![](http://aiimpacts.org/wp-content/uploads/2015/05/groupsAIpredictions.png)](http://aiimpacts.org/wp-content/uploads/2015/05/groupsAIpredictions.png)**Figure 1:** Cumulative distributions of predictions made since 2000 by different groups of people
The groups’ predictions look pretty different, and mostly in ways you might expect: futurists and AGI researchers are more optimistic than other AI researchers, who are more optimistic than ‘others’. The median years given by different groups span seventy years, though this is mostly due to ‘other’, which is a small group. Medians for AI and AGI are eighteen years apart.

The ‘futurist’ and ‘other’ categories are twelve people together, and the line between being a futurist and merely pronouncing on the future sometimes seems blurry. It is interesting that the futurists here look very different from the the ‘others’, but I wouldn’t read that much into it. It may just be that Paul’s perception of who is a futurist depends on degree of confidence about futuristic technology.

Most of the predictors are in the AI or AGI categories. These groups have markedly different expectations. About 85% of AGI researchers are more optimistic than the median AI researcher. This is particularly important because ‘expert predictions’ about AI usually come from some combination of AI and AGI researchers, and it looks like what the combination is may alter the median date by around two decades.

Why would AGI researchers be systematically more optimistic than other AI researchers? There are perhaps too many plausible explanations for the discrepancy.

Maybe AGI researchers are—like many—overoptimistic about their own project. Planning fallacy is ubiquitous, and planning fallacy about building AGI naturally shortens overall AGI timelines.

Another possibility is expertise: perhaps human-level AI really will arrive soon, and the AGI researchers are close enough to the action to see this, while it takes time for the information to percolate to others. The AI researchers are also somewhat informed, so their predictions are partway between those of the AGI researchers, and those of the public.

Another reason is selection bias. AI researchers who are more optimistic about AGI will tend to enter the subfield of AGI more often than those who think human-level AI is a long way off. Naturally then, AGI researchers will always be more optimistic about AGI than AI researchers are, even if they are all reasonable and equally well informed. It seems hard to imagine some of the effect not being caused by this.

It matters which explanations are true: expertise means we should listen to AGI researchers above others. Planning fallacy and selection bias suggest we should not listen to them so much, or at least not directly. If we want to listen to them in those cases, we might want to make different adjustments to account for biases.

How can we tell which explanations are true? The shapes of the curves could give some evidence. What would we expect the curves to look like if the different explanations were true? Planning fallacy might look like the entire AI curve being shifted fractionally to the left to produce the AGI curve – e.g. so all of the times are halved. Selection bias would make the AGI curve look like the bottom of the AI curve, or the AI curve with its earlier parts heavily weighted. Expertise could look like dates that everyone in the know just doesn’t predict. Or the predictions might just form a narrower, more accurate, band. In fact all of these would lead to pretty similar looking graphs, and seem to roughly fit the data. So I don’t think we can infer much this way.

Do you favor any of the hypotheses I mentioned? Or others? How do you distinguish between them?

---

*Our page about demographic differences in AI predictions is [here](http://aiimpacts.org/group-differences-in-ai-predictions/ "Group Differences in AI Predictions").*

*Our page about the MIRI AI predictions dataset is [here](http://aiimpacts.org/miri-ai-predictions-dataset/ "MIRI AI Predictions Dataset").*


Group Differences in AI Predictions

*Updated 9 November 2020*

In 2015 AGI researchers appeared to expect human-level AI substantially sooner than other AI researchers. The difference ranges from about five years to at least about sixty years as we move from highest percentiles of optimism to the lowest. Futurists appear to be around as optimistic as AGI researchers. Other people appear to be substantially more pessimistic than AI researchers.

Details
-------

### MIRI dataset

We categorized predictors in the [MIRI dataset](http://aiimpacts.org/miri-ai-predictions-dataset/ "MIRI AI Predictions Dataset") as AI researchers, AGI (artificial general intelligence) researchers, Futurists and Other. We also interpreted their statements into a common format, roughly corresponding to the first year in which the person appeared to be suggesting that human-level AI was more likely than not (see ‘minPY’ described [here](http://aiimpacts.org/miri-ai-predictions-dataset/ "MIRI AI Predictions Dataset")).

Recent (since 2000) predictions are shown in the figure below. Those made by people working on AGI specifically tended to be decades more optimistic than those at the same percentile of optimism working in other areas of AI. The difference ranged from around five years to at least around sixty years as we move from the soonest predictions to the latest. Those who worked in AI broadly tended to be at least a decade more optimistic than ‘others’, at any percentile of optimism within their group. Futurists were about as optimistic as AGI researchers.

Note that these predictions were made over a period of at least 12 years, rather than at the same time.

[![xxx](http://aiimpacts.org/wp-content/uploads/2015/05/groupsAIpredictions.png)](http://aiimpacts.org/wp-content/uploads/2015/05/groupsAIpredictions.png)**Figure 1:**Cumulative probability of AI being predicted ([minPY](http://aiimpacts.org/miri-ai-predictions-dataset/ "MIRI AI Predictions Dataset")), for various groups, for predictions made after 2000. See [here](https://sites.google.com/site/aiimpactslibrary/ai-timelines/predictions-of-human-level-ai-dates/miri-ai-predictions-dataset).

Median predictions are shown below (these are also minPY predictions as defined on the [MIRI dataset page](http://aiimpacts.org/miri-ai-predictions-dataset/ "MIRI AI Predictions Dataset"), calculated from ‘cumulative distributions’ sheet in updated dataset spreadsheet also available there).

|  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
| **Median AI predictions** |  AGI |  AI |  Futurist |  Other |  All |
|  Early (pre-2000) (warning: noisy) |  |  1988 |  2031 |  2036 |  2025 |
|  Late (since 2000) |  2033 |  2051 |  2031 |  2101 |  2042 |

### FHI survey data

The [FHI survey](http://aiimpacts.org/fhi-ai-timelines-survey/ "FHI Winter Intelligence Survey") results suggest that people’s views are not very different if they work in computer science or other parts of academia. We have not investigated this evidence in more detail.

Implications
------------

**Biases from optimistic predictors and information asymmetries:**Differences of opinion among groups who predict AI suggest that either some groups have more information, or that biases exist between predictions made by the groups (e.g. even among unbiased but noisy forecasters, if only people most optimistic about a field enter it, then the views of those in the field will be [biased toward optimism](http://aiimpacts.org/bias-from-optimistic-predictors/)) . Either of these is valuable to know about, so that we can either look into the additional information, or try to correct for the biases.

Group Differences in AI Predictions


Supporting AI Impacts

*By Katja Grace, 21 May 2015*

We now have a [donations page](http://aiimpacts.org/donate/ "Donate"). If you like what we are doing as much as anything else you can think of to spend marginal dollars on, I encourage you to support this project! Money will go to more of the kind of thing you see, including AI Impacts’ existence.

Briefly, I think AI Impacts is worth supporting because AI is a really big deal, improving our forecasts of AI is a neglected leg of AI preparations, and [there are](http://aiimpacts.org/possible-investigations/ "Possible Empirical Investigations") [cheap, tractable](http://aiimpacts.org/research-topic-hardware-software-and-ai/ "Research topic: Hardware, software and AI") [projects](http://aiimpacts.org/multipolar-research-projects/ "List of multipolar research projects") which could improve our forecasts. I hope to elaborate on these claims more quantitatively in the future.

If you like what we are doing enough to want to hear about it sometimes, but not enough to want to pay for it, you might want to follow us on [Facebook](https://www.facebook.com/aiimpacts?ref=aymt_homepage_panel) or [Twitter](https://twitter.com/AIImpacts) or RSSs ([blog](http://feeds.feedburner.com/AiImpactsBlog), [featured articles](http://feeds.feedburner.com/AiImpactsFeaturedArticles)). If you don’t like what we are doing even that much, and you think we could do better, [we’d always love to hear](http://aiimpacts.org/feedback/ "Feedback") about it.

*(Image: [Rosario Fiore](https://www.flickr.com/photos/38703275@N06/7030753959))*

Supporting AI Impacts


The Maes-Garreau Law

The Maes-Garreau law posits that people tend to predict exciting future technologies toward the end of their lifetimes. It probably does not hold for predictions of human-level AI.

Clarification
-------------

From [Wikipedia](http://en.wikipedia.org/wiki/Maes%E2%80%93Garreau_law):

> The **Maes–Garreau law** is the statement that “most favorable predictions about future technology will fall within the Maes–Garreau point”, defined as “the latest possible date a prediction can come true and still remain in the lifetime of the person making it”. Specifically, it relates to predictions of a [technological singularity](http://en.wikipedia.org/wiki/Technological_singularity "Technological singularity") or other radical future technologies.
>
>

The law was posited by Kevin Kelly, [here](http://kk.org/thetechnium/2007/03/the-maesgarreau/).

Evidence
--------

In the MIRI dataset, age and predicted time to AI are very weakly anti-correlated, with a correlation of -0.017. That is, older people expect AI very slightly sooner than others. This suggests that if the Maes-Garreau law applies to human-level AI predictions, it is very weak, or is being masked by some other effect. [Armstrong and Sotala](http://intelligence.org/files/PredictingAI.pdf) also interpret an earlier version of the same dataset as evidence against the Maes-Garreau law substantially applying, using a different method of analysis.

Earlier, smaller, informal analyses find evidence of the law, but in different settings. According to Rodney Brooks (according to Kevin Kelly), Pattie Maes observed this effect strongly in a survey of public predictions of human uploading:

> [Maes] took as many people as she could find who had publicly predicted downloading of consciousness into silicon, and plotted the dates of their predictions, along with when they themselves would turn seventy years old. Not too surprisingly, the years matched up for each of them. Three score and ten years from their individual births, technology would be ripe for them to download their consciousnesses into a computer. Just in the nick of time! They were each, in their own minds, going to be remarkably lucky, to be in just the right place at the right time.
>
>

However, according to Kelly, the data was not kept.

Kelly [did](http://kk.org/thetechnium/2007/03/the-maesgarreau/) another small search for predictions of the singularity, which appears to only support a very weakened version of the law: many people predict AI within their lifetime.

The hypothesized reason for this relationship is that people would like to believe they will personally avoid death. If this is true, we might expect the relation to apply much more strongly to predictions of events which might fairly directly save a person from death. Human uploading and the singularity are such events, while human-level AI does not appear to be. Thus it is plausible that this law does apply to some technological predictions, but not human-level AI.

Implications
------------

**Evidence about wishful thinking:** the Maes-Garreau law is a relatively easy to check instance of a larger class of hypotheses to do with AI predictions being directed by wishful thinking. If wishful thinking were a large factor in AI predictions, this would undermine accuracy because it is not related to when human-level AI will appear. That the Maes-Garreau law doesn’t seem to hold is evidence against wishful thinking being a strong determinant of AI predictions. Further evidence might be obtained by observing the correlation between belief that human-level AI will be positive for society and belief that it will come soon.

The Maes-Garreau Law


AI Timeline predictions in surveys and statements

Surveys seem to produce median estimates of time to human-level AI which are roughly a decade later than those produced from voluntary public statements.

Details
-------

We [compared](http://aiimpacts.org/miri-ai-predictions-dataset/ "MIRI AI Predictions Dataset") several surveys to predictions made by similar groups of people in the [MIRI AI predictions dataset](http://aiimpacts.org/miri-ai-predictions-dataset/ "MIRI AI Predictions Dataset"), and found that predictions made in surveys were roughly 0-2 decade later. This was a rough and non-rigorous comparison, and we made no effort to control for most variables.

Stuart Armstrong and Kaj Sotala make a similar comparison [here](http://lesswrong.com/r/discussion/lw/gta/selfassessment_in_expert_ai_predictions/), and also find survey data to give later predictions. However they are comparing non-survey data largely from recent decades with survey data entirely from 1973, which we think makes the groups too different in circumstance to infer much about surveys and statements in particular. Though in the MIRI dataset (that they used), very early predictions [tend to be](http://aiimpacts.org/miri-ai-predictions-dataset/ "MIRI AI Predictions Dataset") more optimistic than later predictions, if anything, so if they had limited themselves to predictions from similar times there would have been a larger difference (though with a very small sample of statements).

Relevance
---------

**Accuracy of AI predictions**: [some biases](http://aiimpacts.org/short-prediction-publication-biases/) which probably exist in public statements about AI predictions are likely to be smaller or not apply in survey data. For instance, public statements are probably more likely to be made by people who believe they have surprising or interesting views, whereas this should much less influence answers to a survey question once someone is taking a survey. Thus comparing data from surveys and voluntary statements can tell us about the strength of such biases. Given that median survey predictions are rarely more than a decade later than similar statements, and survey predictions seem unlikely to be strongly biased in this way, median statements are probably less than a decade early as a result of this bias.


MIRI AI Predictions Dataset

The MIRI AI predictions dataset is a collection of public predictions about human-level AI timelines. We edited the original dataset, as described below. Our dataset is available [here](https://www.dropbox.com/s/x3737sampmb2e8i/siai-fhi_ai_predictions_KG_amended.xlsx), and the original [here](http://lesswrong.com/lw/e79/ai_timeline_prediction_data/).

Interesting features of the dataset include:

* The median dates at which people’s predictions suggest AI is less likely than not and more likely than not are 2033 and 2037 respectively.
* Predictions made before 2000 and after 2000 are distributed similarly, in terms of time remaining when the prediction is made
* Six predictions made before 1980 were probably systematically sooner than predictions made later.
* AGI researchers appear to be more optimistic than AI researchers.
* People predicting AI in public statements (in the MIRI dataset) predict earlier dates than demographically similar survey takers do.
* Age and predicted time to AI are almost entirely uncorrelated: r = -.017.

Details
-------

### History of the dataset

We got the original MIRI dataset from [here](http://lesswrong.com/lw/e79/ai_timeline_prediction_data/). According to the accompanying post, the [Machine Intelligence Research Institute](https://aiimpacts.org/feed/intelligence.org) (MIRI) commissioned Jonathan Wang and Brian Potter to gather the data. Kaj Sotala and Stuart Armstrong analyzed and categorized it (their categories are available in both versions of the dataset). It was used in the papers [Armstrong and Sotala 2012](https://intelligence.org/files/PredictingAI.pdf) and [Armstrong and Sotala 2014](http://www.tandfonline.com/doi/full/10.1080/0952813X.2014.895105#.VLLDZorF8kM). We modified the dataset, as described below. Our version is [here.](https://www.dropbox.com/s/x3737sampmb2e8i/siai-fhi_ai_predictions_KG_amended.xlsx)

#### Our changes to the dataset

These are changes we made to the dataset:

* There were a few instances of summary results from large surveys included as single predictions – we removed these because survey medians and individual public predictions seem to us sufficiently different to warrant considering separately.
* We removed entries which appeared to be duplications of the same data, from different sources.
* We removed predictions made by the same individual within less than ten years.
* We removed some data which appeared to have been collected in a biased fashion, where we could not correct the bias.
* We removed some entries that did not seem to be predictions about general artificial intelligence
* We may have removed some entries for other similar reasons
* We added some predictions we knew of which were not in the data.
* We fixed some small typographic errors.

Deleted entries can be seen in the last sheet of our version of the dataset. Most have explanations in one of the last few columns.

We continue to change the dataset as we find predictions it is missing, or errors in it. The current dataset may not exactly match the descriptions on this page.

#### How did our changes matter?

Implications of the above changes:

* The dataset originally had 95 predictions; our version has 65 at last count.
* Armstrong and Sotala transformed each statement into a ‘median’ prediction. In the original dataset, the mean ‘median’ was 2040 and the median ‘median’ 2030. After our changes, the mean ‘median’ is 2046 and the median ‘median’ remains at 2030. The means are highly influenced by extreme outliers.
* We have not evaluated Armstrong and Sotala’s findings in the updated dataset. One reason is that their findings are mostly qualitative. For instance, it is a matter of judgment whether there is still ‘a visible difference’ between expert and non-expert performance. Our judgment may differ from those authors anyway, so it would be unclear whether the change in data changed their findings. We address some of the same questions by different methods.

#### minPY and maxIY predictions

People say many slightly different things about when human-level AI will arrive. We interpreted predictions into a common format: one or both of a claim about when human-level AI would be less likely than not, and a claim about when human-level AI would be more likely than not. Most people didn’t explicitly use such language, so we interpreted things roughly, as closely as we could. For instance, if someone said ‘AI will not be here by 2080’ we would interpret this as AI being less likely to exist than not by that date.

Throughout this page, we use ‘minimum probable year’ (minPY) to refer to the minimum time when a person is interpreted as stating that AI is more likely than not. We use ‘maximum improbable year’ (maxIY) to refer to the maximum time when a person is interpreted as stating that AI is less likely than not. To be clear, these are not necessarily the earliest and latest times that a person holds the requisite belief – just the earliest and latest times that is implied by their statement. For instance, if a person says ‘I disagree that we will have human-level AI in 2050’, then we interpret this as a maxIY prediction of 2050, though they may well also believe AI is less likely than not in 2065 also. We would not interpret this statement as implying any minPY. We interpreted predictions like ‘AI will arrive in about 2045’ as 2045 being the date at which AI would become more likely than not, so both minPY and a maxIY of 2045.

This is different to the ‘median’ interpretation Armstrong and Sotala provided. Which is not necessarily to disagree with their measure: as [Armstrong](http://lesswrong.com/lw/e79/ai_timeline_prediction_data/) points out, it is useful to have independent interpretations of the predictions. Both our measure and theirs could mislead in different circumstances. People who say ‘AI will come in about 100 years’ and ‘AI will come within about 100 years’ probably don’t mean to point to estimates 50 years apart (as they might be seen to in Armstrong and Sotala’s measure). On the other hand, if a person says ‘AI will obviously exist before 3000AD’ we will record it as ‘AI is more likely than not from 3000AD’ and it may be easy to forget that in the context this was far from the earliest date at which they thought AI was more likely than not.

|  |  |  |  |  |
| --- | --- | --- | --- | --- |
|  |  Original A&S ‘median’ |  Updated A&S ‘median’ | minPY |  maxIY |
|  Mean | 2040 | 2046 | 2067 | 2067 |
|  Median | 2030 | 2030 | 2037 | 2033 |

***Table 1:** Summary of mean and median AI predictions under different interpretations*

As shown in Table 1, our median dates are a few years later than Armstrong & Sotala’s original or updated dates, and only four years from one another.

#### Categories used in our analysis

##### Timing

‘Early’ throughout refers to before 2000. ‘Late’ refers to 2000 onwards. We split the predictions in this way because often we are interested in recent predictions, and 2000 is a relatively natural recent cutoff. We chose this date without conscious attention to the data beyond the fact that there have been plenty of predictions since 2000.

##### Expertise

We categorized people as ‘AGI’, ‘AI’, ‘futurist’ and ‘other’ as best we could, according to their apparent research areas and activities. These are ambiguous categories, but the ends to which we put such categorization do not require that they be very precise.

### Findings

#### Basic statistics

The median minPY is 2037 and median maxIY is 2033 (see  ‘Basic statistics’ sheet). The mean minPY is 2067, which is the same as the mean maxIY (see ‘Basic statistics’ sheet). These means are fairly meaningless, as they are influenced greatly by a few extreme outliers. Figure 1 shows the distribution of most of the predictions.

[![xxx](http://aiimpacts.wpengine.com/wp-content/uploads/2014/12/AI-and-no-AI-Predictions-1-1024x674.png)](http://aiimpacts.wpengine.com/wp-content/uploads/2014/12/AI-and-no-AI-Predictions-1.png)**Figure 1:**minPY (‘AI after’) and maxIY (‘No AI till’) predictions(from ‘Basic statistics’ sheet)

The following figures shows the fraction of predictors over time who claimed that human-level AI is more likely to have arrived by that time than not (i.e. minPY predictions). The first is for all predictions, and the second for predictions since 2000. The first graph is hard to meaningfully interpret, because the predictions were made in very different volumes at very different times. For instance, the small bump on the left is from a small number of early predictions. However it gives a rough picture of the data.

[![xxx](http://aiimpacts.wpengine.com/wp-content/uploads/2014/12/Cumulative-AI-predictions-1.png)](http://aiimpacts.wpengine.com/wp-content/uploads/2014/12/Cumulative-AI-predictions-1.png)**Figure 2**: Fraction of all minPY predictions which say AI will have arrived, over time (From ‘Cumulative distributions’ sheet).

[![xxx](http://aiimpacts.wpengine.com/wp-content/uploads/2014/12/predictions-since-2000-cdf-1.png)](http://aiimpacts.wpengine.com/wp-content/uploads/2014/12/predictions-since-2000-cdf-1.png)**Figure 3**: Fraction of late minPY predictions (made since 2000) which say AI will have arrived, over time (From ‘Cumulative distributions’ sheet).

Remember that these are dates from which people claimed something like AI being more likely than not. Such dates are influenced not only by what people believe, but also by what they are asked. If a person believes that AI is more likely than not by 2020, and they are asked ‘will there be AI in 2060’ they will respond ‘yes’ and this will be recorded as a prediction of AI being more likely than not after 2060. The graph is thus an upper bound for when people predict AI is more likely than not. That is, the graph of when people really predict AI with 50 percent confidence keeps somewhere to the left of the one in figures 2 and 3.

#### Similarity of predictions over time

In general, early and late predictions are distributed fairly similarly over the years following them. For minPY predictions, the correlation between the date of a prediction and number of years until AI is predicted from that time is 0.13 (see ‘Basic statistics’ sheet). Figure 5 shows the cumulative probability of AI being predicted over time, by late and early predictors. At a glance, they are surprisingly similar. The largest difference between the fraction of early and of late people who predict AI by any given distance in the future is about 15% (see ‘Predictions over time 2’ sheet). A difference this large is fairly likely by chance. However most of the predictions were made within twenty years of one another, so it is not surprising if they are similar.

The six very early predictions do seem to be unusually optimistic. They are all below the median 30 years, which would have a 1.6% probability of occurring by chance.

Figures 4-7 illustrate the same data in different formats.

[![xxx](http://aiimpacts.wpengine.com/wp-content/uploads/2014/12/Time-to-AI-more-likely-than-not-1-1024x761.png)](http://aiimpacts.wpengine.com/wp-content/uploads/2014/12/Time-to-AI-more-likely-than-not-1.png)**Figure 4:**Time left until minPY predictions, by date when they were made. (From ‘Basic statistics’ sheet)

[![](http://aiimpacts.wpengine.com/wp-content/uploads/2014/12/time-to-predictions-2-1.png)](http://aiimpacts.wpengine.com/wp-content/uploads/2014/12/time-to-predictions-2-1.png)**Figure 5:**Cumulative probability of AI being predicted (minPY) different distances out for early and late predictors (From ‘Predictions over time 2’ sheet)

[![xxx](http://aiimpacts.wpengine.com/wp-content/uploads/2014/12/time-to-predictions-bins-1-1024x747.png)](http://aiimpacts.wpengine.com/wp-content/uploads/2014/12/time-to-predictions-bins-1.png)**Figure 6:** Fraction of minPY predictions at different distances in the future, for early and late predictors (From ‘Predictions over time’ sheet)

[![Early vs Late CDF (1)](http://aiimpacts.wpengine.com/wp-content/uploads/2014/12/Early-vs-Late-CDF-1.png)](http://aiimpacts.wpengine.com/wp-content/uploads/2014/12/Early-vs-Late-CDF-1.png)**Figure 7:** Cumulative probability of AI being predicted by a given date, for early and late predictors (minPY). (From ‘Cumulative distributions’ sheet)

#### Groups of participants

##### Associations with expertise and enthusiasm

###### Summary

AGI people in this dataset are generally substantially more optimistic than AI people. Among the small number of futurists and others, futurists were optimistic about timing, and others were pessimistic.

###### Details

We classified the predictors as AGI researchers, (other) AI researchers, Futurists and Other, and calculated CDFs of their minPY  predictions, both for early and late predictors. The figures below show a selection of these. Recall that ‘early’ and ‘late’ correspond to before and after 2000.

As we can see in figure 8, Late AGI predictors are substantially more optimistic than late AI predictors: for almost any date this century, at least 20% more AGI people predict AI by then. The median late AI researcher minPY is 18 years later than the median AGI researcher minPY. We haven’t checked whether this is partly caused by predictions by AGI researchers having been made earlier.

There were only 6 late futurists, and 6 late ‘other’ (compared to 13 and 16 late AGI and late AI respectively), so the data for these groups is fairly noisy. Roughly, late futurists in the sample were more optimistic than anyone, while late ‘other’ were more pessimistic than anyone.

There were no early AGI people, and only three early ‘other’. Among seven early AI and eight early futurists, the AI people predicted AI much earlier (70% of early AI people predict AI before any early futurists do), but this seems to be at least partly explained by the early AI people being concentrated very early, and people predicting AI similar distances in the future throughout time.

[![xxx](http://aiimpacts.wpengine.com/wp-content/uploads/2014/12/AI-vs-AGD-CDF-1.png)](http://aiimpacts.wpengine.com/wp-content/uploads/2014/12/AI-vs-AGD-CDF-1.png)**Figure 8:**Cumulative probability of AI being predicted over time, for late AI and late AGI predictors.(See ‘Cumulative distributions’ sheet)

[![](http://aiimpacts.org/wp-content/uploads/2015/05/groupsAIpredictions.png)](http://aiimpacts.org/wp-content/uploads/2015/05/groupsAIpredictions.png)**Figure 9:**Cumulative probability of AI being predicted over time, for all late groups. (See ‘Cumulative distributions’ sheet)

|  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
| ****Median******minPY predictions** |  AGI |  AI |  Futurist |  Other |  All |
|  Early (warning: noisy) |  – |  1988 |  2031 |  2036 |  2024 |
|  Late |  2033 |  2051 |  2030 |  2101 |  2042 |

***Table 2:** Median minPY predictions for all groups, late and early. There were no early AGI predictors.*

##### Statement makers and survey takers

###### Summary

Surveys seem to produce later median estimates than similar individuals making public statements do. We compared some of the [surveys](http://aiimpacts.org/ai-timeline-surveys/ "AI Timeline Surveys") we know of to the demographically similar predictors in the MIRI dataset. We expected these to differ because predictors in the MIRI dataset are mostly choosing to making public statements, while survey takers are being asked, relatively anonymously, for their opinions. Surveys seem to produce median dates on the order of a decade later than statements made by similar groups.

###### Details

We expect surveys and voluntary statements to be subject to different selection biases. In particular, we expect surveys to represent a more even sample of opinion, while voluntary statements to be more strongly concentrated among people with exciting things to say or strong agendas. To learn about the difference between these groups, and thus the extent of any such bias, we below compare median predictions made in surveys to median predictions made by people from similar groups in voluntary statements.

Note that this is rough: categorizing people is hard, and we have not investigated the participants in these surveys more than cursorily. There are very few ‘other’ predictors in the MIRI dataset. The results in this section are intended to provide a ballpark estimate only.

Also note that while both sets of predictions are minPYs, the survey dates are often the actual median year that a person expects AI, whereas the statements could often be later years which the person happens to be talking about.

|  |  |  |  |  |
| --- | --- | --- | --- | --- |
| Survey | Primary participants |  Median minPY prediction in comparable statements in the MIRI data |  Median in survey |  Difference |
|  Kruel (AI researchers) |  AI |  2051 |  2062 | +11 |
|  Kruel (AGI researchers) |  AGI | 2033 |  2031 | -2 |
|  AGI-09 |  AGI |  2033 |  2040 | +7 |
|  FHI |  AGI/other |  2033-2062 |  2050 | in range |
|  Klein |  Other/futurist |  2030-2062 |  2050 | in range |
|  AI@50 |  AI/Other |  2051-2062 |  2056 | in range |
|  Bainbridge |  Other |  2062 |  2085 | +23 |

***Table 3**: median predictions in surveys and statements from demographically similar groups.*

Note that the [Kruel interviews](http://aiimpacts.org/kruel-ai-survey/ "Kruel AI Interviews") are somewhere between statements and surveys, and are included in both data.

It appears that the surveys give somewhat later dates than similar groups of people making statements voluntarily. Around half of the surveys give later answers than expected, and the other half are roughly as expected. The difference seems to be on the order of a decade. This is what one might naively expect in the presence of a bias from people advertising their more surprising views.

##### Relation of predictions and lifespan

Age and predicted time to AI are very weakly anti-correlated: r = -.017 (see Basic statistics sheet, “correlation of age and time to prediction”). This is evidence against a posited bias to predict AI within your existing lifespan, known as the [Maes-Garreau Law](http://en.wikipedia.org/wiki/Maes%E2%80%93Garreau_law).

MIRI AI Predictions Dataset


A new approach to predicting brain-computer parity

*By Katja Grace, 7 May 2015*

How large does a computer need to be before it is ‘as powerful’ as the human brain?

This is a difficult question, which [people have answered](http://aiimpacts.org/preliminary-prices-for-human-level-hardware/ "Preliminary prices for human-level hardware") before, with much uncertainty.

We have a new answer! (Longer description [here](http://aiimpacts.org/brain-performance-in-teps/ "Brain performance in TEPS"); summary in the rest of this post.) This answer is based on ‘traversed edges per second’ ([TEPS](http://en.wikipedia.org/wiki/Traversed_edges_per_second)), a metric which emphasizes communication within a computer, instead of computing operations (like FLOPS). That is, TEPS measures how fast information can move around.

Communication can be a substantial bottleneck for big computers, slowing them down in spite of their powerful computing capacity. It seems plausible that communication is also a bottleneck for the brain, which is both a big computer, and one that spends lots of resources on communication. This is one reason to measure the brain in terms of TEPS: if communication is a bottleneck, then it is especially important to know when computers will achieve similar performance to the brain there, not just on easier aspects of being a successful computer.

The TEPS benchmark [asks the computer](http://www.graph500.org/specifications) to simulate a graph, and then to search through it. The question is how many edges in the graph the computer can follow per second. We can’t ask the brain to run the TEPS benchmark, but the brain is already a graph of neurons, and we can measure edges being traversed in it (action potentials communicating between neurons). So we can count how many edges are traversed in the brain per second, and compare this to existing computer hardware.

The brain seems to have around [1.8-3.2 x 10^14](http://aiimpacts.org/scale-of-the-human-brain/ "Scale of the Human Brain")  synapses. We’d like to know how often these synapses convey spikes, but this has been too hard to discover. So we use neuron firing frequency as a proxy. We [previously calculated](http://aiimpacts.org/rate-of-neuron-firing/ "Neuron firing rates in humans") that each neuron spikes around 0.1-2 times per second. Together with the number of synapses, this suggests the brain performs at around 0.18 – 6.4 \* 1014 TEPS. This assumes many things, and is hazy in many ways, some of which are detailed in our longer [page on the topic](http://aiimpacts.org/brain-performance-in-teps/ "Brain performance in TEPS"). The estimate could be tightened on many fronts with more work.

The [Sequoia supercomputer](http://en.wikipedia.org/wiki/IBM_Sequoia) is currently the [best computer in the world](http://www.graph500.org/results_nov_2014) on the TEPS benchmark. Its record is 2.3 \*1013 TEPS. So the human brain seems to be somewhere between as powerful and thirty times as powerful as the best supercomputer, in terms of TEPS.

At current prices for TEPS, the brain’s performance should cost roughly $4,700 – $170,000/hour. Our previous [fairly wild guess](http://aiimpacts.org/cost-of-teps/ "The cost of TEPS") was that TEPS prices should improve by a factor of ten every four years. If this is true, it should take seven to fourteen years for a computer which costs $100/hour to be competitive with the human brain. At that point, if having human-level hardware in terms of TEPS [were enough](http://aiimpacts.org/how-ai-timelines-are-estimated/ "How AI timelines are estimated") to have [human-level AI](http://aiimpacts.org/human-level-ai/ "Human-Level AI"), human-level AI should be replacing well paid humans.

[Moravec’s and Kurzweil’s](http://aiimpacts.org/preliminary-prices-for-human-level-hardware/ "Preliminary prices for human-level hardware") estimates of computation in the brain suggest human-equivalent hardware should cost $100/hour either some time in the past or in about four years respectively, so our TEPS estimate is actually late relative to those. However they are all pretty close together. Sandberg and Bostrom’s estimates of hardware required to emulate a brain [span from around then to around thirty years later](http://aiimpacts.org/preliminary-prices-for-human-level-hardware/ "Preliminary prices for human-level hardware"), though note that emulating is different from replicating functionally. Altogether ‘human-level’ hardware seems likely to be upon us soon, if it isn’t already. The estimate from TEPS points to the near future even more strongly.

*(Featured image by [MartinGrandjean](https://aiimpacts.org/feed/By%20Martin Grandjean  http:/en.wikipedia.org/wiki/Data_visualization#/media/File:Social_Network_Analysis_Visualization.png))*