Vignettes workshop

*by Daniel Kokotajlo, 15 June 2021*

Plan
----

1. Try to write plausible future histories of the world, focusing on AI-relevant features. (“Vignettes.”)
2. Read each others’ vignettes and critique the implausible bits: “Wouldn’t the US government do something at that point?” “You say twenty nations promise each other not to build agent AI–could you say more about why and how?”
3. Amend and repeat.
4. Better understand your own views about how the development of advanced AI may go down.
(5. Maybe add your vignette to [our collection.](https://aiimpacts.org/partially-plausible-fictional-ai-futures/?fbclid=IwAR3n_Jl2IEVh9FlJQwyqdefp2ieZF4w0PGOfsluNN1cSX5khhsq1CigxSLw))

This event will happen over two days, so you can come Friday if this counts as work for you, Saturday if it counts as play, and both if you are keen. RSVP to particular days is somewhat helpful; let us know in the comments.

Date, Time, Location
--------------------

The event will happen on Friday the 25th of June and Saturday the 26th.

It’ll go from 10am (California time) until probably around 4pm both days.

It will take place online, in the LessWrong Walled Garden. Here are the links to attend:
[Friday](https://www.lesswrong.com/walledGardenPortal/?code=x1ON&event=ai-impacts-vignettes-workshop-day-1&fbclid=IwAR1an9qFL8eLX1MfGFj5awIiHVUvqW3mfC-3_C_BhRQmi_IOAjkPaDcbdxM)
[Saturday](https://www.lesswrong.com/walledGardenPortal/?code=AAmb&event=ai-impacts-vignettes-workshop-day-2&fbclid=IwAR2MuRszV1TN5ditlx7N214IYv5yPBjz8Dg1-F50fob9m0JETDczp76dgGY)

[Facebook event](https://www.facebook.com/events/828493847873343/?post_id=828501734539221&view=permalink&notif_id=1623722140360640&notif_t=feedback_reaction_generic&ref=notif)

FAQ
---

> Do I need literary merit or creativity?
No.

> Do I need to have realistic views about the future?
No, the idea is to get down what you have and improve it.

> Do I need to write stories?
Nah, you can just critique them if you want.

> What will this actually look like?
We’ll meet up online, discuss the project and answer questions, and then spend chunks of time (online or offline) writing and/or critiquing vignettes, interspersed with chatting together.

> Have you done this before? Can I see examples?
Yes, on a small scale. See [here](https://aiimpacts.org/partially-plausible-fictional-ai-futures/?fbclid=IwAR0UWvgkH94N2XKeCoWqHFyjrM3uCnPPfht3lH3bupMaN1E3vjQrjlBpNn4) for some resulting vignettes. We thought it was fun and interesting.

*This event is co-organized by Katja Grace and Daniel Kokotajlo. Thanks to everyone who participated in the trial Vignettes Day months ago. Thanks to John Salvatier for giving us the idea.*

Vignettes workshop


Fiction relevant to AI futurism

This page is an incomplete collection of fiction about the development of advanced AI, and the consequences for society.

Details
-------

Entries are generally included if we judge that they contain enough that is plausible or correctly evocative to be worth considering, in light of AI futurism.

The list includes:

1. works (usually in draft form) belonging to our [AI Vignettes Project](https://aiimpacts.org/ai-vignettes-project/). These are written with the intention of incrementally improving their realism via comments. These are usually in commentable form, and we welcome criticism, especially of departures from realism.
2. works created for the purpose of better understanding the future of AI
3. works from mainstream entertainment, either because they were prominent or recommended to us.[1](https://aiimpacts.org/partially-plausible-fictional-ai-futures/#easy-footnote-bottom-1-2893 "We collected traditional fictional works via requests on social media, here and here")

The list can be sorted and filtered by various traits that aren’t visible by default (see top left options). For instance:

* **Type**, i.e. being mainstream entertainment, futurism, or specifically from our [Vignettes Project](https://aiimpacts.org/ai-vignettes-project/), as described above.
* **Relevant themes**, e.g. ‘failure modes’ or ‘largeness of mindspace’
* **Scenario categories**, e.g. ‘fast takeoff’, ‘government project’, ‘brain emulations’
* **Recommendation rating**: this is roughly how strongly we recommend the piece for people wanting to think about the future of AI. It takes into account a combination of realism, tendency to evoke some specific useful intuition, ease of reading. It is very rough and probably not consistent.

Many entries are only partially filled out. These are marked ‘unfinished’, and so can be filtered out.

We would appreciate further submissions of stories or additional details for stories we have here, reviews of stories in the collection here, or other comments [here](https://aiimpacts.org/feedback/).

### Collection

The collection can also be seen full screen [here](https://airtable.com/shr5EIpLNHB7o2q9Z/tblMVjRvMKVNkoZVg?backgroundColor=cyan&viewControls=on) or as a table [here](https://airtable.com/shrVnjq9U53R5nrxO).

Related
-------

* [AI Vignettes Project](https://aiimpacts.org/ai-vignettes-project/)

Notes
-----

Fiction relevant to AI futurism


What do coherence arguments imply about the behavior of advanced AI?

*Published 8 April 2021*

*This is an initial page, in the process of review, which may not be comprehensive or represent the best available understanding.*

Coherence arguments say that if an entity’s preferences do not adhere to the axioms of expected utility theory, then that entity is susceptible to losing things that it values.

This does not imply that advanced AI systems must adhere to these axioms (‘be coherent’), or that they must be goal-directed.

Such arguments do appear to suggest that there will be non-zero pressure for advanced AI to become more coherent, and arguably also more ‘goal-directed’, given some minimal initial level of goal-directedness.

Details
-------

### Motion toward coherence

#### Expected utility maximization

‘Maximizing expected utility’ is a decision-making strategy, in which you assign a value to each possible ‘outcome’, and assign a probability to each outcome conditional on each of your available actions, then always choose the action whose resulting outcomes have the highest ‘[expected value](https://en.wikipedia.org/wiki/Expected_value)‘ (average value of outcomes, weighted by the probability of those outcomes).

#### Coherence arguments

‘Coherence arguments’[1](https://aiimpacts.org/what-do-coherence-arguments-imply-about-the-behavior-of-advanced-ai/#easy-footnote-bottom-1-2836 "Or ‘coherence theorems’. Discussed further here.") demonstrate that if one’s preferences cannot be understood as ‘maximizing expected utility’, then one can be manipulated into giving up things that one values for no gain.

For instance, one coherence argument notes that if you have ‘circular preferences’ then you will consent to series of decisions that will leave you worse off, given your preferences:

Suppose you prefer:

* apple over pear
* pear over quince
* quince over apple
* any fruit over nothing

Then there is some tiny amount of money you would pay to go from apple to quince, and quince to pear, and pear to apple. At which point, you have spent money and are back where you started. If it is also possible to buy all of these fruit for money, then losing money means you lost some of whatever fruit for nothing, and you do want all of the fruit, by assumption.

If you avoid all such predictable losses, then according to the coherence arguments, you must be maximizing expected utility (as described above).

#### Coherence forces

That a certain characteristic of an entity’s ‘preferences’ makes it vulnerable to manipulation does not mean that it will not have that characteristic. In order for such considerations to change the nature of an entity, ignoring outside intervention, something like the following conditions need to hold:

* The entity can detect the characteristic (which could be difficult, if it is a logical relationship between all of its ‘preferences’ which are perhaps not straightforwardly accessible or well-defined)
* The realistic chance for loss is large enough to cause the entity to prioritize the problem
* The entity is motivated to become coherent by the possibility of loss (versus for instance inferring that losing money is good, since it is equivalent to a set of exchanges that are each good)
* The entity is in a position to alter its own preferences

Similar might apply if versions of the above hold for an outside entity with power over the agent, e.g. its creators, though in that case it is less clear that ‘coherence’ is a further motivator beyond that for having the agent’s preferences align with those of the outside agent (which would presumably coincide with coherence, to the extant that the outside agent had more coherent preferences).

Thus we say there is generally an incentive for coherence, but it may or may not actually cause a an entity to change in the direction of coherence at a particular time. We can also describe this as a ‘coherence force’ or ‘coherence pressure’, pushing minds toward coherence, all things equal, but for all we know, so weakly as to be often irrelevant.

#### Coherence forces apply to entities with ‘preferences’

The coherence arguments only apply to creatures with ‘preferences’ that might be thwarted by their choices, so there are presumably possible entities that are not subject to any coherence forces, due to not having preferences of the relevant type.

### Behavior of coherent creatures

Supposing entities are likely to become more coherent, all things equal, a natural question is how coherent entities differ from incoherent entities.

#### Coherence is consistent with any behavior

If we observe an agent exhibiting any history of behavior, that is consistent with the agent’s being coherent because the agent could have a utility function that rates that history higher than any other history. Rohin Shah [discusses](https://www.alignmentforum.org/s/4dHMdK5TLN6xcqtyc/p/NxF5G6CJiof6cemTw) this.

#### Coherence and goal-directedness

##### Coherence doesn’t logically require goal-directedness

As Rohin Shah [discusses](https://www.alignmentforum.org/s/4dHMdK5TLN6xcqtyc/p/NxF5G6CJiof6cemTw), the above means that coherence does not imply ‘goal-directed’ behavior (however you choose to define that, if it doesn’t include all behavior):

1. Coherence arguments do not exclude any behavior
2. Non-goal-directed behavior is consistent with coherence arguments
3. Thus coherence arguments do not imply goal directed behavior

##### Increasing coherence seems likely to be associated with increased intuitive ‘goal-directedness’

The following hypotheses (quoted from [this blog post](https://aiimpacts.org/coherence-arguments-imply-a-force-for-goal-directed-behavior/)) seem plausible (where goal-directednessRohin means something like ‘that which looks intuitively goal-directed)[2](https://aiimpacts.org/what-do-coherence-arguments-imply-about-the-behavior-of-advanced-ai/#easy-footnote-bottom-2-2836 "Or precisely, from that post: ‘something like Rohin’s preferred usage: roughly, that which seems intuitively goal directed to us, e.g. behaving similarly across situations, and accruing resources, and not flopping around in possible pursuit of some exact history of personal floppage, or peaceably preferring to always take the option labeled ‘A’.’"):

> **1. Coherence-reformed entities will tend to end up looking similar to their starting point but less conflicted**For instance, if a creature starts out being indifferent to buying red balls when they cost between ten and fifteen blue balls, it is more likely to end up treating red balls as exactly 12x the value of blue balls than it is to end up very much wanting the sequence where it takes the blue ball option, then the red ball option, then blue, red, red, blue, red. Or wanting red squares. Or wanting to ride a dolphin.
>
> […]**2. More coherent strategies are systematically less wasteful, and waste inhibits goal-directionRohin, which means more coherent strategies are more forcefully goal-directedRohin on average**
> In general, if you are sometimes a force for A and sometimes a force against A, then you are not moving the world with respect to A as forcefully as you would be if you picked one or the other. Two people intermittently changing who is in the driving seat, who want to go to different places, will not cover distance in any direction as effectively as either one of them. A company that cycles through three CEOs with different evaluations of everything will—even if they don’t actively scheme to thwart one another—tend to waste a lot of effort bringing in and out different policies and efforts (e.g. one week trying to expand into textiles, the next week trying to cut everything not involved in the central business).
>
> **3. Combining points 1 and 2 above, as entities become more coherent, they generally become more goal-directedRohin.**As opposed to, for instance, becoming more goal-directedRohin *on average,* but individual agents being about as likely to become worse as better as they are reformed. Consider: a creature that values red balls at 12x blue balls is very similar to one that values them inconsistently, except a little less wasteful. So it is probably similar but more goal-directedRohin. Whereas it’s fairly unclear how goal-directedRohina creature that wants to ride a dolphin is compared to one that wanted red balls inconsistently much. In a world with lots of balls and no possible access to dolphins, it might be much less goal-directedRohin, in spite of its greater coherence.
>
> **4. Coherence-increasing processes rarely lead to non-goal-directedRohin agents—like the one that twitches on the ground** In the abstract, few starting points and coherence-motivated reform processes will lead to an agent with the goal of carrying out a specific convoluted moment-indexed policy without regard for consequence, like Rohin’s twitching agent, or to valuing the sequence of history-action pairs that will happen anyway, or to being indifferent to everything. And these outcomes will be even less likely in practice, where AI systems with anything like preferences probably start out caring about much more normal things, such as money and points and clicks, so will probably land at a more consistent and shrewd version of that, if 1 is true. (Which is not to say that you couldn’t intentionally create such a creature.)
>
>
>
>

Thus it presently seems likely that coherence arguments correspond to a force for for entities with something like ‘preferences’ to grow increasingly coherent, and generally increasingly goal-directed (intuitively defined).

Thus, to the extent that future advanced AI has preferences of the relevant kind, there appears to be a pressure for it to become more goal-directed. However it is unclear what can be said generally about the strength of this force.

*Primary author: Katja Grace*

Notes
-----


April files

*By Katja Grace, 1 April 2021*

Today we are sharing with our blog readers a collection of yet-to-be-published drafts, in the hope of receiving feedback. We are especially looking for methodological critique, but all comments welcome!

[Human-level performance estimate](https://docs.google.com/document/d/1riFeI_SIO6nCLmaht1hsLiXyOeJWeq1HHsomnYCPxXQ/edit?usp=sharing) (Katja Grace)

[How much hardware will we need to create AGI?](https://docs.google.com/document/d/1_PZ3KxUGq2zRKeXr8Ca_hyS1VG0njOIHkwHqkMDhQhE/edit?usp=sharing) (Asya Bergal, original idea credit to Ronny Fernandez)

[Historic trends in AI Impacts productivity](https://docs.google.com/document/d/1yk9X7gUogaqxTBWDMgNoUZQydM3GG5jG3-njayFbS9k/edit?usp=sharing) (Daniel Kokotajlo)

[Analysis of superbombs as a global threat](https://docs.google.com/document/d/17ddIf4To4R-0C7PtYAqPaihcMLRFir8Wx-kfH5StLS8/edit?usp=sharing) (Katja Grace)

April files


Coherence arguments imply a force for goal-directed behavior

**By Katja Grace, 25 March 2021**

*[Epistemic status: my current view, but I haven’t read all the stuff on this topic even in the LessWrong community, let alone more broadly.]*

There is a line of thought that says that advanced AI will tend to be ‘goal-directed’—that is, consistently doing whatever makes certain favored outcomes more likely—and that this is to do with the ‘coherence arguments’. [Rohin Shah](https://www.lesswrong.com/posts/NxF5G6CJiof6cemTw/coherence-arguments-do-not-imply-goal-directed-behavior), and probably others[1](https://aiimpacts.org/coherence-arguments-imply-a-force-for-goal-directed-behavior/#easy-footnote-bottom-1-2875 "For instance, Richard Ngo agrees here, and Eric Drexler makes a related argument here, section 6.4."), have argued against this. I want to argue against them.

The old argument for coherence implying (worrisome) goal-directedness
---------------------------------------------------------------------

I’d reconstruct the original argument that Rohin is arguing against as something like this (making no claim about my own beliefs here):

1. **‘Whatever things you care about, you are best off assigning consistent numerical values to them and maximizing the expected sum of those values’
‘**Coherence arguments

And since the point of all this is to argue that advanced AI might be hard to deal with, note that we can get to that conclusion with:

5. **‘Highly intelligent goal-directed agents are dangerous’**If AI systems exist that very competently pursue goals, they will likely be better than us at attaining their goals, and therefore to the extent there is a risk of mismatch between their goals and ours, we face a serious risk.

Rohin’s counterargument
-----------------------

Rohin’s counterargument begins with an observation made by others before: any behavior is consistent with maximizing expected utility, given *some* utility function. For instance, a creature just twitching around on the ground may have the utility function that returns 1 if the agent does whatever it in fact does in each situation (where ‘situation’ means, ‘entire history of the world so far’), and 0 otherwise. This is a creature that just wants to make the right twitch in each detailed, history-indexed situation, with no regard for further consequences. Alternately the twitching agent might care about outcomes, but just happen to want the particular holistic unfolding of the universe that is occurring, including this particular series of twitches. Or it could be indifferent between all outcomes.

The basic point is that rationality doesn’t say what ‘things’ you can want. And in particular, it doesn’t say that you have to care about particular atomic units that larger situations can be broken down into. If I try to call you out for first spending money to get to Paris, then spending money to get back from Paris, there is nothing to say you can’t just have wanted to go to Paris for a bit and then to come home. In fact, this is a common human situation. ‘Aha, I money pumped you!’ says the airline, but you aren’t worried. The twitching agent might always be like this—a creature of more refined tastes, who cares about whole delicate histories and relationships, rather than just summing up modular momentarily-defined successes. And given this freedom, any behavior might conceivably be what a creature wants.

Then I would put the full argument, as I understand it, like this:

1. Any observable sequence of behavior is consistent with the entity doing EU maximization (see observation above)
2. Doing EU maximization doesn’t imply anything about what behavior we might observe (from 1)
3. In particular, knowing that a creature is an EU maximizer doesn’t imply that it will behave in a ‘goal-directed’ way, assuming that *that* concept doesn’t apply to all behavior. (from 2)

Is this just some disagreement about the meaning of the word ‘goal-directed’? No, because we can get back to a major difference in physical expectations by adding:

4. Not all behavior in a creature implicates dire risk to humanity, so any concept of goal-directedness that is consistent with any behavior—and so might be implied by the coherence arguments—cannot imply AI risk.

So where the original argument says that the coherence arguments plus some other assumptions imply danger from AI, this counterargument says that they do not.

(There is also at least some variety in the meaning of ‘goal-directed’. I’ll use goal-directedRohin to refer to what I think is Rohin’s [preferred](https://www.lesswrong.com/s/4dHMdK5TLN6xcqtyc/p/DfcywmqRSkBaCB6Ma) usage: roughly, that which seems intuitively goal directed to us, e.g. behaving similarly across situations, and accruing resources, and not flopping around in possible pursuit of some exact history of personal floppage, or peaceably preferring to always take the option labeled ‘A’.[2](https://aiimpacts.org/coherence-arguments-imply-a-force-for-goal-directed-behavior/#easy-footnote-bottom-2-2875 "I am not sold on this usage myself for ‘goal-directed’—there is an appeal to using that phrase for ‘pursues goals’ in its most basic sense, but I am also tentatively in favor of having as many concepts as possible."))

My counter-counterarguments
---------------------------

What’s wrong with Rohin’s counterargument? It sounded tight.

In brief, I see two problems:

1. The whole argument is in terms of logical implication. But what seems to matter is changes in probability. Coherence doesn’t need to rule out any behavior to matter, it just has to change the probabilities of behaviors. Understood in terms of probability, argument 2 is a false inference: just because any sequence of behavior is consistent with EU maximization doesn’t mean that EU maximization says nothing about what behavior we will see, probabilistically. All it says is that the probability of a behavioral sequence is never reduced to zero by considerations of coherence alone, which is hardly saying anything.

You might then think that a probabilistic version still applies: since every entity appears to be in good standing with the coherence arguments, the arguments don’t exert any force, probabilistically, on what entities we might see. But:

2. An outside observer being able to rationalize a sequence of observed behavior as coherent doesn’t mean that the behavior is actually coherent. Coherence arguments constrain combinations of external behavior and internal features—‘preferences’[3](https://aiimpacts.org/coherence-arguments-imply-a-force-for-goal-directed-behavior/#easy-footnote-bottom-3-2875 "It seems perhaps misleading to call these ‘preferences’, if they are incoherent, and so do not together implicate orderings of outcomes being better than one another. If a creature is not coherent, what are even the objects of its decision calculus? I am inclined to think in terms of ‘decision criteria’, e.g. ‘given X and Y, choose X’, and ‘given Y and Z, choose Y’, which don’t necessarily imply anything about ‘given Z and X, choose …’, but I haven’t thought about this much, and it seems like a technical detail of the creature in question. Whatever they are though, if the creature has behavior, then there are internal dynamics that produce it. When exactly an aspect of these should be considered a ‘preference’ for the sake of this argument isn’t entirely clear to me, but would seem to depend on something like whether it tends to produce actions favoring certain outcomes over other outcomes across a range of circumstances (similar to the unclear definition of ‘agent’).") and beliefs. So whether an actor is coherent depends on what preferences and beliefs it actually has. And if it isn’t coherent in light of these, then coherence pressures will apply, whether or not its behavior *looks* coherent. And in many cases, revision of preferences due to coherence pressures will end up affecting external behavior. So 2) is not only not a sound inference from 1), but actually a wrong conclusion: if a system moves toward EU maximization, that does imply things about the behavior that we will observe (probabilistically).

Perhaps Rohin only meant to argue about whether it is *logically possible* to be coherent and not goal-directed-seeming, for the purpose of arguing that humanity can construct creatures in that perhaps-unlikely-in-nature corner of mindspace, if we try hard. In which case, I agree that it is logically possible. But I think his argument is often taken to be relevant more broadly, to questions of whether advanced AI will tend to be goal-directed, or to be goal-directed in places where they were not intended to be.

I take 1) to be fairly clear. I’ll lay out 2) in more detail.

My counter-counterarguments in more detail
------------------------------------------

### **How might coherence arguments affect creatures?**

Let us step back.

How would coherence arguments affect an AI system—or anyone—anyway? They’re not going to fly in from the platonic realm and reshape irrational creatures.

The main routes, as I see it, are via implying:

1. incentives for the agent itself to reform incoherent preferences
2. incentives for the processes giving rise to the agent (explicit design, or selection procedures directed at success) to make them more coherent
3. some advantage for coherent agents in competition with incoherent agents

To be clear, the agent, the makers, or the world are not necessarily thinking about the arguments here—the arguments correspond to incentives in the world, which these parties are responding to. So I’ll often talk about ‘incentives for coherence’ or ‘forces for coherence’ rather than ‘coherence arguments’.

I’ll talk more about 1 for simplicity, expecting 2 and 3 to be similar, though I haven’t thought them through.

### ***Looking*** **coherent isn’t enough: if you aren’t coherent inside, coherence forces apply**

If self-adjustment is the mechanism for the coherence, this doesn’t depend on what a sequence of actions looks like from the outside, but from what it looks like from the inside.

Consider the aforementioned creature just twitching sporadically on the ground. Let’s call it Alex.

As noted earlier, there is a utility function under which Alex is maximizing expected utility: the one that assigns utility 1 to however Alex in fact acts in every specific history, and utility 0 to anything else.

But from the inside, this creature you excuse as ‘maybe just wanting that series of twitches’ has—let us suppose—actual preferences and beliefs. And if its preferences do not in fact prioritize this elaborate sequence of twitching in an unconflicted way, and it has the self-awareness and means to make corrections, then it will make corrections[4](https://aiimpacts.org/coherence-arguments-imply-a-force-for-goal-directed-behavior/#easy-footnote-bottom-4-2875 "The ‘right’ way to correct your own incoherent preferences seems complicated and not obviously well defined or existent, and perhaps there is not much more to say than that what you do will depend on your design. It’s also not clear to me that a genuinely incoherent creature should necessarily want to reform, by its own lights, but that is a question for another time—here I’m assuming that the coherence arguments do have this implication that seems commonly attributed to them. My guess is that in practice, such creatures often do want to reform, and exactly how they do it doesn’t matter for my argument here."). And having done so, its behavior will change.

Thus excusable-as-coherent Alex is still moved by coherence arguments, even while the arguments have no complaints about its behavior *per se*.

For a more realistic example: suppose Assistant-Bot is observed making this sequence of actions:

* Offers to buy gym membership for $5/week
* Consents to upgrade to gym-pro membership for $7/week, which is like gym membership but with added morning classes
* Takes discounted ‘off-time’ deal, saving $1 per week for only using gym in evenings

This is consistent with coherence: Assistant-Bot might prefer that exact sequence of actions over all others, or might prefer incurring gym costs with a larger sum of prime factors, or might prefer talking to Gym-sales-bot over ending the conversation, or prefer agreeing to things.

But suppose that *in fact*, in terms of the structure of the internal motivations producing this behavior, Assistant-Bot just prefers you to have a gym membership, and prefers you to have a better membership, and prefers you to have money, but is treating these preferences with inconsistent levels of strength in the different comparisons. Then there appears to be a coherence-related force for Assistant-Bot to change. One way that that could look is that since Assistant-Bot’s overall behavioral policy currently entails giving away money for nothing, and also Assistant-Bot prefers money over nothing, that preference gives Assistant-Bot reason to alter its current overall policy, to avert the ongoing exchange of money for nothing.[5](https://aiimpacts.org/coherence-arguments-imply-a-force-for-goal-directed-behavior/#easy-footnote-bottom-5-2875 "I’m describing the force potentially felt by Assistant-Bot itself, but to the extent that its makers, or users also have preferences for money over nothing, and wish to use Assistant-Bot, and can alter it, they would seem to have similar incentives to mitigate its self-defeating behavior.") And if its behavioral policy is arising from something like preferences, then the natural way to alter it is via altering those preferences, and in particular, altering them in the direction of coherence.

One issue with this line of thought is that it’s not obvious in what sense there is anything inside a creature that corresponds to ‘preferences’. Often when people posit preferences, the preferences are defined in terms of behavior. Does it make sense to discuss different possible ‘internal’ preferences, distinct from behavior? I find it helpful to consider the behavior and ‘preferences’ of groups:

Suppose two cars are parked in driveways, each containing a couple. One couple are just enjoying hanging out in the car. The other couple are dealing with a conflict: one wants to climb a mountain together, and the other wants to swim in the sea together, and they aren’t moving because neither is willing to let the outing proceed as the other wants. ‘Behaviorally’, both cars are the same: stopped. But their internal parts (the partners) are importantly different. And in the long run, we expect different behavior: the car with the unconflicted couple will probably stay where it is, and the conflicted car will (hopefully) eventually resolve the conflict and drive off.

I think here it makes sense to talk about internal parts, separate from behavior, and real. And similarly in the single agent case: there are physical mechanisms producing the behavior, which can have different characteristics, and which in particular can be ‘in conflict’—in a way that motivates change—or not. I think it is also worth observing that humans find their preferences ‘in conflict’ and try to resolve them, which is suggests that they at least are better understood in terms of both behavior and underlying preferences that are separate from it.

So we have: even if you can excuse any seizuring as consistent with coherence, coherence incentives still exert a force on creatures that are *in fact* incoherent, given their real internal state (or would be incoherent if created). At least if they or their creator have machinery for noticing their incoherence, caring about it, and making changes.

Or put another way, coherence doesn’t exclude overt behaviors alone, but does exclude combinations of preferences, and preferences beget behaviors. This changes how specific creatures behave, even if it doesn’t entirely rule out any behavior ever being correct for some creature, somewhere.

That is, the coherence theorems may change what behavior is *likely* to appear amongst creatures with preferences.

### **Reform for coherence probably makes a thing more goal-directed****Rohin**

Ok, but moving toward coherence might sound totally innocuous, since, per Rohin’s argument, coherence includes all sorts of things, such as absolutely any sequence of behavior.

But the relevant question is again whether a coherence-increasing reform process is likely to result in some kinds of behavior over others, probabilistically.

This is partly a practical question—what kind of reform process is it? Where a creature ends up depends not just on what it incoherently ‘prefers’, but on what kinds of things its so-called ‘preferences’ are at all[6](https://aiimpacts.org/coherence-arguments-imply-a-force-for-goal-directed-behavior/#easy-footnote-bottom-6-2875 "The creature’s ‘preferences’ can’t be in terms of consistent numerical values assigned to everything, because those would be consistent. So what are they? For instance, one might imagine that they are pairwise comparisons between some kind of things (which can include ‘A > B’ and ‘B > C’ and ‘C > A’), or that they are a set of ‘situation—>action’ mappings, or they are a noisy ‘options—>feelings’ mapping combined with a set of deontological constraints over actions and feelings (‘choose things you feel better about, except don’t choose things out of selfishness, except when you feel more than 50% scared…’, etc."), and what mechanisms detect problems, and how problems are resolved.

My guess is that there are also things we can say in general. It’s is too big a topic to investigate properly here, but some initially plausible hypotheses about a wide range of coherence-reform processes:

1. **Coherence-reformed entities will tend to end up looking similar to their starting point but less conflicted**For instance, if a creature starts out being indifferent to buying red balls when they cost between ten and fifteen blue balls, it is more likely to end up treating red balls as exactly 12x the value of blue balls than it is to end up very much wanting the sequence where it takes the blue ball option, then the red ball option, then blue, red, red, blue, red. Or wanting red squares. Or wanting to ride a dolphin.

(I agree that if a creature starts out valuing Tuesday-red balls at fifteen blue balls and yet all other red balls at ten blue balls, then it faces no obvious pressure from within to become ‘coherent’, since it is not incoherent.)
2. **More coherent strategies are systematically less wasteful, and waste inhibits goal-directionRohin, which means more coherent strategies are more forcefully goal-directedRohin on average**
In general, if you are sometimes a force for A and sometimes a force against A, then you are not moving the world with respect to A as forcefully as you would be if you picked one or the other. Two people intermittently changing who is in the driving seat, who want to go to different places, will not cover distance in any direction as effectively as either one of them. A company that cycles through three CEOs with different evaluations of everything will—even if they don’t actively scheme to thwart one another—tend to waste a lot of effort bringing in and out different policies and efforts (e.g. one week trying to expand into textiles, the next week trying to cut everything not involved in the central business).

3. **Combining points 1 and 2 above, as entities become more coherent, they generally become more goal-directed****Rohin****.** As opposed to, for instance, becoming more goal-directedRohin *on average,* but individual agents being about as likely to become worse as better as they are reformed. Consider: a creature that values red balls at 12x blue balls is very similar to one that values them inconsistently, except a little less wasteful. So it is probably similar but more goal-directedRohin. Whereas it’s fairly unclear how goal-directedRohin a creature that wants to ride a dolphin is compared to one that wanted red balls inconsistently much. In a world with lots of balls and no possible access to dolphins, it might be much less goal-directedRohin, in spite of its greater coherence.

4. **Coherence-increasing processes rarely lead to non-goal-directed****Rohin** **agents—like the one that twitches on the ground**In the abstract, few starting points and coherence-motivated reform processes will lead to an agent with the goal of carrying out a specific convoluted moment-indexed policy without regard for consequence, like Rohin’s twitching agent, or to valuing the sequence of history-action pairs that will happen anyway, or to being indifferent to everything. And these outcomes will be even less likely in practice, where AI systems with anything like preferences probably start out caring about much more normal things, such as money and points and clicks, so will probably land at a more consistent and shrewd version of that, if 1 is true. (Which is not to say that you couldn’t intentionally create such a creature.)

These hypotheses suggest to me that the changes in behavior brought about by coherence forces favor moving toward goal-directednessRohin, and therefore at least weakly toward risk.

Does this mean advanced AI will be goal-directedRohin?
------------------------------------------------------

Together, this does not imply that advanced AI will tend to be goal-directedRohin. We don’t know how strong such forces are. Evidently not so strong that humans[7](https://aiimpacts.org/coherence-arguments-imply-a-force-for-goal-directed-behavior/#easy-footnote-bottom-7-2875 "For instance, humans are insistently incoherent on the Allais paradox."), or our other artifacts, are whipped into coherence in mere hundreds of thousands of years[8](https://aiimpacts.org/coherence-arguments-imply-a-force-for-goal-directed-behavior/#easy-footnote-bottom-8-2875 "That seems pretty interesting—but note that well-designed computers have been known to do things that took humans participating in biological and cultural evolution hundreds of thousands of years before, so inference here not straightforward, and the forces of coherence depend on the costs of reform, which depend on the machinery for it. Also, we don’t know what other forces were in play—there might even have been forces for apparent incoherence, e.g. insofar as hypocrisy can benefit social animals, and dishonesty is complicated (The Elephant in the Brain discusses such ideas)."). If a creature doesn’t have anything like preferences (beyond a tendency to behave certain ways), then coherence arguments don’t obviously even apply to it (though discrepancies between the creature’s behavior and its makers’ preferences probably produce an analogous force[9](https://aiimpacts.org/coherence-arguments-imply-a-force-for-goal-directed-behavior/#easy-footnote-bottom-9-2875 "For instance, the coherent creature that evaluates red balls differently on Tuesday and Wednesday might be in conflict with its creators, if they have a more consistent red ball evaluation, giving them reason to reform it. You might class this under the question, ‘what kinds of advanced AI will people want?’, but the reason for it is very similar to the reasons for internal pressure for coherence. If you refuse to pay $13 for a red ball, and your AI then goes out and buys you one for $15 because it is Tuesday, then the pair of you together could have done better.") and competitive pressures probably produce a similar force for coherence in valuing resources instrumental to survival). Coherence arguments mark out an aspect of the incentive landscape, but to say that there is an incentive for something, all things equal, is not to say that it will happen.

In sum
------

1) Even though any behavior could be coherent in principle, if it is not coherent in combination with an entity’s internal state, then coherence arguments point to a real force for different (more coherent) behavior.

2) My guess is that this force for coherent behavior is also a force for goal-directed behavior. This isn’t clear, but seems likely, and also isn’t undermined by Rohin’s argument, as seems commonly believed.

.

[![](http://aiimpacts.org/wp-content/uploads/2021/03/Two_dogs_attached_to_the_same_leash_are_pulling_in_different_Wellcome_V0021880-1024x804.jpeg)](http://aiimpacts.org/wp-content/uploads/2021/03/Two_dogs_attached_to_the_same_leash_are_pulling_in_different_Wellcome_V0021880-scaled.jpeg)*Two dogs attached to the same leash are pulling in different directions*. [Etching by J. Fyt](https://commons.wikimedia.org/wiki/File:Two_dogs_attached_to_the_same_leash_are_pulling_in_different_Wellcome_V0021880.jpg), [1642](https://catalogue.wellcomelibrary.org/record=b1199389)
.

---

.


AI Impacts 2020 review

*Published Dec 21, 2020*

This is a list of work done at AI Impacts published in 2020.

Major projects
--------------

### **Discontinuities project**

**Primary authors:** Katja Grace, Richard Korzekwa, Asya Bergal, Daniel Kokotajlo
**Main page:** [Discontinuous progress investigation](https://aiimpacts.org/discontinuous-progress-investigation/)
**Blog post:** [Discontinuous progress in history: an update](https://aiimpacts.org/discontinuous-progress-in-history-an-update/)

Other pages added in 2020: [Effect of AlexNet on historic trends in image recognition](https://aiimpacts.org/effect-of-alexnet-on-historic-trends-in-image-recognition/), [Historic trends in transatlantic message speed](https://aiimpacts.org/historic-trends-in-transatlantic-message-speed/), [Historic trends in long-range military payload delivery](https://aiimpacts.org/historic-trends-in-long-range-military-payload-delivery/), [Historic trends in bridge span length](https://aiimpacts.org/historic-trends-in-bridge-span-length/), [Historic trends in light intensity](https://aiimpacts.org/historic-trends-in-light-intensity/), [Historic trends in book production](https://aiimpacts.org/historic-trends-in-book-production/), [Historic trends in telecommunications performance](https://aiimpacts.org/historic-trends-in-telecommunications-performance/), [Historic trends in slow light technology](https://aiimpacts.org/historic-trends-in-slow-light-technology/), [Penicillin and historic syphilis trends](https://aiimpacts.org/penicillin-and-historic-syphilis-trends/), [Historic trends in the maximum superconducting temperature](https://aiimpacts.org/historic-trends-in-the-maximum-superconducting-temperature/), [Historic trends in chess AI](https://aiimpacts.org/historic-trends-in-chess-ai/), [Effect of Eli Whitney’s cotton gin on historic trends in cotton ginning](https://aiimpacts.org/effect-of-eli-whitneys-cotton-gin-on-historic-trends-in-cotton-ginning/), [Historic trends in flight airspeed records](https://aiimpacts.org/historic-trends-in-flight-airspeed-records/)

### **Evolution engineering comparison**

******Primary authors******: Ronny Fernandez
**Main page:** [How energy efficient are human-engineered flight designs relative to natural ones?](https://aiimpacts.org/are-human-engineered-flight-designs-better-or-worse-than-natural-ones/)

Other pages added in 2020: [Energy efficiency of monarch butterfly flight](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/), [Energy efficiency of wandering albatross flight](https://aiimpacts.org/energy-efficiency-of-wandering-albatross-flight/), [Energy efficiency of paramotors](https://aiimpacts.org/energy-efficiency-of-paramotors/), [Energy efficiency of The Spirit of Butt’s Farm](https://aiimpacts.org/energy-efficiency-of-the-spirit-of-butts-farm/), [Energy efficiency of MacCready Gossamer Albatross](https://aiimpacts.org/maccready-gossamer-albatross/), [Energy efficiency of Boeing 747-400](https://aiimpacts.org/energy-efficiency-of-boeing-747-400/), [Energy efficiency of Airbus A320](https://aiimpacts.org/energy-efficiency-of-airbus-a320/), [Energy efficiency of North American P-51 Mustang](https://aiimpacts.org/energy-efficiency-of-north-american-p-51-mustang/), [Energy efficiency of Vickers Vimy plane](https://aiimpacts.org/energy-efficiency-of-north-american-p-51-mustang/), [Energy efficiency of Wright model B](https://aiimpacts.org/energy-efficiency-of-wright-model-b/), [Energy efficiency of Wright Flyer](https://aiimpacts.org/energy-efficiency-of-wright-flyer/)

### **Time to cross human performance range**

********Primary authors****:**** Richard Korzekwa
**Pages added in 2020:**

* [Time for AI to cross the human range in English draughts](https://aiimpacts.org/time-for-ai-to-cross-the-human-range-in-english-draughts/)
* [Time for AI to cross the human range in StarCraft](https://aiimpacts.org/time-for-ai-to-cross-the-human-range-in-starcraft/)
* [Time for AI to cross the human performance range in ImageNet image classification](https://aiimpacts.org/time-for-ai-to-cross-the-human-performance-range-in-imagenet-image-classification/)
* [Time for AI to cross the human performance range in chess](https://aiimpacts.org/time-for-ai-to-cross-the-human-performance-range-in-chess/)
* [Time for AI to cross the human performance range in Go](https://aiimpacts.org/time-for-ai-to-cross-the-human-performance-range-in-go/)

### **Historic hardware trends**

******Primary authors****:** Asya Bergal
**Pages added in 2020:**

* [2019 recent trends in GPU price per FLOPS](https://aiimpacts.org/2019-recent-trends-in-gpu-price-per-flops/)
* [2019 recent trends in Geekbench score per CPU price](https://aiimpacts.org/2019-recent-trends-in-geekbench-score-per-cpu-price/)
* [Trends in DRAM price per gigabyte](https://aiimpacts.org/trends-in-dram-price-per-gigabyte/)

### **Interviews on plausibility of AI safety by default**

******Primary authors****:** Asya Bergal, Robert Long
**Main page:** [Interviews on plausibility of AI safety by default](https://aiimpacts.org/interviews-on-plausibility-of-ai-safety-by-default/)
**Blog post:** [Takeaways from safety by default interviews](https://aiimpacts.org/takeaways-from-safety-by-default-interviews/)

### **Shallow survey of prescient actions**

****Primary authors**:** Richard Korzekwa
**Main page:** [Preliminary survey of prescient actions](https://aiimpacts.org/survey-of-prescient-actions/)

Other work
----------

### **Pages added in 2020**

* [AI Impacts key questions of interest](https://aiimpacts.org/ai-impacts-key-questions-of-interest/)— Primary author: Katja Grace
* [Was the industrial revolution a drastic departure from historic trends?](https://aiimpacts.org/was-the-industrial-revolution-a-drastic-departure-from-historic-trends/) — Primary author: Katja Grace
* [Surveys on fractional progress towards HLAI](https://aiimpacts.org/surveys-on-fractional-progress-towards-hlai/) — Primary author: Asya Bergal
* [Precedents for economic n-year doubling before 4n-year doubling](https://aiimpacts.org/precedents-for-economic-n-year-doubling-before-4n-year-doubling/) — Primary author: Daniel Kokotajlo
* [Resolutions of mathematical conjectures over time](https://aiimpacts.org/resolutions-of-mathematical-conjectures-over-time/) — Primary author: Asya Bergal

### **Blog posts added in 2020**

* [Misalignment and misuse: whose values are manifest?](https://aiimpacts.org/misalignment-and-misuse-whose-values-are-manifest/) — Katja Grace
* [Automated intelligence is not AI](https://aiimpacts.org/automated-intelligence-is-not-ai/) — Katja Grace
* [Relevant pre-AGI possibilities](https://aiimpacts.org/relevant-pre-agi-possibilities/) — Daniel Kokotajlo, Asya Bergal
* [Description vs simulated prediction](https://aiimpacts.org/description-vs-simulated-prediction/) — Richard Korzekwa
* [Atari early](https://aiimpacts.org/atari-early/) — Katja Grace
* [Three kinds of competitiveness](https://aiimpacts.org/three-kinds-of-competitiveness/) — Daniel Kokotajlo
* [AGI in a vulnerable world](https://aiimpacts.org/agi-in-a-vulnerable-world/) — Asya Bergal
* [Cortes, Pizarro, and Afonso as precedents for takeover](https://aiimpacts.org/cortes-pizarro-and-afonso-as-precedents-for-ai-takeover/) — Daniel Kokotajlo

AI Impacts 2020 review


AI Impacts key questions of interest

*Updated Dec 17, 2020*

This is a list of questions that AI Impacts focuses on answering.

Details
-------

We are interested in understanding how the development of advanced AI will proceed and how it may affect humanity, especially insofar as these are relevant to efforts to improve the outcomes. This is a list of questions within this topic that we currently consider particularly important to answer.

### List

* **AI RISK:** What type and degree of risk will be posed to humanity by advanced AI systems?
* **CHARACTER:** What will early advanced AI systems be like?
	+ **ARCHITECTURE:** What types of algorithms will advanced AI systems use?
	+ **AGENCY:** Will the bulk of advanced AI systems be in the form of ‘agents’? (If so, in what sense? Will they pursue ‘goals’? Can we say anything about the nature the goals or the pursuit?)
	+ **PRICE:** How much will the first [human-level AI](https://aiimpacts.org/human-level-ai/) systems cost?
* **TIMELINES:** When will [human-level AI](https://aiimpacts.org/human-level-ai/) be developed? (When will other important AI milestones take place?)
* **TAKE-OFF SPEED:** How rapid is the development of AI likely to be near human-level?
	+ **DISCONTINUITY:** Will there be abrupt progress in AI development at around human-level performance?
	+ **INTELLIGENCE EXPLOSION:** How much will AI development be accelerated by feedback from AI-based automation of the process?
* **PRE-AI DEVELOPMENTS:** What developments will take place before advanced AI is developed?
	+ **PATHS TO HLAI:** By what methods is advanced AI likely to come about? (e.g. will human-level AI be developed via brain emulation before it is developed via machine learning? Will neuroscientific understanding play a large role in development?)
	+ **CONTEMPORARY EVENTS:** How will the world be relevantly different at the time that advanced AI is developed?
	+ **WARNING SIGNS:** Should we expect advance notice of disruptive change from AI? (What would it look like?)
* **POST-AI SOCIETY:** If advanced AI transforms the world, what will the world look like afterwards? (e.g. What will the economic impacts be? Will humans flourish? What roles will AI systems play in society?)
* **ACTIONS:** What can we say about the impact of contemporary choices on long-term outcomes?

AI Impacts key questions of interest


How energy efficient are human-engineered flight designs relative to natural ones?

*Updated Dec 10, 2020*

***This page is out-of-date. Visit the [updated version of this page](https://wiki.aiimpacts.org/doku.php?id=power_of_evolution:evolution_engineering_comparison:how_energy_efficient_are_human-engineered_flight_designs_relative_to_natural_ones) on our [wiki](https://wiki.aiimpacts.org/doku.php?id=start).***
Among two animals and nine machines:

* In terms of mass⋅distance/energy, the most efficient animal was 2-8x more efficient than the most efficient machine. All entries fell within two orders of magnitude.
* In terms of distance/energy, the most efficient animal was 3,000-20,000x more efficient than the most efficient machine. Both animals were more efficient than all machines. Entries ranged over more than eight orders of magnitude.

Details
-------

### Background

This case study is part of [research](https://aiimpacts.org/comparison-of-naturally-evolved-and-engineered-solutions/) that intends to compare the performance of human engineers and natural evolution on problems where both have developed solutions. The goal of this is to inform our expectations about the performance of future artificial intelligence relative to biological minds.

### Metrics

We consider two metrics:

1. Distance per energy used (meters / kilojoule).
2. Mass times distance per energy used (kilograms⋅meters / joule).

These operationalize the problem of flight into two more specific problems. There are many other aspects of flight performance that one could measure, such as energy efficiency of acceleration in a straight line, turning, hovering, vertical acceleration, vertical distance, landing, taking off, time flying per energy, and our same measures with fewer or further restrictions on acceptable entries. For instance, we might look at the problem of flying with flapping wings, or without the restriction that the solutions we consider are heavier than air and self powered.

We did not require that the flight of an entry be constantly powered. Solutions that spend some time gliding as well as some time using powered flight were allowed. Both [albatrosses](https://aiimpacts.org/energy-efficiency-of-wandering-albatross-flight/) and [butterflies](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/) use air currents to fly further.[1](https://aiimpacts.org/are-human-engineered-flight-designs-better-or-worse-than-natural-ones/#easy-footnote-bottom-1-2715 "“Albatrosses and other large seabirds use dynamic soaring to gain sufficient energy from the wind to travel large distances rapidly and with little apparent effort.“

Richardson, Philip L., Ewan D. Wakefield, and Richard A. Phillips. “Flight Speed and Performance of the Wandering Albatross with Respect to Wind.” Movement Ecology 6, no. 1 (March 7, 2018): 3. https://doi.org/10.1186/s40462-018-0121-9.

See page on monarch butterflies for details of their soaring behavior.") The energy gains from these techniques were not included in the final score, and entries were not penalized for spending a larger fraction of time gliding. It seems likely that paramotor pilots use similar techniques, since paramotors are well suited to gliding (being paragliders with propeller motors strapped to the backs of their pilots). Our energy efficiency estimate for the paramotor came from a record breaking distance flight in which the quantity of available fuel was limited, and so it is likely that some gliding was used to increase the distance traveled as much as possible.

When multiple input values could have been used, such as the takeoff weight and the landing weight, or different estimates for the energetic costs of different kinds of flight for the Monarch butterfly, we generally calculated a high and a low estimate, taking the most optimistic and pessimistic inputs respectively. In all cases, the resulting best and worst estimates differed by less than a factor of ten.

### Selection of case studies

We selected case studies informally, according to judgments about possible high energy efficiencies, and with an eye to exploring a wider range of case studies.

We started by looking at the Boeing 747-400 plane, the Wandering Albatross, and the Monarch Butterfly. We chose the animals for both being known for their abilities to fly long distances, and for both having fairly different body plans.

All three scored surprisingly similarly on distance times weight per energy (details below). This prompted us to look for engineered solutions that were optimized for fuel efficiency. To that end, we looked at paramotors and record breaking flying machines. In the latter category, we found the MacCready Gassomer Albatross, which was a human powered flying device that crossed the English Channel, and the Spirit of Butts’ Farm, which was a model airplane that crossed the Atlantic on one gallon of gasoline.

For reasons that are now obscure, we also included a number of different planes.

We would have liked to include microdrones, since they are different enough from other entries that they might be unusually efficient. However we did not find data on them.

### Case studies

These are the full articles calculating the efficiencies of different flying machines and animals:

* [Wright Flyer](https://aiimpacts.org/energy-efficiency-of-wright-flyer/)
* [Wright model B](https://aiimpacts.org/energy-efficiency-of-wright-model-b/)
* [Vickers Vimy](https://aiimpacts.org/energy-efficiency-of-vickers-vimy-plane/)
* [North American P-51 Mustang](https://aiimpacts.org/energy-efficiency-of-north-american-p-51-mustang/)
* [Paramotors](https://aiimpacts.org/energy-efficiency-of-paramotors/)
* [The Spirit of Butt’s Farm](https://aiimpacts.org/energy-efficiency-of-the-spirit-of-butts-farm/)
* [Monarch butterfly](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/)
* [MacCready Gossamer Albatross](https://aiimpacts.org/maccready-gossamer-albatross/)
* [Airbus A-320](https://aiimpacts.org/energy-efficiency-of-airbus-a320/)
* [Boeing 747-400](https://aiimpacts.org/energy-efficiency-of-boeing-747-400/)
* [Wandering albatross](https://aiimpacts.org/energy-efficiency-of-wandering-albatross-flight/)

### Summary results

Results are available in Table 1 below, and in [this spreadsheet](https://docs.google.com/spreadsheets/d/1hMyKszvJx4A-A-qlL-frQATnb7Wv9bI51ennFbbi_wU/edit?usp=sharing). Figures 1 and 2 below illustrate the equivalent questions of how far each of these animals and machines can fly, given either the same amount of fuel energy, or fuel energy proportional to their body mass.

| Name | natural or human-engineered |  | kg⋅m/J |  |  |  |  | m/kJ |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  |  | worst | mean | best |  |  | worst | mean | best |
| Monarch Butterfly | natural | 0.065 | 0.21 | 0.36 |  |  | 100000 | 350000 | 600000 |
| Wandering Albatross | natural | 1.4 | 2.2 | 3 |  |  | 240 | 240 | 240 |
| The Spirit of Butt’s Farm | human-engineered | 0.086 | 0.12 | 0.16 |  |  | 32 | 32 | 32 |
| MacGready Gossamer Albatross | human-engineered | 0.19 | 0.32 | 0.46 |  |  | 2 | 3.3 | 4.6 |
| Paramotor | human-engineered | 0.058 | 0.079 | 0.1 |  |  | 0.36 | 0.36 | 0.36 |
| Wright model B | human-engineered | 0.036 | 0.078 | 0.12 |  |  | 0.1 | 0.16 | 0.21 |
| Wright Flyer | human-engineered | 0.022 | 0.042 | 0.061 |  |  | 0.080 | 0.13 | 0.18 |
| North American P-51 Mustang | human-engineered | 0.25 | 0.38 | 0.5 |  |  | 0.073 | 0.083 | 0.092 |
| Vickers Vimy | human-engineered | 0.081 | 0.17 | 0.25 |  |  | 0.025 | 0.038 | 0.05 |
| Airbus A320 | human-engineered | 0.33 | 0.47 | 0.61 |  |  | 0.0078 | 0.0078 | 0.0078 |
| Boeing 747-400 | human-engineered | 0.39 | 0.61 | 0.83 |  |  | 0.0021 | 0.0021 | 0.0021 |

**Table 1: Energy efficiency of flight for a variety of natural and man-made flying entities.**

![](https://aiimpacts.org/wp-content/uploads/2020/12/129727480_757428541515114_4350085674692898023_n-1024x434.jpg)**Figure 1: If you give each animal or machine energy proportional to its weight, how far can it fly?**

On mass⋅distance/energy, evolution beats engineers, but they are relatively evenly matched: the albatross (1.4-3.0 kg.m/J) and the Boeing 747-400 (0.39-0.83 kg.m/J) are the best in the natural and engineered classes respectively. Thus the best natural solution we found was roughly 2x-8x more efficient than the human-engineered one.[2](https://aiimpacts.org/are-human-engineered-flight-designs-better-or-worse-than-natural-ones/#easy-footnote-bottom-2-2715 "For the best case for engineers we compare the Boeing 747-400’s best score to the Albatross’s worst, and for the best case for evolution we do the opposite. This gives an advantage for evolution by a factor of somewhere between 1.7 and 7.7.") We found several flying machines more efficient on this metric than the monarch butterfly.

![](https://aiimpacts.org/wp-content/uploads/2020/12/129734907_191799035908993_4248841669315097267_n-1024x435.jpg)**Figure 2: How far animals and machines can fly on the same amount of energy. Note that the vertical axis is log scaled, unlike that of Figure 1, so smaller looking differences are in fact much larger: over eight orders of magnitude (vs less than two in Figure 1).**

On distance/energy, the natural solutions have a much larger advantage. Both are better than all man-made solutions we considered. The best natural and engineered solutions respectively are the monarch butterfly (100,000-600,000 m/kJ) and the Spirit of Butts’ Farm (32 m/kJ), for roughly a 3,000x to 20,000x advantage to natural evolution.

### Interpretation

We take this as weak evidence about the best possible distance/energy and distance.mass/energy measures achievable by human engineers or natural evolution. One reason for this is that this is a small set of examples. Another is that none of these animals or machines were optimized purely for either of these flight metrics—they all had other constraints or more complex goals. For instance, the [paramotor](https://aiimpacts.org/energy-efficiency-of-paramotors/) was competing for a record in which a paramotor had to be used, specifically. For the longest human flight, the flying machine had to be capable of carrying a human. The albatross’ body has many functions. Thus it seems plausible that either engineers or natural evolution could reach solutions far better on our metrics than those recorded here if they were directly aiming for those metrics.

The measurements for distance.mass/energy covered a much narrower band than those for distance/energy: a factor of under two orders of magnitude versus around eight. Comparing best scores between evolution and engineering, the gap is also much smaller, as noted above (a factor of less than one order of magnitude versus three orders of magnitude). This seems like some evidence that that band of performance is natural for some reason, and so that more pointed efforts to do better on these metrics would not readily lead to much higher performance.

*Primary author: Ronny Fernandez*

Notes
-----

Summary: When forecasting AI timelines from <@biological anchors@>(@Draft report on AI timelines@), one important subquestion is how well we expect human-made artifacts to compare to natural artifacts (i.e. artifacts made by evolution). This post gathers empirical data for flight, by comparing the Monarch butterfly and the Wandering Albatross to various types of planes. The albatross is the most efficient, with a score of 2.2 kg-m per Joule (that is, a ~7 kg albatross spends ~3 Joules for every meter it travels). This is 2-8x better than the most efficient manmade plane that the authors considered, the Boeing 747-400, which in turn is better than the Monarch butterfly. (The authors also looked at distance per Joule without considering mass, in which case unsurprisingly the butterfly wins by miles; it is about 3 orders of magnitude better than the albatross, which is in turn better than all the manmade solutions.)


Energy efficiency of monarch butterfly flight

*Updated Nov 25, 2020*

According to very rough estimates, the monarch butterfly:

* can fly around 100,000-600,000 m/kJ
* and move mass at around 0.065-0.36 kg⋅m/J

Details
-------

The **Monarch Butterfly** is a butterfly known for its migration across North America.[1](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-1-2776 "“The monarch butterfly, Danaus plexippus, is famous for its spectacular annual migration across
North America”

Zhan, Shuai, Wei Zhang, Kristjan Niitepõld, Jeremy Hsu, Juan Fernández Haeger, Myron P. Zalucki, Sonia Altizer, Jacobus C. de Roode, Steven M. Reppert, and Marcus R. Kronforst. “The Genetics of Monarch Butterfly Migration and Warning Colouration.” Nature 514, no. 7522 (October 2014): 317–21. https://doi.org/10.1038/nature13812.")

### Mass

The average mass of a monarch butterfly prior to its annual migration has been estimated to be 600mg[2](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-2-2776 "“Each year in late summer and fall in southern Ontario, monarch butterflies, Danaus plexippus plexippus L., engage in migratory flight to the southern U.S.A. and Mexico…

…Late summer monarchs average approximately 600mg.”

Gibo, David L., and Pallett, Megan J., “Soaring Flight of Monarch Butterflies, Danaus Plexippus (Lepidoptera: Danaidae), during the Late Summer Migration in Southern Ontario.” Canadian Journal of Zoology, July 1979. https://doi.org/10.1139/z79-180.")
### Distance per Joule

The following table gives some very rough estimates of energy expenditures, speeds and distances for several modes of flight, based on confusing information from a small number of papers (see footnotes for details).

|  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- |
| Activity | Description | Energy expenditure per mass ( J/g⋅hr) | Energy expenditure for 600mg butterfly (J/s) | Speed (m/s) | distance/energy (m/J) |
| Soaring/gliding | Unpowered flight, including gradual decline and ascent via air currents | 8-33[3](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-3-2776 "Gibo and Pallet (1979) appear to think energy expenditure of soaring is well approximated by energy expenditure for resting:

“The most efficient flying technique, in terms of
cost per unit of distance travelled, is soaring (Pennycuick 1969, 1975). During soaring flight, altitude
is gained or maintained by gliding in rising air currents. Since a soaring animal is actually gliding,
the wings are held more or less motionless and
the high-energy expenditure of powered flight is
avoided…

…if soaring requires approximately
the basal level of metabolic expenditure, an average D. piesippus with an initial fat supply of 140 mg,
which could fly under power for only I1 h, may be
able to soar for 1060 h.”

Gibo, David L., and Pallett, Megan J., “Soaring Flight of Monarch Butterflies, Danaus Plexippus (Lepidoptera: Danaidae), during the Late Summer Migration in Southern Ontario.” Canadian Journal of Zoology, July 1979. https://doi.org/10.1139/z79-180.

They say, “Flying Lepidoptera have a metabolic rate that is as
much as 100 times above the basal rate (Zebe 1954).”

Zhan et al 2014 measure metabolic rates for monarch butterflies specifically, but only report that the resting rate is 25x lower than a flying rate they prompted:

“…we tested it by measuring flight metabolic
rates. We found active flight to be exceptionally demanding energetically, utilizing 25 times
more energy than resting…”

Zhan, Shuai, Wei Zhang, Kristjan Niitepõld, Jeremy Hsu, Juan Fernández Haeger, Myron P. Zalucki, Sonia Altizer, Jacobus C. de Roode, Steven M. Reppert, and Marcus R. Kronforst. “The Genetics of Monarch Butterfly Migration and Warning Colouration.” Nature 514, no. 7522 (October 2014): 317–21. https://doi.org/10.1038/nature13812.

Also, their data makes very little sense to us. For instance, the figure suggests that flying is less energy intensive than resting. However it seems likely that this is a misunderstanding on our part, since for instance their y-axis in not labeled. Given this and the earlier claim that the difference between resting and flying can be 100x for butterflies, we use the relative number 25-100x, in combination with the flight and cruising energy estimates given below.

high estimate:
837/25 J/g⋅hr = 33 J/g⋅hr (see below for flying rate)

The lowest estimate would be the cruising speed energy expenditure divided by 100, but it seems like the 100 figure was meant as an upper bound on the difference, so probably referring to rest vs. maximal flight energy.

Thus we take the lowest of the other combinations, as low end estimate:
min(209/25, 837/100) = min(8.4,  8.4) = 8.4 J/g⋅hr") | ~0.0014 – 0.0056[4](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-4-2776 "8-33 J/g⋅hr * 0.6g= ~5-20 J/hr = 0.0014 – 0.0056 J/s") | Very roughly 2.5-3.6 on average[5](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-5-2776 "Gibo & Pallett (1979) assume this in estimating flight range possible via soaring:

“Consequently, if soaring requires approximately
the basal level of metabolic expenditure, an average D. piesippus with an initial fat supply of 140 mg,
which could fly under power for only I1 h, may be
able to soar for 1060 h. For gliding speeds ranging
from approximately 9 to 13 km/hr, the theoretical
maximum range, without pauses for feeding, would
fall between 9500 and 13 800 km.”

Gibo, David L., and Pallett, Megan J., “Soaring Flight of Monarch Butterflies, Danaus Plexippus (Lepidoptera: Danaidae), during the Late Summer Migration in Southern Ontario.” Canadian Journal of Zoology, July 1979. https://doi.org/10.1139/z79-180.

9-13km/hr = 2.5-3.6 m/s") | 446- 2571[6](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-6-2776 "2.5/.0056 = 446

3.6/.0014 = 2571

These numbers sound surprisingly high, but they are at least in line with Gibo & Pallett’s estimate for maximal distance possibly flown with the 140mg of fat in a butterfly, if it could soar for the entire time:

“Consequently, if soaring requires approximately
the basal level of metabolic expenditure, an average D. piesippus with an initial fat supply of 140 mg,
which could fly under power for only 11 h, may be
able to soar for 1060 h. For gliding speeds ranging
from approximately 9 to 13 km/hr, the theoretical
maximum range, without pauses for feeding, would
fall between 9500 and 13 800 km.”

Gibo, David L., and Pallett, Megan J., “Soaring Flight of Monarch Butterflies, Danaus Plexippus (Lepidoptera: Danaidae), during the Late Summer Migration in Southern Ontario.” Canadian Journal of Zoology, July 1979. https://doi.org/10.1139/z79-180.

Our understanding is that this much soaring without the need for powered flight is implausible, but this efficiency figure should still be approximately correct for shorter distances where soaring is feasible.") |
| Cruising | Low speed powered flight | Very roughly 209[7](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-7-2776 "Gibo and Pallett estimate cruising flight to be about one fourth as costly as sustained flapping flight:

“Since profile drag is proportional to the square of the speed, doubling the speed increases the profile drag by four (Irving 1977). Consequently, it seems reasonable to assume that the slower, less energetic cruising flight of D. plexippus requires approximately 25% of the energy expenditure of vigorous flight.”

Gibo, David L., and Pallett, Megan J., “Soaring Flight of Monarch Butterflies, Danaus Plexippus (Lepidoptera: Danaidae), during the Late Summer Migration in Southern Ontario.” Canadian Journal of Zoology, July 1979. https://doi.org/10.1139/z79-180.

Given an estimate of 837 J/g⋅hr for sustained flapping flight (see below), this gives us 209 J/g⋅hr for cruising.") | 0.035[8](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-8-2776 "209 J/g⋅hr * 0.6g = 125J/hr = 0.035 J/s") | Maximum: >5[9](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-9-2776 "“However, during migration D.
plexippus adults usually adopt a slower cruising
flight at an airspeed of 18 km/hr (Urquhart 1960), a
flying strategy that should result in less fuel being
consumed as a result of reduced drag at lower
airspeeds.”

Gibo, David L., and Pallett, Megan J., “Soaring Flight of Monarch Butterflies, Danaus Plexippus (Lepidoptera: Danaidae), during the Late Summer Migration in Southern Ontario.” Canadian Journal of Zoology, July 1979. https://doi.org/10.1139/z79-180.") | Maximum: >143[10](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-10-2776 ">5/0.035 = 143") |
| Sustained flapping | High speed powered flight | Very roughly 837[11](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-11-2776 "We were not able to find the flight metabolic rate of monarch butterflies. Instead we use the flight metabolic rate of 200 calories per gram per hour (837 J/g⋅hr) that was measured as a minimum for another butterfly species during energetic flight, as Gibo and Pallet (1979) seem to think is somewhat reasonable, in their paper on soaring flight of monarch butterflies:

“If we assume that the metabolic rate of D. plexippus during sustained flapping flight is only 200 cal/g h^-1, the minimal value determined by Zebe (1954) for another butterfly (Vanessa sp.)…”

Gibo, David L., and Pallett, Megan J., “Soaring Flight of Monarch Butterflies, Danaus Plexippus (Lepidoptera: Danaidae), during the Late Summer Migration in Southern Ontario.” Canadian Journal of Zoology, July 1979. https://doi.org/10.1139/z79-180.") | ~0.14[12](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-12-2776 "837 J/g⋅hr * 0.6g = 502 J/hr = 0.14 J/s") | Maximum: >13.9[13](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-13-2776 "Maximal observed in one study:

“On some days, the butterflies were achieving estimated ground speeds of more than 50 km/hr.”

Gibo, David L., and Pallett, Megan J., “Soaring Flight of Monarch Butterflies, Danaus Plexippus (Lepidoptera: Danaidae), during the Late Summer Migration in Southern Ontario.” Canadian Journal of Zoology, July 1979. https://doi.org/10.1139/z79-180.

Also, airspeed during sustained flapping is around 40km/hr, and it seems unlikely that the motion of air can’t add a further 10km/hr to ground speed:

“In D. plexippus vigorous flight of this type produces an airspeed
of approximately 40 kmlh (Urquhart 1960).”

Gibo, David L., and Pallett, Megan J., “Soaring Flight of Monarch Butterflies, Danaus Plexippus (Lepidoptera: Danaidae), during the Late Summer Migration in Southern Ontario.” Canadian Journal of Zoology, July 1979. https://doi.org/10.1139/z79-180.
") | Maximum: >99[14](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-14-2776 ">13.9/0.14 = 99") |

**Table 1: Statistics for several modes of flight. All figures are very rough estimates, based on incomplete and confusing information from a small number of papers (see footnotes for details).**

Soaring is estimated to be potentially very energy efficient (see Table 1), since it mostly makes use of air currents for energy. It seems likely that at least a small amount of powered flight is needed for getting into the air, however monarch butterflies can apparently fly for hundreds of kilometers in a day[15](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-15-2776 "“The farthest ranging monarch butterfly recorded traveled 265 miles in one day.” [265 miles = 426km]

“Monarch Butterfly Migration and Overwintering.” Accessed November 25, 2020. https://www.fs.fed.us/wildflowers/pollinators/Monarch_Butterfly/migration/index.shtml."), so supposing that they don’t stop many times in a day, taking off seems likely a negligible part of the flight.[16](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-16-2776 "For instance, if it pessimistically takes 100 meters of powered flight to take off, then it would be .02% of the distance, so if it was 25x as much energy as usual for that distance, it would add .5% to the total energy use, which is far within the margin of error for this very rough calculation.")
This would require ideal wind conditions, and our impression is that in practice, butterflies do not often fly very long distances without using at least a small amount of powered flight.[17](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-17-2776 "For instance, in this video of soaring butterflies, most of them appear to flap their wings occasionally: https://www.youtube.com/watch?v=a-8SPgG–6I

Usa Monarch. Soaring MONARCH BUTTERFLIES at Their Mexico Migration Site, 2019. https://www.youtube.com/watch?v=a-8SPgG–6I.")
There is stronger evidence that monarch butterflies can realistically soar around 85% of the time, from Gibo & Pallett, who report their observations of butterflies under relatively good conditions.[18](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-18-2776 "“A description of the thermal soaring activity observed on September 7, a particularly favourable day, provides a clear picture of the relative importance of this type of flight. On this day we recorded the greatest proportion of soaring flights for any single day of observation.

Soaring accounted for 1964 s or 83.5% of this time, and powered flight for 359 s or 15.3%.”

Gibo, David L., and Pallett, Megan J., “Soaring Flight of Monarch Butterflies, Danaus Plexippus (Lepidoptera: Danaidae), during the Late Summer Migration in Southern Ontario.” Canadian Journal of Zoology, July 1979. https://doi.org/10.1139/z79-180.") So as a high estimate, we use this fraction of the time for soaring, and suppose that the remaining time is the relatively energy-efficient cruising, and take the optimistic end of all ranges. This gives us:

One second of flight = 0.15 seconds cruising + 0.85 seconds soaring

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_= 0.15s \* 5 m/s cruising + 0.85s \* 3.6m/s soaring

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_= 0.75m cruising + 3.06m soaring

\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_\_= 3.81m total

This also gives us:

= 0.75m / 143 m/J cruising + 3.06m / 2571 m/J soaring

= 0.0064 J total

Thus we have:

distance/energy = 3.81m/0.0064 J = 595 m/J

For a low estimate of efficiency, we will assume that all of the powered flight is the most energetic flight, that powered flight is required half the time on average, and that the energy cost of gliding is twice that of resting. This gives us:

Energy efficiency = (50% \* soaring distance + 50% \* powered distance) / (50% \* soaring energy + 50% \* powered energy)

= (50% \* soaring distance/time + 50% \* powered distance/time) / (50% \* soaring energy/time + 50% \* powered energy/time)

= (0.5 \* 2.5m/s + 0.5 \* 13.9m/s) / (0.5 \* (0.0056 \* 2) J/s + 0.5 \* 0.14 J/s)

= 108 m/J

Thus we have, very roughly:

distance/energy = 100,000-600,000 m/kJ

For concreteness, a kJ is the energy in around a quarter of a raspberry. [19](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-19-2776 "“FoodData Central.” Accessed December 10, 2020. https://fdc.nal.usda.gov/fdc-app.html#/food-details/167755/nutrients.")
### Mass⋅distance per Joule

As noted earlier, the average mass of a monarch butterfly prior to its annual migration has been estimated to be 600mg[20](https://aiimpacts.org/energy-efficiency-of-monarch-butterfly-flight/#easy-footnote-bottom-20-2776 "“Each year in late summer and fall in southern Ontario, monarch butterflies, Danaus plexippus plexippus L., engage in migratory flight to the southern U.S.A. and Mexico…

…Late summer monarchs average approximately 600mg.”

Gibo, David L., and Pallett, Megan J., “Soaring Flight of Monarch Butterflies, Danaus Plexippus (Lepidoptera: Danaidae), during the Late Summer Migration in Southern Ontario.” Canadian Journal of Zoology, July 1979. https://doi.org/10.1139/z79-180.")
Thus we have:

mass\*distance/energy = 0.0006 kg \* 108 — 0.0006 kg \* 595 m/J

= 0.065 — 0.36 kg⋅m/J

*Primary author: Ronny Fernandez*

Notes
-----


Energy efficiency of wandering albatross flight

*Updated Nov 25, 2020*

The wandering albatross:

* can fly around 240m/kJ
* and move mass at around 1.4—3.0kg.m/J

Details
-------

The wandering albatross is a very large seabird that flies long distances on wings with the largest span of any bird.[1](https://aiimpacts.org/energy-efficiency-of-wandering-albatross-flight/#easy-footnote-bottom-1-2772 "“The wandering albatross, snowy albatross, white-winged albatross or goonie[3] (Diomedea exulans) is a large seabird from the family Diomedeidae, which has a circumpolar range in the Southern Ocean…It is one of the largest, best known, and most studied species of bird in the world, with it possessing the greatest known wingspan of any living bird. …Some individual wandering albatrosses are known to circumnavigate the Southern Ocean three times, covering more than 120,000 km (75,000 mi), in one year”

“Wandering Albatross.” In Wikipedia, October 27, 2020. https://en.wikipedia.org/w/index.php?title=Wandering_albatross&oldid=985673754.")

### Distance per Joule

#### Speed

In a study of wandering albatrosses flying in various wind speeds and directions, average ground speed was 12 m/s, though the fastest ground speed measured appears to be around 24m/s, [2](https://aiimpacts.org/energy-efficiency-of-wandering-albatross-flight/#easy-footnote-bottom-2-2772 "“The average ground speed is 12.0 (± 0.1) m/s.”

See Figure 3 for all ground speed measurements. Though also,

“Notably, due to the combination of fast airspeeds and leeway the fastest ground speeds (~ 22 m/s) tend to be located in the diagonal downwind direction.

Richardson, Philip L., Ewan D. Wakefield, and Richard A. Phillips. “Flight Speed and Performance of the Wandering Albatross with Respect to Wind.” Movement Ecology 6, no. 1 (March 7, 2018): 3. https://doi.org/10.1186/s40462-018-0121-9.") We use average ground speed for this estimate because we only have data on average energy expenditure, though it is likely that higher ground speeds involve more energy efficient flight, since albatross flight speed is dependent on wind and it appears that higher speeds are substantially due to favorable winds.[3](https://aiimpacts.org/energy-efficiency-of-wandering-albatross-flight/#easy-footnote-bottom-3-2772 "“Notably, due to the combination of fast airspeeds and leeway the fastest ground speeds (~ 22 m/s) tend to be located in the diagonal downwind direction.”

Richardson, Philip L., Ewan D. Wakefield, and Richard A. Phillips. “Flight Speed and Performance of the Wandering Albatross with Respect to Wind.” Movement Ecology 6, no. 1 (March 7, 2018): 3. https://doi.org/10.1186/s40462-018-0121-9.")
#### Energy expenditure

One study produced an estimate that when flying, albatrosses use 2.35 times their basal metabolic rate[4](https://aiimpacts.org/energy-efficiency-of-wandering-albatross-flight/#easy-footnote-bottom-4-2772 "“Energy cost of flight was estimated to be 2.35 times measured BMR.”

Adams, N. J., C. R. Brown, and K. A. Nagy. “Energy Expenditure of Free-Ranging Wandering Albatrosses Diomedea                Exulans.” Physiological Zoology 59, no. 6 (November 1, 1986): 583–91. https://doi.org/10.1086/physzool.59.6.30158606.") which same paper implies is around 1,833 kJ/bird.day.[5](https://aiimpacts.org/energy-efficiency-of-wandering-albatross-flight/#easy-footnote-bottom-5-2772 "“This is equivalent to an overall energy expenditure of 3,354 kJ bird⁻¹ day⁻¹ or 1.83 times measured basal metabolic rate (BMR).”

Adams, N. J., C. R. Brown, and K. A. Nagy. “Energy Expenditure of Free-Ranging Wandering Albatrosses Diomedea                Exulans.” Physiological Zoology 59, no. 6 (November 1, 1986): 583–91. https://doi.org/10.1086/physzool.59.6.30158606.

From this we infer that the basal metabolic rate is 1,833 kJ/ bird.day.")

That gives us a flight cost for flying of 0.050 kJ/second.[6](https://aiimpacts.org/energy-efficiency-of-wandering-albatross-flight/#easy-footnote-bottom-6-2772 "2.35 * 3,354 kJ/1.83 * 1/86400 seconds/day")

#### Distance per Joule calculation

This gives us a distance per energy score of:

distance/energy

= 12 m/s / 0.050 kJ/s

= 240m/kJ

### Mass.distance per Joule

Albatrosses weigh 5.9 to 12.7 kg.[7](https://aiimpacts.org/energy-efficiency-of-wandering-albatross-flight/#easy-footnote-bottom-7-2772 "“Adults can weigh from 5.9 to 12.7 kg (13 to 28 lb)”

“Wandering Albatross.” In Wikipedia, October 27, 2020. https://en.wikipedia.org/w/index.php?title=Wandering_albatross&oldid=985673754.")
Thus we can estimate:

mass.distance/Joule

= 5.9kg \* 240 m/kJ to 12.7kg\*240 m/kJ

= 1.4—3.0kg.m/J

*Primary author: Ronny Fernandez*

Notes
-----


Energy efficiency of paramotors

*Updated Nov 24, 2020*

We estimate that a record-breaking two-person paramotor:

* covered around = 0.36 m/kJ
* and moved mass at around 0.058 – 0.10 kg⋅m/J

Details
-------

Paramotors are powered parachutes that allow the operator to steer.[1](https://aiimpacts.org/energy-efficiency-of-paramotors/#easy-footnote-bottom-1-2765 "“Powered paragliding, also known as paramotoring or PPG, is a form of ultralight aviation where the pilot wears a back-mounted motor (a paramotor) which provides enough thrust to take off using a paraglider. It can be launched in still air, and on level ground, by the pilot alone — no assistance is required….Powered paragliders are smaller, use more efficient (but more difficult to manage) paraglider wings, and steer with brake toggles like sport parachutists.”

“Powered Paragliding.” In Wikipedia, October 31, 2020. https://en.wikipedia.org/w/index.php?title=Powered_paragliding&oldid=986450866.")
### Distance per Joule

The Fédération Aéronautique Internationale (FAI) maintains records for a number of classes of paramotor contest. We look at subclass RPF2T—(Paramotors : Paraglider Control / Foot-launched / Flown with two persons / Thermal Engine)—which is appears to be the most recent paramotor record for ‘Distance in a straight line with limited fuel’.[2](https://aiimpacts.org/energy-efficiency-of-paramotors/#easy-footnote-bottom-2-2765 "Recent paramotor records for distance in a straight line with limited fuel:

“Records | World Air Sports Federation.” Accessed November 18, 2020. https://www.fai.org/records?f%5B0%5D=field_record_sport%3A2025&f%5B1%5D=field_type_of_record%3A488&record=&order=field_date_single_custom&sort=desc.

Details of Mark Morgan’s 2013 record:

“Mark Morgan (GBR) (16956),” October 10, 2017. https://www.fai.org/record/16956.")
The record distance was 123.18 km.[3](https://aiimpacts.org/energy-efficiency-of-paramotors/#easy-footnote-bottom-3-2765 "“Mark Morgan (GBR) (16956),” October 10, 2017. https://www.fai.org/record/16956.") The FAI rules state that no more than 7.5 kg of fuel may be used.[4](https://aiimpacts.org/energy-efficiency-of-paramotors/#easy-footnote-bottom-4-2765 "“Special rules for distance in a straight line with limited fuel…

The aircraft must carry no more than 7.5 kg of fuel which may be used as required.”

FAI. “FAI Sporting Code: Section 10 – Microlights and Paramotors,” 2017.") We will assume that in the process of breaking this record, all of the available fuel was used. We will also assume that regular gasoline was used. Gasoline has an energy density of 45 MJ/kg.[5](https://aiimpacts.org/energy-efficiency-of-paramotors/#easy-footnote-bottom-5-2765 "“Energy Density of Gasoline – The Physics Factbook.” Accessed November 18, 2020. https://hypertextbook.com/facts/2003/ArthurGolnik.shtml.")
Distance per energy = 123.18 km / (7.5 kg \* 45 MJ/kg)

= 0.36 m/kJ

### Mass.distance per Joule

The weight of an entire paramotoring apparatus appears to be the weights of the passengers plus motor plus wing plus clothing and incidentals, based on forum posts.[6](https://aiimpacts.org/energy-efficiency-of-paramotors/#easy-footnote-bottom-6-2765 "“You are 102kg, the motor is probably 25kg, wing weight say 5kg, clothing and incidentals about 8kg. So you total is 140kg.”

www.Paramotorclub.org. “Is This Wing for Me.” Accessed November 24, 2020. https://www.paramotorclub.org/topic/13451-is-this-wing-for-me/.

“Wife at 59kg, plus a 20kg motor (light) plus clothing and instruments say 7kg plus weight of wing (should be included) 5.6kg, that equals 91.6kg.”

www.Paramotorclub.org. “Roadster 3 Weight/Sizing Question.” Accessed November 24, 2020. https://www.paramotorclub.org/topic/12973-roadster-3-weightsizing-question/.") These posts put clothing and incidentals at around 8kg, but are estimates for single person flying, whereas this record was a two person flight. We guess that two people need around 1.5x as much additional weight, for 12kg.

Wikipedia says that the weight of a paramotor varies from 18kg to 34 kg.[7](https://aiimpacts.org/energy-efficiency-of-paramotors/#easy-footnote-bottom-7-2765 "“A typical paramotor will weigh on average around 50 lbs. (23 kg) with some models as light at 40 lbs. (18 kg) and some models as high as 75 lbs. (34 kg.)”

“Paramotor.” In Wikipedia, October 1, 2020. https://en.wikipedia.org/w/index.php?title=Paramotor&oldid=981354167.") However it is unclear whether this means the motor itself, or all of the equipment involved.

The glider used appears to be MagMax brand, a typical example of which weighs around 8kg, though this may have been different in 2013, or they may have used a different specific glider.[8](https://aiimpacts.org/energy-efficiency-of-paramotors/#easy-footnote-bottom-8-2765 "“Aircraft            Mag Max”

“Mark Morgan (GBR) (16956),” October 10, 2017. https://www.fai.org/record/16956.

“Glider Weight (kg)     8     8.4”

Ozone Paramotor. “MagMax.” Accessed November 18, 2020. https://www.flyozone.com/paramotor/products/gliders/magmax.") To account for this uncertainty, we shall add the glider weight to the high estimate, and so estimate the weight of the glider and motor together at 18-42kg.

We will assume that the apparently male pilots weighed between 65 and 115 kgs each, based on normal male weights[9](https://aiimpacts.org/energy-efficiency-of-paramotors/#easy-footnote-bottom-9-2765 "Pilots appear to be male in this picture: https://www.fai.org/sites/default/files/record/recs-documents/16956-2_0.jpg

10th to 90th percentile male weights, from:

DQYDJ – Don’t Quit Your Day Job… “Weight Percentile Calculator for Men and Women in the United States,” February 13, 2019. https://dqydj.com/weight-percentile-calculator-men-women/.").

Thus we have:

weight = motor + wing + people + clothing and incidentals

weight (low estimate) = 18 + 65\*2 + 12 = 160kg

weight (high estimate) = 42 + 115\*2 + 12 = 284kg

High efficiency estimate:

284kg \* 0.36 m/kJ  = 0.10 kg⋅m/J

Low efficiency estimate:

160kg \* 0.36 m/kJ  = .058 kg⋅m/J

This gives us a range of 0.058 – 0.10 kg⋅m/J

*Primary author: Ronny Fernandez*

Notes
-----

Energy efficiency of paramotors


Misalignment and misuse: whose values are manifest?

*By Katja Grace, 18 November 2020, Crossposted from [world spirit sock puppet](https://worldspiritsockpuppet.com/).*

AI related disasters are often categorized as involving misaligned AI, or misuse, or accident. Where:

* misuse means the bad outcomes were wanted by the people involved,
* misalignment means the bad outcomes were wanted by AI (and not by its human creators), and
* accident means that the bad outcomes were not wanted by those in power but happened anyway due to error.

In thinking about specific scenarios, these concepts seem less helpful.

I think a likely scenario leading to bad outcomes is that AI can be made which gives a set of people things they want, at the expense of future or distant resources that the relevant people do not care about or do not own.

For example, consider autonomous business strategizing AI systems that are profitable additions to many companies, but in the long run accrue resources and influence and really just want certain businesses to nominally succeed, resulting in a worthless future. Suppose Bob is considering whether to get a business strategizing AI for his business. It will make the difference between his business thriving and struggling, which will change his life. He suspects that within several hundred years, if this sort of thing continues, the AI systems will control everything. Bob probably doesn’t hesitate, in the way that businesses don’t hesitate to use gas vehicles even if the people involved genuinely think that climate change will be a massive catastrophe in hundreds of years.

When the business strategizing AI systems finally plough all of the resources in the universe into a host of thriving 21st Century businesses, was this misuse or misalignment or accident? The strange new values that were satisfied were those of the AI systems, but the entire outcome only happened because people like Bob chose it knowingly (let’s say). Bob liked it more than the long glorious human future where his business was less good. That sounds like misuse. Yet also in a system of many people, letting this decision fall to Bob may well have been an accident on the part of others, such as the technology’s makers or legislators.

Outcomes are the result of the interplay of choices, driven by different values. Thus it isn’t necessarily sensical to think of them as flowing from one entity’s values or another’s. Here, AI technology created a better option for both Bob and some newly-minted misaligned AI values that it also created—‘Bob has a great business, AI gets the future’—and that option was worse for the rest of the world. They chose it together, and the choice needed both Bob to be a misuser and the AI to be misaligned. But this isn’t a weird corner case, this is a natural way for the future to be destroyed in an economy.

*Thanks to Joe Carlsmith for conversation leading to this post.*


Energy efficiency of The Spirit of Butt’s Farm

*Updated Nov 18, 2020*

The Spirit of Butt’s Farm:

* covered around 31.67 m/kJ
* and moved mass at around 0.16 – 0.086 kg⋅m/J

Details
-------

The Spirit of Butt’s Farm was a record setting model airplane that crossed the Atlantic on one gallon of fuel.[1](https://aiimpacts.org/energy-efficiency-of-the-spirit-of-butts-farm/#easy-footnote-bottom-1-2759 "“The Spirit of Butts’ Farm (also known as TAM 5) was the first model aircraft to cross the Atlantic Ocean on August 11, 2003…Fuel tank: Approx. 118 US fluid ounces (3.5 l)” [118 US fluid ounces = 0.92 US liquid gallons according to Google]

“The Spirit of Butts’ Farm.” In Wikipedia, September 29, 2019. https://en.wikipedia.org/w/index.php?title=The_Spirit_of_Butts%27_Farm&oldid=918639855.") Fully fueled it weighed 4.987 kg, dry it weighed 2.705 kg.[2](https://aiimpacts.org/energy-efficiency-of-the-spirit-of-butts-farm/#easy-footnote-bottom-2-2759 "From Wikipedia:

Weight:	Dry: 5.96 lb (2.705 kg); Fully fueled: 10.99 lb (4.987 kg)[11]

“The Spirit of Butts’ Farm.” In Wikipedia, September 29, 2019. https://en.wikipedia.org/w/index.php?title=The_Spirit_of_Butts%27_Farm&oldid=918639855.")
 The record setting flight used 117.1 fluid ounces of fuel.[3](https://aiimpacts.org/energy-efficiency-of-the-spirit-of-butts-farm/#easy-footnote-bottom-3-2759 "“The flight used 99.2% of its fuel…Fuel tank: Approx. 118 US fluid ounces (3.5 l)”

“The Spirit of Butts’ Farm.” In Wikipedia, September 29, 2019. https://en.wikipedia.org/w/index.php?title=The_Spirit_of_Butts%27_Farm&oldid=918639855.

Multiplying these values we find that the vehicle crossed the Atlantic using 99.2%*118 = 117.1 fluid ounces") The straight line distance of the flight was 3,028.1 km.[4](https://aiimpacts.org/energy-efficiency-of-the-spirit-of-butts-farm/#easy-footnote-bottom-4-2759 "“It was recognized by the FAI as a double world record[2] flight for its duration of 38h 52 min 19 sec[3] and straight-line distance of 1,881.6 mi (3,028.1 km) using an autopilot,[4]…”

“The Spirit of Butts’ Farm.” In Wikipedia, September 29, 2019. https://en.wikipedia.org/w/index.php?title=The_Spirit_of_Butts%27_Farm&oldid=918639855.") It was powered by 88% Coleman lantern fuel, mixed with lubricant.[5](https://aiimpacts.org/energy-efficiency-of-the-spirit-of-butts-farm/#easy-footnote-bottom-5-2759 "From Wikipedia:

Fuel:	Coleman lantern fuel with 16 US fl oz (470 ml) of Indopol L-50 lubricant additive per 1 US gal (3,785 ml). Single fuel tank in the fuselage at the CG point [12] (normal: alcohol)

“The Spirit of Butts’ Farm.” In Wikipedia, September 29, 2019. https://en.wikipedia.org/w/index.php?title=The_Spirit_of_Butts%27_Farm&oldid=918639855.") Coleman fuel is based on naphtha [6](https://aiimpacts.org/energy-efficiency-of-the-spirit-of-butts-farm/#easy-footnote-bottom-6-2759 "“…Coleman Camp Fuel, which is a common naphtha-based fuel used in many lanterns and stoves”

“Coleman Fuel.” In Wikipedia, August 22, 2019. https://en.wikipedia.org/w/index.php?title=Coleman_fuel&oldid=911926038."), so we can use the energy density of naphtha—31.4 MJ/L[7](https://aiimpacts.org/energy-efficiency-of-the-spirit-of-butts-farm/#easy-footnote-bottom-7-2759 "“

Liquid Fuel			MJ / liter

…

Naphtha		31.4

“List of Common Conversion Factors (Engineering Conversion Factors) – IOR Energy Pty Ltd.” Accessed November 17, 2020. http://w.astro.berkeley.edu/~wright/fuel_energy.html.")—as a rough guide to its energy content, though naphtha appears to vary in its content, and it is unclear whether Coleman fuel consists entirely of naphtha.

From all this, we have:

Distance per energy = 3,028.1 km / (117.1 fl oz \* 0.88 \* 31.4 MJ/L)

= 31.67 m/kJ

For weight times distance per energy we will calculate a best and a worst score. To calculate the best score we will use the fully fueled weight, and to calculate the worst score we will use the dry weight. All other values are the same in both calculations.

Best score:

Distance\*mass/energy = 4.987 kg \* 31.67 m/kJ

= 0.16 kg⋅m/J

(4.987 kg \* 3,028.1km) / (117.1 US fluid ounces \* 31.4MJ/litre) = 0.1389 kg\*m/j

Worst score:

Distance\*mass/energy = 2.705 kg \* 31.67 m/kJ

= .086 kg⋅m/J

*Primary author: Ronny Fernandez*

Notes
-----

[Photo](https://commons.wikimedia.org/wiki/File:Tam5.jpg) by Ronan Coyne, licensed under the [Creative Commons](https://en.wikipedia.org/wiki/en:Creative_Commons) [Attribution-Share Alike 3.0 Unported](https://creativecommons.org/licenses/by-sa/3.0/deed.en) license, unaltered.