Probability distribution: Motivated definition

When discussing probabilities, people will often (informally) say things like "well, the probability $\mathbb P(sick)$ of the patient being sick is about 20%." What does this $\mathbb P(sick)$ notation mean?

Intuitively, $\mathbb P(sick)$ is supposed to denote the probability that a particular person is sick (on a scale from 0 to 1). But how is $\mathbb P(sick)$ defined? Is there an objective probability of sickness? If not, where does the number come from?

At first you might be tempted to say $\mathbb P(sick)$ is defined by the surrounding population: If 1% of people are sick at any given time, then maybe $\mathbb P(sick)$ should be 1%. But what if this person is currently running a high fever and complaining about an upset stomach? Then we should probably assign a probability higher than 1%.

Next you might be tempted to say that the _true_ probability of the person being sick is either 0 or 1 (because they're either sick or they aren't), but this observation doesn't really help us manage our _own_ uncertainty. It's all well and good to say "either they sick or they aren't," but if you're a doctor who has to choose which medication to prescribe (and different ones have different drawbacks), then you need some way of talking about how sick they _seem_ to be (given what you've seen).

This leads us to the notion of [4vr subjective probability]. _Your_ probability that a person is sick is a fact about _you._ They are either sick or healthy, and as you observe more facts about them (such as "they're running a fever"), your _personal_ belief in their health vs sickness changes. This is the idea that used to define notation like $\mathbb P(sick).$

Formally, $\mathbb P(sick)$ is defined to be the probability that $\mathbb P$ assigns to $sick,$ where $\mathbb P$ is a type of object known as a "probability distribution", which is an object designed for keeping track of (and managing) uncertainty. Specifically, probability distributions are objects that distribute a finite amount of "stuff" across a large number of "states," and $\mathbb P(sick)$ measures how much stuff $\mathbb P$ _in particular_ puts on $sick$-type states. For example, the states could be cups with labels on them, and the stuff could be water, in which case $\mathbb P(sick)$ would be the proportion of all water in the $sick$-labeled cups.

The "stuff" and "states" may be arbitrary: you can build a probability distribution out of water in cups, clay in cubbyholes, abstract numbers represented in a computer, or weightings between neurons in your head. The stuff is called "probability mass," the states are called "possibilities."

To be even more concrete, imagine you build $\mathbb P$ out of cups and water, and that you give some of the cups suggestive labels like $sick$ and $healthy$. Then you can talk about the proportion of all probability-water that's in the $sick$ cup vs the $healthy$ cup. This is a probability distribution, but it's not a very useful one. In practice, we want to model more than one thing at a time. Let's say that you're a doctor at an immigration center who needs to assess a person's health, age, and country of origin. Now the set of possibilities that you want to represent aren't just $sick$ and $healthy,$ they're _all combinations_ of health, age, and origin:

$$
\begin{align}
sick, \text{age }1, \text{Afghanistan} \\
healthy, \text{age }1, \text{Afghanistan} \\
sick, \text{age }2, \text{Afghanistan} \\
\vdots \\
sick, \text{age }29, \text{Albania} \\
healthy, \text{age }29, \text{Albania} \\
sick, \text{age }30, \text{Albania} \\
\vdots
\end{align}
$$

and so on. If you build this probability distribution out of cups, you're going to need a lot of cups. If there are 2 possible health states ($sick$ and $healthy$), 150 possible ages, and 196 possible countries, then the total number of cups you need in order to build this probability distribution is $2 \cdot 150 \cdot 196 = 58800,$ which is rather excessive. (There's a reason we do probabilistic reasoning using transistors and/or neurons, as opposed to cups with water in them).

In order to make this proliferation of possibilities manageable, the possibilities are usually arranged into columns, such as the "Health", "Age", and "Country" columns above. This columns are known as "[random_variable variables]" of the distribution. Then, $\mathbb P(sick)$ is an abbreviation for $\mathbb P(\text{Health}=sick),$ which counts the proportion of all probability mass (water) allocated to possibilities (cups) that have $sick$ in the Health column of their label.

What's the point of doing all this setup? Once we've made a probability distribution, we can hook it up to the outside world such that, when the world interacts with the probability distribution, the probability mass is shifted around inside the cups. For example, if you have a rule which says "whenever a person shows me a passport from country X, I throw out all water except the water in cups with X in the Country column", then, whenever you see a passport, the probability distribution will get more accurate.

The natural question here is, what are the best ways to manipulate the probability mass in $\mathbb P$ (in response to observations), if the goal is to have $\mathbb P$ get more and more accurate over time? That's exactly the sort of question that [-1bv] can be used to answer (and it has implications both for artificial intelligence, and for understanding human intelligence — after all, _we ourselves_ are a physical system that manages uncertainty, and updates beliefs in response to observations).

At this point, there are two big objections to answer. First objection:

> Whoa now, the number of cups in $\mathbb P$ got pretty big pretty quickly, and this was a simple example. In a realistic probability distribution $\mathbb P$ intended to represent the real world (which has way more than 3 variables worth tracking), the number of necessary possibilities would be _ridiculous._ Why do we define probabilities in terms of these huge impractical "probability distributions"?

This is an important question, which is answered by three points:

1. In practice, there are a number of tricks for exploiting regularities in the structure of the world in order to drastically reduce the number of cups you need to track. We won't be covering those tricks in this guide, but you can check out [ Realistic probabilities] and [ Arbital's guide to Bayes nets] if you're interested in the topic.
2. Even so, full-fledged probabilistic reasoning _is_ computationally infeasible on complex problems. In practice, physical reasoning systems (such as brains or artificial intelligence algorithms) use lots of approximations and shortcuts.
3. Nevertheless, reasoning according to a full probability distribution is the _theoretical ideal_ for how to do good reasoning. [ You can't do better than probabilistic reasoning] (unless you're born knowing the right answers to everything), and [ insofar as you don't use probabilistic reasoning, you can be exploited]. Even if complex probability distribution are too big to manage in practice, they tell give lots of hints about how to reason right or wrong that we can follow in our day-to-day lives.

Second objection:

> You basically just said "given a bunch of cups and some water, we define the probability of a person being sick as the amount of water in some suggestively-labeled cups." How does that have anything to do with whether or not the person is actually sick? Just because you put a $sick$ label on there doesn't magically give the water meaning!

This is an important point. For $\mathbb P$ to be useful, we want to design a reasoning procedure such that the more we interact with a person, the more probability mass starts to reflect how healthy the person actually is. That is, we want the water to go into $sick$ cups if they're sick, and $healthy$ cups if they're healthy. If our reasoning procedure has that property, and we have $\mathbb P$ interact with the world for a while, then its probabilities will get pretty accurate — at which point $\mathbb P$ can be used to answer questions and/or make decisions. (This is the principle that makes brains and artificial intelligence algorithms tick.)

How do we design reasoning mechanisms that cause $\mathbb P$ to become more accurate the more it interacts with the world? That's a [how_to_get_accurate_probabilities big question], and the answer has many parts. One of the most important parts of the answer, though, is a law of probability theory which tells us the correct way to move the probability mass around in response to new observations (assuming the goal is to make $\mathbb P$ more accurate). For more on that law, see [1lz Bayes' rule].


Proof of Bayes' rule: Probability form

Let $\mathbf H$ be a [random_variable variable] in $\mathbb P$ for the true hypothesis, and let $H_k$ be the possible values of $\mathbf H,$ such that $H_k$ is [-1rd]. Then, Bayes' theorem states:

$$\mathbb P(H_i\mid e) = \dfrac{\mathbb P(e\mid H_i) \cdot \mathbb P(H_i)}{\sum_k \mathbb P(e\mid H_k) \cdot \mathbb P(H_k)},$$

with a proof that runs as follows. By the definition of [-1rj],

$$\mathbb P(H_i\mid e) = \dfrac{\mathbb P(e \wedge H_i)}{\mathbb P(e)} = \dfrac{\mathbb P(e \mid  H_i) \cdot \mathbb P(H_i)}{\mathbb P(e)}$$

By the law of [law_of_marginal_probability marginal probability]:

$$\mathbb P(e) = \sum_{k} \mathbb P(e \wedge H_k)$$

By the definition of conditional probability again:

$$\mathbb P(e \wedge H_k) = \mathbb P(e\mid H_k) \cdot \mathbb P(H_k)$$

Done.

Note that this proof of Bayes' rule is less general than the [1xr proof] of the [1x5 odds form of Bayes' rule].

## Example

Using the [22s Diseasitis] example problem, this proof runs as follows:

$$\begin{array}{c}
\mathbb P({sick}\mid {positive}) = \dfrac{\mathbb P({positive} \wedge {sick})}{\mathbb P({positive})} \\[0.3em]
= \dfrac{\mathbb P({positive} \wedge {sick})}{\mathbb P({positive} \wedge {sick}) + \mathbb P({positive} \wedge \neg {sick})} \\[0.3em]
= \dfrac{\mathbb P({positive}\mid {sick}) \cdot \mathbb P({sick})}{(\mathbb P({positive}\mid {sick}) \cdot \mathbb P({sick})) + (\mathbb P({positive}\mid \neg {sick}) \cdot \mathbb P(\neg {sick}))}
\end{array}
$$

Numerically:

$$3/7 = \dfrac{0.18}{0.42} = \dfrac{0.18}{0.18 + 0.24} = \dfrac{90\% * 20\%}{(90\% * 20\%) + (30\% * 80\%)}$$

Using red for sick, blue for healthy, and + signs for positive test results, the proof above can be visually depicted as follows:

![bayes theorem probability](https://i.imgur.com/H9im04o.png?0)

%todo: if we replace the other Venn diagram for the proof of Bayes' rule, we should probably update this one too.%


Likelihood function

Let's say you have a piece of evidence $e$ and a set of hypotheses $\mathcal H.$ Each $H_i \in \mathcal H$ assigns some [56v likelihood] to $e.$ The function $\mathcal L_{e}(H_i)$ that reports this likelihood for each $H_i \in \mathcal H$ is known as a "likelihood function."

For example, let's say that the evidence is $e_c$ = "Mr. Boddy was killed with a candlestick," and the hypotheses are $H_S$ = "Miss Scarlett did it," $H_M$ = "Colonel Mustard did it," and $H_P$ = "Mrs. Peacock did it." Furthermore, if Miss Scarlett was the murderer, she's 20% likely to have used a candlestick. By contrast, if Colonel Mustard did it, he's 10% likely to have used a candlestick, and if Mrs. Peacock did it, she's only 1% likely to have used a candlestick. In this case, the likelihood function is

$$\mathcal L_{e_c}(h) =
\begin{cases}
0.2 & \text{if $h = H_S$} \\
0.1 & \text{if $h = H_M$} \\
0.01 & \text{if $h = H_P$} \\
\end{cases}
$$

For an example using a continuous function, consider a possibly-biased coin whose bias $b$ to come up heads on any particular coinflip might be anywhere between $0$ and $1$.  Suppose we observe the coin to come up heads, tails, and tails. We will denote this evidence $e_{HTT}.$ The likelihood function over each hypothesis $H_b$ =  "the coin is biased to come up heads $b$ portion of the time" for $b \in [0, 1]$ is:

$$\mathcal L_{e_{HTT}}(H_b) = b\cdot (1-b)\cdot (1-b).$$

There's no reason to [1rk normalize] likelihood functions so that they sum to 1 — they aren't probability distributions, they're functions expressing each hypothesis' propensity to yield the observed evidence. For example, if the evidence was really obvious ($e_s$ = "the sun rose this morning,") it might be the case that almost all hypotheses have a very high likelihood, in which case the sum of the likelihood function will be much more than 1.

Likelihood functions carry _absolute_ likelihood information, and therefore, they contain information that [1rq relative likelihoods] do not. Namely, absolute likelihoods can be used to check a hypothesis for [227 strict confusion].


Likelihood ratio

[summary: Given a piece of evidence $e$ and two hypothsese $H_i$ and $H_j,$ the likelihood ratio between them is the ratio of the [-56v] each hypothesis assigns to $e$ For example, let $e$ = "Mr. Boddy was knifed", and say that Professor Plum is 25% likely to use a knife while Mrs. White is only 5% likely to use a knife. Then the likelihood ratio of $e$ between the hypotheses "Plum did it" and "Mrs. White did it" is 25/5 = 5/1. See also [1rq].]

Given a piece of evidence $e_0$ and two hypothsese $H_i$ and $H_j,$ the likelihood ratio between them is the ratio of the [-56v] each hypothesis assigns to $e_0.$

For example, imagine the evidence is $e$ = "Mr. Boddy was knifed", and the hypotheses are $H_P$ = "Professor Plum did it" and $H_W$ = "Mrs. White did it." Let's say that, if Professor Plum were the killer, we're 25% sure he would have used a knife. Let's also say that, if Mrs. White were the killer, there's only a 5% chance she would have used a knife. Then the likelihood ratio of $e_0$ between $H_P$ and $H_W$ is 25/5 = 5, which says that $H_P$ assigns five times as much likelihood to $e$ as does $H_W,$ which means that the evidence supports the "Plum did it" hypothesis five times as much as it supports the "Mrs. White did it" hypothesis.

A likelihood ratio of 5 denotes [1rq relative likelihoods] of $(5 : 1).$ Relative likelihoods can be multiplied by [1rb odds] in order to update those odds, as per [1lz].


Likelihood

[summary:  "Likelihood", when speaking of Bayesian reasoning, denotes *the probability of an observation, supposing some hypothesis to be correct.*

Suppose our piece of evidence $e$ is that "Mr. Boddy was shot."  One of our suspects is Miss Scarlett, and we denote by $H_S$ the hypothesis that Miss Scarlett shot Mr. Boddy.  Suppose that if Miss Scarlett *were* the killer, we'd have predicted in advance a 20% probability she would use a gun, and an 80% chance she'd use some other weapon.

Then the *likelihood* from the evidence, to Miss Scarlett being the killer, is 0.20.  Using [1rj conditional probability notation], $\mathbb P(e \mid H_S) = 0.20.$

This doesn't mean Miss Scarlett has a 20% chance of being the killer; it means that if she is the killer, our observation had a probability of 20%.

Relative likelihoods are a key ingredient for [1ly Bayesian reasoning] and one of the quantities plugged into [1lz Bayes's Rule].]

Consider a piece of evidence $e,$ such as "Mr. Boddy was shot." We might have a number of different hypotheses that explain this evidence, including $H_S$ = "Miss Scarlett killed him", $H_M$ = "Colonel Mustard killed him", and so on.

Each of those hypotheses assigns a different probability to the evidence. For example, imagine that _if_ Miss Scarlett _were_ the killer, there's a 20% chance she would use a gun, and an 80% chance she'd use some other weapon. In this case, the "Miss Scarlett" hypothesis assigns a *likelihood* of 20% to $e.$

When reasoning about different hypotheses using a [-probability_distribution probability distribution] $\mathbb P$, the likelihood of evidence $e$ given hypothesis $H_i$ is often written using the [1rj conditional probability] $\mathbb P(e \mid H_i).$ When reporting likelihoods of many different hypotheses at once, it is common to use a [-likelihood_function,] sometimes written [51n $\mathcal L_e(H_i)$].

[1rq Relative likelihoods] measure the degree of support that a piece of evidence $e$ provides for different hypotheses. For example, let's say that if Colonel Mustard were the killer, there's a 40% chance he would use a gun. Then the absolute likelihoods of $H_S$ and $H_M$ are 20% and 40%, for _relative_ likelihoods of (1 : 2). This says that the evidence $e$ supports $H_M$ twice as much as it supports $H_S,$ and that the amount of support would have been the same if the absolute likelihoods were 2% and 4% instead.

According to [1lz Bayes' rule], relative likelihoods are the appropriate tool for measuring the [22x strength of a given piece evidence]. Relative likelihoods are one of two key constituents of belief in [bayesian_reasoning Bayesian reasoning], the other being [1rm prior probabilities].

While absolute likelihoods aren't necessary when updating beliefs by Bayes' rule, they are useful when checking for [227 confusion]. For example, say you have a coin and only two hypotheses about how it works: $H_{0.3}$ = "the coin is random and comes up heads 30% of the time", and $H_{0.9}$ = "the coin is random and comes up heads 90% of the time." Now let's say you toss the coin 100 times, and observe the data HTHTHTHTHTHTHTHT... (alternating heads and tails). The _relative_ likelihoods strongly favor $H_{0.3},$ because it was less wrong. However, the _absolute_ likelihood of $H_{0.3}$ will be much lower than expected, and this deficit is a hint that $H_{0.3}$ isn't right. (For more on this idea, see [227].)


Bayes' rule: Beginner's guide

This arc teaches basic understanding of [1lz] via [1x5]. This arc should work for [1r5]-type readers.


Proposed A-Class

Post pages which you think should be A-Class in a comment here. Subscribe to this page to get updates about new proposals.

Once the number of approvals from [5ft reviewers] reaches $3-rejections*2$ the page can be given an [4yf A-Class] tag.


Opinion page

Opinion pages represent one view or opinion on a topic, and are not necessarily balanced or a reflection of consensus.


Comprehensive guide to Bayes' Rule

Click below to start reading!


Introduction to Bayes' Rule odds form

Click below to start reading!


Bayes' Rule and its different forms

Click below to start reading!


Bayes' Rule and its implications

Click below to start reading!


Wants to get straight to Bayes

A simple requisite page to mark whether the user has selected wanting to get straight into Bayes on the [1zq Bayes' Rule Guide start page].


High-speed intro to Bayes's rule

(This is a high-speed introduction to [1lz] for people who want to get straight to it and are good at math.  If you'd like a gentler or more thorough introduction, try starting at the [1zq Bayes' Rule Guide] page instead.)

# Percentages, frequencies, and waterfalls

Suppose you're screening a set of patients for a disease, which we'll call Diseasitis.%note:Lit. "inflammation of the disease".% Your initial test is a tongue depressor containing a chemical strip, which usually turns black if the patient has Diseasitis.

- Based on prior epidemiology, you expect that around 20% of patients in the screening population have Diseasitis.
- Among patients with Diseasitis, 90% turn the tongue depressor black.
- 30% of the patients without Diseasitis will also turn the tongue depressor black.

What fraction of patients with black tongue depressors have Diseasitis?

%%hidden(Answer): 3/7 or 43%, quickly obtainable as follows:  In the screened population, there's 1 sick patient for 4 healthy patients.  Sick patients are 3 times more likely to turn the tongue depressor black than healthy patients.  $(1 : 4) \cdot (3 : 1) = (3 : 4)$ or 3 sick patients to 4 healthy patients among those that turn the tongue depressor black, corresponding to a probability of $3/7 = 43\%$ that the patient is sick.%%

(Take your own stab at answering this question, then please click "Answer" above to read the answer before continuing.)

Bayes' rule is a theorem which describes the general form of the operation we carried out to find the answer above.  In the form we used above, we:

- Started from the *prior odds* of (1 : 4) for sick versus healthy patients;
- Multiplied by the *likelihood ratio* of (3 : 1) for sick versus healthy patients blackening the tongue depressor;
- Arrived at *posterior odds* of (3 : 4) for a patient with a positive test result being sick versus healthy.

Bayes' rule in this form thus states that **the prior odds times the likelihood ratio equals the posterior odds.**

We could also potentially see the positive test result as **revising** a *prior belief* or *prior probability* of 20% that the patient was sick, to a *posterior belief* or *posterior probability* of 43%.

To make it clearer that we did the correct calculation above, and further pump intuitions for Bayes' rule, we'll walk through some additional visualizations.

## Frequency representation

The *frequency representation* of Bayes' rule would describe the problem as follows:  "Among 100 patients, there will be 20 sick patients and 80 healthy patients."

![prior frequency](https://i.imgur.com/gvcoqQN.png?0)

"18 out of 20 sick patients will turn the tongue depressor black.  24 out of 80 healthy patients will blacken the tongue depressor."

![posterior frequency](https://i.imgur.com/0XDbrYi.png?0)

"Therefore, there are (18+24)=42 patients who turn the tongue depressor black, among whom 18 are actually sick. (18/42)=(3/7)=43%."

(Some experiments show %%note: E.g. "[Probabilistic reasoning in clinical medicine](https://faculty.washington.edu/jmiyamot/p548/eddydm%20prob%20reas%20i%20clin%20medicine.pdf)" by David M. Eddy (1982).%% that this way of explaining the problem is the easiest for e.g. medical students to understand, so you may want to remember this format for future use.  Assuming you can't just send them to Arbital!)

## Waterfall representation

The *waterfall representation* may make clearer why we're also allowed to transform the problem into prior odds and a likelihood ratio, and multiply (1 : 4) by (3 : 1) to get posterior odds of (3 : 4) and a probability of 3/7.

The following problem is isomorphic to the Diseasitis one:

"A waterfall has two streams of water at the top, a red stream and a blue stream.  These streams flow down the waterfall, with some of each stream being diverted off to the side, and the remainder pools at the bottom of the waterfall."

![unlabeled waterfall](https://i.imgur.com/D8EhY65.png?0)

"At the top of the waterfall, there's around 20 gallons/second flowing from the red stream, and 80 gallons/second flowing from the blue stream.  90% of the red water makes it to the bottom of the waterfall, and 30% of the blue water makes it to the bottom of the waterfall.  Of the purplish water that mixes at the bottom, what fraction is from the red stream versus the blue stream?"

![labeled waterfall](https://i.imgur.com/QIrtuVU.png?0)

We can see from staring at the diagram that the *prior odds* and *likelihood ratio* are the only numbers we need to arrive at the answer:

- The problem would have the same answer if there were 40 gallons/sec of red water and 160 gallons/sec of blue water (instead of 20 gallons/sec and 80 gallons/sec).  This would just multiply the total amount of water by a factor of 2, without changing the ratio of red to blue at the bottom.
- The problem would also have the same answer if 45% of the red stream and 15% of the blue stream made it to the bottom (instead of 90% and 30%).  This would just cut down the total amount of water by a factor of 2, without changing the *relative* proportions of red and blue water.

![wide vs narrow waterfall](https://i.imgur.com/6FOndjc.png?0)

So only the *ratio* of red to blue water at the top (prior odds of the proposition), and only the *ratio* between the percentages of red and blue water that make it to the bottom (likelihood ratio of the evidence), together determine the *posterior* ratio at the bottom:  3 parts red to 4 parts blue.

## Test problem

Here's another Bayesian problem to attempt.  If you successfully solved the earlier problem on your first try, you might try doing this one in your head.

10% of widgets are bad and 90% are good.  4% of good widgets emit sparks, and 12% of bad widgets emit sparks.  What percentage of sparking widgets are bad?

%%hidden(Answer):
- There's $1 : 9$ bad vs. good widgets.  (9 times as many good widgets as bad widgets; widgets are 1/9 as likely to be bad as good.)
- Bad vs. good widgets have a $12 : 4$ relative likelihood to spark, which simplifies to $3 : 1.$  (Bad widgets are 3 times as likely to emit sparks as good widgets.)
- $(1 : 9) \cdot (3 : 1) = (3 : 9) \cong (1 : 3).$  (1 bad sparking widget for every 3 good sparking widgets.)
- Odds of $1 : 3$ convert to a probability of $\frac{1}{1+3} = \frac{1}{4} = 25\%.$ (25% of sparking widgets are bad.) %%

(If you're having trouble using odds ratios to represent uncertainty, see [561 this intro] or [1rb this page].)

# General equation and proof

To say exactly what we're doing and prove its validity, we need to introduce some notation from [1rf probability theory].

If $X$ is a proposition, $\mathbb P(X)$ will denote $X$'s probability, our quantitative degree of belief in $X.$

$\neg X$ will denote the negation of $X$ or the proposition "$X$ is false".

If $X$ and $Y$ are propositions, then $X \wedge Y$ denotes the proposition that both X and Y are true.  Thus $\mathbb P(X \wedge Y)$ denotes "The probability that $X$ and $Y$ are both true."

We now define [1rj conditional probability]:

$$\mathbb P(X|Y) := \dfrac{\mathbb P(X \wedge Y)}{\mathbb P(Y)}   \tag*{(definition of conditional probability)}$$

We pronounce $\mathbb P(X|Y)$ as "the conditional probability of X, given Y".  Intuitively, this is supposed to mean "The probability that $X$ is true, *assuming* that proposition $Y$ is true".

Defining conditional probability in this way means that to get "the probability that a patient is sick, given that they turned the tongue depressor black" we should put all the sick *plus* healthy patients with positive test results into a bag, and ask about the probability of drawing a patient who is sick *and* got a positive test result from that bag.  In other words, we perform the calculation $\frac{18}{18+24} = \frac{3}{7}.$

![diseasitis frequency](https://i.imgur.com/0XDbrYi.png?0)

Rearranging [1rj the definition of conditional probability], $\mathbb P(X \wedge Y) = \mathbb P(Y) \cdot \mathbb P(X|Y).$  So to find "the fraction of all patients that are sick *and* get a positive result", we multiply "the fraction of patients that are sick" times "the probability that a sick patient blackens the tongue depressor".

We're now ready to prove Bayes's rule in the form, "the prior odds times the likelihood ratio equals the posterior odds".

The "prior odds" is the ratio of sick to healthy patients:

$$\frac{\mathbb P(sick)}{\mathbb P(healthy)}  \tag*{(prior odds)}$$

The "likelihood ratio" is how much more *relatively* likely a sick patient is to get a positive test result (turn the tongue depressor black), compared to a healthy patient:

$$\frac{\mathbb P(positive | sick)}{\mathbb P(positive | healthy)} \tag*{(likelihood ratio)}$$

The "posterior odds" is the odds that a patient is sick versus healthy, *given* that they got a positive test result:

$$\frac{\mathbb P(sick | positive)}{\mathbb P(healthy | positive)} \tag*{(posterior odds)}$$

Bayes's theorem asserts that *prior odds times likelihood ratio equals posterior odds:*

$$\frac{\mathbb P(sick)}{\mathbb P(healthy)} \cdot \frac{\mathbb P(positive | sick)}{\mathbb P(positive | healthy)} = \frac{\mathbb P(sick | positive)}{\mathbb P(healthy | positive)}$$

We will show this by proving the general form of Bayes's Rule.  For any two hypotheses $H_j$ and $H_k$ and any piece of new evidence $e_0$:

$$
\frac{\mathbb P(H_j)}{\mathbb P(H_k)}
\cdot
\frac{\mathbb P(e_0 | H_j)}{\mathbb P(e_0 | H_k)}
=
\frac{\mathbb P(e_0 \wedge H_j)}{\mathbb P(e_0 \wedge H_k)}
=
\frac{\mathbb P(e_0 \wedge H_j)/\mathbb P(e_0)}{\mathbb P(e_0 \wedge H_k)/\mathbb P(e_0)}
=
\frac{\mathbb P(H_j | e_0)}{\mathbb P(H_k | e_0)}
$$

In the Diseasitis example, this corresponds to performing the operations:

$$
\frac{0.20}{0.80}
\cdot
\frac{0.90}{0.30}
=
\frac{0.18}{0.24}
=
\frac{0.18/0.42}{0.24/0.42}
=
\frac{0.43}{0.57}
$$

Using red for sick, blue for healthy, grey for a mix of sick and healthy patients, and + signs for positive test results, the proof above can be visualized as follows:

![bayes venn](https://i.imgur.com/YBc2nYo.png?0)

%todo: less red in first circle (top left).  in general, don't have prior proportions equal posterior proportions graphically!%

## Bayes' theorem

An alternative form, sometimes called "Bayes' theorem" to distinguish it from "Bayes' rule" (although not everyone follows this convention), uses absolute probabilities instead of ratios.  The [marginal_probability law of marginal probability] states that for any set of [1rd mutually exclusive and exhaustive] possibilities $\{X_1, X_2, ..., X_i\}$ and any proposition $Y$:

$$\mathbb P(Y) = \sum_i \mathbb P(Y \wedge X_i)   \tag*{(law of marginal probability)}$$

Then we can derive an expression for the absolute (non-relative) probability of a proposition $H_k$ after observing evidence $e_0$ as follows:

$$
\mathbb P(H_k | e_0)
= \frac{\mathbb P(H_k \wedge e_0)}{\mathbb P(e_0)}
= \frac{\mathbb P(e_0 \wedge H_k)}{\sum_i P(e_0 \wedge H_i)}
= \frac{\mathbb P(e_0 | X_k) \cdot \mathbb P(X_k)}{\sum_i \mathbb P(e_0 | X_i) \cdot \mathbb P(X_i)}
$$

The equation of the first and last terms above is what you will usually see described as Bayes' theorem.

To see why this decomposition might be useful, note that $\mathbb P(sick | positive)$ is an *inferential* step, a conclusion that we make after observing a new piece of evidence.  $\mathbb P(positive | sick)$ is a piece of *causal* information we are likely to have on hand, for example by testing groups of sick patients to see how many of them turn the tongue depressor black.  $\mathbb P(sick)$ describes our state of belief before making any new observations.  So Bayes' theorem can be seen as taking what we already believe about the world (including our prior belief about how different imaginable states of affairs would generate different observations), plus an actual observation, and outputting a new state of belief about the world.

## Vector and functional generalizations

Since the proof of Bayes' rule holds for *any pair* of hypotheses, it also holds for relative belief in any number of hypotheses.  Furthermore, we can repeatedly multiply by likelihood ratios to chain together any number of pieces of evidence.

Suppose there's a bathtub full of coins:

- Half the coins are "fair" and have a 50% probability of coming up Heads each time they are thrown.
- A third of the coins are biased to produce Heads 25% of the time (Tails 75%).
- The remaining sixth of the coins are biased to produce Heads 75% of the time.

You randomly draw a coin, flip it three times, and get the result **HTH**.  What's the chance this is a fair coin?

We can validly calculate the answer as follows:

$$
\begin{array}{rll}
  & (3 : 2 : 1) & \cong (\frac{1}{2} : \frac{1}{3} : \frac{1}{6}) \\
\times & (2 : 1 : 3) & \cong ( \frac{1}{2} : \frac{1}{4} : \frac{3}{4} ) \\
\times & (2 : 3 : 1) & \cong ( \frac{1}{2} : \frac{3}{4} : \frac{1}{4} ) \\
\times & (2 : 1 : 3) & \\
= & (24 : 6 : 9) & \cong (8 : 2 : 3) \cong (\frac{8}{13} : \frac{2}{13} : \frac{3}{13})
\end{array}
$$

So the posterior probability the coin is fair is 8/13 or ~62%.

This is one reason it's good to know the [1x5 odds form] of Bayes' rule, not just the [554 probability form] in which Bayes' theorem is often given.%%note: Imagine trying to do the above calculation by repeatedly applying the form of the theorem that says: $$\mathbb P(H_k | e_0) = \frac{\mathbb P(e_0 | X_k) \cdot \mathbb P(X_k)}{\sum_i \mathbb P(e_o | X_i) \cdot \mathbb P(X_i)}$$ %%

We can generalize further by writing Bayes' rule in a functional form.  If $\mathbb O(H_i)$ is a relative belief vector or relative belief function on the variable $H,$ and $\mathcal L(e_0 | H_i)$ is the likelihood function giving the relative chance of observing evidence $e_0$ given each possible state of affairs $H_i,$ then relative posterior belief $\mathbb O(H_i | e_0)$ is given by:

$$\mathbb O(H_i | e_0) = \mathcal L(e_0 | H_i) \cdot \mathbb O(H_i)$$

If we [1rk normalize] the relative odds $\mathbb O$ into absolute probabilities $\mathbb P$ - that is, divide through $\mathbb O$ by its sum or integral so that the new function sums or integrates to $1$ - then we obtain Bayes' rule for probability functions:

$$\mathbb P(H_i | e_0) \propto \mathcal L(e_0 | H_i) \cdot \mathbb P(H_i)  \tag*{(functional form of Bayes' rule)}$$

# Applications of Bayesian reasoning

This general Bayesian framework - prior belief, evidence, posterior belief - is a lens through which we can view a *lot* of formal and informal reasoning plus a large amount of entirely nonverbal cognitive-ish phenomena.%%note:This broad statement is widely agreed.  Exactly *which* phenomena are good to view through a Bayesian lens is sometimes disputed.%%

Examples of people who might want to study Bayesian reasoning include:

- Professionals who use statistics, such as scientists or medical doctors.
- Computer programmers working in the field of machine learning.
- Human beings trying to think.

The third application is probably of the widest general interest.

## Example human applications of Bayesian reasoning

Philip Tetlock found when studying "superforecasters", people who were especially good at predicting future events:

"The superforecasters are a numerate bunch: many know about Bayes' theorem and could deploy it if they felt it was worth the trouble. But they rarely crunch the numbers so explicitly. What matters far more to the superforecasters than Bayes' theorem is Bayes' core insight of gradually getting closer to the truth by constantly updating in proportion to the weight of the evidence." — Philip Tetlock and Dan Gardner, [https://en.wikipedia.org/wiki/Superforecasting _Superforecasting_]

This is some evidence that *knowing about* Bayes' rule and understanding its *qualitative* implications is a factor in delivering better-than-average intuitive human reasoning.  This pattern is illustrated in the next couple of examples.

### The OKCupid date.

One realistic example of Bayesian reasoning was deployed by one of the early test volunteers for a much earlier version of a guide to Bayes' rule.  She had scheduled a date with a 96% OKCupid match, who had then cancelled that date without other explanation.  After spending some mental time bouncing back and forth between "that doesn't seem like a good sign" versus "maybe there was a good reason he canceled", she decided to try looking at the problem using that Bayes thing she'd just learned about.  She estimated:

- A 96% OKCupid match like this one, had prior odds of 2 : 5 for being a desirable versus undesirable date.  (Based on her prior experience with 96% OKCupid matches, and the details of his profile.)
- Men she doesn't want to go out with are 3 times as likely as men she might want to go out with to cancel a first date without other explanation.

This implied posterior odds of 2 : 15 that this was an undesirable date, which was unfavorable enough not to pursue him further.%%note: She sent him what might very well have been the first explicitly Bayesian rejection notice in dating history, reasoning that if he wrote back with a Bayesian counterargument, this would promote him to being interesting again.  He didn't write back.%%

The point of looking at the problem this way is not that she knew *exact* probabilities and could calculate that the man had an exactly 88% chance of being undesirable.  Rather, by breaking up the problem in that way, she was able to summarize what she thought she knew in compact form, see what those beliefs already implied, and stop bouncing back and forth between imagined reasons why a good date might cancel versus reasons to protect herself from potential bad dates.  An answer *roughly* in the range of 15/17 made the decision clear.

### Internment of Japanese-Americans during World War II

From Robyn Dawes's [Rational Choice in an Uncertain World](https://amazon.com/Rational-Choice-Uncertain-World-Robyn/dp/0155752154):

> Post-hoc fitting of evidence to hypothesis was involved in a most grievous chapter in United States history: the internment of Japanese-Americans at the beginning of the Second World War.  When California governor Earl Warren testified before a congressional hearing in San Francisco on February 21, 1942, a questioner pointed out that there had been no sabotage or any other type of espionage by the Japanese-Americans up to that time.  Warren responded, "I take the view that this lack [of subversive activity] is the most ominous sign in our whole situation. It convinces me more than perhaps any other factor that the sabotage we are to get, the Fifth Column activities are to get, are timed just like Pearl Harbor was timed... I believe we are just being lulled into a false sense of security."

You might want to take your own shot at guessing what Dawes had to say about a Bayesian view of this situation, before reading further.

%%hidden(Answer):Suppose we put ourselves into the shoes of this congressional hearing, and imagine ourselves trying to set up this problem.

- The [1rm prior] [1rb odds] that there would be a conspiracy of Japanese-American saboteurs.
- The [56t likelihood] of the observation "no visible sabotage or any other type of espionage", *given* that a Fifth Column actually existed.
- The likelihood of the observation "no visible sabotage from Japanese-Americans", in the possible world where there is *no* such conspiracy.

As soon as we set up this problem, we realize that, whatever the probability of "no sabotage" being observed if there is a conspiracy, the likelihood of observing "no sabotage" if there *isn't* a conspiracy must be even higher.  This means that the likelihood ratio:

$$\frac{\mathbb P(\neg \text{sabotage} | \text {conspiracy})}{\mathbb P(\neg \text {sabotage} | \neg \text {conspiracy})}$$

...must be *less than 1,* and accordingly:

$$
\frac{\mathbb P(\text {conspiracy} | \neg \text{sabotage})}{\mathbb P(\neg \text {conspiracy} | \neg \text{sabotage})}
<
\frac{\mathbb P(\text {conspiracy})}{\mathbb P(\neg \text {conspiracy})}
\cdot
\frac{\mathbb P(\neg \text{sabotage} | \text {conspiracy})}{\mathbb P(\neg \text {sabotage} | \neg \text {conspiracy})}
$$

Observing the total absence of any sabotage can only decrease our estimate that there's a Japanese-American Fifth Column, not increase it.  (It definitely shouldn't be "the most ominous" sign that convinces us "more than any other factor" that the Fifth Column exists.)

Again, what matters is not the *exact* likelihood of observing no sabotage given that a Fifth Column actually exists.  As soon as we set up the Bayesian problem, we can see there's something *qualitatively* wrong with Earl Warren's reasoning.%%

# Further reading

This has been a very brief and high-speed presentation of Bayes and Bayesianism.  It should go without saying that a vast literature, nay, a universe of literature, exists on Bayesian statistical methods and Bayesian epistemology and Bayesian algorithms in machine learning.  Staying inside Arbital, you might be interested in moving on to read:

## More on the technical side of Bayes' rule

- A sadly short list of [22w example Bayesian word problems].  Want to add more?  (Hint hint.)
- [1zm __Bayes' rule: Proportional form.__] The fastest way to present a step in Bayesian reasoning in a way that will sound sort of understandable to somebody who's never heard of Bayes.
- [1zh __Bayes' rule: Log-odds form.__] A simple transformation of Bayes' rule reveals tools for measuring degree of belief, and strength of evidence.
- [1zg __The "Naive Bayes" algorithm.__]  (Scroll down to the middle.)  The original simple Bayesian spam filter.
- [1zg __Non-naive multiple updates.__]  (Scroll down past Naive Bayes.)  How to avoid double-counting the evidence, or worse, when considering multiple items of *correlated* evidence.
- [21c __Laplace's Rule of Succession.__]  The classic example of an [21b inductive prior].

## More on intuitive implications of Bayes' rule

- [220 __A Bayesian view of scientific virtues.__] Why is it that science relies on bold, precise, and falsifiable predictions? Because of Bayes' rule, of course.
- [update_by_inches __Update by inches.__] It's virtuous to change your mind in response to overwhelming evidence. It's even more virtuous to shift your beliefs a little bit at a time, in response to *all* evidence (no matter how small).
- [1y6 __Belief revision as probability elimination.__] Update your beliefs by throwing away large chunks of probability mass.
- [552 __Shift towards the hypothesis of least surprise.__] When you see new evidence, ask: which hypothesis is *least surprised?*
- [21v __Extraordinary claims require extraordinary evidence.__] The people who adamantly claim they were abducted by aliens do provide *some* evidence for aliens. They just don't provide quantitatively *enough* evidence.
- [ __Ideal reasoning via Bayes' rule.__] Bayes' rule is to reasoning as the [https://en.wikipedia.org/wiki/Carnot_cycle Carnot cycle] is to engines: Nobody can be a perfect Bayesian, but Bayesian reasoning is still the theoretical ideal.
- [4xx __Likelihoods, p-values, and the replication crisis.__]  Arguably, a large part of the replication crisis can ultimately be traced back to the way journals treat p-values, and a large number of those problems can be summed up as "P-values are not Bayesian."


Low-speed explanation

Use this tag to indicate that a page offers a relatively slower, more gentle, or more wordy explanation.

Note that the speed of an explanation is not the same as its technical level. An explanation can assume a high technical level and still take its time going into details and so deserve the low-speed tag.


High-speed explanation

Use this tag to indicate that a page offers a relatively faster and more terse explanation.

Note that the speed of an explanation is not the same as its technical level. An explanation can assume a low technical level and still be brief and so deserve the high-speed tag.


Concept

Add this tag to pages which represent a concept. A concept page doesn't have an explanation, it merely describes what the concept is. Take a look at the text that comes before the table of contents on any Wikipedia page to see what a good example of a concept page might contain. Concepts are also expected to be used as requisites.


Odds: Technical explanation

Odds express relative belief: we write "the odds for X versus Y are $17 : 2$" when we think that proposition X is 17/2 = 8.5 times as likely as proposition Y.%%note: The colon denotes that we are forming a set of odds. It does not denote division, as it might in French or German.%%

Odds don't say anything about how likely X or Y is in absolute terms. X might be "it will hail tomorrow" and Y might be "there will be a hurricane tomorrow." In that case, it might be the case that the odds for X versus Y are $17 : 2$, despite the fact that both X and Y are very unlikely. [1lz Bayes' rule] is an example of an important operation that makes use of relative belief.

Odds can be expressed between many different propositions at once. For example, let Z be the proposition "It will rain tomorrow," the odds for X vs Y vs Z might be $(17 : 2 : 100).$ When odds are expressed between only two propositions, they can be expressed using a single [-ratio]. For example, above, the odds ratio between X and Y is 17/2, the odds ratio between X and Z is 17/100, and the odds ratio between Y and Z is 2/100 = 1/50. This asserts that X is 8.5x more likely than Y, and that Z is 50x more likely than Y. When someone says "the odds ratio of sick to healthy is 2/3", they mean that the odds of sickness vs health are $2 : 3.$

[toc:]

# Formal definition

Given $n$ propositions $X_1, X_2, \ldots X_n,$ a set of odds between the propositions is a list $(x_1, x_2, \ldots, x_n)$ of non-negative [4bc real] numbers. Each $x_i$ in the set of odds is called a "term." Two sets of odds $(x_1, x_2, \ldots, x_n)$ and $(y_1, y_2, \ldots, y_n)$ are called "equivalent" if there is an $\alpha > 0$ such that $ \alpha x_i = y_i$ for all $i$ from 1 to $n.$

When we write a set of odds using colons, like $(x_1 : x_2 : \ldots : x_n),$ it is understood that the '=' sign denotes this equivalence. Thus, $(3 : 6) = (9 : 18).$

A set of odds with only two terms can also be written as a fraction $\frac{x}{y},$ where it is understood that $\frac{x}{y}$ denotes the odds $(x : y).$ These fractions are often called "odds ratios."

# Example

Suppose that in some forest, 40% of the trees are rotten and 60% of the trees are healthy.  There are then 2 rotten trees for every 3 healthy trees, so we say that the relative *odds* of rotten trees to healthy trees is 2 : 3. If we selected a tree at random from this forest, the *probability* of getting a rotten tree would be 2/5, but the *odds* would be 2 : 3 for rotten vs. healthy trees.

![2 sick trees, 3 healthy trees](https://i.imgur.com/GVZnz2c.png?0)

# Conversion between odds and probabilities

Consider three propositions, $X,$ $Y,$ and $Z,$ with odds of $(3 : 2 : 6).$ These odds assert  that $X$ is half as probable as $Z.$

When the set of propositions are [1rd mutually exclusive and exhaustive], we can convert a set of odds into a set of [1rf probabilities] by [1rk normalizing] the terms so that they sum to 1.  This can be done by summing all the components of the ratio, then dividing each component by the sum:

$$(x_1 : x_2 : \dots : x_n) = \left(\frac{x_1}{\sum_{i=1}^n x_i} : \frac{x_2}{\sum_{i=1}^n x_i} : \dots : \frac{x_n}{\sum_{i=1}^n x_i}\right)$$

For example, to obtain probabilities from the odds ratio 1/3, w write:

$$(1 : 3) = \left(\frac{1}{1+3}:\frac{3}{1+3}\right) = ( 0.25 : 0.75 )$$

which corresponds to the probabilities of 25% and 75%.

To go the other direction, recall that $\mathbb P(X) + \mathbb P(\neg X) = 1,$ where $\neg X$ is the negation of $X.$  So the odds for $X$ vs $\neg X$ are $\mathbb P(X) : \mathbb P(\neg X)$ $=$ $\mathbb P(X) : 1 - \mathbb P(X).$ If Alexander Hamilton has a 20% probability of winning the election, his odds for winning vs losing are $(0.2 : 1 - 0.2)$ $=$ $(0.2 : 0.8)$ $=$ $(1 : 4).$

# Bayes' rule

Odds are exceptionally convenient when reasoning using [1lz Bayes' rule], since the [1rm prior] odds can be term-by-term multiplied by a set of [relative_likelihoods relative likelihoods] to yield the [1rp posterior] odds.  (The posterior odds in turn can be normalized to yield posterior probabilities, but if performing repeated updates, it's [1zg more convenient] to multiply by all the likelihood ratios under consideration before normalizing at the end.)

$$\dfrac{\mathbb{P}(H_i\mid e_0)}{\mathbb{P}(H_j\mid e_0)} = \dfrac{\mathbb{P}(e_0\mid H_i)}{\mathbb{P}(e_0\mid H_j)} \cdot \dfrac{\mathbb{P}(H_i)}{\mathbb{P}(H_j)}$$

As a more striking illustration, suppose we receive emails on three subjects:  Business (60%), personal (30%), and spam (10%).  Suppose that business, personal, and spam emails are 60%, 10%, and 90% likely respectively to contain the word "money"; and that they are respectively 20%, 80%, and 10% likely to contain the word "probability".  Assume for the sake of discussion that a business email containing the word "money" [naive_bayes is thereby no more or less likely] to contain the word "probability", and similarly with personal and spam emails.  Then if we see an email containing both the words "money" and "probability":

$$(6 : 3 : 1) \times (6 : 1 : 9) \times (2 : 8 : 1) = (72 : 24 : 9) = (24 : 8: 3)$$

...so the posterior odds are 24 : 8 : 3 favoring the email being a business email, or roughly 69% probability after [1rk normalizing].

# Log odds

The odds $\mathbb{P}(X) : \mathbb{P}(\neg X)$ can be viewed as a dimensionless scalar quantity $\frac{\mathbb{P}(X)}{\mathbb{P}(\neg X)}$ in the range $[0, +\infty]$.  If the odds of Alexander Hamilton becoming President are 0.75 to 0.25 in favor, we can also say that Andrew Jackson is 3 times as likely to become President as not.  Or if the odds were 0.4 to 0.6, we could say that Alexander Hamilton was 2/3rds as likely to become President as not.

The **log odds** are the logarithm of this dimensionless positive quantity, $\log\left(\frac{\mathbb{P}(X)}{\mathbb{P}(\neg X)}\right),$ e.g., $\log_2(1:4) = \log_2(0.25) = -2.$  Log odds fall in the [range_notation range] $[-\infty, +\infty]$ and are finite for probabilities inside the range $(0, 1).$

When using a log odds form of [1lz Bayes' rule], the posterior log odds are equal to the prior log odds plus the log likelihood.  This means that the change in log odds can be identified with [ the strength of the evidence].  If the probability goes from 1/3 to 4/5, our odds have gone from 1:2 to 4:1 and the log odds have shifted from -1 bits to +2 bits.  So we must have seen evidence with a strength of +3 bits (a likelihood ratio of 8:1).

The convenience of this representation is what Han Solo refers to in *Star Wars* when he shouts:  "Never tell me the odds!", implying that he would much prefer to be told the logarithm of the odds ratio.

## Direct representation of infinite certainty

In the log odds representation, the probabilities $0$ and $1$ are represented as $-\infty$ and $+\infty$ respectively.

This exposes the specialness of the classical probabilities $0$ and $1,$ and the ways in which these "infinite certainties" sometimes behave qualitatively differently from all finite credences.  If we don't start by being absolutely certain of a proposition, it will require infinitely strong evidence to shift our belief all the way out to infinity.  If we do start out absolutely certain of a proposition, no amount of ordinary evidence no matter how great can ever shift us away from infinity.

This reasoning is part of the justification of [Cromwell's rule](https://en.wikipedia.org/wiki/Cromwell%27s_rule) which states that probabilities of exactly $0$ or $1$ should be avoided except for logical truths and falsities (and maybe [not even then](http://lesswrong.com/lw/mo/infinite_certainty/)).  It also demonstrates how log odds are a good fit for measuring *strength of belief and evidence,* even if classical probabilities are a better representation of *degrees of caring* and betting odds.

%%%comment: We are checking to see if users will click this button, even though we don't have the content for it yet.%%%
%%hidden(Check my understanding):
Coming soon!
%%


Arbital Labs

This is a temporary group created as an experiment in both software and community design. It's meant to act as a seed for multiple future Arbital communities. Currently membership is by invite only.


Stub

Stub-Class pages are tiny, often a single paragraph, or a couple of sentences.  Stubs may have been created to provide a [-5xs] for parenting, tagging, or reference purposes in a section under construction.  An editor expanding a Stub page might start over from scratch.

Contrast:

- [3rk], which is a larger page, but still visibly incomplete or disorganized.
- [4gs], for brief pages which only contain a formal definition and don't try to explain the topic.

**Also** use the tag [4v] if the page is being actively edited.

**[4yg Quality scale]**

* [4ym]
* [4gs]
* [5xq]
* [72]
* [3rk]
* [4y7]
* [4yd]
* [4yf]
* [4yl]


Invisible background fallacies

[summary:  Considering *all* the implications often gives us a very different picture from considering *some* of the implications, and the latter is a common species of fallacy.

Suppose that you went outdoors and jumped up into the air, and then, as you were jumping, gravity suddenly switched off.  Would your feet hit the ground again?

You might initially imagine that gravity no longer pulls you down toward the Earth, and that therefore your jump would keep you continuing up into the air.  If you were thinking more broadly, you might imagine that the air on Earth's surface, previously kept pressurized by gravity, might start to rush off into space.

Actually, the ground under your feet is also currently being pulled downward by gravity.  It follows that other pressures on the ground immediately below your feet--from the air above it, and from the rock below--must net out to an upward force that would, if *not* for gravity, push the ground upward at $9.8m/s^2.$  So if you jumped up into the air, and then gravity switched off, the ground and everything previously motionless around you would accelerate upward at $9.8m/s^2,$ causing your feet to hit the ground at the expected time.  That is, if gravity switched off everywhere in the universe, you wouldn't notice anything for a couple of minutes.]

Suppose that you went outdoors and jumped up into the air, and then, as you were jumping, the law of gravity suddenly switched off--for whatever weird reason, masses everywhere in the Universe stopped attracting one another.  Would your feet hit the ground again?

As an initial thought about this problem, you might imagine that gravity no longer pulls you down toward the Earth, and that therefore your jump would keep you continuing up into the air.  If you were thinking more broadly, you might realize that the concept of 'air pressure' means that the air on Earth's surface is kept pressurized by gravity, and in the absence of gravity holding down all that air, the atmosphere might start to rush off into space as it expanded to relieve the pressure.  Although (you then reason further) the speed of sound is finite, so the atmosphere might start rushing out at the edges, and would need to disappear there before the air started to expand away from the Earth's surface.  And then that rushing air would only carry you further away from the ground.

This answer is incorrect.  Why?  Because the ground under your feet is also currently being pulled downward by gravity.

Indeed, the ground under your feet is currently in a state of [ equilibrium] despite being pulled downward by gravity at what would otherwise be an acceleration of $9.8m/s^2.$  It follows that other pressures on the ground immediately below your feet--from the air above it, and from the rock below--must net out to an upward force that would, if not for gravity, push the ground upward at $9.8m/s^2.$  This logic applies to the dirt below your feet, the air around you, and the rocks immediately underneath the dirt under your feet.  Everything around you that seemed motionless was previously in equilibrium with gravity, so as soon as gravity vanishes, it all accelerates upward in near unison.

So your feet hit the ground again, as the Earth expands beneath you.  And then the Earth continues to accelerate upward at $9.8m/s^2$, pressuring your feet and accelerating you as well.  So in fact, if gravity had switched off everywhere in the universe 10 seconds earlier as you read this, you probably would not have noticed anything different, yet.  (Though you *would* notice shortly afterward as things began to expand far enough to change the balance of forces, and as the Sun finished exploding.)

In many situations and scenarios, there are rules or laws or considerations that properly apply to *everything*--including background objects and processes and concepts we don't usually think about--and then considering the *complete* effect gives us a very different picture from thinking only about the effect on *visible* things, with the latter being a common species of fallacy.

An essay now famous in the field of [ economics] is Frederic Bastiat's "[That Which Is Seen, And That Which Is Not Seen](http://bastiat.org/en/twisatwins.html)" which illustrated this idea and introduced the now-famous 'broken window fallacy'.  Suppose you heave a rock through somebody's window; have you done a good deed or a bad deed?  You can imagine somebody arguing that they've done a good deed, despite the initial inconvenience, because the homeowner needs to pay a glazier to fix the window, and then the glazier spends money at the baker, and so the whole economy is helped.  This is a fallacy because, if the window hadn't been broken, the homeowner would have spent the money somewhere else.  But the broken window and the payment to the glazier are highly visible; while the disappearing of whatever economic activity would otherwise have occurred, is not seen.  If we consider the effect on the whole picture, including things that are ordinarily hard to see or fading into the background or invisible for some other reason, we arrive at a very different picture of the net effect of breaking somebody's window.


Ideological Turing test

The [Ideological Turing Test](https://en.wikipedia.org/wiki/Ideological_Turing_Test) is when, to try to make sure you really understand an opposing position, you explain it such that *an advocate of the position cannot tell whether the explanation was written by you or a real advocate.*  This is often run online as a formal test where an audience tries to distinguish which pro-X and anti-X essays have been written by real pro-X and anti-X advocates, and which essays were actually written by advocates for the opposing side.  If you're pro-X, and you write an intelligent essay advocating anti-X, and other people can't tell that you were really pro-X, you win the Ideological Turing Test by having demonstrated that you really and honestly understand the arguments for anti-X.


Gotcha button

A "gotcha button" is a conversational point which, when pressed, causes some substantial fraction of listeners to leap up and shout "Gotcha!  That's obviously wrong and shows you don't know what you're talking about!", after which they have [motivated_skepticism license to dismiss] the conversation.

This behavior does not *necessarily* mean the listener is wrong.  If you're describing your brilliant invention to a physics-literate person, and you mention that it can accelerate forward without pushing anything else in the opposite direction, the listener will shout "Gotcha!  You can't violate Conservation of Momentum!"  They'll be right, and their implied inference about your level of physics literacy will probably also be right.

Nonetheless, there are a *lot* of ideas that function as Gotcha Buttons for some significant fraction of various audiences, which do *not* correspond to a claimed known falsity.

- Some true/plausible claims will be seen as 'obviously false' by many listeners.
- Some phrases or ideas will be predictably misinterpreted to a claim that actually is false.

Example of the first type:  If you were explaining evolutionary biology to a novice audience, you might be wise to start with some example other than how hominids in particular evolved from primate ancestors.  This isn't wrong, but a fraction of the audience thinks it's obviously wrong and diagnostic of you as someone they shouldn't listen to.

Example of the second type:  If you were discussing [42g Artificial General Intelligence], or AI with [7mt par-human performance] in some domain, you might be wise to avoid the term "human-level AI"; because that particular phrase will cause some fraction of the audience to jump up and shout "Gotcha!  AIs won't have a humanlike balance of abilities!  There's no such thing as the human level!"  (They can then dismiss the rest of your argument, without trying to carefully consider whether your further arguments are actually refuted by accepting "AIs won't have a humanlike balance of abilities", because you've made such an obvious and severe mistake that you just lose.)


Harmless supernova fallacy

[summary:  "Harmless supernova" fallacies are arguments that [3tc can equally be used to conclude] than any real phenomenon, including a supernova, is harmless.

- **Bounded, therefore harmless:**  "A supernova isn't *infinitely* energetic--that would violate the laws of physics! Just wear a flame-retardant jumpsuit and you'll be fine."
- **Continuous, therefore harmless:**  "Temperature is continuous; there's no qualitative threshold where something becomes 'super' hot. We just need better versions of our existing heat-resistant materials."
- **Varying, therefore harmless:**  "A supernova wouldn't heat up all areas of the solar system to exactly the same temperature!  It'll be hotter closer to the center, and cooler toward the outside. We just need to stay on a part of Earth that's further way from the Sun."
- **Mundane, therefore harmless** aka **straw superpower**:  "Contrary to what many non-astronomers seem to believe, a supernova can't burn hot enough to sear through time itself.  So we'll be fine."
- **Precedented, therefore harmless:** "Really, we've already had supernovas around for a while; there already exist devices that produce tons of energy by fusion, and they're called thermonuclear weapons.  Society has proven itself able to handle those."
- **Undefinable, therefore harmless:**  "What exactly is heat?  Different people could call different temperatures 'too warm'.  When exactly does a sun shining become so 'super' that it's a supernova?"]

Harmless supernova fallacies are a class of arguments, usually a subspecies of false dichotomy or continuum fallacy, which [3tc can equally be used to argue] that almost any physically real phenomenon--including a supernova--is harmless / manageable / safe / unimportant.

- **Bounded, therefore harmless:**  "A supernova isn't infinitely energetic--that would violate the laws of physics! Just wear a flame-retardant jumpsuit and you'll be fine."  (All physical phenomena are finitely energetic; some are nonetheless energetic enough that a flame-retardant jumpsuit won't stop them.  "Infinite or harmless" is a false dichotomy.)
- **Continuous, therefore harmless:**  "Temperature is continuous; there's no qualitative threshold where something becomes 'super' hot. We just need better versions of our existing heat-resistant materials, which is a well-understood engineering problem."  (Direct instance of the standard continuum fallacy: the existence of a continuum of states between two points does not mean they are not distinct.  Some temperatures, though not qualitatively distinct from lower temperatures, exceed what we can handle using methods that work for lower temperatures.  "Quantity has a quality all of its own.")
- **Varying, therefore harmless:**  "A supernova wouldn't heat up all areas of the solar system to exactly the same temperature!  It'll be hotter closer to the center, and cooler toward the outside. We just need to stay on a part of Earth that's further way from the Sun."  (The temperatures near a supernova vary, but they are all quantitatively high enough to be far above the greatest manageable level; there is no nearby temperature low enough to form a survivable valley.)
- **Mundane, therefore harmless** or **straw superpower**: "Contrary to what many non-astronomers seem to believe, a supernova can't burn hot enough to sear through time itself, so we'll be fine."  (False dichotomy: the divine ability is not required for the phenomenon to be dangerous / non-survivable.)
- **Precedented, therefore harmless:**  "Really, we've already had supernovas around for a while: there are already devices that produce 'super' amounts of heat by fusing elements low in the periodic table, and they're called thermonuclear weapons.  Society has proven well able to regulate existing thermonuclear weapons and prevent them from being acquired by terrorists; there's no reason the same shouldn't be true of supernovas."  (Reference class tennis / noncentral fallacy / continuum fallacy: putting supernovas on a continuum with hydrogen bombs doesn't make them able to be handled by similar strategies, nor does finding a class such that it contains both supernovas and hydrogen bombs.)
- **Undefinable, therefore harmless:**  "What is 'heat', really?  Somebody from Greenland might think 295 Kelvin was 'warm', somebody from the equator might consider the same weather 'cool'.  And when exactly does a 'sun shining' become a 'supernova'?  This whole idea seems ill-defined."  (Someone finding it difficult to make exacting definitions about a physical process doesn't license the conclusion that the physical process is harmless. %%note: It also happens that the distinction between the runaway process in a supernova and a sun shining is empirically sharp; but this is not why the argument is invalid--a super-hot ordinary fire can also be harmful even if we personally are having trouble defining "super" and "hot". %%)


Error getting page

Error getting page


Rationality

The subject domain for [ epistemic] and [ instrumental] rationality.