Elementary Algebra

How do we describe relations between different things? How can we figure out new true things from true things we already know?
How can we find and think about patterns we notice with numbers? Algebra is a unified framework for thinking about these questions, and gives us lots of tools to help us answer them, which work in an extremely wide variety of situations.

#Equations
Algebra is based on the arithmetic of numbers, and the relations between them.
Let's look at a basic equation: $$2 + 2 = 4$$
The 'equals' sign (=) tells us that both sides of the equation are actually the same. If we have two, and add two more to it, then we'll have four.

Some other relations are 'less than' (<), for example $2 < 4$. or 'greater than' (>), for example $5 > 1$.

##The right order
In equations, parenthesis tell us the right order to do things - things inside of parenthesis have to be done before the rest. This is important because doing things in different orders can give us different answers!
$$2 + (3 	imes 4) = 14$$
$$(2 + 3) 	imes 4 = 20$$

It's annoying to have to use parentheses all the time (though it might be helpful if you find yourself getting confused about something). It would be nice if we could just write $2 + 3 	imes 4$ and have everyone know that we meant $2 + (3 	imes 4)$ . There's a standard [54s order of operations] that everyone uses so that we don't have to use too many parentheses.

#Balancing the truth
If we know one equation, what are some ways we could get _new_ equations from it, that will still be true?
We could make a change to one side, but then the equation would stop being true... unless we did the same change to the other side also! For example, we know $2+2=4$. What if we add three to both sides? If we check
$(2 + 2) + 3 = 4 + 3$, we can see that it's still true!

##Substitution
One way of getting new true things is really important. If we know two things are the same, we can always substitute one of them for the other, and this automatically will give us an equality relation between the two things!

This is really helpful for breaking down calculations into manageable pieces. For example, if we want to calculate $2^3 	imes 2^4$, we can first expand $2^3 = 2 	imes 2 	imes 2$, and then calculate $2 	imes 2 	imes 2 = 8$. Now, we combine these last two equations to see that $2^3 = 8$. Often, people will do both of these in one step, so if you ever are having a hard time figuring out how someone got an equation, you can try breaking it down like this to see if that helps. Next, we can do the same thing to see that $2^4 = 2 	imes 2 	imes 2 	imes 2 = 16$. We can then substitute both of these back into the original expression, to get $2^3 	imes 2^4 = 8 	imes 16$. One final calculation lets us see that $2^3	imes 2^4 = 128$.

#Naming numbers
What are all those letters doing in math, anyway?

When you first learn someone's name, do you know everything there is to know about them? Sometimes, we know that there's _some_ number that fits in an equation, but we don't know what particular number it is yet. It's really helpful to be able to talk about the number anyway, in fact, giving it a name is often the first step to figuring out what it really is.

This kind of name is called a **variable**.

## Doing lots of things at once
Another way names are useful is if we want to say something about lots of numbers at once!
For example, you might notice that $0 	imes 3 = 0$, $0 	imes -4 = 0$, $0 	imes 1224 = 0$ and so on. In fact, it's true that zero times _any_ number is zero. We could write this as $0 	imes 	ext{any number} = 0$. Or if we need to keep referring to that number, we could give it a shorter name, while mentioning it could be any number: $0 	imes x = 0$, where $x$ can be any number. This is really useful because it allows us to express patterns much more easily!

#Important patterns
These patterns hold for all [45h natural numbers], [48l integers], and [4zq rational numbers], including for variables that are known to be one of these types of numbers, and much more! For an in-depth exploration of these patterns and their consequences, see the page on [3gq rings].
##Commutativity
$$ a + b = b + a$$
$$ a 	imes b = b	imes a$$
##Identity
$$ 0 + a = a$$
$$ 1 	imes a = a$$
##Associativity
$$ (a + b) + c = a + (b + c)$$
$$ (a 	imes b ) 	imes c = a 	imes (b	imes c)$$
##Distributivity
$$ a 	imes (b + c) = a	imes b + a	imes c$$
##Additive inverse
$$ a + (-a) = a - a = 0 $$

#Next steps
* [Solving_equations]
* [3jy Functions]


The reals (constructed as classes of Cauchy sequences of rationals) form a field

The real numbers, when [50d constructed as equivalence classes of Cauchy sequences] of [4zq rationals], form a [540 totally ordered] [481 field], with the inherited field structure given by

- $[a_n] + [b_n] = [a_n+b_n]$
- $[a_n] 	imes [b_n] = [a_n 	imes b_n]$
- $[a_n] \leq [b_n]$ if and only if either $[a_n] = [b_n]$ or for sufficiently large $n$, $a_n \leq b_n$.

# Proof

Firstly, we need to show that those operations are even well-defined: that is, if we pick two different representatives $(x_n)_{n=1}^{\infty}$ and $(y_n)_{n=1}^{\infty}$ of the same equivalence class $[x_n] = [y_n]$, we don't somehow get different answers.

## Well-definedness of $+$

We wish to show that $[x_n]+[a_n] = [y_n] + [b_n]$ whenever $[x_n] = [y_n]$ and $[a_n] = [b_n]$; this is an exercise.
%%hidden(Show solution):
Since $[x_n] = [y_n]$, it must be the case that both $(x_n)$ and $(y_n)$ are Cauchy sequences such that $x_n - y_n 	o 0$ as $n 	o \infty$.
Similarly, $a_n - b_n 	o 0$ as $n 	o \infty$.

We require $[x_n+a_n] = [y_n+b_n]$; that is, we require $x_n+a_n - y_n-b_n 	o 0$ as $n 	o \infty$.

But this is true: if we fix rational $\epsilon > 0$, we can find $N_1$ such that for all $n > N_1$, we have $|x_n - y_n| < rac{\epsilon}{2}$; and we can find $N_2$ such that for all $n > N_2$, we have $|a_n - b_n| < rac{\epsilon}{2}$.
Letting $N$ be the maximum of the two $N_1, N_2$, we have that for all $n > N$, $|x_n + a_n - y_n - b_n| \leq |x_n - y_n| + |a_n - b_n|$ by the [-triangle_inequality], and hence $\leq \epsilon$.
%%

## Well-definedness of $	imes$

We wish to show that $[x_n] 	imes [a_n] = [y_n] 	imes [b_n]$ whenever $[x_n] = [y_n]$ and $[a_n] = [b_n]$; this is also an exercise.
%%hidden(Show solution):
We require $[x_n a_n] = [y_n b_n]$; that is, $x_n a_n - y_n b_n 	o 0$ as $n 	o \infty$.

Let $\epsilon > 0$ be rational.
Then $$|x_n a_n - y_n b_n| = |x_n (a_n - b_n) + b_n (x_n - y_n)|$$ using the very handy trick of adding the expression $x_n b_n - x_n b_n$.

By the triangle inequality, this is $\leq |x_n| |a_n - b_n| + |b_n| |x_n - y_n|$.

We now use the fact that [cauchy_sequences_are_bounded], to extract some $B$ such that $|x_n| < B$ and $|b_n| < B$ for all $n$;
then our expression is less than $B (|a_n - b_n| + |x_n - y_n|)$.

Finally, for $n$ sufficiently large we have $|a_n - b_n| < rac{\epsilon}{2 B}$, and similarly for $x_n$ and $y_n$, so the result follows that $|x_n a_n - y_n b_n| < \epsilon$.
%%

## Well-definedness of $\leq$

We wish to show that if $[a_n] = [c_n]$ and $[b_n] = [d_n]$, then $[a_n] \leq [b_n]$ implies $[c_n] \leq [d_n]$.

Suppose $[a_n] \leq [b_n]$, but suppose for [46z contradiction] that $[c_n]$ is not $\leq [d_n]$: that is, $[c_n]
ot = [d_n]$ and there are arbitrarily large $n$ such that $c_n > d_n$.
Then there are two cases.

- If $[a_n] = [b_n]$ then $[d_n] = [b_n] = [a_n] = [c_n]$, so the result follows immediately. %%note:We didn't need the extra assumption that $[c_n]
ot \leq [d_n]$ here.%%
- If for all sufficiently large $n$ we have $a_n \leq b_n$, then [todo: this part]

## Additive [3jb commutative] [3gd group structure] on $\mathbb{R}$

The additive identity is $[0]$ (formally, the equivalence class of the sequence $(0, 0, \dots)$).
Indeed, $[a_n] + [0] = [a_n+0] = [a_n]$.

The additive inverse of the element $[a_n]$ is $[-a_n]$, because $[a_n] + [-a_n] = [a_n-a_n] = [0]$.

The operation is commutative: $[a_n] + [b_n] = [a_n+b_n] = [b_n+a_n] = [b_n] + [a_n]$.

The operation is closed, because the sum of two Cauchy sequences is a Cauchy sequence (exercise).
%%hidden(Show solution):
If $(a_n)$ and $(b_n)$ are Cauchy sequences, then let $\epsilon > 0$.
We wish to show that there is $N$ such that for all $n, m > N$, we have $|a_n+b_n - a_m - b_m| < \epsilon$.

But $|a_n+b_n - a_m - b_m| \leq |a_n - a_m| + |b_n - b_m|$ by the triangle inequality; so picking $N$ so that $|a_n - a_m| < rac{\epsilon}{2}$ and $|b_n - b_m| < rac{\epsilon}{2}$ for all $n, m > N$, the result follows.
%%

The operation is associative: $$[a_n] + ([b_n] + [c_n]) = [a_n] + [b_n+c_n] = [a_n+b_n+c_n] = [a_n+b_n] + [c_n] = ([a_n]+[b_n])+[c_n]$$

## [3gq Ring structure]

The multiplicative identity is $[1]$ (formally, the equivalence class of the sequence $(1,1, \dots)$).
Indeed, $[a_n] 	imes [1] = [a_n 	imes 1] = [a_n]$.

$	imes$ is closed, because the product of two Cauchy sequences is a Cauchy sequence (exercise).
%%hidden(Show solution):
If $(a_n)$ and $(b_n)$ are Cauchy sequences, then let $\epsilon > 0$.
We wish to show that there is $N$ such that for all $n, m > N$, we have $|a_n b_n - a_m b_m| < \epsilon$.

But $$|a_n b_n - a_m b_m| = |a_n (b_n - b_m) + b_m (a_n - a_m)| \leq |b_m| |a_n - a_m| + |a_n| |b_n - b_m|$$ by the triangle inequality.

Cauchy sequences are bounded, so there is $B$ such that $|a_n|$ and $|b_m|$ are both less than $B$ for all $n$ and $m$.

So picking $N$ so that $|a_n - a_m| < rac{\epsilon}{2B}$ and $|b_n - b_m| < rac{\epsilon}{2B}$ for all $n, m > N$, the result follows.
%%

$	imes$ is clearly commutative: $[a_n] 	imes [b_n] = [a_n 	imes b_n] = [b_n 	imes a_n] = [b_n] 	imes [a_n]$.

$	imes$ is associative: $$[a_n] 	imes ([b_n] 	imes [c_n]) = [a_n] 	imes [b_n 	imes c_n] = [a_n 	imes b_n 	imes c_n] = [a_n 	imes b_n] 	imes [c_n] = ([a_n] 	imes [b_n]) 	imes [c_n]$$

$	imes$ distributes over $+$: we need to show that $[x_n] 	imes ([a_n]+[b_n]) = [x_n] 	imes [a_n] + [x_n] 	imes [b_n]$.
But this is true: $$[x_n] 	imes ([a_n]+[b_n]) = [x_n] 	imes [a_n+b_n] = [x_n 	imes (a_n+b_n)] = [x_n 	imes a_n + x_n 	imes b_n] = [x_n 	imes a_n] + [x_n 	imes b_n] = [x_n] 	imes [a_n] + [x_n] 	imes [b_n]$$

## Field structure

To get from a ring to a field, it is necessary and sufficient to find a multiplicative inverse for any $[a_n]$ not equal to $[0]$.

Since $[a_n]
ot = 0$, there is some $N$ such that for all $n > N$, $a_n
ot = 0$.
Then defining the sequence $b_i = 1$ for $i \leq N$, and $b_i = rac{1}{a_i}$ for $i > N$, we obtain a sequence which induces an element $[b_n]$ of $\mathbb{R}$; and it is easy to check that $[a_n] [b_n] = [1]$.
%%hidden(Show solution):
$[a_n] [b_n] = [a_n b_n]$; but the sequence $(a_n b_n)$ is $1$ for all $n > N$, and so it lies in the same equivalence class as the sequence $(1, 1, \dots)$.
%%

## Ordering on the field

We need to show that:

- if $[a_n] \leq [b_n]$, then for every $[c_n]$ we have $[a_n] + [c_n] \leq [b_n] + [c_n]$;
- if $[0] \leq [a_n]$ and $[0] \leq [b_n]$, then $[0] \leq [a_n] 	imes[b_n]$.

We may assume that the inequalities are strict, because if equality holds in the assumption then everything is obvious.
%%hidden(Show obvious bits):
If $[a_n] = [b_n]$, then for every $[c_n]$ we have $[a_n] + [c_n] = [b_n] + [c_n]$ by well-definedness of addition.
Therefore $[a_n] + [c_n] \leq [b_n] + [c_n]$.

If $[0] = [a_n]$ and $[0] \leq [b_n]$, then $[0] = [0] 	imes [b_n] = [a_n] 	imes [b_n]$, so it is certainly true that $[0] \leq [a_n] 	imes [b_n]$.
%%

For the former: suppose $[a_n] < [b_n]$, and let $[c_n]$ be an arbitrary equivalence class.
Then $[a_n] + [c_n] = [a_n+c_n]$; $[b_n] + [c_n] = [b_n+c_n]$; but we have $a_n + c_n \leq b_n + c_n$ for all sufficiently large $n$, because $a_n \leq b_n$ for sufficiently large $n$.
Therefore $[a_n] + [c_n] \leq [b_n] + [c_n]$, as required.

For the latter: suppose $[0] < [a_n]$ and $[0] < [b_n]$.
Then for sufficiently large $n$, we have both $a_n$ and $b_n$ are positive; so for sufficiently large $n$, we have $a_n b_n \geq 0$.
But that is just saying that $[0] \leq [a_n] 	imes [b_n]$, as required.


Likelihood notation

The likelihood of a piece of evidence $e$ according to a hypothesis $H,$ known as "the likelihood of $e$ given $H$", is often written either $\mathcal L_e(H)$ or $\mathcal L(H \mid e).$ The latter notation is confusing, because then $\mathcal L(H \mid e) = \mathbb P(e \mid H).$ Many students of statistics find it hard enough to keep the difference between $\mathbb P(H \mid e)$ and $\mathbb P(e \mid H)$ straight in their heads if we _don't_ occasionally swap the order of the arguments when talking about similar functions, so on Arbital, we much prefer the notation $\mathcal L_e(H) = \mathbb P(e \mid H).$

[fixme: Make this a child of 'likelihood' when 'likelihood' exists.]


Modal logic

Modal logic is a [formal_system] in which we expand [-propositional_calculus] with two new operators meant to represent necessity ($\square$) and possibility ($\diamond$).%%note:There are more [ general modal logics] with different modal predicates out there.%%

Those two operators can be defined in terms of each other. For example, we have that something is possible iff it is not necessary that its opposite is true; ie, $\diamond A \iff
eg \square
eg A$.

Thus in modal logic, you can express sentences as $\square A$ to express that it is necessary that $A$ is true. You can also go more abstract and say $
eg\square ot$ to express that it is not necessary that false is true. If we read the box operator as "there is no mathematical proof of" then the previous sentence becomes "there is no mathematical proof of falsehood", which encodes the [ consistency of arithmetic].

There are many systems of modal logic %note: See for example [t_modal_logic T],[b_modal_logic B],[s5_modal_logic S5],[s4_modal_logic S4],[k_modal_logic K], [gls_modal_logic GLS]%, which differ in the sentences they take as [-axioms] and which [ rules of inference] they allow.

Of particular interest are the systems called [5j8 normal systems of modal logic], and especially the [5l3 $Gödel-Löb$] system (called $GL$ for short, also known as the logic of provability). The main interest of $GL$ comes from being a [ decidable] logic which allows us to reason about sentences of [3ft Peano arithmetic] talking about [5gt provability] via [ translations], as proved by the [ arithmetical adequacy theorems of Solomonov].

Another widely studied system of modal logic is $GLS$, for which Solomonov proved [ adequacy for truth] in the [-standard_model_of_arithmetic].

The [semantics_mathematical semantics] of the normal systems of modal logic come in the form of [Kripke_models]: digraph structures composed of worlds over which a visibility relation is defined.

##Sentences
Well formed sentences of modal logic are defined as follows:

* $ot$ is a well-formed sentence of modal logic.
* The sentence letters $p,q,...$ are well-formed sentences of modal logic.
* If $A$ is a well-formed sentence, then $\square A$ is a well-formed sentence.
* If $A$ and $B$ are well formed sentences, then $A	o B$ is a well-formed sentence.

The rest of [ logical connectors] can then be defined as abbreviations for structures made from implication and $ot$. For example , $
eg A = A	o ot$. The possibility operator $\diamond$ is defined as $
eg\square
eg$ as per the previous discussion.


Integers: Intro (Math 0)

[summary: The integers are an extension of the [45h natural numbers] into the *negatives*.

In this article we'll be exploring a novel way to think of negative numbers, using collections of cows and anti-cows. *Mooooo!*]

The integers are an extension of the [45h natural numbers] into the *negatives*.

## Negative numbers

Negative numbers are numbers less than zero. They are notated as numbers with a minus sign in front of them, like $-2$ or $-15$ or $-6387$.

When many people are first introduced to the concept of negative numbers, one question they have is, "how do negative numbers exist? How can we have less than nothing of something?"

Usually this question is answered with something about travelling along the number line and then going in the opposite direction past zero. But we can also answer this question from the traditional perspective of quantities, using _antimatter_.

## Arithmetic with anti-cows

Matter and antimatter are substances that annihilate each other — they cancel each other out. In the same way, positive and negative quantities cancel each other out to some extent when added together.

[todo: cow artwork]

Let's create a physical example of this antimatter phenomenon, and consider a collection of cows. [comment: diagram of cows] If a natural number is how many cows we have, then negative numbers are the presence of *anti-cows*. [comment: For the sake of easy reading, let's colour a cow red, and an anti-cow blue: diagram of red cow = cow, blue cow = anticow]  For example, if you have $3$ anti-cows, you have $-3$ cows. If you have $128$ anti-cows, you have $-128$ cows, and so on.

When a cow and an anti-cow come into contact, they annihilate each other and nothing is left (except technically an enormous burst of energy that could cause a magnitude 9 earthquake, but we won't get into the nitty-gritty physics of antimatter annihilation here). [comment: diagram of cow/anti-cow annihilation] Moreover, each cow will annihilate exactly one anti-cow, and vice versa.

This lets us perform subtraction by adding anti-cows instead of taking away cows. If you want to perform $6 - 4$, you imagine you have a collection of six cows, and you add four anti-cows to the mix, which annihilate four of the cows leaving you with two. This is exactly the same as if you'd taken away four cows (except that you've now released enough energy to power the entire world's energy demands for seven months). [comment: diagram of 6 - 4]

This form of adding negatives is slightly more flexible than regular subtraction. In regular subtraction, you couldn't take away more cows than there already were. But when you add negatives, you can keep adding anti-cows after there are no cows left, at which point you just have more and more anti-cows, giving you a more and more negative number. So if you wanted to subtract $4 - 6$, you couldn't take away six cows from four, but you could add six anti-cows, and four of them would annihilate the four cows, leaving you with two anti-cows or a final answer of $-2$. [comment: diagram of 4 - 6]

## Useful properties of negative numbers

Extending the natural numbers into negatives gives us lots of useful properties when doing arithmetic.

### Addition of negative numbers

Negative numbers add up just like positive ones. If you have $-3$ and $-7$, they add up to $-10$, just like three anti-cows and seven anti-cows would come together to make ten anti-cows. [comment: diagram of (-3) + (-7)]

Knowing this, how would you calculate $(-6) + (-8) + (-12) + (-20)$?

%%hidden(Show solution):
Simply add up all the negative quantities together. $$6 + 8 + 12 + 20 = 46 	o (-6) + (-8) + (-12) + (-20) = -46$$
%%

### Subtraction as additive inversion

If we represent a natural number as a number of cows, then the same number of anti-cows is called the *additive inverse* of that number, because a number and its additive inverse add up to zero (equal numbers of cows and anti-cows will all annihilate each other to give nothing), which is the "additive [54p identity]" (or "the number that changes nothing when you add it to something").

Using additive inverses allows us to express subtraction in the form of addition — for example, $5 - 2$ can be rewritten as $5 + (-2)$ (which just means that instead of taking away two cows, you're adding two anti-cows). When we do this, suddenly we can rearrange subtraction operations along with addition operations even though subtraction isn't [3jb commutative].

Then, if you have a giant addition and subtraction problem you want to solve, you can add up all the numbers you want to add (into a big collection of cows), then add up all the numbers you want to subtract (into a big collection of anti-cows), and just do one subtraction (annihilating cow/anti-cow pairs) to get your answer. For example, $6 - 2 + 7 - 5$ is really just $6 + (-2) + 7 + (-5)$ which is equal to $6 + 7 + (-5) + (-2)$. Then, you add up the number of cows (13) and anti-cows (7), and annihilate them to get the final answer of $6$.

Here's another exercise for you: How would you calculate $13 + 8 - 5 + 6 + 4 - 12 - 9 + 1$?

%%hidden(Show solution):
First, rewrite each subtraction operation as addition of a negative number: $$13 + 8 + (-5) + 6 + 4 + (-12) + (-9) + 1$$

Then rearrange the equation to group all the positive and negative numbers (cows and anti-cows) together: $$13 + 8 + 6 + 4 + 1 + (-5) + (-12) + (-9)$$

Then add up all the positive and negative numbers separately: $$(13 + 8 + 6 + 4 + 1) + ((-5) + (-12) + (-9)) = 32 + (-26)$$

And finally, subtract it all at once: $$32 + (-26) = 32 - 26 = 6$$

And voilà, you have six cows.
%%

What about $8 - 6 + 4 - 13 + 7 - 5 - 9 + 12$?

%%hidden(Show solution):
Same as before, we rewrite, regroup, add up, and subtract:

$$8 + (-6) + 4 + (- 13) + 7 + (- 5) + (- 9) + 12 \ \Downarrow$$

$$8 + 4 + 7 + 12 + (-6) + (-13) + (-5) + (-9) \ \Downarrow$$

$$(8 + 4 + 7 + 12) + ((-6) + (-13) + (-5) + (-9)) = 31 + (-33)$$

$$31 + (-33) = 31 - 33$$

But hold on — now you have more anti-cows than cows, so you can't subtract directly! Not to worry — simply flip the subtraction problem around, and subtract the cows from the anti-cows, knowing that the final result will be negative instead.

$$31 - 33 = -(33 - 31) = -2$$
%%

## The set of integers

Back to defining the integers. The set of integers is simply the set of natural numbers and their additive inverses — i.e. the set of numbers of cows you can have if you can have anti-cows as well. This set branches out to infinity on both sides, and we usually write it as $\{ \ldots, -2, -1, 0, 1, 2, \ldots \}$.

[todo: cow multiplication with herds and anti-herds]

[todo: check readability scores to make sure it's in grade 7-8 range]


The reals (constructed as Dedekind cuts) form a field

The real numbers, when [50g constructed as Dedekind cuts] over the [4zq rationals], form a [481 field].

We shall often write the one-sided [dedekind_cut Dedekind cut] $(A, B)$ %%note:Recall: "one-sided" means that $A$ has no greatest element.%% as simply $\mathbf{A}$ (using bold face for Dedekind cuts); we can do this because if we already know $A$ then $B$ is completely determined.
This will make our notation less messy.

The field structure, together with the [total_order total ordering] on it, is as follows (where we write $\mathbf{0}$ for the Dedekind cut $(\{ r \in \mathbb{Q} \mid r < 0\}, \{ r \in \mathbb{Q}  \mid r \geq 0 \})$):

- $(A, B) + (C, D) = (A+C, B+D)$
- $\mathbf{A} \leq \mathbf{C}$ if and only if everything in $A$ lies in $C$.
- Multiplication is somewhat complicated.
 - If $\mathbf{0} \leq \mathbf{A}$, then $\mathbf{A} 	imes \mathbf{C} = \{ a c \mid a \in A, a > 0, c \in C \}$. [todo: we've missed out the complement in this notation, and can't put set-builder sets in boldface]
 - If $\mathbf{A} < \mathbf{0}$ and $\mathbf{0} \leq \mathbf{C}$, then $\mathbf{A} 	imes \mathbf{C} = \{ a c \mid a \in A, c \in C, c > 0 \}$.
 - If $\mathbf{A} < \mathbf{0}$ and $\mathbf{C} < \mathbf{0}$, then $\mathbf{A} 	imes \mathbf{C} = \{\} $ [todo: write down the form of the set]

where $(A, B)$ is a one-sided [dedekind_cut Dedekind cut] (so that $A$ has no greatest element).

(Here, the "set sum" $A+C$ is defined as "everything that can be made by adding one thing from $A$ to one thing from $C$": namely, $\{ a+c \mid a \in A, c \in C \}$ in [-3lj]; and $A 	imes C$ is similarly $\{ a 	imes c \mid a \in A, c \in C \}$.)

# Proof

## Well-definedness

We need to show firstly that these operations do in fact produce [dedekind_cut Dedekind cuts].

### Addition
Firstly, we need everything in $A+C$ to be less than everything in $B+D$.
This is true: if $a+c \in A+C$, and $b+d \in B+D$, then since $a < b$ and $c < d$, we have $a+c < b+d$.

Next, we need $A+C$ and $B+D$ together to contain all the rationals.
This is true: [todo: this, it's quite boring]

Finally, we need $(A+C, B+D)$ to be one-sided: that is, $A+C$ needs to have no top element, or equivalently, if $a+c \in A+C$ then we can find a bigger $a' + c'$ in $A+C$.
This is also true: if $a+c$ is an element of $A+C$, then we can find an element $a'$ of $A$ which is bigger than $a$, and an element $c'$ of $C$ which is bigger than $C$ (since both $A$ and $C$ have no top elements, because the respective Dedekind cuts are one-sided); then $a' + c'$ is in $A+C$ and is bigger than $a+c$.

### Multiplication
[todo: this section]

### Ordering
[todo: this section]

## Additive [3jb commutative] [3gd group structure]

[todo: identity, associativity, inverse, commutativity]

## [3gq Ring structure]

[todo: multiplicative identity, associativity, distributivity]

## [481 Field structure]

[todo: inverses]

## Ordering on the field

[todo: a <= b implies a+c <= b+c, and 0 <= a, 0 <= b implies 0 <= ab]


Equivalence relation

[summary: An **equivalence relation** is a binary [-3nt] that is [reflexive_relation reflexive], [symmetric_relation symmetric], and [573 transitive]. You can think of it as a way to say when two elements of a [-3jz] are "the same" or "equivalent," despite not being literally the same element. It can be used to [set_partition partition] a set into equivalence classes.]

An **equivalence relation** is a binary [-3nt] $\sim$ on a [-3jz] $S$ that can be used to say whether [element_of_a_set elements] of $S$ are equivalent.

An equivalence relation satisfies the following properties:

1. For any $x \in S$, $x \sim x$ (the [reflexive_relation reflexive] property).
2. For any $x,y \in S$, if $x \sim y$ then $y \sim x$ (the [symmetric_relation symmetric] property).
3. For any $x,y,z \in S$, if $x \sim y$ and $y \sim z$ then $x \sim z$ (the [573 transitive] property).

Intuitively, any element is equivalent to itself, equivalence isn't directional, and if two elements are equivalent to the same third element then they're equivalent.

## Equivalence classes

Whenever we have a set $S$ with an equivalence relation $\sim$, we can divide $S$ into *equivalence classes*, i.e. sets of mutually equivalent elements. The equivalence class of some $x \in S$ is the set of elements of $S$ equivalent to $x$, written $[x]=\{y \in S \mid x \sim y\}$, and we say that $x$ is a "representative" of $[x]$. We call the set of equivalence classes $S/\sim = \{[x] \mid x \in S\}$.

From the definition of an equivalence relation, it's easy to show that $x \in [x]$ and $[x]=[y]$ if and only if $x \sim y$.

If you have a set already [set_partition partitioned] into subsets ($S$ is the [-disjoint_union] of the elements of a collection $A$), you can go the other way and define a relation with two elements related whenever they're in the same subset ($x \sim y$ when there is some $U \in A$ with $x,y \in U$). Then this is an equivalence relation, and the partition of the set is the set of equivalence classes under this relation (that is, $[x] \in A$ and $A=S/\sim$).

##Defining functions on equivalence classes

Suppose you have a [-3jy] $f: S 	o U$ and want to define a corresponding function $f^*: S/\sim 	o U$, where $U$ can be any set. You define $f^*([x])$ to be $f(x)$. This could be a problem; what if $x \sim y$ but $f(x)
eq f(y)$? Then $f^*([x])=f^*([y])$ wouldn't be [-well_defined well-defined]. Whenever you define a function on equivalence classes in terms of representatives, you have to make sure the definition doesn't depend on which representative you happen to pick. In fact, one way you might arrive at an equivalence relation is to say that $x \sim y$ whenever $f(x)=f(y)$.

If you have a function $f: S 	o S$ and want to define $f^*: S/\sim 	o S/\sim$ by $f^*([x])=[f(x)]$, you have to verify that whenever $x \sim y$, $[f(x)]=[f(y)]$, equivalently $f(x) \sim f(y)$.

#Examples

Consider the [48l integers] with the relation $x \sim y$ if $n|x-y$, for some $n \in \mathbb N$. This is the integers [modular_arithmetic mod] $n$. The [-addition] and [-multiplication] operations can be inherited from the integers, so it makes sense to talk about addition and multiplication mod $n$.

The [4bc real numbers] can be defined as equivalence classes of [50d Cauchy sequences].

[4f4] is an equivalence relation (unless the objects form a [-proper_class]).


Totally ordered set

[summary:
A totally ordered set is a pair $(S, \le)$ of a set $S$ and a *total order* $\le$ on $S$, which is a [3nt binary relation] that satisfies the following properties:

1. For all $a, b \in S$, if $a \le b$ and $b \le a$, then $a = b$. (the [antisymmetric_relation antisymmetric] property)
2. For all $a, b, c \in S$, if $a \le b$ and $b \le c$, then $a \le c$. (the [573 transitive] property)
3. For all $a, b \in S$, either $a \le b$ or $b \le a$, or both. (the [total_relation totality] property)]

A **totally ordered set** is a pair $(S, \le)$ of a set $S$ and a *total order* $\le$ on $S$, which is a [-binary_relation] that satisfies the following properties:

1. For all $a, b \in S$, if $a \le b$ and $b \le a$, then $a = b$. (the [antisymmetric_relation antisymmetric] property)
2. For all $a, b, c \in S$, if $a \le b$ and $b \le c$, then $a \le c$. (the [573 transitive] property)
3. For all $a, b \in S$, either $a \le b$ or $b \le a$, or both. (the [total_relation totality] property)

A totally ordered set is a special type of [3rb partially ordered set] that satisfies the total property — in general, posets only satisfy the [5dy reflexive] property, which is that $a \le a$ for all $a \in S$.

## Examples of totally ordered sets

The [4bc real numbers] are a totally ordered set. So are any of the subsets of the real numbers, such as the [4zq rational numbers] or the [48l integers].

## Examples of not totally ordered sets

The [4zw complex numbers] do not have a canonical total ordering, and especially not a total ordering that preserves all the properties of the ordering of the real numbers, although one can define a total ordering on them quite easily.


Intro to Number Sets

There are several common [3jz sets] of [54y numbers] that mathematicians use in their studies. In order from simple to complex, they are:

1. The [506 natural numbers] $\mathbb{N}$

2. The [53r integers] $\mathbb{Z}$

3. The [rational_number_math0 rational numbers] $\mathbb{Q}$

4. The [real_number_math0 real numbers] $\mathbb{R}$

5. The [complex_number_math0 complex numbers] $\mathbb{C}$

Each set is constructed in some way from the previous one, and this path will show you how they are constructed from the most basic numbers. You may have come across these terms in a math class that you attended, and may have had other definitions given to you. In this path, you will obtain a firm, complete understanding of these sets, how they are constructed, and what they mean in mathematics.

## Why are number sets important?

Before we go any further though, it would be nice to know the motivation behind defining the number sets first.

A [3jz set] is a fancy name for a collection of objects. Some collections of objects have special properties — such as the set of all blue things, which are special in that they're all blue. In math, if a set of objects all have a certain property, we can make **inferences** about them — that is, there are certain things we can say about them that you can deduce logically. For example:

> In a [481 field], every nonzero number has a multiplicative inverse.

You don't need to know what a field is yet (it's a special type of set), but now you can make inferences about them without restricting yourself to a specific example when talking about them. For example, you know that if a set _is_ a field, then every number in that set that isn't zero can divide into another number in that set (by multiplying by its "multiplicative inverse") and produce yet another number in that set.

Conversely, you can also tell when a set is or isn't a field based on whether it satisfies the properties a field has. For example, since you can't divide $3$ by $2$ (because the result is $1.5$ which is not a natural number), you now know that the natural numbers are not a field.

Now let's turn to our first set: the natural numbers.


The square root of 2 is irrational

$\sqrt 2$, the unique [-positive] [-4bc] whose square is 2, is not a [-4zq].

#Proof

Suppose $\sqrt 2$ is rational. Then $\sqrt 2=rac{a}{b}$ for some integers $a$ and $b$; [-without_loss_of_generality] let $rac{a}{b}$ be in [-lowest_terms], i.e. $\gcd(a,b)=1$. We have

$$\sqrt 2=rac{a}{b}$$

From the definition of $\sqrt 2$,

$$2=rac{a^2}{b^2}$$
$$2b^2=a^2$$

So $a^2$ is a multiple of $2$. Since $2$ is [4mf prime], $a$ must be a multiple of 2; let $a=2k$. Then

$$2b^2=(2k)^2=4k^2$$
$$b^2=2k^2$$

So $b^2$ is a multiple of $2$, and so is $b$. But then $2|\gcd(a,b)$, which contradicts the assumption that $rac{a}{b}$ is in lowest terms! So there isn't any way to express $\sqrt 2$ as a fraction in lowest terms, and thus there isn't a way to express $\sqrt 2$ as a ratio of integers at all. That is, $\sqrt 2$ is irrational.


Order relation

An **order relation** (also called an **order** or **ordering**) is a [3nt binary relation] $\le$ on a [3jz set] $S$ that can be used to order the elements in that set.

An order relation satisfies the following properties:

1. For all $a \in S$, $a \le a$. (the [reflexive_relation reflexive] property)
2. For all $a, b \in S$, if $a \le b$ and $b \le a$, then $a = b$. (the [antisymmetric_relation antisymmetric] property)
3. For all $a, b, c \in S$, if $a \le b$ and $b \le c$, then $a \le c$. (the [transitive_relation transitive] property)

A set that has an order relation is called a [3rb partially ordered set] (or "poset"), and $\le$ is its *partial order*.

## Totality of an order

There is also a fourth property that distinguishes between two different types of orders:

4. For all $a, b \in S$, either $a \le b$ or $b \le a$ or both. (the [total_relation total] property)

The total property implies the reflexive property, by setting $a = b$.

If the order relation satisfies the total property, then $S$ is called a [-540], and $\le$ is its *total order*.

## Well-ordering

A fifth property that extends the idea of a "total order" is that of the [55r well-ordering]:

5. For every subset $X$ of $S$, $X$ has a least element: an element $x$ such that for all $y \in X$, we have $x \leq y$.

Well-orderings are very useful: they are the orderings we can perform [mathematical_induction induction] over. (For more on this viewpoint, see the page on [structural_induction].)

# Derived relations

The order relation immediately affords several other relations.

## Reverse order

We can define a *reverse order* $\ge$ as follows: $a \ge b$ when $b \le a$.

## Strict order

From any poset $(S, \le)$, we can derive a *strict order* $<$, which disallows equality. For $a, b \in S$, $a < b$ when $a \le b$ and $a
eq b$. This strict order is still antisymmetric and transitive, but it is no longer reflexive.

We can then also define a reverse strict order $>$ as follows: $a > b$ when $b \le a$ and $a
eq b$.

## Incomparability

In a poset that is not totally ordered, there exist elements $a$ and $b$ where the order relation is undefined. If neither $a \leq b$ nor $b \leq a$ then we say that $a$ and $b$ are *incomparable*, and write $a \parallel b$.

## Cover relation

From any poset $(S, \leq)$, we can derive an underlying *cover relation* $\prec$, defined such that for $a, b \in S$, $a \prec b$ whenever the following two conditions are satisfied:

1. $a < b$.
2. For all $s \in S$, $a \leq s < b$ implies that $a = s$.

Simply put, $a \prec b$ means that $b$ is the smallest element of $S$ which is strictly greater than $a$.
$a \prec b$ is pronounced "$a$ is covered by $b$", or "$b$ covers $a$", and $b$ is said to be a *cover* of $a$.


Identity element

[summary: An identity [5xy element] in a set $S$ with a [-3kb binary operation] $*$ is an element $i$ that leaves any element $a \in S$ unchanged when combined with it in that operation.]

An identity [5xy element] in a set $S$ with a [-3kb binary operation] $*$ is an element $i$ that leaves any element $a \in S$ unchanged when combined with it in that operation.

Formally, we can define an element $i$ to be an identity element if the following two statements are true:

1. For all $a \in S$, $i * a = a$. If only this statement is true then $i$ is said to be a *left identity*.
2. For all $a \in S$, $a * i = a$. If only this statement is true then $i$ is said to be a *right identity*.

The existence of an identity element is a property of many algebraic structures, such as [3gd groups], [3gq rings], and [481 fields].


Proof that there are infinitely many primes

The proof that there are infinitely many [4mf prime numbers] is a [-46z].

First, we note that there is indeed a prime: namely $2$.
We will also state a lemma: that every number greater than $1$ has a prime which divides it.
(This is the easier half of the [5rh], and the slightly stronger statement that "every number may be written as a product of primes" is proved there.)

Now we can proceed to the meat of the proof.
Suppose that there were finitely many prime numbers $p_1, p_2, \ldots, p_n$.
Since we know $2$ is prime, we know $2$ appears in that list.

Then consider the number $P = p_1p_2\ldots p_n + 1$ — that is, the product of all primes plus 1.
Since $2$ appeared in the list, we know $P \geq 2+1 = 3$, and in particular $P > 1$.

The number $P$ can't be divided by any of the primes in our list, because it's 1 more than a multiple of them.
But there is a prime which divides $P$, because $P>1$; we stated this as a lemma.
This is immediately a contradiction: $P$ cannot be divided by any prime, even though all integers greater than $1$ can be divided by a prime.

# Example

There is a common misconception that $p_1 p_2 \dots p_n+1$ must be prime if $p_1, \dots, p_n$ are all primes.
This isn't actually the case: if we let $p_1, \dots, p_6$ be the first six primes $2,3,5,7,11,13$, then you can check by hand that $p_1 \dots p_6 + 1 = 30031$; but $30031 = 59 	imes 509$.
(These are all somewhat ad-hoc; there is no particular reason I knew that taking the first six primes would give me a composite number at the end.)
However, we *have* discovered a new prime this way (in fact, two new primes!): namely $59$ and $509$.

In general, this is a terrible way to discover new primes.
The proof tells us that there must be some new prime dividing $30031$, without telling us anything about what those primes are, or even if there is more than one of them (that is, it doesn't tell us whether $30031$ is prime or composite).
Without knowing in advance that $30031$ is equal to $59 	imes 509$, it is in general very difficult to *discover* those two prime factors.
In fact, it's an open problem whether or not prime factorisation is "easy" in the specific technical sense of there being a polynomial-time algorithm to do it, though most people believe that prime factorisation is "hard" in this sense.


Order of operations

The **order of operations** is a notational convention employed in arithmetical expressions to disambiguate the order in which operations should be performed.

It is also referred to PEMDAS, BEDMAS, BODMAS, or other such acronyms depending on where you live. The acronym is always 6 letters, and refers to the following things in order:

1. [parentheses] (or Brackets), in order of inside-most parentheses first
2. [exponent Exponents] (or Orders or Indices), in order of inside-most (or rightmost) exponents first
3. [multiplication] and [division], in order from left to right
4. [addition] and [subtraction], in order from left to right

## Motivation

The infix notation we use to write expressions of mathematical operators is inherently ambiguous. Only in some very special examples where the expression consists of only a single [3h4 associative] operation do all possible orders of evaluation evaluate to the same thing. When considering the expression $2 - 4 + 3$, do we do the $2 - 4$ first or the $4 + 3$ first? The result is either $1$ or $-5$ depending on which you choose. When considering the expression $7 + 8 	imes 9 - 6$, which operation do we do first? The result could be anywhere from $31$ to $129$ depending on the order you do them in.

Therefore, to relieve mathematicians of placing brackets around every expression, there are some standard, intuitive conventions put in place.

## Example expressions

Consider the expression $3 + 7 	imes 2^{(6 + 8)}$. We do parentheses first: $$3 + 7 	imes 2^{14}$$ Then exponents: $$3 + 7 	imes 16384$$ Then multiplication and division: $$3 + 114688$$ And finally addition and subtraction: $$114691$$

## Notable ambiguous cases

The expression $48 \div 2 (9 + 3)$ is a controversial ambiguity in the order of operations that results from an ambiguity in multiplication by juxtaposition — which is, the convention that numbers next to each other in brackets should be multiplied together. For example, $2(3+5)$ is equal to $2 	imes (3 + 5)$ or $16$.

If multiplication by juxtaposition is taken as a _parenthetical_ operation due to the parentheses in the operands, then the result is 2:

$$egin{align*}
48 \div 2 (9 + 3) &= 48 \div 2 (12) \
&= 48 \div 24 \
&= 2
\end{align*}$$

However, if multiplication by juxtaposition is taken as simply a regular _multiplication_ operation, the result is 288:

$$egin{align*}
48 \div 2 (9 + 3) &= 48 \div 2 	imes 12 \
&= 24 	imes 12 \
&= 288
\end{align*}$$

The second step is different because the 48 and 2 are operated on first, as the left most division or multiplication operation in the expression.


Whole number

[summary: An intuitive concept of a number that is not fractional. Depending on the author, the set of whole numbers is defined as either the [45h natural numbers], the natural numbers and [-zero], or the [48l integers].]

A **whole number** is an intuitive concept of a number that is not [ fractional]. There are three different definitions of the whole numbers, whose usage depends heavily on the author:

1. The [45h natural numbers].

2. The natural numbers and [-zero] (in definitions of the [-number_sets] where the natural numbers do not contain zero).

3. The [48l integers].

The set of whole numbers, when it is separate from the natural numbers and the integers, is usually notated as either $\mathbb{N}^0$ or $\mathbb{W}$.

This page is a disambiguation page for three different definitions.


Blue oysters

You're collecting exotic oysters in Nantucket, and there are two different bays you could harvest oysters from. In both bays, 11% of the oysters contain valuable pearls and 89% are empty. In the first bay, 4% of the pearl-containing oysters are blue, and 8% of the non-pearl-containing oysters are blue. In the second bay, 13% of the pearl-containing oysters are blue, and 26% of the non-pearl-containing oysters are blue. You created a special device that helps you find blue oysters. Would you rather harvest blue oysters from the first bay or the second bay?

You're encouraged to try to solve this problem yourself, and to refrain from looking at the answer. The answer can be found in [1zh].


Example problem

Tag for pages that provide an example problem referenced by a number of other pages.

The summary of the page should contain the entire text of the problem (so that the problem can be referenced by greenlink). The body of the page should either contain the answer (e.g., hidden behind an 'Answer' button), or should contain a link to the page that contains the answer. The body of the page should not explain the solution to the problem, but may link to the page that does.


Number

A **number** is an object used to represent quantity or value.

Examples of numbers include the [45h natural numbers], the [48l integers], the [4zq rational numbers], and the [4bc real numbers].

[todo: fill out with more examples]


Irrational number

An irrational number is a [-4bc] that is not a [-4zq]. This set is generally denoted using either $\mathbb{I}$ or $\overline{\mathbb{Q}}$, the latter of which represents it as the [set_complement complement] of the rationals within the reals.

In the [50d Cauchy sequence definition] of real numbers, the irrational numbers are the equivalence classes of Cauchy sequences of rational numbers that do not converge in the rationals. In the [50g Dedekind cut definition], the irrational numbers are the one-sided Dedekind cuts where the set $\mathbb{Q}^\ge$ does not have a least element.

## Properties of irrational numbers

Irrational numbers have decimal expansions (and indeed, representations in any base $b$) that do not repeat or terminate.

The set of irrational numbers is [2w0 uncountable].


Shift towards the hypothesis of least surprise

The [1zh log-odds form of Bayes' rule] says that strength of belief and strength of evidence can both be measured in [3y2 bits]. These evidence-bits can also be used to measure a quantity called "Bayesian surprise", which yields [fixme: One final, if this is the last thing in the path]yet another intuition for understanding Bayes' rule.

Roughly speaking, we can measure how surprised a hypothesis $H_i$ was by the evidence $e$ by measuring how much probability it put on $e.$ If $H_i$ put 100% of its probability mass on $e$, then $e$ is completely unsurprising (to $H_i$). If $H_i$ put 0% of its probability mass on $e$, then $e$ is _as surprising as possible._ Any measure of $\mathbb P(e \mid H_i),$ the probability $H_i$ assigned to $e$, that obeys this property, is worthy of the label "surprise." Bayesian surprise is $-\!\log(\mathbb P(e \mid H_i)),$ which is a quantity that obeys these intuitive constraints and has some other interesting features.

Consider again the [-54v] problem. Consider the hypotheses $H$ and $\lnot H$, which say "the oyster will contain a pearl" and "no it won't", respectively. To keep the numbers easy, let's say we draw an oyster from a third bay, where $rac{1}{8}$ of pearl-carrying oysters are blue and $rac{1}{4}$ of empty oysters are blue.

Imagine what happens when the oyster is blue. $H$ predicted blueness with $rac{1}{8}$ of its probability mass, while $\lnot H$ predicted blueness with $rac{1}{4}$ of its probability mass. Thus, $\lnot H$ did better than $H,$ and goes up in probability. Previously, we've been combining both $\mathbb P(e \mid H)$ and $\mathbb P(e \mid \lnot H)$ into unified likelihood ratios, like $\left(rac{1}{8} : rac{1}{4}
ight)$ $=$ $(1 : 2),$ which says that the 'blue' observation carries 1 bit of evidence $H.$ However, we can also take the logs first, and combine second.

Because $H$ assigned only an eighth of its probability mass to the 'blue' observation, and because [1y6 Bayesian update works by eliminating incorrect probability mass], we have to adjust our belief in $H$ by $\log_2\left(rac{1}{8}
ight) = -3$ bits away from $H.$ (Each negative bit means "throw away half of $H$'s probability mass," and we have to do that 3 times in order to remove the probability that $H$ failed to assign to $e$.)

Similarly, because $\lnot H$ assigned only a quarter of its probability mass to the 'blue' observation, we have to adjust our belief in $H$ by $\log_2\left(rac{1}{4}
ight) = -2$ bits away from $\lnot H.$

Thus, when the 'blue' observation comes in, we move our belief (measured in bits) 3 notches away from $H$ and then two notches back towards $H.$ On net, our belief shifts 1 notch away from $H$.

![hypotheses emitting surprise](https://i.imgur.com/ZXGB8x0.png)

_$H$ assigned 1/8th of its probability mass to blueness, so it emits $-\!\log_2\left(rac{1}{8}
ight)=3$ bits of surprise pushing away from $H$. $\lnot H$ assigned 1/4th of its probability mass to blueness, so it emits $-\!\log_2\left(rac{1}{4}
ight)=2$ bits of surprise pushing away from $\lnot H$ (and towards $H$). Thus, belief in $H$ moves 1 bit towards $\lnot H$, on net._

If instead $H$ predicted blue with probability 4% (penalty $\log_2(0.04) pprox -4.64$) and $\lnot H$ predicted blue with probability 8% (penalty $\log_2(0.08) pprox -3.64$), then we would have shifted a bit over 4.6 notches towards $\lnot H$ and a bit over 3.6 notches back towards $H,$ but we would have shifted the same number of notches _on net._ This is why it's only the _relative_ difference between the number of bits docked from $H$ and the number of bits docked from $\lnot H$ that matters.

In general, given an observation $e$ and a hypothesis $H,$ the number of bits we need to dock from our belief in $H$ is $\log_2(\mathbb P(e \mid H)),$ that is, the log of the probability that $H$ assigned to $e.$ This quantity is never positive, because the logarithm of $x$ for $0 \le x \le 1$ is in the range $[-\infty, 0]$. If we negate it, we get a non-negative quantity that relates $H$ to $e$, which is 0 when $H$ was certain that $e$ was going to happen, and which is infinite when $H$ was certain that $e$ wasn't going to happen, and which is measured in the same units as evidence and belief. Thus, this quantity is often called "surprise," and intuitively, it measures how surprised the hypothesis $H$ was by $e$ (in bits).

There is some correlation between Bayesian surprise and the times when a human would feel surprised (at seeing something that they thought was unlikely), but, of course, the human emotion is quite different. (A human can feel surprised for other reasons than "my hypotheses failed to predict the data," and humans are also great at ignoring evidence instead of feeling surprised.)

Given this definition of Bayesian surprise, we can view Bayes' rule as saying that _surprise repels belief._ When you make an observation $e,$ each hypothesis emits repulsive "surprise" signals, which shift your hypothesis. Referring again to the image above, when $H$ predicts the observation you made with $rac{1}{8}$ of its probability mass, and $\lnot H$ predicts it with $rac{1}{4}$ of its probability mass, we can imagine $H$ emitting a surprise signal with a strength of 3 bits away from $H$ and $\lnot H$ emitting a surprise signal with a strength of 2 bits away from $\lnot H$. Both those signals push the belief in $H$ in different directions, and it ends up 1 bit closer to $\lnot H$ (which emitted the weaker surprise signal).

In other words, whenever you find yourself feeling surprised by something you saw, think of the _least surprising explanation_ for that evidence — and then award that hypothesis a few bits of belief.


Bayes' rule: Definition

Bayes' rule is the mathematics of [1rf probability theory] governing how to update your beliefs in the light of new evidence.

[toc:]

## [1y9 Notation]

In much of what follows, we'll use the following [1y9 notation]:

- Let the hypotheses being considered be $H_1$ and $H_2$.
- Let the evidence observed be $e_0.$
- Let $\mathbb P(H_i)$ denote the [1rm prior probability] of $H_i$ before observing the evidence.
- Let the [1rj conditional probability] $\mathbb P(e_0\mid H_i)$ denote the [1rq likelihood] of observing evidence $e_0$ assuming $H_i$ to be true.
- Let the [1rj conditional probability] $\mathbb P(H_i\mid e_0)$ denote the [1rp posterior probability] of $H_i$ after observing $e_0.$

## [1x5 Odds]/[1zm proportional] form

Bayes' rule in the [1x5 odds form] or [1zm proportional form] states:

$$\dfrac{\mathbb P(H_1)}{\mathbb P(H_2)} 	imes \dfrac{\mathbb P(e_0\mid H_1)}{\mathbb P(e_0\mid H_2)} = \dfrac{\mathbb P(H_1\mid e_0)}{\mathbb P(H_2\mid e_0)}$$

In other words, the [1rm prior] [1rb odds] times the [1rq likelihood ratio] yield the [1rp posterior] odds.  [1rk Normalizing] these odds will then yield the posterior probabilities.

In [1zm other other words]:  If you initially think $h_i$ is $lpha$ times as probable as $h_k$, and then see evidence that you're $eta$ times as likely to see if $h_i$ is true as if $h_k$ is true, you should update to thinking that $h_i$ is $lpha \cdot eta$ times as probable as $h_k.$

Suppose that Professor Plum and Miss Scarlet are two suspects in a murder, and that we start out thinking that Professor Plum is twice as likely to have committed the murder as Miss Scarlet ([1rm prior] [1rb odds] of 2 : 1).  We then discover that the victim was poisoned.  We think that Professor Plum is around one-fourth as likely to use poison as Miss Scarlet ([1rq likelihood ratio] of 1 : 4).  Then after observing the victim was poisoned, we should think Plum is around half as likely to have committed the murder as Scarlet: $2 	imes \dfrac{1}{4} = \dfrac{1}{2}.$  This reflects [1rp posterior] odds of 1 : 2, or a posterior probability of 1/3, that Professor Plum did the deed.

## [1xr Proof]

The [1xr proof of Bayes' rule] is by the definition of [1rj conditional probability] $\mathbb P(X\wedge Y) = \mathbb P(X\mid Y) \cdot \mathbb P(Y):$

$$
\dfrac{\mathbb P(H_i)}{\mathbb P(H_j)} 	imes  \dfrac{\mathbb P(e\mid H_i)}{\mathbb P(e\mid H_j)}
= \dfrac{\mathbb P(e \wedge H_i)}{\mathbb P(e \wedge H_j)}
= \dfrac{\mathbb P(e \wedge H_i) / \mathbb P(e)}{\mathbb P(e \wedge H_j) / \mathbb P(e)}
= \dfrac{\mathbb P(H_i\mid e)}{\mathbb P(H_j\mid e)}
$$

## [1zh Log odds form]

The [1zh log odds form of Bayes' rule] states:

$$\log \left ( \dfrac
   {\mathbb P(H_i)}
   {\mathbb P(H_j)}

ight )
+
\log \left ( \dfrac
   {\mathbb P(e\mid H_i)}
   {\mathbb P(e\mid H_j)}

ight )
 =
\log \left ( \dfrac
   {\mathbb P(H_i\mid e)}
   {\mathbb P(H_j\mid e)}

ight )
$$

E.g.:  "A study of Chinese blood donors found that roughly 1 in 100,000 of them had HIV (as determined by a very reliable gold-standard test). The non-gold-standard test used for initial screening had a sensitivity of 99.7% and a specificity of 99.8%, meaning that it was 500 times as likely to return positive for infected as non-infected patients."  Then our prior belief is -5 orders of magnitude against HIV, and if we then observe a positive test result, this is evidence of strength +2.7 orders of magnitude for HIV.  Our posterior belief is -2.3 orders of magnitude, or odds of less than 1 to a 100, against HIV.

In log odds form, the same [22x strength of evidence] (log [1rq likelihood ratio]) always [1zh moves us the same additive distance] along a line representing strength of belief (also in log odds).  If we measured distance in probabilities, then the same 2 : 1 likelihood ratio might move us a different distance along the probability line depending on whether we started with prior 10% probability or 50% probability.

## Visualizations

Graphical of visualizing Bayes' rule include [1wy frequency diagrams, the waterfall visualization], the [1zm spotlight visualization], the [1zh magnet visualization], and the [1xr Venn diagram for the proof].

## Examples

Examples of Bayes' rule may be found [1wt here].

## [1zg Multiple hypotheses and updates]

The [1x5 odds form of Bayes' rule] works for odds ratios between more than two hypotheses, and applying multiple pieces of evidence.  Suppose there's a bathtub full of coins.  1/2 of the coins are "fair" and have a 50% probability of producing heads on each coinflip; 1/3 of the coins produce 25% heads; and 1/6 produce 75% heads.  You pull out a coin at random, flip it 3 times, and get the result HTH.  You may legitimately calculate:

$$egin{array}{rll}
(1/2 : 1/3 : 1/6) \cong & (3 : 2 : 1) & \
	imes & (2 : 1 : 3) & \
	imes & (2 : 3 : 1) & \
	imes & (2 : 1 : 3) & \
= & (24 : 6 : 9) & \cong (8 : 2 : 3)
\end{array}$$

Since multiple pieces of evidence may not be [conditional_independence conditionally independent] from one another, it is important to be aware of the [naive_bayes_assumption Naive Bayes assumption] and whether you are making it.

## [554 Probability form]

As a formula for a single probability $\mathbb P(H_i\mid e),$ Bayes' rule states:

$$\mathbb P(H_i\mid e) = \dfrac{\mathbb P(e\mid H_i) \cdot \mathbb P(H_i)}{\sum_k \mathbb P(e\mid H_k) \cdot \mathbb P(H_k)}$$

## [1zj Functional form]

In [1zj functional form], Bayes' rule states:

$$\mathbb P(\mathbf{H}\mid e) \propto \mathbb P(e\mid \mathbf{H}) \cdot \mathbb P(\mathbf{H}).$$

The posterior probability function over hypotheses given the evidence, is *proportional* to the likelihood function from the evidence to those hypotheses, times the prior probability function over those hypotheses.

Since posterior probabilities over [1rd mutually exclusive and exhaustive] possibilities must sum to $1,$ [1rk normalizing] the product of the likelihood function and prior probability function will yield the exact posterior probability function.


Bayes' rule: Probability form

[summary:  The formulation of [1lz] you are most likely to see in textbooks says:

$$\mathbb P(H_i\mid e) = \dfrac{\mathbb P(e\mid H_i) \cdot \mathbb P(H_i)}{\sum_k \mathbb P(e\mid H_k) \cdot \mathbb P(H_k)}$$

This follows from the definition of [-1rj] which states that $\mathbb P(X \mid Y) = rac{\mathbb P(X \wedge Y)}{\mathbb P (Y)}$, and the [law_marginal_probability law of marginal probability] which says that $\mathbb P(Y) = \sum_k \mathbb P(Y \wedge X_k)$.

We can think of the corresponding advice as saying, "Think of how much each hypothesis in $H$ contributed to our expectation of seeing the evidence $e$, including both the [56v likelihood] of seeing $e$ if $H_k$ is true, and the [1rm prior] probability of $H_k$.  The [1rp posterior] of $H_i$ after seeing $e,$ is the amount $H_i$ contributed to our expectation of seeing $e,$ within the total expectation of seeing $e$ contributed by every hypothesis in $H.$]

The formulation of [1lz] you are most likely to see in textbooks runs as follows:

$$\mathbb P(H_i\mid e) = \dfrac{\mathbb P(e\mid H_i) \cdot \mathbb P(H_i)}{\sum_k \mathbb P(e\mid H_k) \cdot \mathbb P(H_k)}$$

Where:

- $H_i$ is the hypothesis we're interested in.
- $e$ is the piece of evidence we observed.
- $\sum_k (	ext {expression containing } k)$ [summation_notation means] "Add up, for every $k$, the sum of all the (expressions containing $k$)."
- $\mathbf H$ is a set of [1rd mutually exclusive and exhaustive] hypotheses that include $H_i$ as one of the possibilities, and the expression $H_k$ inside the sum ranges over all the possible hypotheses in $\mathbf H$.

As a quick example, let's say there's a bathtub full of potentially biased coins.

- Coin type 1 is fair, 50% heads / 50% tails.  40% of the coins in the bathtub are type 1.
- Coin type 2 produces 70% heads.  35% of the coins are type 2.
- Coin type 3 produces 20% heads.  25% of the coins are type 3.

We want to know the [1rp posterior] probability that a randomly drawn coin is of type 2, after flipping the coin once and seeing it produce heads once.

Let $H_1, H_2, H_3$ stand for the hypotheses that the coin is of types 1, 2, and 3 respectively.  Then using [1rj conditional probability notation], we want to know the probability $\mathbb P(H_2 \mid heads).$

The probability form of Bayes' theorem says:

$$\mathbb P(H_2 \mid heads) = rac{\mathbb P(heads \mid H_2) \cdot \mathbb P(H_2)}{\sum_k \mathbb P(heads \mid H_k) \cdot \mathbb P(H_k)}$$

Expanding the sum:

$$\mathbb P(H_2 \mid heads) = rac{\mathbb P(heads \mid H_2) \cdot \mathbb P(H_2)}{[\mathbb P(heads \mid H_1) \cdot \mathbb P(H_1)] + [\mathbb P(heads \mid H_2) \cdot \mathbb P(H_2)] + [\mathbb P(heads \mid H_3) \cdot \mathbb P(H_3)]}$$

Computing the actual quantities:

$$\mathbb P(H_2 \mid heads) = rac{0.70 \cdot 0.35 }{[0.50 \cdot 0.40] + [0.70 \cdot 0.35] + [0.20 \cdot 0.25]} = rac{0.245}{0.20 + 0.245 + 0.05} = 0.\overline{49}$$

This calculation was big and messy.  Which is fine, because the probability form of Bayes' theorem is okay for directly grinding through the numbers, but not so good for doing things in your head.

# Meaning

We can think of the advice of Bayes' theorem as saying:

"Think of how much each hypothesis in $H$ contributed to our expectation of seeing the evidence $e$, including both the [56v likelihood] of seeing $e$ if $H_k$ is true, and the [1rm prior] probability of $H_k$.  The [1rp posterior] of $H_i$ after seeing $e,$ is the amount $H_i$ contributed to our expectation of seeing $e,$ within the total expectation of seeing $e$ contributed by every hypothesis in $H.$"

Or to say it at somewhat greater length:

Imagine each hypothesis $H_1,H_2,H_3\ldots$ as an expert who has to distribute the probability of their predictions among all possible pieces of evidence.  We can imagine this more concretely by visualizing "probability" as a lump of clay.

The total amount of clay is one kilogram (probability $1$).  Each expert $H_k$ has been allocated a fraction $\mathbb P(H_k)$ of that kilogram.  For example, if $\mathbb P(H_4)=rac{1}{5}$ then expert 4 has been allocated 200 grams of clay.

We're playing a game with the experts to determine which one is the best predictor.

Each time we're about to make an observation $E,$ each expert has to divide up all their clay among the possible outcomes $e_1, e_2, \ldots.$

After we observe that $E = e_j,$ we take away all the clay that wasn't put onto $e_j.$  And then our new belief in all the experts is the relative amount of clay that each expert has left.

So to know how much we now believe in expert $H_4$ after observing $e_3,$ say, we need to know two things:  First, the amount of clay that $H_4$ put onto $e_3,$ and second, the total amount of clay that all experts (including $H_4$) put onto $e_3.$

In turn, to know *that,* we need to know how much clay $H_4$ started with, and what fraction of its clay $H_4$ put onto $e_3.$  And similarly, to compute the total clay on $e_3,$ we need to know how much clay each expert $H_k$ started with, and what fraction of their clay $H_k$ put onto $e_3.$

So Bayes' theorem here would say:

$$\mathbb P(H_4 \mid e_3) = rac{\mathbb P(e_3 \mid H_4) \cdot \mathbb P(H_4)}{\sum_k \mathbb P(e_3 \mid H_k) \cdot \mathbb P(H_k)}$$

What are the incentives of this game of clay?

On each round, the experts who gain the most are the experts who put the most clay on the observed $e_j,$ so if you know for certain that $e_3$ is about to be observed, your incentive is to put all your clay on $e_3.$

But putting *literally all* your clay on $e_3$ is risky; if $e_5$ is observed instead, you lose all your clay and are out of the game. Once an expert's amount of clay goes all the way to zero, there's no way for them to recover over any number of future rounds. That hypothesis is done, dead, and removed from the game.  ("Falsification," some people call that.)  If you're not certain that $e_5$ is literally impossible, you'd be wiser to put at least a *little* clay on $e_5$ instead.  That is to say: if your mind puts some probability on $e_5,$ you'd better put some clay there too!

([bayes_score As it happens], if at the end of the game we score each expert by the logarithm of the amount of clay they have left, then each expert is incentivized to place clay exactly proportionally to their honest probability on each successive round.)

It's an important part of the game that we make the experts put down their clay in advance.  If we let the experts put down their clay afterwards, they might be tempted to cheat by putting down all their clay on whichever $e_j$ had actually been observed.  But since we make the experts put down their clay in advance, they have to *divide up* their clay among the possible outcomes: to give more clay to $e_3,$ that clay has to be taken away from some other outcome, like $e_5.$  To put a very high probability on $e_3$ and gain a lot of relative credibility if $e_3$ is observed, an expert has to stick their neck out and risk losing a lot of credibility if some other outcome like $e_5$ happens instead.  *If* we force the experts to make advance predictions, that is!

We can also derive from this game that the question "does evidence $e_3$ support hypothesis $H_4$?" depends on how well $H_4$ predicted $e_3$ _compared to the competition._ It's not enough for $H_4$ to predict $e_3$ well if every other hypothesis also predicted $e_3$ well--your amazing new theory of physics gets no points for predicting that the sky is blue.  $H_k$ only goes up in probability when it predicts $e_j$ better than the alternatives.  And that means we have to ask what the alternative hypotheses predicted, even if we think those hypotheses are false.

If you get in a car accident, and don't want to relinquish the hypothesis that you're a great driver, then you can find all sorts of reasons ("the road was slippery! my car freaked out!") why $\mathbb P(e \mid GoodDriver)$ is not too low. But $\mathbb P(e \mid BadDriver)$ is also part of the update equation, and the "bad driver" hypothesis *better* predicts the evidence. Thus, your first impulse, when deciding how to update your beliefs in the face of a car accident, should not be "But my preferred hypothesis allows for this evidence!" It should instead be "Points to the 'bad driver' hypothesis for predicting this evidence better than the alternatives!"  (And remember, you're allowed to [update_beliefs_incrementally increase $\mathbb P(BadDriver)$ a little bit], while still thinking that it's less than 50% probable.)

# Proof

The proof of Bayes' theorem follows from the definition of [-1rj]:

$$\mathbb P(X \mid Y) = rac{\mathbb P(X \wedge Y)}{\mathbb P (Y)}$$

And from the [law_marginal_probability law of marginal probability]:

$$\mathbb P(Y) = \sum_k \mathbb P(Y \wedge X_k)$$

Therefore:

$$
\mathbb P(H_i \mid e) = rac{\mathbb P(H_i \wedge e)}{\mathbb P (e)}  	ag{defn. conditional prob.}
$$

$$
\mathbb P(H_i \mid e) = rac{\mathbb P(e \wedge H_i)}{\sum_k \mathbb P (e \wedge H_k)} 	ag {law of marginal prob.}
$$

$$
\mathbb P(H_i \mid e) = rac{\mathbb P(e \mid H_i) \cdot \mathbb P(H_i)}{\sum_k \mathbb P (e \mid H_k) \cdot \mathbb P(H_k)} 	ag {defn. conditional prob.}
$$

QED.


Odds form to probability form

The odds form of Bayes' rule works for any two hypotheses $H_i$ and $H_j,$ and looks like this:

$$rac{\mathbb P(H_i \mid e)}{\mathbb P(H_j \mid e)} = rac{\mathbb P(H_i)}{\mathbb P(H_j)} 	imes rac{\mathbb P(e \mid H_i)}{\mathbb P(e \mid H_j)} 	ag{1}.$$

The probabilistic form of Bayes' rule requires a hypothesis set $H_1,H_2,H_3,\ldots$ that is [-1rd], and looks like this:

$$\mathbb P(H_i\mid e) = rac{\mathbb P(e\mid H_i) \cdot \mathbb P(H_i)}{\sum_k \mathbb P(e\mid H_k) \cdot \mathbb P(H_k)} 	ag{2}.$$

We will now show that equation (2) follows from equation (1). Given a collection $H_1,H_2,H_3,\ldots$ of mutually exclusive and exhaustive hypotheses and a hypothesis $H_i$ from that collection, we can form another hypothesis $\lnot H_i$ consisting of all the hypotheses $H_1,H_2,H_3,\ldots$ _except_ $H_i.$ Then, using $\lnot H_i$ as $H_j$ and multiplying the fractions on the right-hand side of equation (1), we see that

$$rac{\mathbb P(H_i \mid e)}{\mathbb P(\lnot H_i \mid e)} = rac{\mathbb P(H_i) \cdot \mathbb P(e \mid H_i)}{\mathbb P(\lnot H_i)\cdot \mathbb P(e \mid \lnot H_i)}.$$

$\mathbb P(\lnot H_i)\cdot \mathbb P(e \mid \lnot H_i)$ is the prior probability of $\lnot H_i$ times the degree to which $\lnot H_i$ predicted $e.$ Because $\lnot H_i$ is made of a bunch of mutually exclusive hypotheses, this term can be calculated by summing $\mathbb P(H_k) \cdot \mathbb P(e \mid H_k)$ for every $H_k$ in the collection except $H_i.$ Performing that replacement, and swapping the order of multiplication, we get:

$$rac{\mathbb P(H_i \mid e)}{\mathbb P(\lnot H_i \mid e)} = rac{\mathbb P(e \mid H_i) \cdot \mathbb P(H_i)}{\sum_{k
eq i} \mathbb P(e \mid H_k) \cdot \mathbb P(H_k)}.$$

These are the posterior odds for $H_i$ versus $\lnot H_i.$ Because $H_i$ and $\lnot H_i$ are mutually exclusive and exhaustive, we can convert these odds into a probability for $H_i,$ by calculating numerator / (numerator + denominator), in the same way that $3 : 4$ odds become a 3 / (3 + 4) probability. When we do so, equation (2) drops out:

$$\mathbb P(H_i\mid e) = rac{\mathbb P(e\mid H_i) \cdot \mathbb P(H_i)}{\sum_k \mathbb P(e\mid H_k) \cdot \mathbb P(H_k)}.$$

Thus, we see that the probabilistic formulation of Bayes' rule follows from the odds form, but is less general, in that it only works when the set of hypotheses being considered are mutually exclusive and exhaustive.

We also see that the probabilistic formulation converts the posterior odds into a posterior probability. When computing [bayes_rule_vector multiple updates in a row], you actually only need to perform this "normalization" step once at the very end of your calculations — which means that the odds form of Bayes' rule is also more efficient in practice.


Logistic function

[summary: The logistic function is a [-sigmoid] function that maps the [4bc real numbers] to the unit interval $(0, 1)$ using the formula $\displaystyle f(x) = rac{1}{1 + e^{-x}}$ or variants of this formula.]

The logistic function is a [-sigmoid] function that maps the [4bc real numbers] to the unit interval $(0, 1)$ using the formula $\displaystyle f(x) = rac{1}{1 + e^{-x}}$.

More generally, there exists a [family_of_functions family] of logistic functions that can be written as $\displaystyle f(x) = rac{M}{1 + lpha^{c(x_0 - x)}}$, where:

* $M$ is the upper bound of the function (in which case the function maps to the interval $(0, M)$ instead). When $M = 1$, the logistic function is usually being used to calculate some [-1rf] or [-4w3] of a total.

* $x_0$ is the inflection point of the curve, or the value of $x$ where the function's growth stops speeding up and starts slowing down.

* $lpha$ is a variable controlling the steepness of the curve.

* $c$ is a scaling factor for the distance.

## Applications

* The logistic function is used to model growth rates of populations in an ecosystem with a limited carrying capacity.

* The inverse logistic function (with $lpha = 2$) is used to convert a probability to log-odds form for use in [1lz].

* The logistic function (with $lpha = 10$ and $c = 1/400$) is used to calculate the expected probability of a player winning given a specific difference in rating in the Elo rating system.


Sparking widgets

10% of widgets are bad and 90% are good. 4% of good widgets emit sparks, and 12% of bad widgets emit sparks. If a widget is sparking, find an easy way to calculate the chance that it's a bad widget, keeping the operations as simple as possible so that it's easy to do the calculation entirely in your head.

You're encouraged to solve the problem yourself without looking at the answer. an answer can be found in [bayes_extraordinary_claims].


Sock-dresser search

You left your socks somewhere in your room.  You think there's a 4/5 chance that they're in your dresser, so you start looking through your dresser's 8 drawers.  After checking 6 drawers at random, you haven't found your socks yet.  What is the probability you will find your socks in the next drawer you check?

You're encouraged to solve the problem yourself before looking up the answer. The answer can be found in [1y6].


Ordinary claims require ordinary evidence

A corollary to [21v extraordinary claims require extraordinary evidence] is "ordinary claims require only ordinary evidence"%%note: Attributed to [Gwern Branwen](https://www.gwern.net/), who hasn't popularized this claim nearly as much as Carl Sagan popularized the extraordinary version.%%

This corollary is probably more important in practice, as, in our day-to-day lives, we encounter more ordinary claims than extraordinary claims, but may be more tempted to reject many of those ordinary claims by demanding extraordinary evidence (a bias known as "motivated skepticism").

# Example: A failing new employee

One contributed example:

> A few years back, a senior person at my workplace told me that a new employee wasn't getting his work done on time, and that she'd had to micromanage him to get any work out of him at all.  This was an unpleasant fact for a number of reasons; I'd liked the guy, and I'd advocated for hiring him to our Board of Directors just a few weeks earlier (which is why the senior manager was talking to me).  I could have demanded more evidence, I could have demanded that we give him more time to work out, I could have demanded a videotape and signed affidavits… but a new employee not working out, just *isn't that improbable*.  Could I have named the exact prior odds of an employee not working out, could I have said [1zm how much more likely] I was to hear that exact report of a long-term-bad employee than a long-term-good employee?  No, but 'somebody hires the wrong person' happens all the time, and I'd seen it happen before.  It wasn't an extraordinary claim, and I wasn't licensed to ask for extraordinary evidence.  To put numbers on it, I thought the proportion of bad to good employees was on the order of 1 : 4 at the time, and the likelihood ratio for the manager's report seemed more like 10 : 1.

Or to put it another way:  The rule is 'extraordinary claims require extraordinary evidence', not 'inconvenient but ordinary claims require extraordinary evidence'.

In everyday life, we consider many more ordinary claims than extraordinary claims — including many ordinary claims whose inconvenience might tempt us to dismiss their adequate but ordinary evidence. In those cases, it's important to remember that ordinary claims only require ordinary evidence.


Ordered ring

[summary: An **ordered ring** is a [-3gq] that is [540 totally ordered], where the ordering agrees with the ring operations. In particular, adding something to two elements doesn't change which of them is bigger, and the product of two [-positive] elements is positive.]

An **ordered ring** is a [-3gq] $R=(X,\oplus,\otimes)$ with a [540 total order] $\leq$ compatible with the ring structure. Specifically, it must satisfy these axioms for any $a,b,c \in X$:

- If $a \leq b$, then $a \oplus c \leq b \oplus c$.
- If $0 \leq a$ and $0 \leq b$, then $0 \leq a \otimes b$

An element $a$ of the ring is called "[-positive]" if $0<a$ and "[-negative]" if $a<0$. The second axiom, then, says that the product of nonnegative elements is nonnegative.

An ordered ring that is also a [481 field] is an [-ordered_field].

#Basic Properties

- For any element $a$, $a \leq 0$ if and only if $0 \leq -a$.

%%hidden(Show proof):
First suppose $a \leq 0$. Using the first axiom to add $-a$ to both sides, $a+(-a) = 0 \leq -a$. For the other direction, suppose $0 \leq -a$. Then $a \leq -a+a = 0$.
%%

- The product of nonpositive elements is nonnegative.

%%hidden(Show proof):
Suppose $a$ and $b$ are nonpositive elements of $R$, that is $a,b \leq 0$. From the first axiom, $a+(-a) = 0 \leq -a$, and similarly $0 \leq -b$. By the second axiom $0 \leq -a \otimes -b$. But $-a \otimes -b = a \otimes b$, so $0 \leq a \otimes b$.
%%

- The [-square] of any element is nonnegative.

%%hidden(Show proof):
Let $a$ be such an element. Since the ordering is total, either $0 \leq a$ or $a \leq 0$. In the first case, the second axiom gives $0 \leq a^2$. In the second case, the previous property gives $0 \leq a^2$, since $a$ is nonpositive. Either way we have $0 \leq a^2$.
%%

- The additive [54p identity] $1 \geq 0$. (Unless the ring is trivial, $1>0$.)

%%hidden(Show proof):
Clearly $1 = 1 \otimes 1$. So $1$ is a square, which means it's nonnegative.
%%

# Examples

The [4bc real numbers] are an ordered ring (in fact, an [-ordered_field]), as is any [-subring] of $\mathbb R$, such as [4zq $\mathbb Q$].

The [4zw complex numbers] are not an ordered ring, because there is no way to define the order between $0$ and $i$. Suppose that $0 \le i$, then, we have $0 \le i 	imes i = -1$, which is false. Suppose that $i \le 0$, then $0 = i + (-i) \le 0 + (-i)$, but then we have $0 \le (-i) 	imes (-i) = -1$, which is again false. Alternatively, $i^2=-1$ is a square, so it must be nonnegative; that is, $0 \leq -1$, which is a contradiction.


Addition of rational numbers (Math 0)

[summary: Addition is "putting two quantities next to each other and working out how much they come to when combined".]

We've already met the idea that rational numbers are what we can make by putting together the building blocks which we get by dividing a single apple into some number of equally-sized pieces; and remember that the building blocks have a special notation, as $rac{1}{	ext{number}}$.
Recall also that if we take $5$ of a certain building block, for instance, we write $rac{5}{	ext{number}}$.

It's clear that if you take one apple and another apple, and put them together, you'll get two apples.
So we should hope that that's the same if we divided up the apples into pieces first (without removing any of the pieces).

We write $a+b$ for "take the rational number $a$ and put it next to the rational number $b$, and count up what you've got".
Because I'm bad at drawing, we'll pretend apples are just perfect circles. %%note:This is actually a design decision: eventually we'll want to get away from considering apples, and this more abstract representation will be useful.%%

![Example of a sum of two rational numbers](http://i.imgur.com/GUgmCwb.png)

----------

For example, we should hope that $rac{2}{2} + rac{3}{3} = 2$.

![Two halves and three thirds](http://i.imgur.com/6vdRdcT.png)

What about in cases which aren't of the form $rac{n}{n}$ for some integer $n$?
Well, how about $rac{5}{3} + rac{8}{3}$.

![Five thirds and eight thirds](http://i.imgur.com/7dPqmCP.png)

If we had five $rac{1}{3}$-blocks, and we put them together with another eight $rac{1}{3}$-blocks, we would hope to have $5+8=13$ blocks.
So we should hope that $rac{5}{3} + rac{8}{3} = rac{13}{3}$.

![Thirds, separated out](http://i.imgur.com/foVkEiS.png)

And in general, if both our quantities are made up from the same size of building-block (in the above case, the blocks are $rac{1}{3}$-sized), we should just be able to take the two numerators %%note:Remember, that was the word mathematicians use for the number of blocks we have.%% and add them together.

----------

Very well. But what if the two quantities are not made up from the same size of building-block? That is, the denominators are different?
For example, $rac{5}{3} + rac{5}{4}$?

![Five thirds and five quarters](http://i.imgur.com/xFtKEnU.png)

![Five thirds and five quarters, rearranged](http://i.imgur.com/Av984T0.png)

Now it's not so clear.
How might we approach this? You should muse on this for thirty seconds before reading on; it will be good for your soul.

%%hidden(Show solution):
The way we do it is to find some *smaller* block-size, out of which we can make *both* building-blocks.
%%

# Example

Our example is $rac{5}{3} + rac{5}{4}$.
So we want to make the $rac{1}{3}$ block and the $rac{1}{4}$ block out of some smaller block.

Now, I won't tell you yet how I got this, but you can check for yourself that $rac{1}{12}$-blocks will make both $rac{1}{3}$- and $rac{1}{4}$-blocks: because three $rac{1}{12}$-blocks together make something the same size as a $rac{1}{4}$-block, and four $rac{1}{12}$-blocks together make something the same size as a $rac{1}{3}$-block.

![Thirds, divided into twelfths](http://i.imgur.com/k2iz8uR.png)

![Quarters, divided into twelfths](http://i.imgur.com/gmHK3oj.png)

Therefore $rac{1}{3} = rac{4}{12}$:

![One third, divided into twelfths](http://i.imgur.com/fFX4Dmc.png)

And $rac{1}{4} = rac{3}{12}$:

![One quarter, divided into twelfths](http://i.imgur.com/gRqmfAc.png)

Now, if $rac{1}{3} = rac{4}{12}$ - that is, if one $rac{1}{3}$-piece is the same size as four $rac{1}{12}$-pieces - then it should be the case that five $rac{1}{3}$-pieces come to five lots of four $rac{1}{12}$-pieces.
That is, to twenty $rac{1}{12}$-pieces: $rac{5}{3} = rac{20}{12}$.
[todo: picture which I haven't got round to detailing yet]

Similarly, $rac{5}{4} = rac{15}{12}$, because five $rac{1}{4}$-pieces is the same as five lots of three sets of $rac{1}{12}$-pieces, and $5 	imes 3 = 15$.

Therefore $rac{5}{3} + rac{5}{4}$ should be just the same as $rac{20}{12} + rac{15}{12}$, which we know how to calculate!
It is $rac{35}{12}$.

# General procedure

The preceding example might suggest a general way for adding two fractions together. (In the process, this should put to rest the existential dread mentioned in [4zx the intro to rational numbers], about whether it makes sense to add two rational numbers at all.)

The general procedure is as follows:

 1. Find a building-block size such that the building-blocks of both fractions can be made out of that building-block
 2. Express each fraction individually in terms of that building-block
 3. Add the two together (which we can now do, because they are both made of the same size of building-block).

The interesting part is really the first step.

## Finding a building-block that will neatly make two other differently-sized blocks

Let's be concrete here: just as above, we showed that $rac{1}{12}$ works for blocks of size $rac{1}{3}$ and $rac{1}{4}$, so we'll now consider $rac{1}{2}$ and $rac{1}{5}$.

We're trying to divide up $rac{1}{2}$ into copies of a smaller building-block, such that with copies of that *same* building block, we can also make $rac{1}{5}$.
Here is the key insight that tells us how to do it: let us pretend that we are dividing up the *same* object, this time a square instead of a circle, into two pieces along one edge and into five pieces along the other edge.
How many pieces have we made?

![Square divided into tenths](http://i.imgur.com/qkMkQvZ.png)

There are $2 	imes 5 = 10$ pieces, each of them the same size (namely the $rac{1}{10}$-block size), and note that as if by magic we've just produced a way to subdivide $rac{1}{2}$ into pieces made up of the $rac{1}{10}$-block!

![Square divided into tenths, one half indicated](http://i.imgur.com/n9dYZBA.png)

And we also have the $rac{1}{5}$ subdivided in the same way!

![Square divided into tenths, one fifth indicated](http://i.imgur.com/0ny9RnS.png)

This might suggest a general way to do this; you should certainly ponder for thirty seconds before continuing.
%%hidden(Show the general way):
[todo: find a way to do this without algebra]
Let's say we are trying to make both $rac{1}{m}$ and $rac{1}{n}$-sized blocks out of smaller blocks.
(So above, $m$ is standing for $2$ and $n$ is standing for $5$.)

Then we can do this by splitting both up into $rac{1}{m 	imes n}$-blocks.

Indeed, $rac{1}{n} = rac{m}{m 	imes n}$ (i.e. we can make a $rac{1}{n}$-block out of $m$ of the $rac{1}{m 	imes n}$ tiny-blocks),
and $rac{1}{m} = rac{n}{m 	imes n}$ (i.e. we can make a $rac{1}{m}$-block out of $n$ of the $rac{1}{m 	imes n}$ tiny-blocks).
%%

## Aside: arithmetical rules

The description of the "general way" above has become pretty full of $rac{1}{	ext{thing}}$, and refers less and less to the building-blocks we originally considered.
The reason for this is that $rac{1}{	ext{thing}}$ is an extremely convenient shorthand, and the "general way" can be written as a very simple rule for manipulating that shorthand.
Once we have such a convenient rule, we can compute with fractions without having to go through the bother of working out what each fraction really and truly *means* (in terms of the building-blocks); this saves a lot of time in practice.
As an analogy, it's easier for you to drive a car if, every time you change gears, you *don't* have to work out what's happening inside the gearbox.

The simple rule we have just seen is $$rac{1}{m} + rac{1}{n} = rac{n}{m 	imes n} + rac{m}{m 	imes n}$$
And we've already had the rule that $$rac{a}{m} + rac{b}{m} = rac{a+b}{m}$$
(Remember, if these are not obvious to you, you should think back to what they really mean, in terms of the building-blocks. The first rule is to do with dividing up a big square into little squares, from the previous subsection; the second rule is just taking two collections of building-blocks of the same size and putting them together.)
[todo: first rule has picture: square divided into n-by-m little squares, with the leftmost vertical column highlighted somehow, indicating that 1/n = m/(mn); second rule has picture: $a$ things of size 1/m, next to $b$ things of size 1/m]

This is almost every rule we need to add any pair of fractions!
We can now add single building blocks of any size together.

To add any pair of rational numbers together - say of building-block size $rac{1}{m}$ and $rac{1}{n}$ respectively - we just need to express them each in terms of the smaller building-block $rac{1}{m 	imes n}$, and add them together as normal.

%%hidden(Example):
Earlier, we considered $rac{5}{4} + rac{5}{3}$.

A building-block which can make both the $rac{1}{4}$ and $rac{1}{3}$-sized blocks is the $rac{1}{12}$-block, because $3 	imes 4 = 12$.

Therefore we need to express both our fractions as being made from $rac{1}{12}$-sized blocks.

$rac{5}{4}$ is $rac{15}{12}$, because in each of the five $rac{1}{4}$-blocks, there are three $rac{1}{12}$-blocks, so we have a total of $5 	imes 3$ blocks of size $rac{1}{12}$.

$rac{5}{3}$ is $rac{20}{12}$, because in each of the five $rac{1}{3}$-blocks, there are four $rac{1}{12}$-blocks, so we have a total of $5 	imes 4 = 20$ blocks of size $rac{1}{12}$.

Therefore our final answer is $rac{15}{12} + rac{20}{12} = rac{35}{12}$.
%%

To make this into a general rule: $$rac{a}{m} + rac{b}{n} = rac{a 	imes n}{m 	imes n} + rac{b 	imes m}{m 	imes n} = rac{a 	imes n + b 	imes m}{m 	imes n}$$

(Recall the order of operations in the integers: the notation $a 	imes n + b 	imes m$ means "do $a 	imes n$; then do $b 	imes m$; then finally add them together". Multiplication comes before addition.)

And that's it! That's how we add rational numbers together, and it works even when any or all of $a, b, m, n$ are negative.
(Remember, though, that $m$ and $n$ can't be zero, because it makes no sense to divide something up into no pieces.)

It would be wise now to try the [55p exercises]. One learns mathematics through doing, much more than through simply reading; your understanding will be cemented by going through some concrete examples.


Addition of rational numbers exercises

[summary: This page consists of exercises designed to help you get to grips with the addition of rational numbers.]

This page consists of exercises designed to help you get to grips with the addition of rational numbers.

# Core exercises

## First exercise: $rac{1}{10} + rac{1}{5}$

%%hidden(Show some possible solutions):
Using the instant rule from the text (which is actually a bit unwieldy here): $$rac{1}{10} + rac{1}{5} = rac{1 	imes 5 + 10 	imes 1}{10 	imes 5} = rac{5+10}{50} = rac{15}{50}$$
Notice that this can actually be made simpler: it is the same thing as $rac{3}{10}$, because when we take $rac{3}{10}$ and split each $rac{1}{10}$-block into five equal pieces, we obtain $15$ copies of the $rac{1}{50}$-block.

Alternatively, one could spot that $rac{1}{10}$-blocks actually can already be used to make $rac{1}{5}$-blocks: $rac{1}{5} = rac{2}{10}$.
Therefore we're actually trying to find $rac{1}{10} + rac{2}{10}$, which is easy: it's $rac{3}{10}$.
%%

## Second exercise: $rac{1}{15} + rac{1}{10}$

%%hidden(Show some possible solutions):
Using the instant rule from the text: $$rac{1}{10} + rac{1}{15} = rac{1 	imes 15 + 10 	imes 1}{10 	imes 15} = rac{25}{150} = rac{1}{6}$$

Were you expecting a big denominator, or at least a multiple of 5? From this example, we can see that the final answer of a rational addition problem can have a denominator which doesn't even seem related to the others.

Alternatively, one could spot that $rac{1}{30}$-blocks will make up $rac{1}{10}$- and $rac{1}{15}$-blocks.
Then we are actually trying to find $rac{3}{30} + rac{2}{30} = rac{5}{30}$; it's a bit easier to see that $rac{5}{30} = rac{1}{6}$ than it is to see that $rac{25}{150} = rac{1}{6}$.
%%

## Third exercise: $rac{1}{10} + rac{1}{15}$
You might find this exercise a little familiar…

%%hidden(Show possible solution):
Using the instant rule from the text: $$rac{1}{15} + rac{1}{10} = rac{1 	imes 10 + 15 	imes 1}{15 	imes 10} = rac{25}{150} = rac{1}{6}$$

You may notice that we've basically done the same calculations as in the second exercise.
In fact, addition doesn't care which way round the numbers go: $rac{1}{10} + rac{1}{15} = rac{1}{15} + rac{1}{10}$, even if we don't already know that that number is $rac{1}{6}$.

This is intuitive from the fact that addition is the idea of "place the apples next to each other and count up the total": just putting the apples down in a different order doesn't change the total amount of apple.
%%

## Fourth exercise: $rac{0}{5} + rac{2}{5}$

%%hidden(Show possible solution):
Notice that both the denominators are the same (namely $5$), so we can just combine the $rac{1}{5}$-sized pieces straight away.
We have $0$ pieces and $2$ pieces, so the total is $2$ pieces.

That is, the answer is $rac{2}{5}$.
%%

## Fifth exercise: $rac{0}{7} + rac{2}{5}$

%%hidden(Show possible solution):
If you spot that this is "no $rac{1}{7}$-pieces" next to "two $rac{1}{5}$-pieces", then you might just immediately write down that the answer is $rac{2}{5}$ because there aren't any $rac{1}{7}$-pieces to change the answer; and you'd be correct.

To use the instant rule from the text: $$rac{0}{7} + rac{2}{5} = rac{0 	imes 5 + 2 	imes 7}{5 	imes 7} = rac{0 + 14}{35} = rac{14}{35}$$

But that is the same as $rac{2}{5}$ (simply expressed with each $rac{1}{5}$-piece subdivided further into sevenths).
%%

# Extension exercises

These exercises are meant to be harder and to stretch your conceptual understanding.
Give them a proper go, but don't worry too much if you don't get the same answers as me.
Mine are, in a technical sense, "right", but no matter what you end up with, you will derive a lot of benefit from trying to work out what the answers are yourself without having been told exactly how.
The learning of mathematics is much more about thinking and understanding (usually guided by examples) than it is about just repeatedly carrying out calculations.

## First extension exercise: $rac{1}{5} + rac{-1}{10}$
Yes, it is possible to add a negative number of chunks.
Try using the instant rule and see what happens.

%%hidden(Show possible solution):
Using the instant rule from the text: $$rac{1}{15} + rac{-1}{10} = rac{1 	imes 10 + 15 	imes (-1)}{15 	imes 10} = rac{10 - 15}{150} = rac{-5}{150} = rac{-1}{30}$$

What has happened here? What have we "really done" with our chunks of apple?
Have a think; we'll see a lot more of this when we get to [56x subtraction].
%%

## Second extension exercise: what rational number must we add to $rac{7}{8}$ to obtain $rac{13}{8}$?

You're looking for an answer that looks like $rac{a}{b}$ where there are integers in place of $a$ and $b$.

%%hidden(Show some possible solutions):

Something you can do to make this question easier is to notice that both the numbers have the same chunk-size (namely $rac{1}{8}$), so we might try adding some number of $rac{1}{8}$-chunks.
Then we're trying to get from $7$ chunks to $13$ chunks, so we need to add $6$ chunks.

That is, the final number is $rac{6}{8}$ (which is also $rac{3}{4}$).
%%

## Third extension exercise: what rational number must we add to $rac{7}{8}$ to obtain $rac{13}{7}$?

You're looking for an answer that looks like $rac{a}{b}$ where there are integers in place of $a$ and $b$.

%%hidden(Show some possible solutions):

Now, the numbers are no longer of the same chunk-size, so we should make it so that they are of the same size.

The chunk-size to use is $rac{1}{8 	imes 7} = rac{1}{56}$. The reason for this is the same as the reasoning we saw when working out how to add $rac{1}{8}$ and $rac{1}{7}$.

Then the two numbers are $rac{7 	imes 7}{7 	imes 8} = rac{49}{56}$ and $rac{8 	imes 13}{8 	imes 7} = rac{104}{56}$, so the answer is that we need to get from $49$ to $104$; to do that, we need to add $55$ chunks of size $rac{1}{56}$, so the answer is $rac{55}{56}$.
%%


Well-ordered set

A **well-ordered set** is a [-540] $(S, \leq)$, such that for any nonempty subset $U \subset S$ there is some $x \in U$ such that for every $y \in U$, $x \leq y$; that is, every nonempty subset of $S$ has a least element.

Any finite totally ordered set is well-ordered. The simplest [infinity infinite] well-ordered set is [45h $\mathbb N$], also called [ordinal_omega $\omega$] in this context.

Every well-ordered set is [4f4 isomorphic] to a unique [-ordinal_number], and thus any two well-ordered sets are comparable.

The order $\leq$ is called a "well-ordering," despite the fact that "well" is usually an adverb.

#Induction on a well-ordered set

[mathematical_induction] works on any well-ordered set. On well-ordered sets longer than $\mathbb N$, this is called [-transfinite_induction].

Induction is a method of proving a statement $P(x)$ for all elements $x$ of a well-ordered set $S$. Instead of directly proving $P(x)$, you prove that if $P(y)$ holds for all $y < x$, then $P(x)$ is true. This suffices to prove $P(x)$ for all $x \in S$.

%%hidden(Show proof):
Let $U = \{x \in S \mid
eg P(x) \}$ be the set of elements of $S$ for which $P$ doesn't hold, and suppose $U$ is nonempty. Since $S$ is well-ordered, $U$ has a least element $x$. That means $P(y)$ is true for all $y < x$, which implies $P(x)$. So $x
ot\in U$, which is a contradiction. Hence $U$ is empty, and $P$ holds on all of $S$.
%%


Needs parent

This page is not attached to an appropriate parent page.


Löb's theorem

[summary: If $PAdash Prv_{PA}(A)\implies A$ then $PAdash A$]

We trust Peano Arithmetic to correctly capture certain features of the [ standard model of arithmetic]. Furthermore, we know that Peano Arithmetic is expressive enough to [31z talk about itself] in meaningful ways. So it would certainly be great if Peano Arithmetic asserted what now is an intuition: that everything it proves is certainly true.

In formal notation, let $Prv$ stand for the [-5gt] of $PA$. Then, $Prv(T)$ is true if and only if there is a proof from the axioms and rules of inference of $PA$ of $T$. Then what we would like $PA$ to say is that $Prv(S)\implies S$ for every sentence $S$.

But alas, $PA$ suffers from a problem of self-trust.

Löb's theorem states that if $PAdash Prv(S)\implies S$ then $PAdash S$. This immediately implies that if $PA$ is consistent, the sentences $PAdash Prv(S)\implies S$ are not provable when $S$ is false, even though according to our intuitive understanding of the standard model every sentence of this form must be true.

Thus, $PA$ is incomplete, and fails to prove a particular set of sentences that would increase massively our confidence in it.

Notice that [godels_second_incompleteness_theorem Gödel's second incompleteness theorem] follows immediately from Löb's theorem, as if $PA$ is consistent, then by Löb's $PA
vdash Prv(0= 1)\implies 0= 1$, which by the propositional calculus implies $PA
vdash
eg Prv(0= 1)$.

It is worth remarking that Löb's theorem does not only apply to the standard provability predicate, but to every predicate satisfying the [ Hilbert-Bernais derivability conditions].