Log as the change in the cost of communicating

When interpreting logarithms as [416 a generalization of the notion of "length"] and as [427 digit exchange rates], in both cases, multiplying the input to the logarithm base 10 by a factor of 10 caused the output to go up by one. Multiplying a number by 10 makes it one digit longer. If a [-42d] is worth \$1, then a 1000-digit is worth exactly \$1 more than a 100-digit, because you can [emulating_digits build] a 1000-digit out of a 100-digit and a 10-digit. Thus, by symmetry _dividing_ an input to the logarithm base 10 by 10 makes the output go down by one: If you divide a number by 10, it gets one digit shorter; and any $n$-digit is worth \$1 more than a $\frac{n}{10}$-digit, because you can build an $n$-digit out of a $\frac{n}{10}$-digit and a 10-digit.

This strongly implies that $\log_{10}(\frac{1}{10})$ should equal $-1$. If a 1000-digit costs \$3, and a 100-digit costs \$2, and a 10-digit costs \$1, and a [1digit 1-digit is worthless], then, extrapolating the pattern, a $\frac{1}{10}$ should cost $-\$1.$ But what does that mean? What sort of digit is worth negative money? Can we give this extrapolation a physical intuition?

Yes, we can, by thinking in terms of how difficult it is to communicate information. Let's say that you and I are in separate rooms, connected only by a conveyor belt, upon which I can place physical objects like coins, dice, and [42d digit wheels] that you can read. Let's imagine also that a third party is going to show me the whodunit of a game of [https://en.wikipedia.org/wiki/Cluedo Clue], and then let me put some objects on the conveyor belt, and then send those objects into your room, and then ask you for the information. If you can reproduce it successfully, then we both win a lot of money. However, I have to pay for every object that I put on the conveyor belt, using the [427 fair prices].

Consider how much I have to pay to tell you the result of a clue game. The "whodunit" in a clue game consists of three pieces of information:

1. The name of the murderer, which is one of: Miss Scarlett, Professor Plum, Mrs. Peacock, Reverend Green, Colonel Mustard, or Mrs. White.
2. The room in which the murder occurred, which is either the kitchen, the ballroom, the conservatory, the dining room, the cellar, the billiard room, the library, the lounge, the hall, or the study.
3. The murder weapon, which is either the candlestick, the dagger, the lead pipe, poison, the revolver, the rope, or the wrench.

Thus, a typical whodunit might look like "Professor Plum, in the conservatory, with the revolver." That sentence is 55 letters long, so one way for me to transmit the message would be to purchase fifty five 29-digits (capable of holding any one of 26 letters, or a space, or a comma, or a period), and send you that sentence directly. However, that might be a bit excessive, as there are in fact only $6 \cdot 10 \cdot 7 = 420$ different possibilities (six possible murderers, ten possible locations, seven possible weapons). As such, I only actually need to buy a 6-digit, a 10-digit, and a 7-digit. Equivalently, I could purchase a single 420-digit (if such things are on sale). We have to agree in advance what the digits mean — for example, "the 6-digit corresponds to the murderer, in the order listed above; the 10-digit corresponds to the room, in the order listed above; the 7-digit corresponds to the weapon, in the order listed above;" but assuming we do, I can get away with much less than fifty five 29-digits.

__Exercise:__ If the only storage devices on sale are coins, how many do I need to buy to communicate the whodunit?
%%hidden(Answer):
Nine. 8 coins only gets you 256 possibilities, and we need at least 420.
%%

__Exercise:__ If the only storage devices on sale are dice, how many do I need to buy?
%%hidden(Answer):
Four. $6^3 < 420 < 6^4.$
%%

__Exercise:__ If I have to choose between all coins or all dice, which should I choose, at the fair prices?
%%hidden(Answer):
The coins. Four dice cost as much as $\log_2(6) * 4 \approx 10.33$ coins, and we can do the job with nine coins instead.
%%

__Exercise:__ If I can mix coins, dice, and digit wheels, what's the cheapest way to communicate the whodunit?
%%hidden(Answer):
One coin and three dice let you send the message at a cost of only $\log_2(2) + 3\cdot \log_2(6) \approx 8.75$ coins.
%%

Now, consider what happens when the third party tells you "Actually, in order to win, you also have to communicate the name of my favorite Clue suspect, which is Colonel Mustard. I already told the person in the other room that you need to communicate two suspects, and that you'll communicate my favorite Clue suspect second. I didn't tell them who my favorite Clue suspect was, though."

Now, the space of possible messages has gone up by a factor of six: There are 420 possible whodunits, and each can be paired with one of six possible "favorite suspects," for a total of 2520 possible messages. How does this impact my cost of communicating with you? My cost goes up by 1 die ($= \log_2(6)$ coins $= \log_{10}(6)$ digit wheels). When the space of possibilities goes up by a factor of 6, my costs of communication (measured, say, in coins) go up by $\log_2(6).$

Now let's say that the third party comes back in the room and tells you "Actually, I gave the person in the other room a logic puzzle that told them which room the murder happened in; they solved it, and now they know that the murder happened in the conservatory."

This _reduces_ the space of possible messages I need to send, by a factor of 10. Now that both you and I know that the murder happened in the conservatory, I only need to transmit the murderer, the weapon, and the favorite suspect — one of 252 possibilities. The space of possibilities was cut into a tenth of its former size, and my cost of communicating dropped by 1 digit wheel ($= \log_6(10)$ dice $= \log_2(10)$ coins).

On this interpretation, logarithms are measuring how much it costs to transmit information, in terms of some "base" medium (such as coins, dice, or digit wheels). Every time the space of possibilities increases by a factor of $n$, my communication costs increase by $\log_2(n)$ coins. Every time the space of possibilities decreases by a factor of $n$, my communication costs _drop_ by $\log_2(n)$ coins.

This is the physical interpretation of logarithms that you can put your weight on: $\log_b(x)$ measures how much more or less costly it will be to send a message (in terms of $b$-digits) when the space of possible messages changes by a factor of $x$. Paired with a physical interpretation of [44l fractional digits], it can explain most of the [log_properties basic properties of the logarithm]:

1. $\log_b(1) = 0,$ because increasing (or decreasing) the space of possible messages by a factor of 1 doesn't affect your communication costs at all.
2. $\log_b(b) = 1,$ because increasing the space of possible messages by a factor of $b$ will increase your communication costs by exactly one $b$-digit.
3. $\log_b\left(\frac{1}{b}\right) = -1,$ because decreasing the space of possible messages by a factor of $b$ saves you one $b$-digit worth of communication costs.
4. $\log_b(x\cdot y) = \log_b(x) + \log_b(y),$ because if $n = x \cdot y$ then one $n$-digit is exactly large enough to store one $x$-message and one $y$-message. Thus, when communicating, an $x\cdot y$-digit is worth the same amount as one $x$-digit plus one $y$-digit.
5. $\log_b(x^n) = n \cdot log_b(x),$ because $n$ $x$-digits can be used to [emulating_digits emulate] one $x^n$-digit.

You might be thinking to yourself:

> Wait, what does it mean for the space of possible messages to go up or down by a factor of $x$? This isn't always clear. What if you're really good at guessing who people's favorite suspect is? For that matter, what if we haven't established a convention like "0 = Miss Scarlett; 1 = Professor Plum; ..."? If I see an observation, the amount by which it changes the space of possible messages is subjective; it depends on my beliefs and on the beliefs of the person I'm communicating with and on the conventions that we set up beforehand. How do you actually formalize this idea?

Those are great questions. Down that path lies [3qq], a field which measures communication costs using logarithms, and which lets us formalize (and quantize) ideas such as the amount of information carried by a message (to a given observer). See the [information_theory_tutorial] for more on this subject.

With regard to logarithms, the key idea here is an interpretation of what $\log_b(x)$ is "really doing." Given an input like "how many possible messages are there," such that your costs go up by 1 unit every time the input space increases by a factor of $b$, $\log_b(x)$ measures the change in cost when the input space increases by a factor of $x$. As we will see next, this idea generalizes beyond the domain of "set of possible messages vs cost of communicating," to _any_ scenario where some measure $\mu$ increases by $1$ every time some object scales by a factor of $b$, in which case $\log_b(x)$ measures the change in $\mu$ when the object scales by a factor of $x$. This is the defining characteristic of logarithms, and now that we have some solid physical interpretations of what it means, we're ready to start exploring logarithms in the abstract.


Lattice (Order Theory)

[summary:
A **lattice** is a [-3rb] that is closed under binary [3rc joins and meets]. It follows from this definition that lattices are closed under all *finite* joins and meets, and that for all lattices $L$ and elements $p, q \in L$, the following algebraic properties are satisfied:

* [3h4 Associativity] of joins and meets: $(p \vee q) \vee r = p \vee (q \vee r)$, and $(p \wedge q) \wedge r = p \wedge (q \wedge r)$
* [3jb Commutativity] of joins and meets: $p \vee q = q \vee p$ and $p \wedge q = q \wedge p$
* Idempotency of joins and and meets: $p \vee p = p$ and $p \wedge p = p$
* Absorption: $p \vee (p \wedge q) = p$ and $p \wedge (p \vee q) = p$
]

A **lattice** is a [-3rb] that is closed under binary [3rc joins and meets].

Let $L$ be a lattice. Then for all $p,q,r \in L$ the following properties are necessarily satisfied.

* [3h4 Associativity] of joins and meets: $(p \vee q) \vee r = p \vee (q \vee r)$, and $(p \wedge q) \wedge r = p \wedge (q \wedge r)$
* [3jb Commutativity] of joins and meets: $p \vee q = q \vee p$ and $p \wedge q = q \wedge p$
* Idempotency of joins and and meets: $p \vee p = p$ and $p \wedge p = p$
* Absorption: $p \vee (p \wedge q) = p$ and $p \wedge (p \vee q) = p$

%%hidden(Proofs):

Lemma 1: Let $P$ be a poset, $S \subseteq P$, and $p \in P$. If both $\bigvee S$ and $(\bigvee S) \vee p$ exist then $\bigvee (S \cup \{p\})$ exists as well, and $(\bigvee S) \vee p = \bigvee (S \cup \{p\})$.

Proof: See the *Join fu* exercise in [4ll].

## Associativity
Let $L$ be a lattice and $p,q,r,s \in L$ such that $s = p \vee (q \vee r)$. We apply the above lemma, along with commutativity and closure of lattices under binary joins, to get
$$p \vee (q \vee r) = (q \vee r) \vee p = (\bigvee \{q, r\}) \vee p = \bigvee (\{q, r\} \cup \{p\}) =$$
$$\bigvee \{ q, r, p \} = \bigvee (\{p, q\} \cup \{r\}) = (\bigvee \{p, q\}) \vee r = (p \vee q) \vee r.$$

By duality, we also have the associativity of binary meets.

## Commutativity

Let $L$ be a lattice and $p,q \in L$. Then $p \vee q = \bigvee \{ p, q \} = q \vee p$. Binary joins are therefore commutative. By duality, binary meets are also commutative.

## Idempotency

Let $L$ be a lattice and $p \in L$. Then $p \vee p = \bigvee \{ p \} = p$. The property that for all $p \in L$, $p \vee p = p$ is called *idempotency*. By duality, we also have the idempotency of meets: for all $p \in L$, $p \wedge p = p$.

## Absorption

Since $p \wedge q$ is the greatest *lower bound* of $\{p,q\}$, $p \wedge q \leq p$. Because $p \leq p$ and $(p \wedge q) \leq p$, $p$ is an upper bound of $\{p, p \wedge q\}$, and so $p \vee (p \wedge q) \leq p$. On the other hand, $p \vee (p \wedge q)$ is the least *upper bound* of $\{p, p \wedge q\}$, and so $p \leq p \vee (p \wedge q)$. By anti-symmetry,  $p = p \vee (p \wedge q)$.

%%

Closure under finite joins and meets
--------------------------------------------------------------

Let $L$ be a lattice and $S = \{ s_1, ..., s_n \}$ be some finite subset of $L$. Then an inductive argument shows that $\bigvee S$ exists.

%%hidden(Proof):
Here again, we will need Lemma 1, stated in the proofs of the four lattice properties.

Our proof proceeds by induction on the cardinality of $S$.

The base case is $\bigvee \{ s_1 \} = s_1 \in L$. For the inductive step, we suppose that $\bigvee \{s_1, ..., s_i \}$ exists. Then, applying lemma 1, we have $\bigvee \{s_1, ..., s_{i+1} \} = \bigvee \{s_1, ..., s_i \} \vee s_{i+1}$. Applying our inductive hypothesis and closure under binary joins, we have $\bigvee \{s_1, ..., s_i \} \vee s_{i+1}$ exists. Lattices are therefore closed under all *finite* joins, not just binary ones. Dually, lattices are closed under all finite meets.
%%

Basic positive examples
--------------

Here are two Hasse diagrams of posets which are lattices.

![A diamond shaped lattice](http://i.imgur.com/OlQnU07.png)

%%%comment:

dot source:

digraph G {
  node [width = 0.1, height = 0.1]
  edge [arrowhead = "none"]
  a [label = ""]
  b [label = ""]
  c [label = ""]
  d [label = ""]

  rankdir = BT;
  a -> b
  a -> c
  b -> d
  c -> d
}
%%%

![A cube shaped lattice](http://i.imgur.com/L0x074n.png)

%%%comment:

dot source:

digraph G {
  node [width = 0.1, height = 0.1]
  edge [arrowhead = "none"]
  a [label = ""]
  b [label = ""]
  c [label = ""]
  d [label = ""]
  e [label = ""]
  f [label = ""]
  g [label = ""]
  h [label = ""]

  rankdir = BT;
  a -> b
  a -> c
  a -> d
  b -> e
  b -> f
  c -> e
  c -> g
  d -> f
  d -> g
  e -> h
  f -> h
  g -> h
}

%%%

Basic negative examples
------------------------------------

Here are two Hasse diagrams of posets which are *not* lattices.

![A simple non-lattice](http://i.imgur.com/DAeuYz0.png)

%%%comment:

digraph G {
  node [width = 0.1, height = 0.1]
  edge [arrowhead = "none"]
  a [label = ""]
  b [label = ""]
  c [label = ""]
  d [label = ""]
  e [label = ""]
  f [label = ""]
  g [label = ""]
  h [label = ""]

  rankdir = BT;
  a -> b
  a -> c
  a -> d
  b -> e
  b -> f
  c -> e
  c -> g
  d -> f
  d -> g
  e -> h
  f -> h
  g -> h
}

%%%

In the above diagram, the two bottom elements have no common lower bounds. Therefore they have no meet, and so the depicted poset is not a lattice. However, it should be easy to verify that this poset is closed under binary joins.

![Another simple non-lattice](http://i.imgur.com/5Vqk87u.png)

%%%comment:

dot source:

digraph G {
  node [width = 0.1, height = 0.1]
  edge [arrowhead = "none"]
  a [label = ""]
  b [label = ""]
  c [label = ""]
  d [label = ""]

  rankdir = BT;
  a -> b
  c -> d
}

%%%

The Hasse diagram of this poset has two connected components. No element from the left component will have a meet or a join with any element from the right component. The depicted poset is therefore *not* a lattice.

The connecting lemma
--------------------

The connecting lemma states that for any lattice $L$ and $p,q \in L$, $p \vee q = p \Leftrightarrow q \leq p$ and dually, $p \wedge q = p \Leftrightarrow q \geq p$. This simple but important lemma is so named because it establishes a connection between the lattice's join operator and its underlying poset order.

%%hidden(Proof):

We prove $p \vee q = p \Leftrightarrow q \leq p$; the other part follows from duality. If $p \vee q = p$, then $p$ is an upper bound of both $p$ and $q$, and so $q \leq p$. Going the other direction, suppose $q \leq p$. Since $p$ is and upper bound of itself by reflexivity, it then follows that $p$ is an upper bound of $\{p, q\}$. There cannot be a lesser upper bound of $\{p, q\}$ because it would not be an upper bound of $p$. Hence, $p \vee q = p$.

%%

Lattices as algebraic structures
-------------------------------------

It's also possible to formulate lattices as [3gx algebraic structures] $\langle L, \vee, \wedge \rangle$ satisfying the associativity, commutativity, idempotency, and absorption laws described above. A poset $\langle L, \leq \rangle$ can then be defined such that for $p, q \in L$, $p \leq q$ whenever $p \vee q = q$. It can be shown that this poset is closed under binary meets and joins, and that these meets and joins are equal to the corresponding meets and joins of the algebraic lattice.

Additional material
-------------------------------

For more examples of lattices, see [574].
For some exercises involving the concepts introduced on this page, see [5ff].


Uncomputability

Imagine you were tasked with the mission of finding out which things can and cannot be calculated with a computer, given that you had infinite memory and infinite time.

Given the generality of computers, you might be tempted to say everything, but it turns out that this is not the case.

To begin tackling this problem, we need a formal notion of what it means to compute something. We are going to focus on computing **natural functions**; that is, [3jy functions] which transform [45h natural numbers] to natural numbers.

As natural numbers can be used to encode pretty much anything, this class of computations is far wider than one might initially think. For example, we can [-encode] finite sequences of arbitrary length of natural numbers in a single natural number, and we can encode words in sequences of numbers.

We are going to say that a computer program computes a certain natural function if on input $n$ it returns the result of applying such function to $n$, for every natural $n$. We note that programs do not have to halt on inputs; they may just run forever. In that case, we will say that the program is computing a partial function, which is not defined on some inputs.

%%%knows-requisite([2w0]):
Now, the first thing to notice is that the set of possible functions from $\mathbb{N}$ to $\mathbb{N}$ is not [-enumerable], while the set of possible programs, which are finite sequences of a finite quantity of symbols, is enumerable. Thus, there cannot be a one-to-one correspondence between functions and programs; there must necessarily exist a function which is not computed by any program.

So computers cannot compute everything!
%%%

----
##The diagonal function

We are going to construct a concrete example of a function which is not computable.

Imagine an infinite list of every possible Python program, associated with the infinite outputs each would produce if we feed it the sequence of natural numbers $1,2,3,...$.

Thus, we may end up with a table such as this:

program #1: 0->1, 1->X, 2->3,...

program #2:

etc...

Where an X means that the program does not halt on that input or does not return an output, and thus the function it represents is not defined there.

Now, we are going to construct an explicit function which is not computable using this table.

Let $DIAG$ be such that:
$$
DIAG(n) = \left\{
  \begin{array}{lr}
    1\ if\ M_n(n) = 0\\
    0\ if\ M_n(n)\mbox{ is undefined or defined and greater than 0}
  \end{array}
\right.
$$

The notation $M_n$ means *"the $n$th program in our enumeration of programs"*.
Let's see what we have done there. We are looking at the $n$th entry of every program and saying that this function on input $n$ outputs something different that the $n$th program. This technique is called [-diagonalization], and it is ubiquitous in computability and logic.

The function $DIAG$, known as the [-diagonal_function], is guaranteed to disagree with every program on at least one input. Thus, there cannot be a program which computes $DIAG$!

----
##The halting function
The reader may at this point be however not satisfied with such an unnatural example of an uncomputable function. After all, who is going to want to compute such a weird thing? Do not dismay, for there is a much better example: the halting function.

Let $HALT$ be defined as:
$$
HALT(n,x) = \left\{
  \begin{array}{lr}
    1\ if\ M_n(x)\mbox{ halts}\\
    0\ if\ M_n(x)\mbox{ does not halt }
  \end{array}
\right.
$$

That is, the function $HALT$ decides whether a given program is going to halt on a particular input. Now that is interesting.

To cite one imaginative use of the halting function, one could for example code a program which on any input simply ignores the input and starts looking for a counterexample of the Collatz conjecture, halting [-46m] it finds one. Then we could use the halting function to see whether the conjecture is true or false.

Sadly, $HALT$ is not computable. We are going to give two proofs of this fact, one by **reduction** to the diagonal function and other by **diagonalization**.

###Proof by reduction
Proofs by **reduction** are quite common in computability. We start by supposing that the function we are dealing with, in this case $HALT$, is computable, and then we proceed to use this assumption to show that if that was the case we could compute another function we know is uncomputable.

Suppose we have a program, which we are going to represent by $HALT$, which computes the halting function.

Then we could write a program such as this one:

    DIAGONAL(n):
        if HALT(n,n):
            return 1 if M_n==0 else return 0
    else return 0

Which computes the diagonal function. As we know such a program cannot possibly exist, we conclude by *reductio ad absurdum* that $HALT$ is not computable.

###Exercise
Prove that $HALT$ is reducible to $DIAGONAL$; ie, that if $DIAGONAL$ was computable we could compute $HALT$.

%%hidden(Show solution):
Suppose we want to know whether the program $PROG$ halts on input $n$. Then we first code the auxiliary program:

    AUX(x):
        PROG(n)
        return 0

That is, $AUX$ is the program which on any input first runs $PROG$ on input $n$ and then outputs $0$ if $PROG$ ever halts.

Then $DIAG(\ulcorner AUX\urcorner)==1$ iff $PROG$ halts on input $n$.
%%

###Proof by diagonalization
Another nifty proof of uncomputability comes from diagonalization.

As before, let's suppose $HALT$ is computable. Then we could write a program such as this one:

    prog(n):
        if HALT(n,n)==1:
           loop forever
        else return 0

The program halts on input $n$ iff $M_n$ does not halt on input $n$.

Now, let $\ulcorner prog\urcorner$ represent the number in the list of the program which occupies the $prog$ program. Here comes the question: does $prog$ halt when its input is $\ulcorner prog \urcorner$?

Supposing either yes or no leads to a contradiction, so we conclude that such a program cannot possibly exist, and it follows from the contradiction that $HALT$ is not computable.

---
So now you are ready to start your journey through the realm of computability!

Suggested next steps include examining other uncomputable functions such as the [ busy beaver sequence], or proceeding to [ complexity theory].


Iff

Iff is a shorthand for "if and only if". Its logical symbol is $\leftrightarrow$.

"A iff B" ($A \leftrightarrow B$) is quite distinct from "if A then B" ($A \rightarrow B$). Consider the stipulation "If the dog barks, then it will soon bite". This would not obligate the dog to bark a warning before biting. The "if" relation isn't symmetrical. As such, the dog might sometimes bite spontaneously, with no barking at all.

If we wanted to ensure that biting is always forewarned by barking, we would instead stipulate "**Iff** dog barks, then it will soon bite". This is equivalent to

* "the dog barks *if and only if* it will soon bite"

* "If the dog barks then it will soon bite, and if the dog bites it will have barked beforehand".

* "The dog barks only when it will soon bite"

With "iff", the implication runs in both directions.


Proof by contradiction

A proof by contradiction (a.k.a. *reductio ad absurdum*, reduction to absurdity) is a [5xz strategy] used by mathematicians to show that a mathematical statement is true by proving that the [-negation] of that statement leads to being able to prove that two opposite statements are simultaneously true (a [-contradiction]).

##Outline
The outline of the strategy is as follows:

1. Suppose that what you want to prove is false.
2. Derive a contradiction from it.
3. Conclude that the supposition is wrong.

##Examples
To illustrate the concept, we will do a simple, non rigorous reasoning. Imagine yourself in the next situation:

You are a defense lawyer. Your client is accused of stealing the cookie from the cookie jar. You want to prove her innocence. Lets say you have evidence that the jar is still sealed. Reason as follows:

1. Assume she stole the cookie from the cookie jar.
2. Then she would have had to open the jar.
3. The jar is still sealed.
4. For the jar to be sealed and for her to have opened it is a contradiction.
5. Hence the assumption in 1 is false (given the deductions below it are true).
6. Hence she did not steal the cookie from the cookie jar.

Now we will work through an actual mathematical example: we will show that $\sqrt 2$ is not [4zq rational]; that is, it cannot be expressed as the division of two [45h natural numbers].

1. We suppose that $\sqrt 2$ is rational, which means that there exist $a,b\in\mathbb{N}$ such that $\sqrt 2 = \frac{a}{b}$. Without loss of generality, we can suppose that $a$ and $b$ [coprime have no common divisors], since otherwise we can just divide both numbers by their greatest common divisor to obtain a pair of numbers which satisfy both properties.
2. If this was the case, $b\sqrt2=a$. by squaring both sides, we arrive to $2b^2=a^2$. But then $2$ divides $a$, so we can express $a$ as $2n$ for some $n\in\mathbb{N}$. Substituing in the previous expression, we arrive to $2b^2 = 4n^2\implies b^2 =2 n^2$. By the same reasoning, $2$ must divide $b$, but then $a$ and $b$ have a common divisor! We have a contradiction.
3. We conclude that our original assumption that $\sqrt 2$ is rational must be false, and thus $\sqrt 2$ is not rational.

##When to use it
Proof by contradiction is one of the most useful techniques one can use to prove anything.

In particular, if you get stuck while doing a proof, resorting to proof by contradiction is a great way to keep exploring a problem from a different perspective. Even if you do not get to solve the problem, you may get a useful insight about the problem when performing the procedure of proof by contradiction.

Also, trying to do proof by contradiction may result in a [ counterexample], which dissolves the problem in question.

[todo: exercises]


Derivative

The derivative of $y$ with respect to $x$ describes the [-rate] at which $y$ changes, given a change in $x$. In particular, we consider how tiny changes in one variable affect another variable. To take the derivative of a [-3jy], we can draw a line that is [-tangent] to a graph of the function. The slope of the tangent line is the value of the derivative at that point. The derivative of a function $f(x)$ is itself a function: it returns, for any $x$, the slope of the line that is tangent to $f(x)$ at the point $(x, f(x))$.

[toc:]

## Examples ##

 - The time-derivative of your car's mileage is your car's speed (because your car's speed is how quickly your car's mileage changes *over time*)

 - The time-derivative of your car's speed is your acceleration (because acceleration means how quickly your *speed* is changing over time)

 - The time-derivative of human population size is the birth rate minus the death rate (because if we take the birth rate minus the death rate, that tells how quickly the population is changing over time)

 - The time-derivative of wealth is income minus spending (because if we take income minus spending, that tells us how your wealth is changing over time)

 - The time-derivative of blood pumped through your heart is the flow rate through the aortic valve (rates in general describe how something changes over time)

 - The time-derivative of charge on a capacitor is the current flowing to it (currents in general are time-derivatives of how much total stuff has flowed)

 - The time-derivative of how good of a life you'll have lived is how happy you are right now (according to [hedonic utilitarians](https://foundational-research.org/hedonistic-vs-preference-utilitarianism/))

Okay, let's take another stab at this. Time-derivatives, or derivatives "with respect to time," describe how things change over time. We can take derivatives with respect to other things too.

 - The derivative of your car's mileage with respect to how much fuel you've burned is your miles per gallon (because your mpg describes how your mileage changes per unit of fuel you burn)

 - The derivative of the temperature of a pot of water with respect to how much heat you blast it with is called the heat capacity of water

 - The derivative of [potential energy](https://en.wikipedia.org/wiki/Potential_energy) with respect to altitude is gravitational force (a higher up object has some stored energy that it would release if it fell; a lower down object has less "potential energy." The change in potential energy as you change the altitude is why there's a force in the first place.)

There were two goals with all those examples, one explicit, and one covert. The explicit one was to give you a sense for what derivatives are. The covert one was to quietly suggest that you will never understand the the way the world works unless you understand derivatives. But hey, look at you! You kind of understand derivatives already! Let's get to the math now, shall we?

## Setting Up The Math ##

You just got your new car.

![](http://o.aolcdn.com/commerce/autodata/images/USC30TSC021B021001.jpg)
It's a [Tesla](http://waitbutwhy.com/2015/06/how-tesla-will-change-your-life.html) because you care about the environment almost as much as you care about looking awesome. Your mileage is sitting at 0. The world is your oyster. At time $t = 0$, you put your foot on the accelerator. For the next few seconds, your mileage will be $4.7 t^2$, where your mileage is in meters, and $t$ is in seconds since you pressed your foot on the accelerator. Now the first question is this: if that equation tells us how many meters we've traveled after how many seconds, how fast are we going at any given point in time?

The astute reader will have noticed that this was the first example of a time-derivative that we gave: the time-derivative of your car's mileage is your car's speed. Let's think about this *sans* math for a second. If we know where we are at any time, we should be able to figure out how fast we're going. There isn't any extra information we need. The only question is how. Well, we take the derivative of the mileage with respect to time to get our speed. In other words:

$$\frac{\mathrm{d}}{\mathrm{d} t} mileage = speed$$

That means the derivative with respect to $t$, where $t$ is the time in seconds. But we know what the mileage is, *in terms of $t$*. Our mileage is just $4.7 t^2$. So we can write:

$$\frac{\mathrm{d}}{\mathrm{d} t} 4.7 t^2 = speed$$

## Solving The Math ##

Sorry to leave you hanging for a sec, but we're going to start with something a little simpler.

![](http://i66.tinypic.com/1p87pl.png)
$$distance\ traveled = 2t$$

If this is the graph of how far someone has traveled after how many seconds, we can see that every second they go 2 more meters. In other words, they are traveling 2 meters per second, which you might notice is the slope of this line. In general, the derivative of a function is like the slope of the function when you graph it out.

This works fine if our function is something like $distance\ traveled = 2t$. What if our function isn't a line though. What if it's $distance\ traveled = t^2$?

![](http://i64.tinypic.com/htu0pv.png)

Things that aren't lines don't have slopes. So if this is a graph of our distance traveled over time, it's not as easy to see how fast we were going. But let's say we want to see how fast we were going at $t=1$. If we zoom in enough on that curve, it will start to flatten out into a straight line until we can't tell the difference. The slope of *that* line is what gives us our speed. The process of taking a curve like this one, and getting the "slope" at any given point is called "taking the derivative."

Let's take the derivative of $d = t^2$, where $d$ is the distance and $t$ is the time. (We'll take the derivative with respect to $t$). Prepare yourself, now take a look at the graph down there.

![](http://i67.tinypic.com/mcxloj.png)

We know how to find the slope of a line if we're given two points, so we're going to do that, and then slowly move the points together until they're on top of each other. The coordinates of the points are shown above, and we can calculate the slope pretty easily by doing $\frac{\Delta d}{\Delta t}$. This gives us a slope of 2.

Now let's say that our first point is at $(t,t^2)$, that our second point is $h$ units to the right, so it's coordinates are $((t+h),(t+h)^2)$. Now we have:
$$∆d=(t+h)^2-t^2$$
$$∆t=(t+h) - t$$
Algebradabra:
$$∆d=2ht + h^2$$
$$∆t=h$$
$$\frac{\Delta d}{\Delta t}=\frac{2ht + h^2}{h}=2t+h$$

Now as we make $h$ really small, the points get closer and closer together, and the slope of the line becomes $2t$. So when $t$ is $1$, the slope is $2$. And when $t$ is $5$, the slope is $10$.

We say the derivative of $t^2$ is $2t$. With similar logic, you can show that the derivative of $4.7t^2$ is $9.4t$. And *that* means that if you put your foot on the accelerator of your new Tesla at time $t=0$, your speed after $t$ seconds will be $9.4t$. After 1 second, you'll be traveling 9.4 meters per second. After 3 seconds, you'll be going 28.2 meters per second (or 64 mph).

## Concluding ##

That's what derivatives are. The Tesla case was just one example of actually finding the derivative of something. Obviously, if our distance traveled had been some totally different function of time, the derivative would have been different. We found that the derivative of $t^2$ is $2t$. Below is a list of other derivatives. You can imagine that if the function on the left was our distance traveled after a time $t$, the function on the right would be our speed at a time $t$. (All of the $c$'s and $n$'s are constants.)
$$\frac{\mathrm{d} }{\mathrm{d} t}c=0$$
$$\frac{\mathrm{d} }{\mathrm{d} t}ct=c$$
$$\frac{\mathrm{d} }{\mathrm{d} t}ct^2=2ct$$
$$\frac{\mathrm{d} }{\mathrm{d} t}ct^2=3ct^2$$
$$\frac{\mathrm{d} }{\mathrm{d} t}ct^n=nct^{n-1}$$
$$\frac{\mathrm{d} }{\mathrm{d} t}e^t=e^t$$
$$\frac{\mathrm{d} }{\mathrm{d} t}sin(t)=cos(t)$$
$$\frac{\mathrm{d} }{\mathrm{d} t}cos(t)=-sin(t)$$

If you're up for it, try to use the method we showed for solving derivatives to verify some of these. Good luck!

## See also

If you enjoyed this explanation, consider exploring some of [3d Arbital's] other [6gg featured content]!

Arbital is made by people like you, if you think you can explain a mathematical concept then consider [-4d6]!


The characteristic of the logarithm

[summary: Any time you find an output that adds whenever the input multiplies, you're probably looking at a (roughly) logarithmic relationship. For example, imagine storing a number using [42d digit wheels]. Every time the number goes up by a factor of 10, you need one additional digit wheel: It takes 3 wheels to store the number 500; 4 to store the number 5000; 5 to store the number 50000; and so on. Thus, the relationship between the magnitude of a number and the number of digits it takes to write down is logarithmic. This pattern is the key characteristic of the logarithm, and whenever you see an output adding when the input multiplies, you can measure the output using logarithms.]

Consider the interpretation of logarithms as [45q the cost of communicating a message]. Every time the number of possible messages to send doubles, your communication costs increase by the price of a coin, or whatever cheaper [-storage_medium] you have that can communicate one of two messages. It doesn't matter whether the number of possible messages goes from 4 to 8 or whether it goes from 4096 to 8192; in both cases, your costs go up by the price of a coin. It is the factor by which the set grew (or shrank) that affects the cost; not the absolute number of messages added (or removed) from the space of possibilities. If the space of possible messages halves, your costs go down by one coin, regardless of how many possibilities there were before the halving.

Algebraically, writing $f$ for the function that measures your costs, $c(x \cdot 2) =$ $c(x) + c(2),$ and, in general, $c(x \cdot y) =$ $c(x) + c(y),$ where we can interpret $x$ as the number of possible messages before the increase, $y$ as the factor by which the possibilities increased, and $x \cdot y$ as the number of possibilities after the increase.

This is the key characteristic of the logarithm: It says that, when the input goes up by a factor of $y$, the quantity measured goes up by a fixed amount (that depends on $y$). When you see this pattern, you can bet that $c$ is a logarithm function. Thus, whenever something you care about goes up by a fixed amount every time something else doubles, you can measure the thing you care about by taking the logarithm of the growing thing. For example:

- Consider the problem of checking whether a date is contained in a gigantic [sorted_list sorted list] of dates. You can do this by jumping to the middle of the list, seeing whether your date is earlier or later than the date in the middle, and thereby cutting the search space in half. Each time you do this, you cut the list of dates you're searching for in half, and so the total number of elements you need to look at goes up by one every time the size of the list doubles. Thus, the cost of searching an ordered list grows logarithmically in the size of the list. See also [binary_search].
- Consider a colony of bacteria where each bacterium in the colony reproduces once per day. Thus, the size of the colony roughly doubles each day. If you care about how long this colony of bacteria has been growing, you can measure the days by taking the logarithm of the number of bacteria in the colony. The logarithm (base 2) counts how many times the colony has doubled (and the log base 3 counts how many times it has tripled, and so on).
- The length of a number in [4sl] grows more-or-less logarithmically in the magnitude of the number: When the magnitude of the number goes up by a factor of 10, the number of digits it takes to write the number down grows by 1. However, this analogy is not perfect: Sometimes, multiplying a number by two does not increase its length (consider the number 300), and sometimes, dividing a number by 10 does not decrease its length by one digit (consider the number 1). See also [log_v_length Length isn't quite logarithmic].

Conversely, whenever you see a $\log_2$ in an equation, you can deduce that someone wants to measure some sort of thing by counting the number of doublings that another sort of thing has undergone. For example, let's say you see an equation where someone takes the $\log_2$ of a [-1rq]. What should you make of this? Well, you should conclude that there is some quantity that someone wants to measure which can be measured in terms of the number of doublings in that likelihood ratio. And indeed there is! It is known as [bayesian_evidence (Bayesian) evidence], and the key idea is that the strength of evidence for a hypothesis $A$ over its negation $\lnot A$ can be measured in terms of $2 : 1$ updates in favor of $A$ over $\lnot A$. (For more on this idea, see [ What is evidence?]).

In fact, a given function $f$ such that $f(x \cdot y) = f(x) + f(y)$ is almost guaranteed to be a logarithm function — modulo a few technicalities.

[checkbox(proove_things): This puts us in a position where you can derive all the main properties of the logarithm (such as $\log_b(x^n) = n \log_b(x)$ for any $b$) yourself. Check this box if that's something you're interested in doing.
y:
path: [4bz]
n:
]

[fixme: Conditional text depending what's next on the path.]


Group homomorphism

[summary: A group homomorphism is a function between [3gd groups] which "respects the group structure".]

[summary(Technical): Formally, given two groups $(G, +)$ and $(H, *)$ (which hereafter we will abbreviate as $G$ and $H$ respectively), a group homomorphism from $G$ to $H$ is a [-3jy] $f$ from the underlying set $G$ to the underlying set $H$, such that $f(a) * f(b) = f(a+b)$ for all $a, b \in G$.]

A group homomorphism is a function between [3gd groups] which "respects the group structure".

#Definition

Formally, given two groups $(G, +)$ and $(H, *)$ (which hereafter we will abbreviate as $G$ and $H$ respectively), a group homomorphism from $G$ to $H$ is a [-3jy] $f$ from the underlying set $G$ to the underlying set $H$, such that $f(a) * f(b) = f(a+b)$ for all $a, b \in G$.

#Examples

 - For any group $G$, there is a group homomorphism $1_G: G \to G$, given by $1_G(g) = g$ for all $g \in G$. This homomorphism is always [499 bijective].
 - For any group $G$, there is a (unique) group homomorphism into the group $\{ e \}$ with one element and the only possible group operation $e * e = e$. This homomorphism is given by $g \mapsto e$ for all $g \in G$. This homomorphism is usually not [4b7 injective]: it is injective if and only if $G$ is the group with one element. (Uniqueness is guaranteed because there is only one *function*, let alone group homomorphism, from any set $X$ to a set with one element.)
 - For any group $G$, there is a (unique) group homomorphism from the group with one element into $G$, given by $e \mapsto e_G$, the identity of $G$. This homomorphism is usually not [4bg surjective]: it is surjective if and only if $G$ is the group with one element. (Uniqueness is guaranteed this time by the property proved below that the identity gets mapped to the identity.)
 - For any group $(G, +)$, there is a bijective group homomorphism to another group $G^{\mathrm{op}}$ given by taking inverses: $g \mapsto g^{-1}$. The group $G^{\mathrm{op}}$ is defined to have underlying set equal to that of $G$, and group operation $g +_{\mathrm{op}} h := h + g$.
 - For any pair of groups $G, H$, there is a homomorphism between $G$ and $H$ given by $g \mapsto e_H$.
 - There is only one homomorphism between the group $C_2 = \{ e_{C_2}, g \}$ with two elements and the group $C_3 = \{e_{C_3}, h, h^2 \}$ with three elements; it is given by $e_{C_2} \mapsto e_{C_3}, g \mapsto e_{C_3}$. For example, the function $f: C_2 \to C_3$ given by $e_{C_2} \mapsto e_{C_3}, g \mapsto h$ is *not* a group homomorphism, because if it were, then $e_{C_3} = f(e_{C_2}) = f(gg) = f(g) f(g) = h h = h^2$, which is not true. (We have used that the identity gets mapped to the identity.)

# Properties

- The identity gets mapped to the identity. ([49z Proof.])
- The inverse of the image is the image of the inverse. ([4b1 Proof.])
- The [3lh image] of a group under a homomorphism is another group. ([4b4 Proof.])
- The composition of two homomorphisms is a homomorphism. ([4b6 Proof.])


Cyclic group

[summary: The cyclic [3gd groups] are the simplest kind of group; they are the groups which can be made by simply "repeating a single element many times". For example, the rotations of a polygon.]

[summary(technical):  A [3gd group] $G$ is **cyclic** if it has a single [generator_mathematics generator]: there is one [element_mathematics element] $g$ such that every element of the group is a [ power] of $g$.]

# Definition

A cyclic group is a group $(G, +)$ (hereafter abbreviated as simply $G$) with a single generator, in the sense that there is some $g \in G$ such that for every $h \in G$, there is $n \in \mathbb{Z}$ such that $h = g^n$, where we have written $g^n$ for $g + g + \dots + g$ (with $n$ terms in the summand).
That is, "there is some element such that the group has nothing in it except powers of that element".

We may write $G = \langle g \rangle$ if $g$ is a generator of $G$.

# Examples

 - $(\mathbb{Z}, +) = \langle 1 \rangle = \langle -1 \rangle$
 - The group with two elements (say $\{ e, g \}$ with identity $e$ with the only possible group operation $g^2 = e$) is cyclic: it is generated by the non-identity element. Note that there is no requirement that the powers of $g$ be distinct: in this case, $g^2 = g^0 = e$.
 - The integers [modular_arithmetic modulo] $n$ form a cyclic group under addition, for any $n$: it is generated by $1$ (or, indeed, by $n-1$).
 - The [497 symmetric groups] $S_n$ for $n > 2$ are *not* cyclic. This can be deduced from the fact that they are not [3h2 abelian] (see below).

# Properties

## Cyclic groups are [3h2 abelian]
Suppose $a, b \in G$, and let $g$ be a generator of $G$. Suppose $a = g^i, b = g^j$. Then $ab = g^i g^j = g^{i+j} = g^{j+i} = g^j g^i = ba$.

## Cyclic groups are [2w0 countable]
The elements of a cyclic group are nothing more nor less than $\{ g^0, g^1, g^{-1}, g^2, g^{-2}, \dots \}$, which is an enumeration of the group (possibly with repeats).


Integer

An **integer** is a [-54y] that can be represented as either a [-45h] or its [-additive_inverse]. -4, 0, and 1,003 are examples integers. 499.99 is not an integer. Integers are real numbers; they are [-4zq rational numbers]; and they are not [-fraction fractions] or [-decimal decimals].

## A Mathier Definition ##

Instead of describing the properties of an integer we'll describe the membership rules for the [-3jz] $\mathbb{Z}$. After we're done, anything that's been allowed into $\mathbb{Z}$ counts as an integer.

Start by putting $0$ and $1$ into $\mathbb{Z}$. Now, pick an element of $\mathbb{Z}$, pick another element of $\mathbb{Z}$, and add them together (you can pick the same element twice). Is that number in $\mathbb{Z}$ yet? No?! Well let's put it in there fast. We can do the same thing as before except instead of adding, we subtract, and if the difference isn't in $\mathbb{Z}$ yet, we put it in there. Anything that could be let into $\mathbb{Z}$ with these procedures is an integer.

This is not an efficient algorithm for building out $\mathbb{Z}$, but it does show the primary motivation for having integers in the first place. Natural numbers (positive integers) are closed under addition, meaning that if you add any two elements in the set, the sum will be in the set, but natural numbers are not closed under subtraction. Integers are what you get when you expand natural numbers to make a set that is closed under subtraction as well.

## Formal construction

Given access to the set $\mathbb{N}$ of [45h natural numbers], we may construct $\mathbb{Z}$ as follows.
Take the collection of all pairs $(a, b)$ of natural numbers, and take the [-quotient] by the [-53y] $\sim$ such that $(a,b) \sim (c,d)$ if and only if $a+d = b+c$.
(The intuition is that the pair $(a,b)$ stands for the integer $a-b$, and we take the quotient so that any given integer has just one representative.)

Writing $[a,b]$ for the equivalence class of the pair $(a,b)$, we define the [-55j] structure as:

- $[a,b] + [c,d] = [a+c,b+d]$
- $[a, b] \times [c, d] = [ac+bd, bc+ad]$
- $[a,b] \leq [c,d]$ if and only if $a+d \leq b+c$.

This does define the structure of a totally ordered ring ([ proof]).


Square visualization of probabilities on two events

$$
\newcommand{\true}{\text{True}}
\newcommand{\false}{\text{False}}
\newcommand{\bP}{\mathbb{P}}
$$

[summary:
$$
\newcommand{\true}{\text{True}}
\newcommand{\false}{\text{False}}
\newcommand{\bP}{\mathbb{P}}
$$

We can represent a [joint_probability_distribution_on_event probability distribution] $\bP(A,B)$ over  two [event_probability events] $A$ and $B$ as a square:

We could also represent $\bP$ by [factoring_probability factoring], so using $\bP(A,B) = \bP(A)\; \bP(B \mid A)$ we'd make this picture:

]

Say we have two [event_probability events], $A$ and $B$, and a [joint_probability_distribution_on_event probability distribution] $\bP$ over whether or not they happen. We can represent $\bP$ as a square:

So for example, the [1rh probability] $\bP(A,B)$ of both $A$ and $B$ occurring is the ratio of \[the area of the dark red region\] to \[the area of the entire square\]:

Visualizing probabilities in a square is neat because we can draw simple pictures that highlight interesting facts about our probability distribution.

Below are some pictures illustrating:

* [4cf independent events] (What happens if the columns and the rows in our square *both* line up?)

* [marginal_probability marginal probabilities] (If we're looking at a square of probabilities, where's the probability $\bP(A)$ of $A$ or the probability $\bP(\neg B)$?)

* [1rj conditional probabilities] (Can we find in the square the probability $\bP(B \mid A)$ of $B$ if we condition on seeing $A$? What about the conditional probability $\bP(A \mid B)$?)

* [factoring_probability factoring a distribution] (Can we always write $\bP$ as a square? Why do the columns line up but not the rows?)

* the process of computing [1rh joint probabilities] from [factoring_probability factored probabilities]

Independent events
===

Here's a picture of the joint distribution of [4cf two independent events] $A$ and $B$:

Now the rows for $\bP(B)$ and  $\bP(\neg B)$ line up across the two columns. This is because $\bP(B \mid A) = \bP(B) = \bP(B \mid \neg A)$. When $A$ and $B$ are independent, updating on $A$ or $\neg A$ doesn't change the probability of $B$.

For more on this visualization of independent events, see the aptly named [4cl].

Marginal probabilities
===

We can see the [marginal_probability marginal probabilities] of $A$ and $B$ by looking at some of the blocks in our square. For example, to find the probability $\bP(\neg A)$ that $A$ doesn't occur, we just need to add up all the blocks where $\neg A$ happens: $\bP(\neg A) = \bP(\neg A, B) + \bP(\neg A, \neg B)$.

Here's the probability $\bP(A)$ of $A$, and the probability $\bP(\neg A)$ of $\neg A$:

Here's the probability $\bP(\neg B)$ of $\neg B$:

In these pictures we're dividing by the area of the whole square. Since the probability of anything at all happening is 1, we could just leave it out, but it'll be helpful for comparison while we think about conditionals next.

Conditional probabilities
===

We can start with some probability $\bP(B)$, and then *assume* that $A$ is true to get a [1rj conditional probability] $\bP(B \mid A)$ of $B$. Conditioning on $A$ being true is like restricting our whole attention to just the possible worlds where $A$ happens:

Then the conditional probability of $B$ given $A$ is the proportion of these $A$ worlds where $B$ also happens:

If instead we condition on $\neg A$, we get:

So our square visualization gives a nice way to see, at a glance, the conditional probabilities of $B$ given $A$ or given $\neg A$:

 We don't get such nice pictures for $\bP(A \mid B)$:

More on this next.

Factoring a distribution
===

Recall the square showing our joint distribution $\bP$:

Notice that in the above square, the reddish blocks for $\bP(A,B)$ and $\bP(A,\neg B)$ are the same width and form a column; and likewise the blueish blocks for $\bP(\neg A,B)$ and $\bP(\neg A,\neg B)$. This is because we chose to [factoring_probability factor] our probability distribution starting with $A$:

$$\bP(A,B) = \bP(A) \bP( B \mid A)\ .$$

Let's use the [event_variable_equivalence equivalence] between [event_probability events] and [binary_variable binary random variables], so if we say $\bP( B= \true \mid A= \false)$ we mean $\bP(B \mid \neg A)$. For any choice of truth values $t_A \in \{\true, \false\}$ and $t_B \in \{\true, \false\}$, we have

$$\bP(A = t_A,B= t_B) = \bP(A= t_A)\; \bP( B= t_B \mid A= t_A)\ .$$

The first factor $\bP(A = t_A)$ tells us how wide to make the red column $(\bP(A = \true))$ relative to the blue column $(\bP(A = \false))$. Then the second factor $\bP( B= t_B \mid A= t_A)$ tells us the proportions of dark $(B = \true)$ and light $(B = \false)$ within the column for $A = t_A$.

We could just as well have factored by $B$ first:

$$\bP(A = t_A,B= t_B) = \bP(B= t_B)\; \bP( A= t_A \mid B= t_b)\ .$$

Then we'd draw a picture like this:

By the way, earlier when we factored by $A$ first, we got simple pictures of the probabilities $\bP(B \mid A)$ for $B$ conditioned on $A$.  Now that we're factoring by $B$ first, we have simple pictures for the conditional probability $\bP(A \mid B)$:

and for the conditional probability $\bP(A \mid \neg B)$:

Computing joint probabilities from factored probabilities
===

Let's say we know the factored probabilities for $A$ and $B$, factoring by $A$. That is, we know $\bP(A = \true)$, and we also know $\bP(B = \true \mid A = \true)$ and $\bP(B = \true \mid A = \false)$. How can we recover the joint probability $\bP(A = t_A, B = t_B)$ that $A = t_A$ is the case and also $B = t_B$ is the case?

Since

$$\bP(B = \false \mid A = \true) = \frac{\bP(A = \true, B = \false)}{\bP(A = \true)}\ ,$$

we can multiply the prior $\bP(A)$ by the conditional $\bP(\neg B \mid A)$ to get the joint $\bP(A, \neg B)$:

$$\bP(A = \true)\; \bP(B = \false \mid A = \true) = \bP(A = \true, B = \false)\ .$$

If we do this at the same time for all the possible truth values $t_A$ and $t_B$, we get back the full joint distribution:

[todo: information theory. a couple things, then point to another page. eg show example when two things have lots of mutual info.]


Symmetric group

The notion that group theory captures the idea of "symmetry" derives from the notion of the symmetric group, and the very important theorem due to Cayley that every group is a subgroup of a symmetric group.

# Definition

Let $X$ be a [-3jz]. A [499 bijection] $f: X \to X$ is a *permutation* of $X$.
Write $\mathrm{Sym}(X)$ for the set of permutations of the set $X$ (so its elements are functions).

Then $\mathrm{Sym}(X)$ is a group under the operation of composition of functions; it is the *symmetric group on $X$*.
(It is also written $\mathrm{Aut}(X)$, for the *automorphism group*.)

We write $S_n$ for $\mathrm{Sym}(\{ 1,2, \dots, n\})$, the *symmetric group on $n$ elements*.

# Elements of $S_n$

We can represent a permutation of $\{1,2,\dots, n\}$ in two different ways, each of which is useful in different situations.

## Double-row notation

Let $\sigma \in S_n$, so $\sigma$ is a function $\{1,2,\dots,n\} \to \{1,2,\dots,n\}$.
Then we write $$\begin{pmatrix}1 & 2 & \dots & n \\ \sigma(1) & \sigma(2) & \dots & \sigma(n) \\ \end{pmatrix}$$
for $\sigma$.
This has the advantage that it is immediately clear where every element goes, but the disadvantage that it is quite hard to see the properties of an element when it is written in double-row notation (for example, "$\sigma$ cycles round five elements" is hard to spot at a glance), and it is not very compact.

## Cycle notation

[49f Cycle notation] is a different notation, which has the advantage that it is easy to determine an element's order and to get a general sense of what the element does.
Every element of $S_n$ [49k can be expressed in (disjoint) cycle notation in an essentially unique way].

## Product of transpositions

It is a useful fact that every permutation in a (finite) symmetric group [4cp may be expressed] as a product of [4cn transpositions].

# Examples

- The group $S_1$ is the group of permutations of a one-point set. It contains the identity only, so $S_1$ is the trivial group.
- The group $S_2$ is isomorphic to the [-47y] of order $2$. It contains the identity map and the map which interchanges $1$ and $2$.

Those are the only two [3h2 abelian] symmetric groups.
Indeed, in cycle notation, $(123)$ and $(12)$ do not commute in $S_n$ for $n \geq 3$, because $(123)(12) = (13)$ while $(12)(123) = (23)$.

- The group $S_3$ contains the following six elements: the identity, $(12), (23), (13), (123), (132)$. It is isomorphic to the [-4cy] $D_6$ on three vertices. ([group_s3_isomorphic_to_d6 Proof.])

# Why we care about the symmetric groups

A very important (and rather basic) result is [49b Cayley's Theorem], which states the link between group theory and symmetry.

%%%knows-requisite([4bj]):
# Conjugacy classes of $S_n$

It is a useful fact that the conjugacy class of an element in $S_n$ is precisely the set of elements which share its [4cg cycle type]. ([4bh Proof.])
We can therefore [4bk list the conjugacy classes] of $S_5$ and their sizes.
%%%

# Relationship to the [-4hf]

The [-4hf] $A_n$ is defined as the collection of elements of $S_n$ which can be made by an even number of [4cn transpositions]. This does form a group ([4hg proof]).

%%%knows-requisite([4h6]):
In fact $A_n$ is a [-4h6] of $S_n$, obtained by taking the quotient by the [4hk sign homomorphism].
%%%


Bijective function

A bijective function is a [-3jy] which has an inverse. Equivalently, it is both [injective_function injective] and [surjective_function surjective].


Cayley's Theorem on symmetric groups

Cayley's Theorem states that every group $G$ appears as a certain subgroup of the [-497] $\mathrm{Sym}(G)$ on the [-3gz] of $G$.

# Formal statement

Let $G$ be a group.
Then $G$ is [49x isomorphic] to a subgroup of $\mathrm{Sym}(G)$.

# Proof

Consider the left regular [3t9 action] of $G$ on $G$: that is, the function $G \times G \to G$ given by $(g, h) \mapsto gh$.
This [49c induces a homomorphism] $\Phi: G \to \mathrm{Sym}(G)$ given by [-currying]: $g \mapsto (h \mapsto gh)$.

Now the following are equivalent:

- $g \in \mathrm{ker}(\Phi)$ the [49y kernel] of $\Phi$
- $(h \mapsto gh)$ is the identity map
- $gh = h$ for all $h$
- $g$ is the identity of $G$

Therefore the kernel of the homomorphism is trivial, so it is injective.
It is therefore bijective onto its [3lh image], and hence an isomorphism onto its image.

Since [4b4 the image of a group under a homomorphism is a subgroup of the codomain of the homomorphism], we have shown that $G$ is isomorphic to a subgroup of $\mathrm{Sym}(G)$.


Group action induces homomorphism to the symmetric group

Just as we can [currying curry] functions, so we can "curry" [47t homomorphisms] and [3t9 actions].

Given an action $\rho: G \times X \to X$ of group $G$ on set $X$, we can consider what happens if we fix the first argument to $\rho$. Writing $\rho(g)$ for the induced map $X \to X$ given by $x \mapsto \rho(g, x)$, we can see that $\rho(g)$ is a [499 bijection].

Indeed, we claim that $\rho(g^{-1})$ is an inverse map to $\rho(g)$.
Consider $\rho(g^{-1})(\rho(g)(x))$.
This is precisely $\rho(g^{-1})(\rho(g, x))$, which is precisely $\rho(g^{-1}, \rho(g, x))$.
By the definition of an action, this is just $\rho(g^{-1} g, x) = \rho(e, x) = x$, where $e$ is the group's identity.

We omit the proof that $\rho(g)(\rho(g^{-1})(x)) = x$, because it is nearly identical.

That is, we have proved that $\rho(g)$ is in $\mathrm{Sym}(X)$, where $\mathrm{Sym}$ is the [-497]; equivalently, we can view $\rho$ as mapping elements of $G$ into $\mathrm{Sym}(X)$, as well as our original definition of mapping elements of $G \times X$ into $X$.

# $\rho$ is a homomorphism in this new sense

It turns out that $\rho: G \to \mathrm{Sym}(X)$ is a homomorphism.
It suffices to show that $\rho(gh) = \rho(g) \rho(h)$, where recall that the operation in $\mathrm{Sym}(X)$ is composition of permutations.

But this is true: $\rho(gh)(x) = \rho(gh, x)$ by definition of $\rho(gh)$; that is $\rho(g, \rho(h, x))$ because $\rho$ is a group action; that is $\rho(g)(\rho(h, x))$ by definition of $\rho(g)$; and that is $\rho(g)(\rho(h)(x))$ by definition of $\rho(h)$ as required.


Cycle notation in symmetric groups

There is a convenient way to represent the elements of a [-497] on a finite set.

# $k$-cycle

A $k$-cycle is a member of $S_n$ which moves $k$ elements to each other cyclically.
That is, letting $a_1, \dots, a_k$ be distinct in $\{1,2,\dots,n\}$, a $k$-cycle $\sigma$ is such that $\sigma(a_i) = a_{i+1}$ for $1 \leq i < k$, and $\sigma(a_k) = a_1$, and $\sigma(x) = x$ for any $x \not \in \{a_1, \dots, a_k \}$.

We have a much more compact notation for $\sigma$ in this case: we write $\sigma = (a_1 a_2 \dots a_k)$.
(If spacing is ambiguous, we put in commas: $\sigma = (a_1, a_2, \dots, a_k)$.)
Note that there are several ways to write this: $(a_1 a_2 \dots a_k) = (a_2 a_3 \dots a_k a_1)$, for example.
It is conventional to put the smallest $a_i$ at the start.

Note also that a cycle's inverse is extremely easy to find: the inverse of $(a_1 a_2 \dots a_k)$ is $(a_k a_{k-1} \dots a_1)$.

For example, the double-row notation $$\begin{pmatrix}1 & 2 & 3 \\ 2 & 3 & 1 \\ \end{pmatrix}$$
is written as $(123)$ or $(231)$ or $(312)$ in cycle notation.

However, it is unclear without context which symmetric group $(123)$ lies in: it could be $S_n$ for any $n \geq 3$.
Similarly, $(145)$ could be in $S_n$ for any $n \geq 5$.

# General elements, not just cycles

Not every element of $S_n$ is a cycle. For example, the following element of $S_4$ has [4cq order] $2$ so could only be a $2$-cycle, but it moves all four elements:
$$\begin{pmatrix}1 & 2 & 3 & 4 \\ 2 & 1 & 4 & 3 \\ \end{pmatrix}$$

However, it may be written as the composition of the two cycles $(12)$ and $(34)$: it is the result of applying one and then the other.
Note that since the cycles are disjoint (having no elements in common), [49g it doesn't matter in which order we perform them].
It is a very important fact that [49k every permutation may be written as the product of disjoint cycles].
If $\sigma$ is a permutation obtained by first doing cycle $c_1 = (a_1 a_2 \dots a_k)$, then by doing cycle $c_2$, then cycle $c_3$, we write $\sigma = c_3 c_2 c_1$; this is by analogy with function composition, indicating that the first permutation to apply is on the rightmost end of the expression.
(Be aware that some authors differ on this.)

## Order of an element

Firstly, a cycle has [4cq order] equal to its length.
Indeed, the cycle $(a_1 a_2 \dots a_k)$ has the effect of rotating $a_1 \mapsto a_2 \mapsto a_3 \dots \mapsto a_k \mapsto a_1$, and if we do this $k$ times we get back to where we started.
(And if we do it fewer times - say $i$ times - we can't get back to where we started: $a_1 \mapsto a_{i+1}$.)

Now, suppose we have an element in disjoint cycle notation: $(a_1 a_2 a_3)(a_4 a_5)$, say, where all the $a_i$ are different.
Then the order of this element is $3 \times 2 = 6$, because:

- $(a_1 a_2 a_3)$ and $(a_4 a_5)$ are disjoint and hence commute, so $[(a_1 a_2 a_3)(a_4 a_5)]^n = (a_1 a_2 a_3)^n (a_4 a_5)^n$
- $(a_1 a_2 a_3)^n (a_4 a_5)^n$ is the identity if and only if $(a_1 a_2 a_3)^n = (a_4 a_5)^n = e$ the identity, because otherwise (for instance, if $(a_1 a_2 a_3)^n$ is not the identity) it would move $a_1$.
- $(a_1 a_2 a_3)^n$ is the identity if and only if $n$ is divisible by $3$, since $(a_1 a_2 a_3)$'s order is $3$.
- $(a_4 a_5)^n$ is the identity if and only if $n$ is divisible by $2$.

This reasoning generalises: the order of an element in disjoint cycle notation is equal to the [-least_common_multiple] of the lengths of the cycles.

# Examples

- The element $\sigma$ of $S_5$ given by first performing $(123)$ and then $(345)$ is $(345)(123) = (12453)$. Indeed, the first application takes $1$ to $2$ and the second application does not affect the resulting $2$, so $\sigma$ takes $1$ to $2$; the first application takes $2$ to $3$ and the second application takes the resulting $3$ to $4$, so $\sigma$ takes $2$ to $4$; the first application does not affect $4$ and the second application takes $4$ to $5$, so $\sigma$ takes $4$ to $5$; and so on.

This example suggests a general procedure for expressing a permutation which is already in cycle form, in *disjoint* cycle form. It turns out that [49k this can be done in an essentially unique way].

## Cycle type

The [-4cg] is given by taking the list of lengths of the cycles in the disjoint cycle form.


Disjoint cycles commute in symmetric groups

[summary: In a symmetric group, if we are applying a collection of permutations which are each disjoint cycles, we get the same result no matter the order in which we perform the cycles.]

Consider two [49f cycles] $(a_1 a_2 \dots a_k)$ and $(b_1 b_2 \dots b_m)$ in the [-497] $S_n$, where all the $a_i, b_j$ are distinct.

Then it is the case that the following two elements of $S_n$ are equal:

- $\sigma$, which is obtained by first performing the permutation notated by $(a_1 a_2 \dots a_k)$ and then by performing the permutation notated by $(b_1 b_2 \dots b_m)$
- $\tau$, which is obtained by first performing the permutation notated by $(b_1 b_2 \dots b_m)$ and then by performing the permutation notated by $(a_1 a_2 \dots a_k)$

Indeed, $\sigma(a_i) = (b_1 b_2 \dots b_m)[(a_1 a_2 \dots a_k)(a_i)] = (b_1 b_2 \dots b_m)(a_{i+1}) = a_{i+1}$ (taking $a_{k+1}$ to be $a_1$), while $\tau(a_i) = (a_1 a_2 \dots a_k)[(b_1 b_2 \dots b_m)(a_i)] = (a_1 a_2 \dots a_k)(a_i) = a_{i+1}$, so they agree on elements of $(a_1 a_2 \dots a_k)$.
Similarly they agree on elements of $(b_1 b_2 \dots b_m)$; and they both do not move anything which is not an $a_i$ or a $b_j$.
Hence they are the same permutation: they act in the same way on all elements of $\{1,2,\dots, n\}$.

This reasoning generalises to more than two disjoint cycles, to show that disjoint cycles commute.


Disjoint cycle notation is unique

[todo: state and prove the theorem]


Pi

Pi, usually written $π$, is a number equal to the ratio of a circle's [-circumference] to its [-diameter]. The value of $π$ is approximately $3.141593$.

![](http://rlv.zcache.ca/mathematical_definition_of_pi_ceramic_tiles-r78c5d7b1adfa46d7b24e2c8bc3bb3f65_agtk1_8byvr_324.jpg)

If the length of a curve seems like an ill-defined concept to you (maybe you only understand how lines could have lengths), consider bigger and bigger [-regular_polygon regular polygons] that make better and better approximations of the circle. As the number of sides $N$ of the polygon goes to $∞$, the perimeter will approach a length of $π$ times the diameter.

One could also define $π$ to be the area of a circle divided the area of a square, whose edge is the radius of the circle.

## What Kind of Number It Is ##

It's not an [48l].

![](https://sensemadehere.files.wordpress.com/2015/03/pi.png?w=584)

If the diameter here is 1, then the perimeter of the hexagon is 3, the perimeter of the square is 4, and the circumference of the circle is in between. There are no integers between 3 and 4.

It's [not rational](https://en.wikipedia.org/wiki/Proof_that_%CF%80_is_irrational), [nor is it algebraic](https://en.wikipedia.org/wiki/Lindemann%E2%80%93Weierstrass_theorem#Transcendence_of_e_and_.CF.80). It's transcendental.


Complexity theory

[summary: Study of the computational resources needed to solve a problem, usually time and memory]

**Complexity theory** is the study of the efficiency of algorithms with respect to several metrics, usually time and memory usage. Complexity theorists aim to classify different problems into [complexity_class classes of difficulty] and study the relations that hold between the classes.

When studying [ computability], we are concerned with the identification of that which is or not computable in an ideal sense, without worrying about time or memory limitations.

However, often in practice we have to be more pragmatic. A program which takes a [42m googol] years to run is not going to see much use. If you need more [ GbBytes] to solve a computational problem that atoms exist in the universe, you may as well go ahead and declare the problem unsolvable for all practical purposes.

**Complexity theory** raises the standards of computability in drawing the boundary between that which you can do with a computer and that which you cannot. It concerns the study of the [oh_notation asymptotic behavior] of programs when fed inputs of growing size, in terms of the resources they consume. The kind of resources with which complexity theorists work more often are the *time* a program takes to finish and the highest *memory usage* %%note:the memory usage is frequently called **space complexity**%% in any given point of the execution.

Complexity theory allows us to have a deeper understanding of what makes an algorithm efficient, which in turn allows us to develop better and faster algorithms. Surprisingly enough, it turns out that proving that some computational problems are hard to solve has incredibly important practical applications in [-cryptography].

----

The abstract framework in which we develop this theory are [ Turing machines] and [ decision problems]. In this context, the [ time complexity] is associated with the number of steps a TM takes to halt and return an output, while the [ space complexity] corresponds to the length of the tape we would need for the TM to never fall off when moving left or right.

One may worry that the complexity measures are highly dependent on the choice of computational framework used. After all, if we allow our TM to skip two spaces per step each time it moves it is going to take potentially half the time to compute something. However, the asymptotic behavior of complexity is [robustness_of_tm surprisingly robust], though there are [failure_of_strong_CT some caveats].

The most interesting characterization of complexity comes in the form of [ complexity classes], which break down the family of [ decision problems] into sets of problems which can be solved with particular constraints.

Perhaps the most important complexity class is [ $P$], the class of decision problems which can be efficiently computed %%note:For example, checking whether a graph is connected or not%%. The second best known class is [ $NP$], the class of problems whose solutions can be easily checked %%note:An example is factoring a number: it is hard to factor $221$, but easy to multiply $13$ and $17$ and check that $13 \cdot 17 = 221$%% . It is an open problem whether those two classes are [4bd one and the same]; that is, that every problem whose solutions are easy to check is also easy to solve.

There are many more important complexity classes, and it can be daunting to contemplate the sheer variety with which complexity theory deals. Feel free to take a guided tour though the [4b9 complexity zoo] if you want an overview of some of the most relevant.

An important concept is that of a [ reduction]. Some complexity classes have problems such that if you were able to solve them efficiently you could translate other problems in the class to this one and solve them efficiently. Those are called [ complete problems of a complexity class].

-----

This page is meant to be a starting point to learn complexity theory from an entry level. If there is any concept which feels mysterious to you, try exploring the greenlinks in their order of appearance. If you feel like the concepts presented are too basic, try a different lens.


Group isomorphism

A group isomorphism is a [-47t] which is [499 bijective].
We say that two groups are *isomorphic* if there is an isomorphism between them.

It turns out that isomorphism is a much more useful concept than true equality of [-3gd groups], and it captures the idea that "these two objects are the same group": the isomorphism shows us how to relabel the elements to see that they are indeed the same group.

For example, the trivial group is in some sense "the only group with one element", but it can be instantiated in many different ways: as $(\{ a \}, +_a)$, or $(\{ b \}, +_b)$, and so on (where $+_x$ is the [-3kb] taking $(x, x)$ to $x$).
They all behave in exactly the same ways for the purpose of group theory, but they are not literally identical.
They are all isomorphic, though: the map $\{a \} \to \{ b \}$ given by $a \mapsto b$ is an isomorphism of the respective groups.

Two groups are isomorphic if and only if they have the same [cayley_table Cayley table], possibly with rearrangement of rows/columns and with relabelling of elements.


Kernel of group homomorphism

The kernel of a [-47t] $f: G \to H$ is the collection of all elements $g$ in $G$ such that $f(g) = e_H$ the identity of $H$.

It is important to note that the kernel of any group homomorphism $G \to H$ is always a subgroup of $G$.
Indeed:

- if $f(g_1) = e_H$ and $f(g_2) = e_H$ then $e_H = f(g_1) f(g_2) = f(g_1 g_2)$, so the kernel is closed under $G$'s operation;
- if $f(x) = e_H$ then $e_H = f(e_G) = f(x^{-1} x) = f(x^{-1}) f(x) = f(x^{-1})$ (where we have used that [49z the image of the identity is the identity]), so inverses are contained in the putative subgroup;
- $f(e_G) = e_H$ because the image of the identity is the identity, so the identity is contained in the putative subgroup.

It turns out that the notion of "[-4h6]" coincides exactly with the notion of "kernel of homomorphism". ([4h7 Proof.])
The "kernel of homomorphism" viewpoint of normal subgroups is much more strongly motivated from the point of view of [-4c7]; Timothy Gowers [considers this to be the correct way](https://gowers.wordpress.com/2011/11/20/normal-subgroups-and-quotient-groups/) to introduce the teaching of normal subgroups in the first place.


Image of the identity under a group homomorphism is the identity

For any [-47t] $f: G \to H$, we have $f(e_G) = e_H$ where $e_G$ is the identity of $G$ and $e_H$ the identity of $H$.

Indeed, $f(e_G) f(e_G) = f(e_G e_G) = f(e_G)$, so premultiplying by $f(e_G)^{-1}$ we obtain $f(e_G) = e_H$.


Under a group homomorphism, the image of the inverse is the inverse of the image

For any [-47t] $f: G \to H$, we have $f(g^{-1}) = f(g)^{-1}$.

Indeed, $f(g^{-1}) f(g) = f(g^{-1} g) = f(e_G) = e_H$, and similarly for multiplication on the left.


The image of a group under a homomorphism is a subgroup of the codomain

Let $f: G \to H$ be a [-47t], and write $f(G)$ for the set $\{ f(g) : g \in G \}$.
Then $f(G)$ is a group under the operation inherited from $H$.

# Proof

To prove this, we must verify the group axioms.
Let $f: G \to H$ be a group homomorphism, and let $e_G, e_H$ be the identities of $G$ and of $H$ respectively.
Write $f(G)$ for the image of $G$.

Then $f(G)$ is closed under the operation of $H$: since $f(g) f(h) = f(gh)$, so the result of $H$-multiplying two elements of $f(G)$ is also in $f(G)$.

$e_H$ is the identity for $f(G)$: it is $f(e_G)$, so it does lie in the image, while it acts as the identity because $f(e_G) f(g) = f(e_G g) = f(g)$, and likewise for multiplication on the right.

Inverses exist, by "the inverse of the image is the image of the inverse".

The operation remains associative: this is inherited from $H$.

Therefore, $f(G)$ is a group, and indeed is a subgroup of $H$.


Expected value

The expected value of an action is the [-mean] numerical outcome of the possible results weighted by their [-1rf]. It may actually be impossible to get the expected value, for example, if a coin toss decides between you getting \$0 and \$10, then we say you get "\$5 in expectation" even though there is no way for you to get \$5.

The expectation of V (often shortened to "the expected V") is how much V you expect to get on average. For example, the expectation of a payoff, or an expected payoff, is how much money you will get on average; the expectation of the duration of a speech, or an expected duration, is how long the speech will last "on average."

Suppose V has discrete possible values, say $V = x_{1},$ or $V = x_{2}, ..., $ or $V = x_{k}$. Let $P(x_{i})$ refer to the probability that $V = x_{i}$. Then the expectation of V is given by:

$$\sum_{i=1}^{k}x_{i}P(x_{i})$$

Suppose V has continuous possible values, x. For instance, let $x \in \mathbb{R}$. Let $P(x)$ be the continuous probability distribution, or $\lim_{dx \to 0}$ of the probability that $x<V<(x+dx)$ divided by $dx$. Then the expectation of V is given by:

$$\int_{-∞}^{∞}xP(x)dx$$

## Importance ##

A common principle of reasoning under uncertainty is that if you are trying to achieve a good G, you should choose the act that maximizes the expectation of G.


The composition of two group homomorphisms is a homomorphism

Given two [47t group homomorphisms] $f: G \to H$ and $g: H \to K$, the composition $gf: G \to K$ is also a homomorphism.

To prove this, note that $g(f(x)) g(f(y)) = g(f(x) f(y))$ since $g$ is a homomorphism; that is $g(f(xy))$ because $f$ is a homomorphism.


Injective function

A [-3jy] $f: X \to Y$ is *injective* if it has the property that whenever $f(x) = f(y)$, it is the case that $x=y$. Given an element in the [3lm image], it came from applying $f$ to exactly one element of the [3js domain].

This concept is also commonly called being "one-to-one".
That can be a little misleading to someone who does not already know the term, however, because many people's natural interpretation of "one-to-one" (without otherwise having learnt the term) is that every element of the domain is matched up in a one-to-one way with *every* element of the domain, rather than simply with *some* element of the domain.
That is, a rather natural way of interpreting "one-to-one" is as "[499 bijective]" rather than "injective".

# Examples

- The function $\mathbb{N} \to \mathbb{N}$ (where $\mathbb{N}$ is the set of [-45h natural numbers]) given by $n \mapsto n+5$ is injective: since $n+5 = m+5$ implies $n = m$. Note that this function is not [4bg surjective]: there is no natural number $k$ such that $k+5 = 2$, for instance, so $2$ is not in the range of the function.
- The function $f: \mathbb{N} \to \mathbb{N}$ given by $f(n) = 6$ for all $n$ is not injective: since $f(1) = f(2)$ but $1 \not = 2$, for instance.


Complexity theory: Complexity zoo

Welcome, visitor! Get ready to exercise your sense of wonder and expand your mind on the **zoo of complexity**.

That where are the cages of the animals, you ask? No cage could contain our specimens, I must say. But you can get a glimpse of their nature through the chalky definitions in each of those blackboards. Where shall we go first?

----

Oh yes! [ $P$], the class of problems solvable in polynomial time. A popular creature in our installations. It is commonly accepted that this  class encompasses the problems which can be solved efficiently. Now, bear in mind that the classes in this zoo only classify problems attending to how fast the computational power you need to solve them grows compared to the size of the problem you are solving! That means, even if all grow at the same polynomial pace, some of the problems who lurk inside this class need a huge overload of resources from the very beginning, which makes them intractable! Not to talk about the multiplicative factor, which also gets ignored in making this distinction between classes, or the degree of the polynomial which upper bounds the function! Beliefs like that $x$ grows as fast as $1000 x^{42}+10^{100}$ can be misleading if not properly understood.

However, in practice such instances are rare, and $P$ is still used thorough all computer science as a synonym of efficiency - though they pay special attention to the degree and distinguish between functions in $\mathcal{O}(n)$ and functions in $\mathcal{O}(n*log(n))$, even though all programs associated fall within $P$.

----

The next creature is without a doubt the most famous of the zoo! Meet [ $NP$], the class of problems whose solution are easily checkable. This little creature is the responsible of modern cryptography, which relies on [one_way_functions functions easy to compute but hard to invert].

$NP$ also is the main star, together with $P$, of one great show in the zoo. Oh, look! It is about to start!...

*Announcer*: Ladies, gentlemen and children who are today with us! We present to you the main attraction of our bestiary, the infamous [ $P$ vs $NP$]!

*Announcer*: You may already have seen both of those creatures, but are they two different beasts? Or only two aspects of the same thing?

*Announcer*: Look again at them, and realize. If finding a solution to a problem is easy, then checking whether a given solution is true is also easy! You can always just find the solution again from scratch and check that both coincide. So we conclude that $P\subset NP$. Can the reverse inclusion be true as well?

*Announcer*: Imagine for a moment! The hard problems of creativity, solved at once through the clever application of an efficient algorithm! [Entscheidungsproble Automated short proof writing]! Instant reversal of one way functions!

*Announcer*: However, we suspect that they are different, though a proof has remained elusive through the years.

(the public cries: If we have been searching for decades for such a proof and none has been found then you should revise your beliefs and update against that! Why suspect otherwise?)

*Announcer*: We seem to have an exceptionally well educated audience today! Then again, we have been also on the search for a proof of the opposite without fruition. Why give this fact more weight in the Bayesian scheme of the things? There happens to be a somewhat convoluted reason for this. For if $P=NP$, then we would expect proofs to be easier to find, while if $P!=NP$ the opposite is true! So it only makes sense that is more probable that we live in the world where $P!=NP$%%note:For a formalization of this notion check [ natural proofs]. Also, see [4bf Arguments against $P=NP$]%%!

*Announcer*: And that's all for today, folks. Keep enjoying your visit though the zoo!

---

(To be continued)


Real number

A **real number** is any number that can be used to represent a physical quantity.

Intuitively, real numbers are any number that can be found between two [48l integers], such as $0,$ $1,$ $-1,$ $\frac{3}{2},$ $\frac{-7}{2},$ [49r $\pi,$] [e $e$], $100 \cdot \sqrt{2},$ and so on. The set of real numbers is written $\mathbb R.$ You can think of $\mathbb R$ as [4zq $\mathbb Q$] extended to include the [54z irrational numbers] like $\pi$ and $e$ which can be found between rational numbers but which cannot be completely written out in [-4sl].

## Definitions of the real numbers

The most commonly used definitions of the real numbers are constructions as extensions of the [4zq rational numbers], which involve either [53b Cauchy sequences] or [dedekind_cut Dedekind cuts].

### Cauchy sequences

Broadly speaking, a Cauchy sequence is a sequence where as the sequence goes on, all the elements past that point get closer and closer together. In the real numbers, every Cauchy sequence [convergence_analysis converges] to a real number. However, in the set of rational numbers, not all Cauchy sequences converge to a rational number.
In the set of rationals, a Cauchy sequence which does not converge to a rational number cannot really be said to "converge" at all: the set of rationals is "missing some of the points" that would be required to make every Cauchy sequence converge.

For example, the sequence of fractions of consecutive Fibonacci numbers $1/1, 2/1, 3/2, 5/3, 8/5, \ldots$ gets closer and closer to $\frac{1 + \sqrt{5}}{2}$, but cannot be said to converge to that number because it is not in the set of rational numbers.

For each of these non-convergent Cauchy sequences, we define a new irrational number to "fill in the gap", and for the Cauchy sequences that do converge, we define a real number equal to that rational number.

### Dedekind cuts

A Dedekind cut of a [-540] is a [-partition] of that set into two sets so that every element in the first set is [ less than] every element in the second set, and the second set has no smallest element. The latter restriction requires that the set also be a [ perfect set] (have no [isolated_point isolated points]), in the sense used in topology.

In the real numbers, such a partition will always have the first set having a greatest element, which is known as the least-upper-bound property. However, in the rational numbers, we might come across a partition where the first set does not have such an element.

For example, define a Dedekind cut $(A, B)$ of the rational numbers such that $B = \{x \in \mathbb{Q} \ | \ x > 0 \wedge x^2 > 2\}$ and $A$ is the complement of $B$. In plainer language, $B$ consists of all the numbers greater than $\sqrt{2}$, but because $\sqrt{2}$ doesn't exist in the space of rational numbers, we can't use that to formulate our definition. Obviously every element of $A$ is less than every element of $B$, but $A$ has no greatest element either, because we can create a sequence of numbers in $A$ that gets bigger and bigger (as it approaches $\sqrt{2}$) but never stops at a maximum value.

For each of these "strict cuts" where neither set has a "boundary element", we define a new irrational number to "fill in the gap", just like with the Cauchy sequences. For the Dedekind cuts where one of the sets does have a least or greatest element, we define a real number equal to that rational number.

This definition has the advantage that each real number is represented by a unique Dedekind cut, unlike the Cauchy sequences where multiple sequences can converge to the same number.


P vs NP

[summary: Are problems with solutions easy to check also easy to solve? ]

The greatest currently open problem in complexity theory and computer science in general, which talks about the relationship between the class of efficiently solvable problems, [ $P$], and the class of problems with easily checkable solutions, [ $NP$].

It is a basic result that [ $P\subseteq NP$]. However, we still do not know whether the reverse implication, $P \supseteq NP$, holds.

A positive, constructive result would have profound implications through all the fields of science and math, not to mention the whole of society.


P vs NP: Arguments against P=NP

[summary:

 - Proving $P \neq NP$ is harder if true than proving $P=NP$ if true.
 - After 50 years we haven't found any efficient algorithm for $NP$-complete problems.
 - The polynomial hierarchy is compared to the arithmetical hierarchy, and the analogy implies $P \neq NP$
]

#Natural proofs
If $P \neq NP,$ then there are results showing that lower bounds in complexity are inherently harder to prove in a [natural_proof technical yet natural sense]. In other words, if $P \neq NP$ then proving $P \neq NP$ is hard.

The opposite proposition for $P=NP$ is also expected to be true. That is, it would make sense that if $P=NP,$ then it should be easier to prove, since we could build far more efficient [ theorem provers].

#Empirical argument
We have been trying to get a fast algorithm for [ $NP$-complete] problems since the 70s, without success. And take into account that "we" does not only comprise a minor group of theoretical computer scientists, but a whole industry trying to get faster algorithms for commercial purposes.

One could also argue more weakly that if $P=NP$, then evolution could have made use of this advantage to sped up its search process, or create more efficient minds to solve $NP$-complete problems at thoughtspeed.

#The arithmetical hierarchy argument
Some authors have drawn an analogy between the [ polynomial hierarchy] and the [1mg arithmetical hierarchy].

There are results showing that the [ arithmetical hierarchy is strict], and if the analogy holds then we would have that the polynomial hierarchy is strict as well, [collapse_of_the_polynomial_hierarchy which automatically implies $P \neq NP$].


Surjective function

A function $f:A \to B$ is *surjective* if every $b \in B$ has some $a \in A$ such that $f(a) = b$.
That is, its [3lg codomain] is equal to its [3lh image].

This concept is commonly referred to as being "onto", as in "The function $f$ is onto."

# Examples

- The function $\mathbb{N} \to \{ 6 \}$ (where $\mathbb{N}$ is the set of [45h natural numbers]) given by $n \mapsto 6$ is surjective. However, the same function viewed as a function $\mathbb{N} \to \mathbb{N}$ is not surjective, because it does not hit the number $4$, for instance.
- The function $\mathbb{N} \to \mathbb{N}$ given by $n \mapsto n+5$ is *not* surjective, because it does not hit the number $2$, for instance: there is no $a \in \mathbb{N}$ such that $a+5 = 2$.


Conjugacy class is cycle type in symmetric group

In the [-497] on a finite set, the [-4bj] of an element is determined exactly by its [4cg cycle type].

More precisely, two elements of $S_n$ are [group_conjugate conjugate] in $S_n$ if and only if they have the same cycle type.

# Proof

## Same conjugacy class implies same cycle type
Suppose $\sigma$ has the cycle type $n_1, \dots, n_k$; write $$\sigma = (a_{11} a_{12} \dots a_{1 n_1})(a_{21} \dots a_{2 n_2}) \dots (a_{k 1} a_{k 2} \dots a_{k n_k})$$
Let $\tau \in S_n$.

Then $\tau \sigma \tau^{-1}(\tau(a_{ij})) = \tau \sigma(a_{ij}) = a_{i (j+1)}$, where $a_{i (n_i+1)}$ is taken to be $a_{i 1}$.

Therefore $$\tau \sigma \tau^{-1} = (\tau(a_{11}) \tau(a_{12}) \dots \tau(a_{1 n_1}))(\tau(a_{21}) \dots \tau(a_{2 n_2})) \dots (\tau(a_{k 1}) \tau(a_{k 2}) \dots \tau(a_{k n_k}))$$
which has the same cycle type as $\sigma$ did.

## Same cycle type implies same conjugacy class

Suppose $$\pi = (b_{11} b_{12} \dots b_{1 n_1})(b_{21} \dots b_{2 n_2}) \dots (b_{k 1} b_{k 2} \dots b_{k n_k})$$
so that $\pi$ has the same cycle type as the $\sigma$ from the previous direction of the proof.

Then define $\tau(a_{ij}) = b_{ij}$, and insist that $\tau$ does not move any other elements.

Now $\tau \sigma \tau^{-1} = \pi$ by the final displayed equation of the previous direction of the proof, so $\sigma$ and $\pi$ are conjugate.

# Example
This result makes it rather easy to [4bk list the conjugacy classes] of $S_5$.


Conjugacy class

Given an element $g$ of a [-3gd] $G$, the conjugacy class of $g$ in $G$ is $\{ x g x^{-1} : x \in G \}$.
It is the collection of elements to which $g$ is [4gk conjugate].

[todo: examples]
[todo: class equation]
[todo: it is the stabiliser of a certain action, which we can show conditionally on the right requisites]


Conjugacy classes of the symmetric group on five elements

The [-497] $S_5$ on five generators has size $5! = 120$ (where the exclamation mark denotes the [-factorial] function).
By the result that [4bh in a symmetric group, conjugacy classes coincide with cycle types], we can list the conjugacy classes of $S_5$ easily.

# The table

$$\begin{array}{|c|c|c|c|}
\hline
\text{Representative}& \text{Size of class} & \text{Cycle type} & \text{Order of element} \\ \hline
(12345) & 24 & 5 & 5 \\ \hline
(1234) & 30 & 4,1 & 4 \\ \hline
(123) & 20 & 3,1,1 & 3 \\ \hline
(123)(45) & 20 & 3,2 & 6 \\ \hline
(12)(34) & 15 & 2,2,1 & 2 \\ \hline
(12) & 10 & 2,1,1,1 & 2 \\ \hline
e & 1 & 1,1,1,1,1 & 1 \\ \hline
\end{array}$$

# Determining the list of cycle types and sizes

There are five elements to permute; we need to find all the possible ways of grouping them up into disjoint cycles.
We will go through this systematically.
Note first that since there are only five elements to permute, there cannot be any element with a $6$-cycle or higher.

## Those with largest cycle of length $5$

If there is a $5$-cycle, then it permutes everything, so its [4cg cycle type] is $5$.
That is, we can take a representative $(12345)$, and this is the only conjugacy class with a $5$-cycle.

Recall that every cycle of length $5$ may be written in five different ways: $(12345)$ or $(23451)$ or $(34512)$, and so on.
Without loss of generality, we may assume $1$ comes at the start (by cycling round the permutation if necessary).

Then there are $4!$ ways to fill in the remaining slots of the cycle (where the exclamation mark denotes the [-factorial] function).

Hence there are $24$ elements of this conjugacy class.

## Those with largest cycle of length $4$

If there is a $4$-cycle, then it permutes everything except one element, so its cycle type must be $4,1$ (abbreviated as $4$).
That is, we can take a representative $(1234)$, and this is the only conjugacy class with a $4$-cycle.

Either the element $1$ is permuted by the $4$-cycle, or it is not.

- In the first case, we have $4$ possible ways to pick the image $a$ of $1$; then $3$ possible ways to pick the image $b$ of $a$; then $2$ possible ways to pick the image $c$ of $b$; then $c$ must be sent back to $1$.
That is, there are $4 \times 3 \times 2 = 24$ possible $4$-cycles containing $1$.
- In the second case, the cycle does not contain $1$, so there are $3$ possible ways to pick the image $a$ of $2$; then $2$ possible ways to pick the image $b$ of $a$; then $1$ possible way to pick the image $c$ of $b$; then $c$ must be sent back to $2$.
So there are $3 \times 2 \times 1 = 6$ possible $4$-cycles not containing $1$.

This comes to a total of $30$ possible $4$-cycles in this conjugacy class.

## Those with largest cycle of length $3$

Now we have two possible conjugacy classes: $3,1,1$ and $3,2$.

### The $3,1,1$ class

An example representative for this class is $(123)$.

We proceed with a slightly different approach to the $4,1$ case.
Using the notation for the [-binomial_coefficient], we have $\binom{5}{3} = 10$ possible ways to select the numbers which appear in the $3$-cycle.
Each selection has two distinct ways it could appear as a $3$-cycle: the selection $\{1,2,3\}$ can appear as $(123)$ (or the duplicate cycles $(231)$ and $(312)$), or as $(132)$ (or the duplicate cycles $(321)$ or $(213)$).

That is, we have $2 \times 10 = 20$ elements of this conjugacy class.

### The $3,2$ class

An example representative for this class is $(123)(45)$.

Again, there are $\binom{5}{3} = 10$ possible ways to select the numbers which appear in the $3$-cycle; having made this selection, we have no further choice about the $2$-cycle.

Given a selection of the elements of the $3$-cycle, as before we have two possible ways to turn it into a $3$-cycle.

We are also given the selection of the elements of the $2$-cycle, but there is no choice about how to turn this into a $2$-cycle because, for instance, $(12)$ is equal to $(21)$ as cycles.
So this time the selection of the elements of the $3$-cycle has automatically determined the corresponding $2$-cycle.

Hence there are again $2 \times 10 = 20$ elements of this conjugacy class.

## Those with largest cycle of length $2$

There are two possible cycle types: $2,2,1$ and $2,1,1,1$.

### The $2,2,1$ class

An example representative for this class is $(12)(34)$.

There are $\binom{5}{2}$ ways to select the first two elements; then once we have done so, there are $\binom{3}{2}$ ways to select the second two.
Having selected the elements moved by the first $2$-cycle, there is only one distinct way to make them into a $2$-cycle, since (for example) $(12)$ is equal to $(21)$ as permutations; similarly the selection of the elements determines the second $2$-cycle.

However, this time we have double-counted each element, since (for example) the permutation $(12)(34)$ is equal to $(34)(12)$ by the result that [49g disjoint cycles commute].

Therefore there are $\binom{5}{2} \times \binom{3}{2} / 2 = 15$ elements of this conjugacy class.

### The $2,1,1,1$ class

An example representative for this class is $(12)$.

There are $\binom{5}{2}$ ways to select the two elements which this cycle permutes.
Once we have selected the elements, there is only one distinct way to put them into a $2$-cycle, since (for example) $(12)$ is equal to $(21)$ as permutations.

Therefore there are $\binom{5}{2} = 10$ elements of this conjugacy class.

## Those with largest cycle of length $1$

There is only the identity in this class, so it is of size $1$.


Needs work

Meta tag for pages which need content improvement.


Church-Turing thesis

[summary:Every [-effectively_computable] [-3jy] is [Turing_machine Turing-computable].]

The **Church-Turing thesis** (often abbreviated **CT thesis**) states:
> Every effectively computable function is Turing-computable

That is, for every function which can be computed by physical means %%note:So no [ hypercomputers]%% there exists a [ Turing machine] which computes exactly that.

The Church-Turing thesis is not a definite mathematical statement, but an inductive statement which affirms that every sensible model of computation we can come up with is equivalent or at least [reduction reducible] to the model proposed by Turing. Thus, we cannot prove it in a mathematical sense, but we can [ gather evidence for it].

For example, this model was proven to coincide with Church's lambda calculus, another universal model of computation, and the equivalence between Church's lambda calculus and Turing's automatic machines is often taken as [4j7 evidence] that they correctly capture our intuition of "effectively computable".

There are many [ consequences] of the CT thesis for computer science in general, artificial intelligence, [11w epistemology], and other fields of knowledge.