Encoding trits with GalCom bits

There are [3nd $\log_2(3) pprox 1.585$] [3p0 bits] to a [3ww]. Why is it that particular value? Consider the [+3tz] thought experiment, in which you make money by sending information about the stock market across the celestial reaches. Let's say that Company X is about to publish a big earnings report, and that you want to send a message to your stock trader in the Vega star system that says one of three things: "Performance was in the top 1/3 of what I expected," "Performance was in the middle 1/3 of what I expected," or "Performance was in the bottom 1/3 of what I expected." You obviously don't want to send whole sentences; it suffices to transmit a single [3ww] of information.

That is, if you had control of a communication channel that could hold one of three signals (say, a wire that could be positively charged, negatively charged, or uncharged) which was connected to your computer on Vega, then you could program the computer to recognize a positive charge as "top 1/3 performance", a neutral charge as "middle 1/3 performance", and a negative charge as "bottom 1/3 performance." You could then program it to buy, hold, or sell respectively, and make your money.

Unfortunately, you don't control a wire connected to your computer on Vega. The computer on Vega is in a different star system, and the only way to communicate with it is through GalCom, the galactic communications network. And GalCom doesn't sell trits (which let you transmit one of three messages) it sells [3y2 bits] (which let you transmit one of two messages).

So, how many bits on GalCom will it cost you to send your trit?

(You're invited to pause and find some method for encoding a trit using bits.)

You can easily transmit a trit via two bits. If you reserve two bits on GalCom, you get to control two of the pulses of light on the sender, by choosing wether they are present or absent. This affects whether the receiver writes down 0 or 1, and those two binary digits will be sent to your machine on Vega. Thus, if you purchase two bits on GalCom, you can send one of four messages to your machine on Vega: 00, 01, 10, or 11. This is more than enough to encode a trit: You can program the machine such that 00 causes it to sell, 01 causes it to hold, and 10 causes it to buy. Then you have one code left over, namely 11, that you'll never send.

![One trit encoded, inefficiently, using 2 bits](http://i.imgur.com/fKPRo2s.png)

_To encode a bit using trits, you could purchase two bits and then make use of only 3 of the 4 available codes._

However, this is inefficient: You purchased one code (11) that you are never using. Is there any way to repurpose this code to make some of your money back?

(You are invited to pause and find some method of encoding a trit using two bits such that, sometimes, you're able to sell one bit back to GalCom and recover some of your losses.)

Here's one thing you could do: Program your machine on Vega such that it recognizes both 10 and 11 as "top 1/3 performance."

![Encoding a trit using two bits, such that 1/3 of the time you can sell one bit back to GalCom.](http://i.imgur.com/AF49bHM.png)

Then, by the time you've transmitted a 1, the machine on Vega already knows to start purchasing stock: It doesn't need the next bit. This means that it doesn't matter (to you) whether you send 10 or 11; these are two different ways of representing the same message. Thus, if Company X has performance in the top 1/3 of your expectations, you can sell one bit back to GalCom: Send 10 if the next bit they were going to send was a 0, and 11 if the next bit they were going to send was a 1. You've just shaved one bit off of the next message that was going to come through GalCom!

(The above exposition implies that you need to know the first bit of the next message, and that your computer on Vega needs to be set up to forward the second bit to where it needs to go. In practice, this sort of thing would be standard operating procedure on GalCom, which could use [ prefix free codes] to avoid the need for seeing/forwarding bits of following messages.)

If you encode your trit this way, then you reserve 2 bits in advance, and 1/3 of the time you get to sell one back. Thus, if you send lots and lots of trits using this encoding, you only pay $2 - rac{1}{3} pprox 1.67$ galcoins per trit on average (assuming, [assume_independent_equally_likely_messages as we should], that each message occurs with equal frequency).

Can we do better? Yes. (TODO)


Logarithm tutorial overview

[summary: The logarithm tutorial covers the following six subjects:

1. What are logarithms?
2. Logarithms as a measure of length.
3. Logarithms as a measure of data and information.
4. The main charactristics and properties of logarithm functions.
4. All logarithm functions are the same (up to a multiplicative constant)
5. Why do logarithms make some calculations more efficient?]

The following topics are covered in Arbital's [3wj introductory guide to logarithms]:

## 1. Definition of the logarithm

What is a logarithm function, anyway? You may have been told that it's a function with a graph that looks like this:

![](https://upload.wikimedia.org/wikipedia/commons/thumb/1/17/Binary_logarithm_plot_with_ticks.svg/408px-Binary_logarithm_plot_with_ticks.svg.png "By Krishnavedala. Own work, CC BY-SA 3.0, goo.gl/VzKMzu")

which is true, but not the whole story. This tutorial begins by asking: [40j].

## 2. Log as length

Logarithms measure how long a number is, for a specific notion of "length" where fractional lengths are allowed. I don't know what $\log_{10}(\text{2,310,426})$ is, but I can immediately tell you that it's between 6 and 7, because it takes 7 digits to write 2,310,426:

$$\underbrace{\text{2,310,426}}_\text{7 digits}$$

Why is it $\log_{10}$ that measures length? What does it mean for a measure of "length" to be fractional? Why do logarithms say that it is 316 (rather than 500) that is 2.5 digits long? These questions and others are answered in [416].

## 3. Logs measure data

Logarithms measure how hard it is to represent messages using physical objects in the world. Let's say that we're magicians, and I want to tell you what card I'm thinking of by using sleight-of-hand to arrange some dice. How many 6-sided dice do I need to set (without the audience noticing) to send you a message that tells you which card I'm thinking of, assuming there are 52 possibilities? The lower bound is $\log_6(52).$ Why? The answer reveals a link between logarithms and the "data capacity" of physical objects.

This idea is explored across three pages: [427], [44l], and [45q].

## 4. The characteristic of the logarithm

 Any time you see a function $f$ such that $f(x \cdot y) = f(x) + f(y)$ for all $x, y \in$ [positive_reals $\mathbb R^+$], $f$ is a logarithm — more or less. [47k] describes the main idea, and the optional [4bz] page takes you through the proofs and discusses some technicalities.

## 5. There is only one logarithm

While there are many logarithm functions (one for each positive number except 1), there is a sense in which they're all doing exactly the same thing: Tapping into an intricate "[4gp logarithm lattice]".

## 6. Life in log-space

Empirically, logarithms have proved quite a useful tool for people and machines who have to do lots and lots of multiplications. Scientists and engineers used to use giant pre-computed tables of the logs of common numbers, and use those to make their calculations. Today, many modern learning algorithms (such as [https://en.wikipedia.org/wiki/AlphaGo AlphaGo]) manipulate the logs of probabilities instead of manipulating probabilities directly. Why? This tutorial concludes by exploring when and why it is easier to deal with the logarithms of things than it is to deal with the things themselves, on the page [4h0].

---

These six concepts are nowhere near all there is to say on the topic of logarithms. Logarithms have applications to many domains, including physics, computer science, calculus, number theory, and psychology. Logarithms have many interesting properties, such as nice [47d derivatives], nice [integral_calculus integrals], and interesting [calculating_logs approximation algorithms]. One of the bases of the logarithm, $\log_e,$ is "special." The logarithm gets [complex_log quite a bit more interesting] when extended to the [complex_plane complex plane]. Those are all more advanced topics, which aren't covered in this tutorial. If you want to learn more about those sorts of topics, see Arbital's [advanced_log_tutorial advanced logarithm tutorial].

This guide focuses on building a solid intuition for what the logarithm does, and why it has its most important properties.


What is a logarithm?

[summary: Logarithms are a group of [3jy functions] that take a number as input and produce another number. There is a logarithm function for every positive number $b$ [41m except 1], and the logarithm function for $b$ (called "the logarithm base $b$" or just "log base $b$") takes an input $x$ and counts how many times you have to multiply $1$ by $b$ to get $x$. For example, the log base 10 of 1000 is 3, because you have to multiply 1 by 10 three times to get 1000. The log base $b$ of $x$ is written $\log_b(x).$]

Logarithms are a group of [3jy functions] that map numbers to numbers. There is a logarithm function for every positive number $b$ [41m except 1], and the logarithm function for $b$ (called "the logarithm base $b$" or just "log base $b$") takes an input $x$ and counts how many times you have to multiply $1$ by $b$ to get $x$.

For example, the log base 10 of 1000 is 3, because you have to multiply 1 by 10 three times to get 1000, and the log base 2 of 16 is 4, because you have to multiply 1 by 2 four times to get 16. The log base $b$ of $x$ is written $\log_b(x).$

The logarithm function grows slowly. For example, $\log_{10}(x)$ is less than 1 until $x$ reaches 10, and it's less than 2 until $x$ reaches 100, and it's less than 6 until $x$ reaches 1,000,000 and so on. You've got to put a lot of oomph (specifically, a whole factor of 10) into the input in order to make the output go up by 1. Here's a graph of $\log_{10}(x)$ inching its way towards a measly 2 as $x$ makes it all the way out to 100:

![log10](http://i.imgur.com/UhUBlTt.png)

[fixme: Make this graph interactive.]

Logarithms may grow slowly, but they never stop growing: For every number $n$, no matter how large, there is an input $x$ such that $\log_{10}(x) > n.$

## Examples

1. $\log_{10}(10000) = 4,$ because you have to multiply 1 by 10 four times to get ten thousand: $1 \cdot 10 \cdot 10 \cdot 10 \cdot 10 = 10000.$
3. $\log_2(8) = 3,$ because $1 \cdot 2 \cdot 2 \cdot 2 = 8.$
4. $\log_3(9) = 2,$ because $1 \cdot 3 \cdot 3 = 9.$
5. $\log_{b}(1) = 0$ for any $b$, because no matter what $b$ is, you don't need to multiply 1 by $b$ any times to get 1.
6. $\log_{b}(b) = 1$ for any $b$, because no matter what $b$ is, if you multiply 1 by $b$ once, you get $b.$
7. $\log_{1.5}(3.375) = 3,$ because $1 \cdot 1.5 \cdot 1.5 \cdot 1.5 = 3.375.$

Question: What's $\log_3(27)$?

%%hidden(Answer):
Answer: 3, because $1 \cdot 3 \cdot 3 \cdot 3 = 27$.
%%

Question: What's $\log_4(16)$?

%%hidden(Answer):
Answer: 2, because $1 \cdot 4 \cdot 4 = 16$.
%%

Question: What's $\log_{10}(\text{1,000,000})$?

%%hidden(Answer):
Answer: 6, because $1 \cdot 10 \cdot 10 \cdot 10 \cdot 10 \cdot 10 \cdot 10 = 1,000,000$.
%%

All of the examples above are cases where the answer is a whole number, because $x$ is a [-power] of $b.$ Logarithms also give answers in cases where the answer is not a whole number. For example, $\log_{10}(500) \approx 2.7$.

What's _that_ supposed to mean? That if you multiply 1 by 10 about 2.7 times, you get 500?

Yes, precisely! If you multiply 1 by ten once, and multiply the result by ten another time, and then multiply that by ten 0.7 times, the result is roughly 500. What does it mean to multiply a number by ten 0.7 times? Well, is there a number $x$ such that if we multiply 1 by $x$ ten times, it's the same as multiplying 1 by 10 seven times? Yes! That number is roughly 5:

$$\underbrace{5 \cdot 5 \cdot \ldots 5}_\text{10 times} \approx \underbrace{10 \cdot 10 \cdot \ldots 10}_\text{7 times}$$

Multiplying by 5 (ten times) is about the same as multiplying by 10 (seven times), so multiplying by five (once) is about the same as multiplying by 10 (seven-tenths of a time). Thus, multiplying 1 by ten 2.7 times means multiplying it by 10 twice, then multiplying it by approximately 5, for a result of approximately 500.

This is why $\log_{10}(500) \approx 2.7$: that's how many times you need to multiply 1 by 10 to get 500.

If that answer isn't satisfactory yet, don't worry: Throughout the tutorial, we'll explore both (a) what this means, and (b) why this is the "right way" to multiply a number by $b$ fractionally-many times.

You might be saying to yourself, "Ok, I can see how 500 is kinda sorta one times ten 2.7 times, but how do you _figure that out?_ As far as I can tell, you pulled that "$5^{10}$ is approximately $10^7$" thing out of thin air. Where did that come from? How is the logarithm actually calculated?"

The annoying answer is that I calculated $\log_{10}(500)$ by typing "log10(500)" into [http://www.wolframalpha.com/input/?i=log10(500) Wolfram Alpha]. In more seriousness, in the coming posts, you'll learn how to find things like $\log_{10}(500)$ on your own.

That said, the outputs of logarithms are generally difficult to find, but useful once you've found them. Historically, scientists and engineers would pay money for giant tables of pre-computed logarithm values, because those sped up their practical calculations considerably. Why are the outputs of logarithm functions so convenient to use? That question will be answered later in this tutorial.

You might also be saying to yourself, "ok, I buy that there's a way to see 500 as '1 times 10, 2.7 times,' but what does that _mean?_ What's the point of looking at 500 that way?"

It's hard to give a short answer to that question, but only because there are so many good answers. One answer goes something like, "if you think of 500 as 2.7 tens, and you think of 8000 as 3.9 tens, then you can multiply the numbers on the left by adding the representations on the right". For example, $500 \cdot 8000$ is approximately $2.7 + 3.9 = 6.6$ tens.

But that's getting ahead of ourselves a bit. The most intuitive interpretation to start with is probably this one: Logarithms are measuring how "long" a number is, in terms of how many digits it takes to write that number down, for a generalized notion of "length."

It may seem that numbers always take a whole number of digits to write down: 139 and 931 are both written using three digits ('9', '3', and '1'). However, as we will see, there's a sense in which 139 is barely using its third digit, while 931 is using all three of its digits to almost their full extent. Logarithms quantify this intuition, and the precise answer that they give reveals some interesting facts about how many digits it costs to write a given number down.


Log as generalized length

[summary: 139 and 981 are both three digits long, but 139 is "almost" a 2-digit number and 981 is "almost" a 3-digit number. In a sense, 139 is using its three digits to a lesser extent. Logarithms formalize and quantify this intuition. You can estimate the logarithm (base 10) of a number by counting how many digits it has, and logarithms give us a notion of how much a number is actually using those digits.]

Here are a handful of examples of how the [-3nd] base 10 behaves. Can you spot the pattern?
$$
\begin{align}
\log_{10}(2) &\ \approx 0.30 \\
\log_{10}(7) &\ \approx 0.85 \\
\log_{10}(22) &\ \approx 1.34 \\
\log_{10}(70) &\ \approx  1.85 \\
\log_{10}(139) &\ \approx 2.14 \\
\log_{10}(316) &\ \approx 2.50 \\
\log_{10}(123456) &\ \approx 5.09 \\
\log_{10}(654321) &\ \approx 5.82 \\
\log_{10}(123456789) &\ \approx 8.09 \\
\log_{10}(\underbrace{987654321}_\text{9 digits}) &\ \approx 8.99
\end{align}
$$

Every time the input gets one digit longer, the output goes up by one. In other words, the output of the logarithm is roughly the length — measured in digits — of the input. ([437 Why?])

Why is it the log base 10 (rather than, say, the log base 2) that roughly measures the length of a number? Because numbers are normally represented in [-4sl], where each new digit lets you write down ten times as many numbers. The logarithm base 2 would measure the length of a number if each digit only gave you the ability to write down twice as many numbers. In other words, the log base 2 of a number is roughly the length of that number when it's represented in [-56q] (where $13$ is written $\texttt{1101}$ and so on):

$$
\begin{align}
\log_2(3) = \log_2(\texttt{11}) &\ \approx 1.58 \\
\log_2(7) = \log_2(\texttt{111}) &\ \approx 2.81 \\
\log_2(13) = \log_2(\texttt{1101}) &\ \approx 3.70 \\
\log_2(22) = \log_2(\texttt{10110}) &\ \approx 4.46 \\
\log_2(70) = \log_2(\texttt{<PHONE>}) &\ \approx 6.13 \\
\log_2(139) = \log_2(\texttt{<PHONE>}) &\ \approx 7.12 \\
\log_2(316) = \log_2(\texttt{<PHONE>}) &\ \approx 8.30 \\
\log_2(1000) = \log_2(\underbrace{\texttt{<PHONE>}}_\text{10 digits}) &\ \approx 9.97
\end{align}
$$

If you aren't familiar with the idea of numbers represented in other number bases besides 10, and you want to learn more, see the [-number_base_tutorial].

Here's an interactive visualization which shows the link between the length of a number expressed in base $b$, and the logarithm base $b$ of that number:

[visualization(log-length-demo):]

As you can see, if $b$ is an [-48l] greater than 1, then the logarithm base $b$ of $x$ is pretty close to the number of digits it takes to write $x$ in base $b.$

Pretty close, but not exactly. The most obvious difference is that the outputs of logarithms generally have a fractional portion: the logarithm of $x$ always falls a little short of the length of $x.$ This is because, insofar as logarithms act like the "length" function, they generalize the notion of length, making it [continuity continuous].

What does this fractional portion mean? Roughly speaking, logarithms measure not only how long a number is, but also how much that number is _really_ using its digits. 12 and 99 are both two-digit numbers, but intuitively, 12 is "barely" two digits long, whereas 97 is "nearly" three digits. Logarithms formalize this intuition, and tell us that 12 is really only using about 1.08 digits, while 97 is using about 1.99.

Where are these fractions coming from? Also, looking at the examples above, notice that $\log_{10}(316) \approx 2.5.$ Why is it 316, rather than 500, that logarithms claim is "2.5 digits long"? What would it even _mean_ for a number to be 2.5 digits long? It very clearly takes 3 digits to write down "316," namely, '3', '1', and '6'. What would it mean for a number to use "half a digit"?

Well, here's one way to approach the notion of a "partial digit." Let's say that you work in a warehouse recording data using digit wheels like they used to have on old desktop computers.

![A digit wheel](http://www.cl.cam.ac.uk/~djg11/howcomputerswork/mechanical-counter3.jpg)

Let's say that one of your digit wheels is broken, and can't hold numbers greater than 4 — every notch 5-9 has been stripped off, so if you try to set it to a number between 5 and 9, it just slips down to 4. Let's call the resulting digit a [4sj 5-digit], because it can still be stably placed into 5 different states (0-4). We could easily call this 5-digit a "partial 10-digit."

The question is, how much of a partial 10-digit is it? Is it half a 10-digit, because it can store 5 out of 10 values that a "full 10-digit" can store? That would be a fine way to measure fractional digits, but it's not the method used by logarithms. Why? Well, consider a scenario where you have to record lots and lots of numbers on these digits (such that you can tell someone how to read off the right data later), and let's say also that you have to pay me one dollar for every digit that you use. Now let's say that I only charge you 50¢ per 5-digit. Then you should do all your work in 5-digits! Why? Because two 5-digits can be used to store 25 different values (00, 01, 02, 03, 04, 10, 11, ..., 44) for \$1, which is way more data-stored-per-dollar than you would have gotten by buying a 10-digit.%note:You may be wondering, are two 5-digits really worth more than one 10-digit? Sure, you can place them in 25 different configurations, but how do you encode "9" when none of the digits have a "9" symbol written on them? If so, see [ The symbols don't matter].%

In other words, there's a natural exchange rate between $n$-digits, and a 5-digit is worth more than half a 10-digit. (The actual price you'd be willing to pay is a bit short of 70¢ per 5-digit, for reasons that we'll explore shortly). A 4-digit is also worth a bit more than half a 10-digit (two 4-digits lets you store 16 different numbers), and a 3-digit is worth a bit less than half a 10-digit (two 3-digits let you store only 9 different numbers).

We now begin to see what the fractional answer that comes out of a logarithm actually means (and why 300 is closer to 2.5 digits long that 500 is). The logarithm base 10 of $x$ is not answering "how many 10-digits does it take to store $x$?" It's answering "how many digits-of-various-kinds does it take to store $x$, where as many digits as possible are 10-digits; and how big does the final digit have to be?" The fractional portion of the output describes how large the final digit has to be, using this natural exchange rate between digits of different sizes.

For example, the number 200 can be stored using only two 10-digits and one 2-digit. $\log_{10}(200) \approx 2.301,$ and a 2-digit is worth about 0.301 10-digits. In fact, a 2-digit is worth _exactly_ $(\log_{10}(200) - 2)$ 10-digits. As another example, $\log_{10}(500) \approx 2.7$ means "to record 500, you need two 10-digits, and also a digit worth at least $\approx$70¢", i.e., two 10-digits and a 5-digit.

This raises a number of additional questions:

---

__Question:__ Wait, there _is_ no digit that's worth 50¢. As you said, a 3-digit is worth less than half a 10-digit (because two 3-digits can only store 9 things), and a 4-digit is worth more than half a 10-digit (because two 4-digits store 16 things). If $\log_{10}(316) \approx 2.5$ means "you need two 10-digits and a digit worth at least 50¢," then why not just have the $\log_{10}$ of everything between 301 and 400 be 2.60? They're all going to need two 10-digits and a 4-digit, aren't they?

__Answer:__ The natural exchange rates between digits is actually _way more interesting_ than it first appears. If you're trying to store either "301" or "400", and you start with two 10-digits, then you have to purchase a 4-digit in both cases. But if you start with a 10-digit and an 8-digit, then the digit you need to buy is different in the two cases. In the "301" case you can still make do with a 4-digit, because the 10, 8, and 4-digits together give you the ability to store any number up to $10\cdot 8\cdot 4 = 320$. But in the "400" case you now need to purchase a 5-digit instead, because the 10, 8, and 4 digits together aren't enough. The logarithm of a number tells you about _every_ combination of $n$-digits that would work to encode the number (and more!). This is an idea that we'll explore over the next few pages, and it will lead us to a much better understanding of logarithms.

---

__Question:__ Hold on, where did the 2.60 number come from above? How did you know that a 5-digit costs 70¢? How are you calculating these exchange rates, and what do they mean?

__Answer:__ Good question. In [427], we'll explore what the natural exchange rate between digits is, and why.

---

__Question:__ $\log_{10}(100)=2,$ but clearly, 100 is 3 digits long. In fact, $\log_b(b^k)=k$ for any integers $b$ and $k$, but $k+1$ digits are required to represent $b^k$ in base $b$ (as a one followed by $k$ zeroes). Why is the logarithm making these off-by-one errors?

__Answer:__ Secretly, the logarithm of $x$ isn't answering the question "how hard is it to write $x$ down?", it's answering something more like "how many digits does it take to record a whole number less than $x$?" In other words, the $\log_{10}$ of 100 is the number of 10-digits you need to be able to name any one of a hundred numbers, and that's two digits (which can hold anything from 00 to 99).

---

__Question:__ Wait, but what about when the _input_ has a fractional portion? How long is the number 100.87? And also, $\log_{10}(100.87249072)$ is just a hair higher than 2, but 100.87249072 is way harder to write down that 100. How can you say that their "lengths" are almost the same?

__Answer:__ Great questions! The length interpretation on its own doesn't shed any light on how logarithm functions handle fractional inputs. We'll soon develop a second interpretation of logarithms which does explain the behavior on fractional inputs, but we aren't there yet.

Meanwhile, note that the question "how hard is it to write down an integer between 0 and $x$ using digits?" is _very different_ from the question of "how hard is it to write down $x$"? For example, 3 is easy to write down using digits, while [49r $\pi$] is very difficult to write down using digits. Nevertheless, the log of $\pi$ is very close to the log of 3. The concept for "how hard is this number to write down?" goes by the name of [-complexity]; see the [+Kolmogorov_complexity_tutorial] to learn more on this topic.

---

__Question:__ Speaking of fractional inputs, if $0 < x < 1$ then the logarithm of $x$ is _negative._ How does _that_ square with the length interpretation? What would it even mean for the length of the number $\frac{1}{10}$ to be $-1$?

__Answer:__ Nice catch! The length interpretation crashes and burns when the inputs are less than one.

---

The "logarithms measure length" interpretation is imperfect. The connection is still useful to understand, because you _already_ have an intuition for how slowly the length of a number grows as the number gets larger. The "length" interpretation is one of the easiest ways to get a gut-level intuition for what logarithmic growth _means._ If someone says "the amount of time it takes to search my database is logarithmic in the number of entries," you can get a sense for what this means by remembering that logarithmic growth is like how the length of a number grows with the magnitude of that number:

[visualization(log-length-animation):]

The interpretation doesn't explain what's going on when the input is fractional, but it's still one of the fastest ways to make logarithms start feeling like a natural property on numbers, rather than just some esoteric function that "[3wr inverts exponentials]." Length is the quick-and-dirty intuition behind logarithms.

For example, I don't know what the logarithm base 10 of 2,310,426 is, but I know it's between 6 and 7, because 2,310,426 is seven digits long.

$$\underbrace{\text{2,310,426}}_\text{7 digits}$$

In fact, I can also tell you that $\log_{10}(\text{2,310,426})$ is between 6.30 and 6.48. How? Well, I know it takes six 10-digits to get up to 1,000,000, and then we need something more than a 2-digit and less than a 3-digit to get to a number between 2 and 3 million. The natural exchange rates for 2-digits and 3-digits (in terms of 10-digits) are 30¢ and 48¢ respectively, so the cost of 2,310,426 in terms of 10-digits is between \$6.30 and \$6.48.

Next up, we'll be exploring this idea of an exchange rate between different types of digits, and building an even better interpretation of logarithms which helps us understand what they're doing on fractional inputs (and why).


Logarithm base 1

[summary: There is no [-3nd] base 1, because no matter how many times you multiply 1 by 1, you get 1. If there _were_ a log base 1, it would send 1 to 0 (because $\log_b(1)=0$ for every $b$), and it would also send 1 to 1 (because $\log_b(b)=1$ for every $b$), which demonstrates some of the difficulties with $\log_1.$ In fact, it would need to send 1 to every number, because $\log(1 \cdot 1) = \log(1) + \log(1)$ and so on. And it would need to send every $x > 1$ to $\infty$, and every $0 < x < 1$ to $-\infty,$ and those aren't numbers, so there's no logarithm base 1.

But _if there was,_ it would be a [-multifunction] with values in the [extended_reals extended real numbers]. This is actually a perfectly valid way to define $\log_1,$ though doing so is not necessarily a good idea.]

There is no [-3nd] base 1, because no matter how many times you multiply 1 by 1, you get 1. If there _were_ a log base 1, it would send 1 to 0 (because $\log_b(1)=0$ for every $b$), and it would also send 1 to 1 (because $\log_b(b)=1$ for every $b$), which demonstrates some of the difficulties with $\log_1.$ In fact, it would need to send 1 to every number, because $\log(1 \cdot 1) = \log(1) + \log(1)$ and so on. And it would need to send every $x > 1$ to $\infty$, and every $0 < x < 1$ to $-\infty,$ and those aren't numbers, so there's no logarithm base 1.

But if you _really_ want a logarithm base $1$, you can define $\log_1$ to be a multifunction [3js from] [positive_real_numebrs $\mathbb R^+$] [3lg to] $\mathbb R \cup \{ \infty, -\infty \}.$ On the input $1$ it outputs $\mathbb R$. On every input $x > 1$ it outputs $\{ \infty \}$. On every input $0 < x < 1$ it outputs $\{ -\infty \}$. This multifunction can be made to satisfy all the basic [4bz properties of the logarithm], if you interpret $=$ as $\in$, $1^{\{\infty\}}$ as the [interval_notation interval] $(1, \infty)$, and $1^{\{-\infty\}}$ as the interval $(0, 1)$. For example, $0 \in \log_1(1)$, $1 \in \log_1(1)$, and $\log_1(1) + \log_1(1) \in \log_1(1 \cdot 1)$. $7 \in log_1(1^7)$, and $15 \in 1^{\log_1(15)}$. This is not necessarily the best idea ever, but hey, the [complex_log final form] of the logarithm was already a multifunction, so whatever. See also [log_is_a_multifunction].

While you're thinking about weird logarithms, see also [4c8].


Exchange rates between digits

[summary: There is a natural exchange rate between types of digits. For example, a 100-digit is worth exactly twice as much as a 10-digit, because you can build a 100-digit out of two 10-digits. Furthermore, a 10-digit is worth between 3x and 4x as much as a 2-digit, because you can emulate a [-42d] using between 3 and 4 coins. The cost of an $n$-digit in terms of $b$-digits is $\log_b(n).$]

Let's say that you want to [storing_numbers store a lot of data], using physical objects such as coins, dice, or [42d digit wheels]. For concreteness, let's say that you're going to be given an arbitrary number between one and $2^	ext{3,000,000,000,000}$ ([numbers_are_data a.k.a. a terabyte of data]), and you need to make sure you have enough physical objects to write that number down.

Let's say that a coin costs 1¢. Because [ $n$ coins can be used to encode any one of $2^n$ different possibilities], you could get the job done with three trillion coins, which would let you encode the number (in [binary_notation binary]) for the low low cost of thirty billion dollars. (At the time of writing this, three trillion bits of data will actually run you about \$60, but leave that aside for now. Pretend you can't use modern transistors, and you have to use comparatively large objects such as coins to store data.)

Instead of buying coins, you could also buy [42d digit wheels], and use those instead. Because digit wheels can store more values than coins, it takes fewer digit wheels than coins to do the job. If digit wheels also cost 1¢, you should just buy digit wheels; that will cost a mere $9B. If instead digit wheels cost a million dollars apiece, you should probably buy coins instead. Somewhere in between those two prices, digit wheels switch from being a good deal to being a bad deal. The question is, if coins are worth 1¢, then how much is a digit wheel worth? At what price should you buy wheels instead of coins, and at what price should you buy coins instead of wheels?

At a first glance, it may seem like the fair price is 5¢ per wheel. A coin can store two values (by being placed 'heads' or 'tails'), while a digit wheel can store ten. Ten is five times larger than 2, so perhaps the price of a digit wheel should be five times the price of a coin. However, if digit wheels cost 5¢, you should ignore digit wheels entirely and buy coins. Why? Because for only 4¢, you could buy four coins, which can hold $2^4=16$ different values. Four coins store a [4sj 16-digit], and a 16-digit is worth more than a 10-digit, so you would have no reason to even consider buying digit wheels at a price of 5¢ per wheel.

So what is the fair price for a digit wheel? 4¢ is still too high, because that makes a 10-digit cost the same as a 16-digit, and the 16-digit is always better at that price. What about 3¢? At that price, the answer is much less clear. On the one hand, spending 3¢ on coins gets you the ability to write down only 8 possibilities, while spending 3¢ on wheels lets you write down 10 different possibilities. On the other hand, if you're trying to store the number 101, you need either 7 coins (7¢, because $2^6 < 101 < 2^7$) or 3 wheels (one per digit, because a wheel stores a digit, for a total of 9¢), in which case the coins are better.

Given that digit wheels can store ten values and cost 3¢ each, and coins can store two values and cost 1¢ each:

__Question:__ When storing the number 8000, should you buy coins or wheels?

%%hidden(Answer):
Wheels. To store 8000, you need 13 coins (because $2^{12} < 8000 < 2^{13}$) for a cost of 13¢, but only 4 wheels (because 8000 is 4 digits long) for a cost of 12¢.
%%

__Question:__ When storing the number 15,000, should you buy coins or wheels?

%%hidden(Answer):
Coins. Storing 15,000 costs 14 coins ($2^{13} < 15,000 < 2^{14}$) or 5 wheels; given that wheels cost 3¢ and coins cost 1¢, the 14 coins are cheaper.
%%

__Question:__ When storing the number 230,000, should you buy coins or wheels?
%%hidden(Answer):
It doesn't matter. Storing 230,000 takes either 18 coins or 6 wheels, which costs 18¢ either way.
%%

At 3¢ per digit wheel, whether you should buy coins or wheels depends on which number you want to store.

But _once the number you're storing gets large enough,_ the digit wheels become a better deal, and stay that way. In this case, the final transition happens when the number you're storing is 11 digits long or longer. Why? Say that the number $x$ you want to store is $n$ digits long in decimal notation. $n$ digit wheels can always encode more possibilities than $3n$ coins (because $10^n > 2^{3n}$ for all $n$), but sometimes $x$ can be written using fewer than $3n$ coins (as in the case of 15,000). This stops happening once $n$ is so large than $10^n$ is $2^3$ times larger than $2^{3n}$, at which point no $n$-digit number can be encoded using $2^{3(n-1)}$ or fewer coins. This happens once $n \ge 11,$ i.e., once $x$ is at least 11 digits long. ([ Proof.])

This probably isn't too surprising: For 3¢ you can either buy 3 coins or one digit wheel. The coins store 8 possibilities, the wheel stores 10, and it's no surprise that a huge collection of 10-digits is unambiguously better than an equally sized collection of 8-digits.

So digit wheels are worth more than 3x as much as a coin, and less than 4x. How can we find the right price exactly? We could use exactly the same argument as above to check whether 3.5¢ is too high or too low, except that it's not clear what it would mean to buy "three and a half coins." We can get around that problem by considering ten-packs of coins vs ten-packs of digit wheels. If wheels cost 3.5¢ and coins cost 1¢, then with 35¢, you could either buy 10 wheels or 35 coins. Which encodes more possibilities? The coins, because $10^{10} < 2^{35}.$ Thus, by a [ generalized version of the argument above], for numbers sufficiently large, the coins are a better deal than the wheels — at that price, you should buy wheels.

As you can verify, $2^{33} < 10^{10} < 2^{34},$ so the fair price must be between 3.3¢ and 3.4¢. And $2^{332} < 10^{100} < 2^{333},$ so it must be between 3.32¢ and 3.33¢. We could keep going, and keep getting digits of the fair price. And we could keep going forever, because the fair price is [-irrational]. ([logs_are_usually_irrational Why?])

In general, if digit wheels cost $p$ times as much as coins, the [ general argument] we've been using says that if you're storing a large number and $2^p > 10$ then you should buy coins, whereas if $2^p < 10$ then you should buy digit wheels. Thus, the fair price must be the number $p$ such that $2^p = 10,$ and its first three digits are 3.32.

In other words, while a digit wheel holds 5 times as many values as a coin, it is only worth about 3.32 times as much as a coin (in terms of its ability to store data). Why? _Because it takes about 3.32 coins to emulate a digit-wheel._ What does that mean? Well, 3 coins is not enough to emulate a digit wheel (3 give you only 8 possibilities), 4 coins is more than enough (4 give you 16 possibilities). And 33 coins is not enough to emulate 10 digit wheels, but 34 coins are more than enough. And 332 coins are not enough to emulate 100 digit wheels, but 333 coins are more than enough.

A digit wheel isn't worth 5 times as much as a coin because, when you buy multiple coins, their data storage capacity _multiplies,_ rather than adding. Each new coin lets you say _twice as many things,_ not just two more things. With five coins you can store $2 \cdot 2 \cdot 2 \cdot 2 \cdot 2 = 32$ different values, not $2 + 2 + 2 + 2 + 2 = 10.$ When finding the exchange rate between coins and digit wheels, the question is not "How many more values does a digit wheel have written on it?" Rather, the question is "How many coins does it take to emulate a digit wheel?" The answer to _that_ question is "More than 332 coins per hundred digit wheels, less than 333 per hundred."

The fair price $p$ of digit wheels (in terms of coins) is the number $p$ such that $2^p = 10$ exactly. We have a name for this number, and it's $\log_2(10),$ and the argument described above is a [ simple algorithm for computing logarithms]. (Logarithm outputs are "difficult to calculate but useful to have," in general.)

Thus, we can look at $\log_b(x)$ as describing the fair price of $x$-digits in terms of $b$-digits. For example, if coins are worth 1¢, then dice are worth $\log_2(6) pprox 2.58$ cents, as you can verify by checking that $2^2 < 6 < 2^3$ and $2^{25} < 6^{10} < 2^{26}$ and $2^{258} < 6^{100} < 2^{259}.$ (Exercise: Find the next digit of $\log_2(6)$ using this method.)

This interpretation is a great intuition pump for what $\log_b(x)$ "really means" if $b$ and $x$ are whole-numbers. Imagine two sets of objects, one (say, weird dice) that can be placed in $b$ different states, and another (say, weird wheels) that can be placed in $x$ different states. Imagine you're storing a really really large number, and imagine that the $b$-objects cost 1¢. At what price of $x$-objects would you be indifferent between using $x$-objects versus $b$-objects to store data? In other words, how many $b$-objects does it take to [emulating_digits "emulate"] an $x$-object?

Probing this interpretation also explains many of the properties of the logarithm. For example, the fact that $\log_b(x)$ is the fair price of $x$-digits in terms of $b$-digits immediately implies that $\log_x(b) = rac{1}{\log_b(x)}$, because if an $x$-digit is worth three $b$-digits then a $b$-digit must be worth a third of an $x$-digit.

This interpretation still doesn't shed light onto what logarithms are doing when their inputs are not whole numbers. For example, what's $\log_{1.5}(2.5)$? What the heck would an object that can be placed into "1.5 different states" even be? As we will see shortly, this notion of there being a "natural exchange rate" between digits reveals an interpretation of the logarithm that explains what it's doing with fractional inputs.


Digit wheel

[summary:
A mechanical device for storing a number from 0 to 9.

![](http://www.cl.cam.ac.uk/~djg11/howcomputerswork/mechanical-counter3.jpg)]

A mechanical device for storing a number from 0 to 9. Devices such as these were often attached to the desks of engineers, as part of historical "desktop computers."

![](http://www.cl.cam.ac.uk/~djg11/howcomputerswork/mechanical-counter3.jpg)

Each wheel can stably rest in any one of ten states. Digit wheels are often useful when it comes to thinking about how much data a physical object can store, as they were physical objects that were clearly designed for storing data. See also [3y1].


Graham's number

[summary:
Graham's number is a... rather large number. Tim Urban of _Wait but Why_ gives a [http://waitbutwhy.com/2014/11/<PHONE>-grahams-number.html good introduction].

![](http://28oa9i1t08037ue3m1l0i861.wpengine.netdna-cdn.com/wp-content/uploads/2014/11/grahams-number.png)]

Graham's number is a... rather large number. Letting $f(x) = 3\uparrow^n 3$ (in [+knuth_up_arrow_notation]) and $f^n(x) = \underbrace{f(f(f(\cdots f(f(x)) \cdots ))}_{n\text{ applications of }f}$, Graham's number is defined to be $f^{64}(4).$

The result is sizable. For an explanation of how large this is and why, see Tim Urban's [http://waitbutwhy.com/2014/11/<PHONE>-grahams-number.html explanation] at Wait but Why.


A googolplex

The number $10^{10^{100}}$ is named "a googolplex." That is, a googolplex is the number represented (in [-decimal_notation]) as a 1 followed by a [-42m] zeroes, i.e., $10^{googol}$, which is $ 10^{10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000}.$


A googol

$10^{100},$ i.e., 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000.

The search engine Google is named after the number googol. (The difference for the spelling, as the apocryphal story goes, comes from the fact that one of the early Google investors misspelled the name on a check.)


Why is log like length?

If a number $x$ is $n$ digits long (in [-decimal_notation]), then its logarithm (base 10) is between $n-1$ and $n$. This follows directly from the [40j definition of the logarithm]: $\log_{10}(x)$ is the number of times you have to multiply 1 by 10 to get $x;$ and each new digit lets you write down ten times as many numbers. In other words, if you have one digit, you can write down any one of ten different things (0-9); if you have two digits you can write down any one of a hundred different things (00-99); if you have three digits, you can write down any one of a thousand different things (000-999); and in general, each new digit lets you write down ten times as many things. Thus, the number of digits you need to write $x$ is close to the number of times you have to multiply 1 by 10 to get $x$. The only difference is that, when computing logs, you multiply 1 by 10 exactly as many times as it takes to get $x$, which might require [fractional_exponent multiplying by 10 a fraction of a time] (if x is not a power of 10), whereas the number of digits in the base 10 representation of x is always a whole number.


Error getting page

Error getting page


Ackermann function

The Ackermann function works as follows:

One may have noticed that addition, multiplication, and exponentiation seem to increase in "power", that is, when pitted against each other, it is easier to produce enormous numbers with exponentiation than with multiplication, and so on.

The Ackermann function produces a hierarchy of such growth operations, and ascends one step higher in the hierarchy each time.

Addition is the first operator in the hierarchy (though if we wanted, we could define [-addition_as_repeated_succession] and declare addition the second operator in the hierarchy). The next operator in the hierarchy is produced by iterating the previous element in the hierarchy. For instance, the next few operators in the hierarchy are multiplication, exponentiation, and tetration:

Multiplication is iterated addition: $A \cdot B = \underbrace{A + A + \ldots A}_{B \text{ copies of } A}$

Exponentiation is iterated multiplication: $A^B = \underbrace{A \times A \times \ldots A}_{B \text{ copies of } A}$.

($A ^ B$ is written $A \uparrow B$ in [knuth_up_arrow_notation].)

Tetration is iterated exponentiation. $A \uparrow\uparrow B = \underbrace{A^{A^{\ldots^A}}}_{B \text{ copies of } A}$ times.

$\uparrow^n$ just means $n$ up arrows, so we can also write tetration as: $A \uparrow^2 B = \underbrace{A \uparrow^1 (A \uparrow^1 (\ldots A))}_{B \text{ copies of } A}$

Now the pattern can be noticed and generalized, following the rule: $A \uparrow^n B = \underbrace{A \uparrow^{n-1} (A \uparrow^{n-1} (\ldots A))}_{B \text{ copies of } A}$

And now we can define the Ackermann function as $A(n) = n \uparrow^n n$.

This definition is relatively small, but the functions grow at incredible rates. [Wait But Why](http://waitbutwhy.com/2014/11/<PHONE>-grahams-number.html) has a good intuitive description of how incredibly fast tetration, pentation, and hexation grow, but by the time we get to $A(6)$, the Ackermann function already grows far more quickly than that. $A(1)=1$, $A(2)=4$, and $A(3)$ cannot be written in the universe, as it has enormously more digits than our universe has subatomic particles.

Interestingly enough, the Ackermann function grows faster than all primitive recursive functions. This is related to a rather deep connection between the consistency strength of a mathematical theory (which sorts of results they are capable of proving) and the slowest-growing function for which the theory cannot prove "this function is defined on all integers."


Fractional digits

When $b$ and $x$ are [48l integers], $\log_b(x)$ has a few good interpretations. It's roughly the [416 length of $x$] when written in [number_bases base $b$]. More precisely, it's the [427 fair price of an $x$-digit] in terms of [4sj $b$-digits]. But what about when $x$ or $b$ is not a whole number? $\log_{3.16}(5.62) pprox 1.5$; how are we supposed to interpret this fact?

We could appeal to the definition of the logarithm, observe that $3.16^{1.5} pprox 5.62,$ and call it a day. But what does that mean? That 3.16 multiplied by itself one-and-a-half times is roughly 5.62? That 5.62, written in base 3.16, is about 1.5 digits long? That a 5.62-digit is worth about 1.5 times as much as a 3.16-digit? What does it mean to multiply a number by itself "half a time"? How would you even write a number down in "base 3.16"? What would a 3.16-digit even look like?

Let's say that you want to [storing_numbers store a lot of data], and [42d digit wheels] cost \$1. At that price, an object that can store 3 different values costs about 48¢, whereas an object that stores 4 different values costs about 60¢. What storage medium, then, costs exactly 50¢? I can think of two:

1. You and I could buy a digit wheel, split the costs, and share it.
2. You and I could both pay 50¢ each for a digit wheel, and then toss a coin to see who gets to use it.

How could we split the digit wheel? The digit wheel can be put into any one of ten different states, labeled 0, 1, 2, 3, 4, 5, 6, 7, 8, and 9. Here's one way to use it to encode two different messages: I choose a number $a$ between 0 and 1, then you choose a number $b$ between 0 and 4. Then we put the digit wheel into the state $5a + b.$ Thus, if you choose 0, the wheel will either say 0 or 5 (depending on which number I choose). If you choose 3, the wheel will either say 3 or 8 (depending on which number I chose). Each number you want to record (0-4) has two representations, and I choose whether we use the high one or the low one. Thus, the digit wheel records two messages: 7 means (high, 2) and so on.

Of course, this is not a fair way to split the digit wheel — you get to store a [3v9 5-message] whereas I only get to store a 2-message.%%note:Thus we see that a [4sj 10-digit] is worth a 5-digit plus a 2-digit, that is, $\log_{10}(5) + \log_{10}(2) = 1$.%% Can we do something fairer?

Well, the methodology can be used to share arbitrary $n$-digits between two people. If $x$ and $y$ are whole numbers such that $x \cdot y \le n$ then an $n$-digit can be used to store both an $x$-message and a $y$-message at once: You choose the [-quotient] of $n$ divided by $x$, I choose the [-remainder].

__Exercise:__ How would you use an $18$-digit to store both a $3$-message and a $6$-message? How do you store (2, 4) using that encoding?
%%hidden(Answer):
Here's one way: Let $a$ be the value of the 3-message (between 0 and 2) and $b$ be the value of the 6-message (between 0 and 5). Record $6a+b.$ Using this encoding, (2, 4) is stored as the number 16.

You can verify for yourself that there is exactly one way to store each (3-message, 6-message) pair in an 18-message under this encoding.
%%

(From this we can see that, when $n = x \cdot y,$ the cost of a $n$-digit is exactly equal to the cost of an $x$-digit plus the cost of a $y$-digit. In other words, $n = x \cdot y$ then $\log_b(x) + \log_b(y) = \log_b(n),$ no matter what base $b$ we're using.)

The fair split of the 10-digit, then, is to let us both store a 3-message: 3 is the highest integer $x$ such that $x \cdot x < 10.$ This isn't maximally efficient, though; that encoding only uses 9 of the 10 possibilities that we paid for, and therefore it's not worth our money: Why buy what is effectively a 3-message for 50¢ when the fair price of 3-messages is 48¢? This might be worth knowing if the market isn't selling objects that carry 3-messages, and both you and I just need a tiny bit more storage, but in general it's not worth it.

However, it becomes more worthwhile the more 10-digits we split. You might expect that, when splitting three digit wheels, we'd each get one digit wheel to ourselves plus a 3-message each from the third wheel, giving us 30 different possible messages each. However, we can do better than that if we're clever. If you pick a number $a$ between 0 and 30 (for a total of 31 different possibilities), and I also pick a number $b$ between 0 and 30, then the number $31a + b$ can always be stored on 3 digit wheels (because $31 \cdot 30 + 30 = 960 \le 999$), so if we're splitting three digit wheels, we can actually use them to store two 31-messages. ([ Why?]) This still isn't maximally efficient (the values from 961 to 999 are wasted), but it's a little better. And we can keep increasing the efficiency if we keep going: Five digit wheels can hold two 316-messages; 7 digit wheels can hold two 3162-messages, and so on. (You can verify those numbers yourself.)

As you can see, when splitting an $n$-digit, the fair split occurs at the largest whole number $x$ such that $x \cdot x \le n$, in which case we both get to record one $x$-message. This is why it is 316, and not 500, that can most naturally be seen as "about 2.5 decimal digits long:" The number that is 2.5 decimal digits long is the largest number $x$ such that, given five digits, both you and I can use those 5 digits to store independent $x$-messages. If you store a 500-message, that only leaves enough space for me to store a 200-message. If you only store a 300-message, that leaves enough space for me to store a 333-message. $x=316$ is the point where we can both store an $x$-message, because 316 is the largest whole number such that $x^2 \le 100000.$

(From this we can deduce that $\log_b(316) pprox rac{5\log_b(10)}{2}$, no matter what the base: A 316-digit is worth about half as much as five 10-digits put together.)

The more 10-digits you split, the more efficient it gets — if we split 7 digit wheels, each of us gets to record a 3162-message; if we split 9 wheels, we each get to record a 31622-message, and so on. But the split is never maximally efficient, because 10 is not a [-square_number]: An $n$-digit can be split perfectly in half if and only if there is an $x$ such that $x \cdot x = n,$ and this can't happen if $n$ is an odd power of 10. Thus, splitting an odd number of digit wheels never makes the final digit wheel be worth _exactly_ 50¢.

You know what storage medium is worth exactly 50¢? A 50% chance of a 10-digit.

Let's say we pay 50¢ each for a digit wheel, and then toss a coin to see who gets to use it. How much does this decrease your storage costs? Half the time, you get to [ reduce the number you have left to store] by a factor of 10; the other half the time, you reduce the number you have left to store by a factor of 1 (i.e., not at all). How much does that reduce the number you have left to store, [4b5 on average]?

On average, your number gets reduced by the [-geometric_mean] of 1 and 10. Geometric means "average out" multiplications: Let's say that you have a thing and you're going to multiply it by 2, 12 and 9. At the end of the day, your thing is going to be 216 times larger, and you're going to have multiplied it by three numbers. Geometric means ask how big each multiplication was "on average" — instead of multiplying your thing by three different numbers, can we find one number $y$ such that $y \cdot y \cdot y = 216,$ such that multiplying your thing by $y$ three times is the same as multiplying by 2, 12, and 9 separately? The answer is yes, and $y = \sqrt[3]{2 \cdot 12 \cdot 9} = 6$, as you can verify. Thus, multiplying your thing by 6 is equivalent, [4b5 in expectation], to multiplying your thing by a value selected [uniform_random uniformly at random] from the list [2, 12, 9]. (For more on this, see [ Geometric mean: Intuition].)

Similarly, if we might reduce the size of the number we want to store by a factor of 1, and we might reduce it by a factor of 10, then "on average" we're reducing it by a factor of $\sqrt[2]{1 \cdot 10} pprox 3.16.$ Multiplying by $\sqrt[2]{10}$ twice is the same as multiplying by 1 once and 10 once, so it's equivalent [4b5 in expectation] to multiplying by values selected uniformly at random from the list [1, 10].

If _three_ people are sharing a 10-digit, then the amount of digits each one gets to use in expectation is $\sqrt[3]{1 \cdot 1 \cdot 10} pprox 2.15.$ So that's what a 2.15 digit looks like — a 10-digit shared between three people. This can be generalized to any probability, and any $n$-digit for $1 < n \le 10$ can be interpreted as a digit wheel that you have some probability of getting to use ([ How?]).

There are a few different things we can learn from this interpretation. It allows us to give a physical interpretation to facts like $\log_{3.16}(5.62) pprox 1.5$, which says that a 5.62-digit is worth one and a half times as much as a 3.16 digit. What's a 3.16 digit? It's (approximately) a 10-digit shared between two people, i.e., a 10-digit that you have a 50% chance of getting to use yourself. What's a 5.62-digit? It's a 10-digit that you have a 75% chance of getting to use yourself. It's 1.5x as valuable, because 75% is 1.5x as much as 50%. The upgrade from a 3.16-digit to a 5.62-digit constitutes 1.5x increase in my chance of getting to use the digit wheel.

The other thing that this interpretation tells us is that the digit worth exactly half an $n$-digit is the $\sqrt{n}$-digit, both because (1) if you want to use an $n$-digit to store two $x$-messages for some $x$, you need $x \cdot x$ to be at most $n$; and (2) if you have a 50% chance of getting to use an $n$-digit and a 50% chance of [1digit nothing], then that's equivalent [4b5 on average] to a certainty of a $\sqrt{n}$-digit.

In case you haven't noticed, we've been implicitly using the fact that doubling a digit squares the number of possibilities you have available to you: With one 10-digit you can store 10 possibilities, with two you can store $10^2 = 100.$ Because two $n$-digits [ emulate] an $n^2$-digit, it's no surprise that half an $n$-digit is a $\sqrt{n}$-digit.

You might be wondering how these square root signs weaseled into a discussion about logarithms. You also might be starting to suspect that roots, logarithms, and exponentials are secretly three sides of the same coin. If so, [exp_log_root you're right], though that's a topic for another day.

Lest you think that this gives us a complete interpretation of logarithms, note that it still doesn't explain what's going on when the inputs to the logarithm function are between 0 and 1. You might think, "well, now that I know what a 3.16-digit is, I can use that intuition to figure out what a 0.16-digit is" — but it's not quite as easy as that. A 3.16-digit can be interpreted as probabilistic access to a 10-digit, and the same is true for $x$-digits whenever $x > 1.$ But no matter how many people share a 10-digit, it's never worth _less_ than a 1-digit ([1digit which is worthless]); $\sqrt[n]{10} > 1$ no matter how large $n$ gets. To understand $x$-digits for $0 < x < 1,$ we need one final puzzle piece, covered in the next post.


Shannon

The shannon (Sh) is a unit of [-3xd]. One shannon is the difference in [info_entropy entropy] between a [-probability_distribution] on a [-binary_variable] that assigns 50% to each value, and one that assigns 100% to one of the two values.


Natural number

[summary: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, ...

The natural numbers are the numbers we use to count things.]

A **natural number** is a number like 0, 1, 2, 3, 4, 5, 6, ... which can be used to represent the count of an object. The set of natural numbers is $\mathbb N.$ Not all sources include 0 in $\mathbb N.$

Natural numbers are perhaps the simplest type of number. They don't include [48l negative numbers], [4zq fractional numbers], [4bc irrational numbers], [4zw imaginary numbers], or any of those complexities.

Thanks to their simplicity, natural numbers are often the first mathematical concept taught to children. Natural numbers are equipped with a notion of [-addition] ($2 + 3 = 5$ and so on) and [-multiplication] ($2 \cdot 3 = 6$ and so on), these are among the first mathematical operations taught to children.

Despite their simplicity, the natural numbers are a ubiquitous and useful mathematical object. They're quite useful for counting things. They represent all the possible [4w5 cardinalities] of finite [3jz sets]. They're also a useful [data_structure data structure], in that numbers can be used to [numeric_encoding encode] all sorts of data. Almost all of modern mathematics can be built out of natural numbers.

[todo: Add a "formalization" lens with the Peano axioms. I recommend at least one page with just the raw Peano axioms (and very little prose), and another gentler introduction sort of like http://bit.ly/29glDrR and http://bit.ly/29nKYAL, albeit more to-the-point and probably without going all the way up to non-standard number territory.]


Log as the change in the cost of communicating

When interpreting logarithms as [416 a generalization of the notion of "length"] and as [427 digit exchange rates], in both cases, multiplying the input to the logarithm base 10 by a factor of 10 caused the output to go up by one. Multiplying a number by 10 makes it one digit longer. If a [-42d] is worth \$1, then a 1000-digit is worth exactly \$1 more than a 100-digit, because you can [emulating_digits build] a 1000-digit out of a 100-digit and a 10-digit. Thus, by symmetry _dividing_ an input to the logarithm base 10 by 10 makes the output go down by one: If you divide a number by 10, it gets one digit shorter; and any $n$-digit is worth \$1 more than a $rac{n}{10}$-digit, because you can build an $n$-digit out of a $rac{n}{10}$-digit and a 10-digit.

This strongly implies that $\log_{10}(rac{1}{10})$ should equal $-1$. If a 1000-digit costs \$3, and a 100-digit costs \$2, and a 10-digit costs \$1, and a [1digit 1-digit is worthless], then, extrapolating the pattern, a $rac{1}{10}$ should cost $-\$1.$ But what does that mean? What sort of digit is worth negative money? Can we give this extrapolation a physical intuition?

Yes, we can, by thinking in terms of how difficult it is to communicate information. Let's say that you and I are in separate rooms, connected only by a conveyor belt, upon which I can place physical objects like coins, dice, and [42d digit wheels] that you can read. Let's imagine also that a third party is going to show me the whodunit of a game of [https://en.wikipedia.org/wiki/Cluedo Clue], and then let me put some objects on the conveyor belt, and then send those objects into your room, and then ask you for the information. If you can reproduce it successfully, then we both win a lot of money. However, I have to pay for every object that I put on the conveyor belt, using the [427 fair prices].

Consider how much I have to pay to tell you the result of a clue game. The "whodunit" in a clue game consists of three pieces of information:

1. The name of the murderer, which is one of: Miss Scarlett, Professor Plum, Mrs. Peacock, Reverend Green, Colonel Mustard, or Mrs. White.
2. The room in which the murder occurred, which is either the kitchen, the ballroom, the conservatory, the dining room, the cellar, the billiard room, the library, the lounge, the hall, or the study.
3. The murder weapon, which is either the candlestick, the dagger, the lead pipe, poison, the revolver, the rope, or the wrench.

Thus, a typical whodunit might look like "Professor Plum, in the conservatory, with the revolver." That sentence is 55 letters long, so one way for me to transmit the message would be to purchase fifty five 29-digits (capable of holding any one of 26 letters, or a space, or a comma, or a period), and send you that sentence directly. However, that might be a bit excessive, as there are in fact only $6 \cdot 10 \cdot 7 = 420$ different possibilities (six possible murderers, ten possible locations, seven possible weapons). As such, I only actually need to buy a 6-digit, a 10-digit, and a 7-digit. Equivalently, I could purchase a single 420-digit (if such things are on sale). We have to agree in advance what the digits mean — for example, "the 6-digit corresponds to the murderer, in the order listed above; the 10-digit corresponds to the room, in the order listed above; the 7-digit corresponds to the weapon, in the order listed above;" but assuming we do, I can get away with much less than fifty five 29-digits.

__Exercise:__ If the only storage devices on sale are coins, how many do I need to buy to communicate the whodunit?
%%hidden(Answer):
Nine. 8 coins only gets you 256 possibilities, and we need at least 420.
%%

__Exercise:__ If the only storage devices on sale are dice, how many do I need to buy?
%%hidden(Answer):
Four. $6^3 < 420 < 6^4.$
%%

__Exercise:__ If I have to choose between all coins or all dice, which should I choose, at the fair prices?
%%hidden(Answer):
The coins. Four dice cost as much as $\log_2(6) * 4 pprox 10.33$ coins, and we can do the job with nine coins instead.
%%

__Exercise:__ If I can mix coins, dice, and digit wheels, what's the cheapest way to communicate the whodunit?
%%hidden(Answer):
One coin and three dice let you send the message at a cost of only $\log_2(2) + 3\cdot \log_2(6) pprox 8.75$ coins.
%%

Now, consider what happens when the third party tells you "Actually, in order to win, you also have to communicate the name of my favorite Clue suspect, which is Colonel Mustard. I already told the person in the other room that you need to communicate two suspects, and that you'll communicate my favorite Clue suspect second. I didn't tell them who my favorite Clue suspect was, though."

Now, the space of possible messages has gone up by a factor of six: There are 420 possible whodunits, and each can be paired with one of six possible "favorite suspects," for a total of 2520 possible messages. How does this impact my cost of communicating with you? My cost goes up by 1 die ($= \log_2(6)$ coins $= \log_{10}(6)$ digit wheels). When the space of possibilities goes up by a factor of 6, my costs of communication (measured, say, in coins) go up by $\log_2(6).$

Now let's say that the third party comes back in the room and tells you "Actually, I gave the person in the other room a logic puzzle that told them which room the murder happened in; they solved it, and now they know that the murder happened in the conservatory."

This _reduces_ the space of possible messages I need to send, by a factor of 10. Now that both you and I know that the murder happened in the conservatory, I only need to transmit the murderer, the weapon, and the favorite suspect — one of 252 possibilities. The space of possibilities was cut into a tenth of its former size, and my cost of communicating dropped by 1 digit wheel ($= \log_6(10)$ dice $= \log_2(10)$ coins).

On this interpretation, logarithms are measuring how much it costs to transmit information, in terms of some "base" medium (such as coins, dice, or digit wheels). Every time the space of possibilities increases by a factor of $n$, my communication costs increase by $\log_2(n)$ coins. Every time the space of possibilities decreases by a factor of $n$, my communication costs _drop_ by $\log_2(n)$ coins.

This is the physical interpretation of logarithms that you can put your weight on: $\log_b(x)$ measures how much more or less costly it will be to send a message (in terms of $b$-digits) when the space of possible messages changes by a factor of $x$. Paired with a physical interpretation of [44l fractional digits], it can explain most of the [log_properties basic properties of the logarithm]:

1. $\log_b(1) = 0,$ because increasing (or decreasing) the space of possible messages by a factor of 1 doesn't affect your communication costs at all.
2. $\log_b(b) = 1,$ because increasing the space of possible messages by a factor of $b$ will increase your communication costs by exactly one $b$-digit.
3. $\log_b\left(rac{1}{b}
ight) = -1,$ because decreasing the space of possible messages by a factor of $b$ saves you one $b$-digit worth of communication costs.
4. $\log_b(x\cdot y) = \log_b(x) + \log_b(y),$ because if $n = x \cdot y$ then one $n$-digit is exactly large enough to store one $x$-message and one $y$-message. Thus, when communicating, an $x\cdot y$-digit is worth the same amount as one $x$-digit plus one $y$-digit.
5. $\log_b(x^n) = n \cdot log_b(x),$ because $n$ $x$-digits can be used to [emulating_digits emulate] one $x^n$-digit.

You might be thinking to yourself:

> Wait, what does it mean for the space of possible messages to go up or down by a factor of $x$? This isn't always clear. What if you're really good at guessing who people's favorite suspect is? For that matter, what if we haven't established a convention like "0 = Miss Scarlett; 1 = Professor Plum; ..."? If I see an observation, the amount by which it changes the space of possible messages is subjective; it depends on my beliefs and on the beliefs of the person I'm communicating with and on the conventions that we set up beforehand. How do you actually formalize this idea?

Those are great questions. Down that path lies [3qq], a field which measures communication costs using logarithms, and which lets us formalize (and quantize) ideas such as the amount of information carried by a message (to a given observer). See the [information_theory_tutorial] for more on this subject.

With regard to logarithms, the key idea here is an interpretation of what $\log_b(x)$ is "really doing." Given an input like "how many possible messages are there," such that your costs go up by 1 unit every time the input space increases by a factor of $b$, $\log_b(x)$ measures the change in cost when the input space increases by a factor of $x$. As we will see next, this idea generalizes beyond the domain of "set of possible messages vs cost of communicating," to _any_ scenario where some measure $\mu$ increases by $1$ every time some object scales by a factor of $b$, in which case $\log_b(x)$ measures the change in $\mu$ when the object scales by a factor of $x$. This is the defining characteristic of logarithms, and now that we have some solid physical interpretations of what it means, we're ready to start exploring logarithms in the abstract.


Lattice (Order Theory)

[summary:
A **lattice** is a [-3rb] that is closed under binary [3rc joins and meets]. It follows from this definition that lattices are closed under all *finite* joins and meets, and that for all lattices $L$ and elements $p, q \in L$, the following algebraic properties are satisfied:

* [3h4 Associativity] of joins and meets: $(p ee q) ee r = p ee (q ee r)$, and $(p \wedge q) \wedge r = p \wedge (q \wedge r)$
* [3jb Commutativity] of joins and meets: $p ee q = q ee p$ and $p \wedge q = q \wedge p$
* Idempotency of joins and and meets: $p ee p = p$ and $p \wedge p = p$
* Absorption: $p ee (p \wedge q) = p$ and $p \wedge (p ee q) = p$
]

A **lattice** is a [-3rb] that is closed under binary [3rc joins and meets].

Let $L$ be a lattice. Then for all $p,q,r \in L$ the following properties are necessarily satisfied.

* [3h4 Associativity] of joins and meets: $(p ee q) ee r = p ee (q ee r)$, and $(p \wedge q) \wedge r = p \wedge (q \wedge r)$
* [3jb Commutativity] of joins and meets: $p ee q = q ee p$ and $p \wedge q = q \wedge p$
* Idempotency of joins and and meets: $p ee p = p$ and $p \wedge p = p$
* Absorption: $p ee (p \wedge q) = p$ and $p \wedge (p ee q) = p$

%%hidden(Proofs):

Lemma 1: Let $P$ be a poset, $S \subseteq P$, and $p \in P$. If both $igvee S$ and $(igvee S) ee p$ exist then $igvee (S \cup \{p\})$ exists as well, and $(igvee S) ee p = igvee (S \cup \{p\})$.

Proof: See the *Join fu* exercise in [4ll].

## Associativity
Let $L$ be a lattice and $p,q,r,s \in L$ such that $s = p ee (q ee r)$. We apply the above lemma, along with commutativity and closure of lattices under binary joins, to get
$$p ee (q ee r) = (q ee r) ee p = (igvee \{q, r\}) ee p = igvee (\{q, r\} \cup \{p\}) =$$
$$igvee \{ q, r, p \} = igvee (\{p, q\} \cup \{r\}) = (igvee \{p, q\}) ee r = (p ee q) ee r.$$

By duality, we also have the associativity of binary meets.

## Commutativity

Let $L$ be a lattice and $p,q \in L$. Then $p ee q = igvee \{ p, q \} = q ee p$. Binary joins are therefore commutative. By duality, binary meets are also commutative.

## Idempotency

Let $L$ be a lattice and $p \in L$. Then $p ee p = igvee \{ p \} = p$. The property that for all $p \in L$, $p ee p = p$ is called *idempotency*. By duality, we also have the idempotency of meets: for all $p \in L$, $p \wedge p = p$.

## Absorption

Since $p \wedge q$ is the greatest *lower bound* of $\{p,q\}$, $p \wedge q \leq p$. Because $p \leq p$ and $(p \wedge q) \leq p$, $p$ is an upper bound of $\{p, p \wedge q\}$, and so $p ee (p \wedge q) \leq p$. On the other hand, $p ee (p \wedge q)$ is the least *upper bound* of $\{p, p \wedge q\}$, and so $p \leq p ee (p \wedge q)$. By anti-symmetry,  $p = p ee (p \wedge q)$.

%%

Closure under finite joins and meets
--------------------------------------------------------------

Let $L$ be a lattice and $S = \{ s_1, ..., s_n \}$ be some finite subset of $L$. Then an inductive argument shows that $igvee S$ exists.

%%hidden(Proof):
Here again, we will need Lemma 1, stated in the proofs of the four lattice properties.

Our proof proceeds by induction on the cardinality of $S$.

The base case is $igvee \{ s_1 \} = s_1 \in L$. For the inductive step, we suppose that $igvee \{s_1, ..., s_i \}$ exists. Then, applying lemma 1, we have $igvee \{s_1, ..., s_{i+1} \} = igvee \{s_1, ..., s_i \} ee s_{i+1}$. Applying our inductive hypothesis and closure under binary joins, we have $igvee \{s_1, ..., s_i \} ee s_{i+1}$ exists. Lattices are therefore closed under all *finite* joins, not just binary ones. Dually, lattices are closed under all finite meets.
%%

Basic positive examples
--------------

Here are two Hasse diagrams of posets which are lattices.

![A diamond shaped lattice](http://i.imgur.com/OlQnU07.png)

%%%comment:

dot source:

digraph G {
  node [width = 0.1, height = 0.1]
  edge [arrowhead = "none"]
  a [label = ""]
  b [label = ""]
  c [label = ""]
  d [label = ""]

  rankdir = BT;
  a -> b
  a -> c
  b -> d
  c -> d
}
%%%

![A cube shaped lattice](http://i.imgur.com/L0x074n.png)

%%%comment:

dot source:

digraph G {
  node [width = 0.1, height = 0.1]
  edge [arrowhead = "none"]
  a [label = ""]
  b [label = ""]
  c [label = ""]
  d [label = ""]
  e [label = ""]
  f [label = ""]
  g [label = ""]
  h [label = ""]

  rankdir = BT;
  a -> b
  a -> c
  a -> d
  b -> e
  b -> f
  c -> e
  c -> g
  d -> f
  d -> g
  e -> h
  f -> h
  g -> h
}

%%%

Basic negative examples
------------------------------------

Here are two Hasse diagrams of posets which are *not* lattices.

![A simple non-lattice](http://i.imgur.com/DAeuYz0.png)

%%%comment:

digraph G {
  node [width = 0.1, height = 0.1]
  edge [arrowhead = "none"]
  a [label = ""]
  b [label = ""]
  c [label = ""]
  d [label = ""]
  e [label = ""]
  f [label = ""]
  g [label = ""]
  h [label = ""]

  rankdir = BT;
  a -> b
  a -> c
  a -> d
  b -> e
  b -> f
  c -> e
  c -> g
  d -> f
  d -> g
  e -> h
  f -> h
  g -> h
}

%%%

In the above diagram, the two bottom elements have no common lower bounds. Therefore they have no meet, and so the depicted poset is not a lattice. However, it should be easy to verify that this poset is closed under binary joins.

![Another simple non-lattice](http://i.imgur.com/5Vqk87u.png)

%%%comment:

dot source:

digraph G {
  node [width = 0.1, height = 0.1]
  edge [arrowhead = "none"]
  a [label = ""]
  b [label = ""]
  c [label = ""]
  d [label = ""]

  rankdir = BT;
  a -> b
  c -> d
}

%%%

The Hasse diagram of this poset has two connected components. No element from the left component will have a meet or a join with any element from the right component. The depicted poset is therefore *not* a lattice.

The connecting lemma
--------------------

The connecting lemma states that for any lattice $L$ and $p,q \in L$, $p ee q = p \Leftrightarrow q \leq p$ and dually, $p \wedge q = p \Leftrightarrow q \geq p$. This simple but important lemma is so named because it establishes a connection between the lattice's join operator and its underlying poset order.

%%hidden(Proof):

We prove $p ee q = p \Leftrightarrow q \leq p$; the other part follows from duality. If $p ee q = p$, then $p$ is an upper bound of both $p$ and $q$, and so $q \leq p$. Going the other direction, suppose $q \leq p$. Since $p$ is and upper bound of itself by reflexivity, it then follows that $p$ is an upper bound of $\{p, q\}$. There cannot be a lesser upper bound of $\{p, q\}$ because it would not be an upper bound of $p$. Hence, $p ee q = p$.

%%

Lattices as algebraic structures
-------------------------------------

It's also possible to formulate lattices as [3gx algebraic structures] $\langle L, ee, \wedge
angle$ satisfying the associativity, commutativity, idempotency, and absorption laws described above. A poset $\langle L, \leq
angle$ can then be defined such that for $p, q \in L$, $p \leq q$ whenever $p ee q = q$. It can be shown that this poset is closed under binary meets and joins, and that these meets and joins are equal to the corresponding meets and joins of the algebraic lattice.

Additional material
-------------------------------

For more examples of lattices, see [574].
For some exercises involving the concepts introduced on this page, see [5ff].


Uncomputability

Imagine you were tasked with the mission of finding out which things can and cannot be calculated with a computer, given that you had infinite memory and infinite time.

Given the generality of computers, you might be tempted to say everything, but it turns out that this is not the case.

To begin tackling this problem, we need a formal notion of what it means to compute something. We are going to focus on computing **natural functions**; that is, [3jy functions] which transform [45h natural numbers] to natural numbers.

As natural numbers can be used to encode pretty much anything, this class of computations is far wider than one might initially think. For example, we can [-encode] finite sequences of arbitrary length of natural numbers in a single natural number, and we can encode words in sequences of numbers.

We are going to say that a computer program computes a certain natural function if on input $n$ it returns the result of applying such function to $n$, for every natural $n$. We note that programs do not have to halt on inputs; they may just run forever. In that case, we will say that the program is computing a partial function, which is not defined on some inputs.

%%%knows-requisite([2w0]):
Now, the first thing to notice is that the set of possible functions from $\mathbb{N}$ to $\mathbb{N}$ is not [-enumerable], while the set of possible programs, which are finite sequences of a finite quantity of symbols, is enumerable. Thus, there cannot be a one-to-one correspondence between functions and programs; there must necessarily exist a function which is not computed by any program.

So computers cannot compute everything!
%%%

----
##The diagonal function

We are going to construct a concrete example of a function which is not computable.

Imagine an infinite list of every possible Python program, associated with the infinite outputs each would produce if we feed it the sequence of natural numbers $1,2,3,...$.

Thus, we may end up with a table such as this:

program #1: 0->1, 1->X, 2->3,...

program #2:

etc...

Where an X means that the program does not halt on that input or does not return an output, and thus the function it represents is not defined there.

Now, we are going to construct an explicit function which is not computable using this table.

Let $DIAG$ be such that:
$$
DIAG(n) = \left\{
  \begin{array}{lr}
    1\ if\ M_n(n) = 0\\
    0\ if\ M_n(n)\mbox{ is undefined or defined and greater than 0}
  \end{array}
\right.
$$

The notation $M_n$ means *"the $n$th program in our enumeration of programs"*.
Let's see what we have done there. We are looking at the $n$th entry of every program and saying that this function on input $n$ outputs something different that the $n$th program. This technique is called [-diagonalization], and it is ubiquitous in computability and logic.

The function $DIAG$, known as the [-diagonal_function], is guaranteed to disagree with every program on at least one input. Thus, there cannot be a program which computes $DIAG$!

----
##The halting function
The reader may at this point be however not satisfied with such an unnatural example of an uncomputable function. After all, who is going to want to compute such a weird thing? Do not dismay, for there is a much better example: the halting function.

Let $HALT$ be defined as:
$$
HALT(n,x) = \left\{
  \begin{array}{lr}
    1\ if\ M_n(x)\mbox{ halts}\\
    0\ if\ M_n(x)\mbox{ does not halt }
  \end{array}
\right.
$$

That is, the function $HALT$ decides whether a given program is going to halt on a particular input. Now that is interesting.

To cite one imaginative use of the halting function, one could for example code a program which on any input simply ignores the input and starts looking for a counterexample of the Collatz conjecture, halting [-46m] it finds one. Then we could use the halting function to see whether the conjecture is true or false.

Sadly, $HALT$ is not computable. We are going to give two proofs of this fact, one by **reduction** to the diagonal function and other by **diagonalization**.

###Proof by reduction
Proofs by **reduction** are quite common in computability. We start by supposing that the function we are dealing with, in this case $HALT$, is computable, and then we proceed to use this assumption to show that if that was the case we could compute another function we know is uncomputable.

Suppose we have a program, which we are going to represent by $HALT$, which computes the halting function.

Then we could write a program such as this one:

    DIAGONAL(n):
        if HALT(n,n):
            return 1 if M_n==0 else return 0
    else return 0

Which computes the diagonal function. As we know such a program cannot possibly exist, we conclude by *reductio ad absurdum* that $HALT$ is not computable.

###Exercise
Prove that $HALT$ is reducible to $DIAGONAL$; ie, that if $DIAGONAL$ was computable we could compute $HALT$.

%%hidden(Show solution):
Suppose we want to know whether the program $PROG$ halts on input $n$. Then we first code the auxiliary program:

    AUX(x):
        PROG(n)
        return 0

That is, $AUX$ is the program which on any input first runs $PROG$ on input $n$ and then outputs $0$ if $PROG$ ever halts.

Then $DIAG(\ulcorner AUX\urcorner)==1$ iff $PROG$ halts on input $n$.
%%

###Proof by diagonalization
Another nifty proof of uncomputability comes from diagonalization.

As before, let's suppose $HALT$ is computable. Then we could write a program such as this one:

    prog(n):
        if HALT(n,n)==1:
           loop forever
        else return 0

The program halts on input $n$ iff $M_n$ does not halt on input $n$.

Now, let $\ulcorner prog\urcorner$ represent the number in the list of the program which occupies the $prog$ program. Here comes the question: does $prog$ halt when its input is $\ulcorner prog \urcorner$?

Supposing either yes or no leads to a contradiction, so we conclude that such a program cannot possibly exist, and it follows from the contradiction that $HALT$ is not computable.

---
So now you are ready to start your journey through the realm of computability!

Suggested next steps include examining other uncomputable functions such as the [ busy beaver sequence], or proceeding to [ complexity theory].


Iff

Iff is a shorthand for "if and only if". Its logical symbol is $\leftrightarrow$.

"A iff B" ($A \leftrightarrow B$) is quite distinct from "if A then B" ($A
ightarrow B$). Consider the stipulation "If the dog barks, then it will soon bite". This would not obligate the dog to bark a warning before biting. The "if" relation isn't symmetrical. As such, the dog might sometimes bite spontaneously, with no barking at all.

If we wanted to ensure that biting is always forewarned by barking, we would instead stipulate "**Iff** dog barks, then it will soon bite". This is equivalent to

* "the dog barks *if and only if* it will soon bite"

* "If the dog barks then it will soon bite, and if the dog bites it will have barked beforehand".

* "The dog barks only when it will soon bite"

With "iff", the implication runs in both directions.


Proof by contradiction

A proof by contradiction (a.k.a. *reductio ad absurdum*, reduction to absurdity) is a [5xz strategy] used by mathematicians to show that a mathematical statement is true by proving that the [-negation] of that statement leads to being able to prove that two opposite statements are simultaneously true (a [-contradiction]).

##Outline
The outline of the strategy is as follows:

1. Suppose that what you want to prove is false.
2. Derive a contradiction from it.
3. Conclude that the supposition is wrong.

##Examples
To illustrate the concept, we will do a simple, non rigorous reasoning. Imagine yourself in the next situation:

You are a defense lawyer. Your client is accused of stealing the cookie from the cookie jar. You want to prove her innocence. Lets say you have evidence that the jar is still sealed. Reason as follows:

1. Assume she stole the cookie from the cookie jar.
2. Then she would have had to open the jar.
3. The jar is still sealed.
4. For the jar to be sealed and for her to have opened it is a contradiction.
5. Hence the assumption in 1 is false (given the deductions below it are true).
6. Hence she did not steal the cookie from the cookie jar.

Now we will work through an actual mathematical example: we will show that $\sqrt 2$ is not [4zq rational]; that is, it cannot be expressed as the division of two [45h natural numbers].

1. We suppose that $\sqrt 2$ is rational, which means that there exist $a,b\in\mathbb{N}$ such that $\sqrt 2 = rac{a}{b}$. Without loss of generality, we can suppose that $a$ and $b$ [coprime have no common divisors], since otherwise we can just divide both numbers by their greatest common divisor to obtain a pair of numbers which satisfy both properties.
2. If this was the case, $b\sqrt2=a$. by squaring both sides, we arrive to $2b^2=a^2$. But then $2$ divides $a$, so we can express $a$ as $2n$ for some $n\in\mathbb{N}$. Substituing in the previous expression, we arrive to $2b^2 = 4n^2\implies b^2 =2 n^2$. By the same reasoning, $2$ must divide $b$, but then $a$ and $b$ have a common divisor! We have a contradiction.
3. We conclude that our original assumption that $\sqrt 2$ is rational must be false, and thus $\sqrt 2$ is not rational.

##When to use it
Proof by contradiction is one of the most useful techniques one can use to prove anything.

In particular, if you get stuck while doing a proof, resorting to proof by contradiction is a great way to keep exploring a problem from a different perspective. Even if you do not get to solve the problem, you may get a useful insight about the problem when performing the procedure of proof by contradiction.

Also, trying to do proof by contradiction may result in a [ counterexample], which dissolves the problem in question.

[todo: exercises]


Derivative

The derivative of $y$ with respect to $x$ describes the [-rate] at which $y$ changes, given a change in $x$. In particular, we consider how tiny changes in one variable affect another variable. To take the derivative of a [-3jy], we can draw a line that is [-tangent] to a graph of the function. The slope of the tangent line is the value of the derivative at that point. The derivative of a function $f(x)$ is itself a function: it returns, for any $x$, the slope of the line that is tangent to $f(x)$ at the point $(x, f(x))$.

[toc:]

## Examples ##

 - The time-derivative of your car's mileage is your car's speed (because your car's speed is how quickly your car's mileage changes *over time*)

 - The time-derivative of your car's speed is your acceleration (because acceleration means how quickly your *speed* is changing over time)

 - The time-derivative of human population size is the birth rate minus the death rate (because if we take the birth rate minus the death rate, that tells how quickly the population is changing over time)

 - The time-derivative of wealth is income minus spending (because if we take income minus spending, that tells us how your wealth is changing over time)

 - The time-derivative of blood pumped through your heart is the flow rate through the aortic valve (rates in general describe how something changes over time)

 - The time-derivative of charge on a capacitor is the current flowing to it (currents in general are time-derivatives of how much total stuff has flowed)

 - The time-derivative of how good of a life you'll have lived is how happy you are right now (according to [hedonic utilitarians](https://foundational-research.org/hedonistic-vs-preference-utilitarianism/))

Okay, let's take another stab at this. Time-derivatives, or derivatives "with respect to time," describe how things change over time. We can take derivatives with respect to other things too.

 - The derivative of your car's mileage with respect to how much fuel you've burned is your miles per gallon (because your mpg describes how your mileage changes per unit of fuel you burn)

 - The derivative of the temperature of a pot of water with respect to how much heat you blast it with is called the heat capacity of water

 - The derivative of [potential energy](https://en.wikipedia.org/wiki/Potential_energy) with respect to altitude is gravitational force (a higher up object has some stored energy that it would release if it fell; a lower down object has less "potential energy." The change in potential energy as you change the altitude is why there's a force in the first place.)

There were two goals with all those examples, one explicit, and one covert. The explicit one was to give you a sense for what derivatives are. The covert one was to quietly suggest that you will never understand the the way the world works unless you understand derivatives. But hey, look at you! You kind of understand derivatives already! Let's get to the math now, shall we?

## Setting Up The Math ##

You just got your new car.

![](http://o.aolcdn.com/commerce/autodata/images/USC30TSC021B021001.jpg)
It's a [Tesla](http://waitbutwhy.com/2015/06/how-tesla-will-change-your-life.html) because you care about the environment almost as much as you care about looking awesome. Your mileage is sitting at 0. The world is your oyster. At time $t = 0$, you put your foot on the accelerator. For the next few seconds, your mileage will be $4.7 t^2$, where your mileage is in meters, and $t$ is in seconds since you pressed your foot on the accelerator. Now the first question is this: if that equation tells us how many meters we've traveled after how many seconds, how fast are we going at any given point in time?

The astute reader will have noticed that this was the first example of a time-derivative that we gave: the time-derivative of your car's mileage is your car's speed. Let's think about this *sans* math for a second. If we know where we are at any time, we should be able to figure out how fast we're going. There isn't any extra information we need. The only question is how. Well, we take the derivative of the mileage with respect to time to get our speed. In other words:

$$rac{\mathrm{d}}{\mathrm{d} t} mileage = speed$$

That means the derivative with respect to $t$, where $t$ is the time in seconds. But we know what the mileage is, *in terms of $t$*. Our mileage is just $4.7 t^2$. So we can write:

$$rac{\mathrm{d}}{\mathrm{d} t} 4.7 t^2 = speed$$

## Solving The Math ##

Sorry to leave you hanging for a sec, but we're going to start with something a little simpler.

![](http://i66.tinypic.com/1p87pl.png)
$$distance\ traveled = 2t$$

If this is the graph of how far someone has traveled after how many seconds, we can see that every second they go 2 more meters. In other words, they are traveling 2 meters per second, which you might notice is the slope of this line. In general, the derivative of a function is like the slope of the function when you graph it out.

This works fine if our function is something like $distance\ traveled = 2t$. What if our function isn't a line though. What if it's $distance\ traveled = t^2$?

![](http://i64.tinypic.com/htu0pv.png)

Things that aren't lines don't have slopes. So if this is a graph of our distance traveled over time, it's not as easy to see how fast we were going. But let's say we want to see how fast we were going at $t=1$. If we zoom in enough on that curve, it will start to flatten out into a straight line until we can't tell the difference. The slope of *that* line is what gives us our speed. The process of taking a curve like this one, and getting the "slope" at any given point is called "taking the derivative."

Let's take the derivative of $d = t^2$, where $d$ is the distance and $t$ is the time. (We'll take the derivative with respect to $t$). Prepare yourself, now take a look at the graph down there.

![](http://i67.tinypic.com/mcxloj.png)

We know how to find the slope of a line if we're given two points, so we're going to do that, and then slowly move the points together until they're on top of each other. The coordinates of the points are shown above, and we can calculate the slope pretty easily by doing $rac{\Delta d}{\Delta t}$. This gives us a slope of 2.

Now let's say that our first point is at $(t,t^2)$, that our second point is $h$ units to the right, so it's coordinates are $((t+h),(t+h)^2)$. Now we have:
$$∆d=(t+h)^2-t^2$$
$$∆t=(t+h) - t$$
Algebradabra:
$$∆d=2ht + h^2$$
$$∆t=h$$
$$rac{\Delta d}{\Delta t}=rac{2ht + h^2}{h}=2t+h$$

Now as we make $h$ really small, the points get closer and closer together, and the slope of the line becomes $2t$. So when $t$ is $1$, the slope is $2$. And when $t$ is $5$, the slope is $10$.

We say the derivative of $t^2$ is $2t$. With similar logic, you can show that the derivative of $4.7t^2$ is $9.4t$. And *that* means that if you put your foot on the accelerator of your new Tesla at time $t=0$, your speed after $t$ seconds will be $9.4t$. After 1 second, you'll be traveling 9.4 meters per second. After 3 seconds, you'll be going 28.2 meters per second (or 64 mph).

## Concluding ##

That's what derivatives are. The Tesla case was just one example of actually finding the derivative of something. Obviously, if our distance traveled had been some totally different function of time, the derivative would have been different. We found that the derivative of $t^2$ is $2t$. Below is a list of other derivatives. You can imagine that if the function on the left was our distance traveled after a time $t$, the function on the right would be our speed at a time $t$. (All of the $c$'s and $n$'s are constants.)
$$rac{\mathrm{d} }{\mathrm{d} t}c=0$$
$$rac{\mathrm{d} }{\mathrm{d} t}ct=c$$
$$rac{\mathrm{d} }{\mathrm{d} t}ct^2=2ct$$
$$rac{\mathrm{d} }{\mathrm{d} t}ct^2=3ct^2$$
$$rac{\mathrm{d} }{\mathrm{d} t}ct^n=nct^{n-1}$$
$$rac{\mathrm{d} }{\mathrm{d} t}e^t=e^t$$
$$rac{\mathrm{d} }{\mathrm{d} t}sin(t)=cos(t)$$
$$rac{\mathrm{d} }{\mathrm{d} t}cos(t)=-sin(t)$$

If you're up for it, try to use the method we showed for solving derivatives to verify some of these. Good luck!

## See also

If you enjoyed this explanation, consider exploring some of [3d Arbital's] other [6gg featured content]!

Arbital is made by people like you, if you think you can explain a mathematical concept then consider [-4d6]!


The characteristic of the logarithm

[summary: Any time you find an output that adds whenever the input multiplies, you're probably looking at a (roughly) logarithmic relationship. For example, imagine storing a number using [42d digit wheels]. Every time the number goes up by a factor of 10, you need one additional digit wheel: It takes 3 wheels to store the number 500; 4 to store the number 5000; 5 to store the number 50000; and so on. Thus, the relationship between the magnitude of a number and the number of digits it takes to write down is logarithmic. This pattern is the key characteristic of the logarithm, and whenever you see an output adding when the input multiplies, you can measure the output using logarithms.]

Consider the interpretation of logarithms as [45q the cost of communicating a message]. Every time the number of possible messages to send doubles, your communication costs increase by the price of a coin, or whatever cheaper [-storage_medium] you have that can communicate one of two messages. It doesn't matter whether the number of possible messages goes from 4 to 8 or whether it goes from 4096 to 8192; in both cases, your costs go up by the price of a coin. It is the factor by which the set grew (or shrank) that affects the cost; not the absolute number of messages added (or removed) from the space of possibilities. If the space of possible messages halves, your costs go down by one coin, regardless of how many possibilities there were before the halving.

Algebraically, writing $f$ for the function that measures your costs, $c(x \cdot 2) =$ $c(x) + c(2),$ and, in general, $c(x \cdot y) =$ $c(x) + c(y),$ where we can interpret $x$ as the number of possible messages before the increase, $y$ as the factor by which the possibilities increased, and $x \cdot y$ as the number of possibilities after the increase.

This is the key characteristic of the logarithm: It says that, when the input goes up by a factor of $y$, the quantity measured goes up by a fixed amount (that depends on $y$). When you see this pattern, you can bet that $c$ is a logarithm function. Thus, whenever something you care about goes up by a fixed amount every time something else doubles, you can measure the thing you care about by taking the logarithm of the growing thing. For example:

- Consider the problem of checking whether a date is contained in a gigantic [sorted_list sorted list] of dates. You can do this by jumping to the middle of the list, seeing whether your date is earlier or later than the date in the middle, and thereby cutting the search space in half. Each time you do this, you cut the list of dates you're searching for in half, and so the total number of elements you need to look at goes up by one every time the size of the list doubles. Thus, the cost of searching an ordered list grows logarithmically in the size of the list. See also [binary_search].
- Consider a colony of bacteria where each bacterium in the colony reproduces once per day. Thus, the size of the colony roughly doubles each day. If you care about how long this colony of bacteria has been growing, you can measure the days by taking the logarithm of the number of bacteria in the colony. The logarithm (base 2) counts how many times the colony has doubled (and the log base 3 counts how many times it has tripled, and so on).
- The length of a number in [4sl] grows more-or-less logarithmically in the magnitude of the number: When the magnitude of the number goes up by a factor of 10, the number of digits it takes to write the number down grows by 1. However, this analogy is not perfect: Sometimes, multiplying a number by two does not increase its length (consider the number 300), and sometimes, dividing a number by 10 does not decrease its length by one digit (consider the number 1). See also [log_v_length Length isn't quite logarithmic].

Conversely, whenever you see a $\log_2$ in an equation, you can deduce that someone wants to measure some sort of thing by counting the number of doublings that another sort of thing has undergone. For example, let's say you see an equation where someone takes the $\log_2$ of a [-1rq]. What should you make of this? Well, you should conclude that there is some quantity that someone wants to measure which can be measured in terms of the number of doublings in that likelihood ratio. And indeed there is! It is known as [bayesian_evidence (Bayesian) evidence], and the key idea is that the strength of evidence for a hypothesis $A$ over its negation $\lnot A$ can be measured in terms of $2 : 1$ updates in favor of $A$ over $\lnot A$. (For more on this idea, see [ What is evidence?]).

In fact, a given function $f$ such that $f(x \cdot y) = f(x) + f(y)$ is almost guaranteed to be a logarithm function — modulo a few technicalities.

[checkbox(proove_things): This puts us in a position where you can derive all the main properties of the logarithm (such as $\log_b(x^n) = n \log_b(x)$ for any $b$) yourself. Check this box if that's something you're interested in doing.
y:
path: [4bz]
n:
]

[fixme: Conditional text depending what's next on the path.]