How many bits to a trit?

[summary: $\log_2(3) \approx 1.585.$ This can be interpreted a few different ways:

1. If you multiply the number of messages you might want to send by 3, then the cost of encoding the message will go up by 1.58 bits on average.
2. If you pack $n$ of independent and equally likely [3v9 3-messages] together into one giant $3^n$ message, then the cost (in bits) per individual 3-message drops as $n$ grows, ultimately converging to $\log_2(3)$ bits per 3-message as $n$ gets very large.
3. The infinite expansion of $\log_2(3) = 1.58496250072\ldots$ tells us not just how many bits it takes to send one 3-message $(\approx \lceil 1.585 \rceil = 2)$ but also how long it takes to send any number of 3-messages put together. For example, it costs 2 bits to send one 3-message; 16 bits to send 10; 159 bits to send 1000; 1585 to send 10,000; 15850 to send 100,000; 158497 to send 1,000,000; and so on.
]

There are [3nd $\log_2(3) \approx 1.585$] [3p0 bits] to a [3ww]. This can be interpreted a few different ways:

1. If you multiply the number of messages you might want to send by 3, then the cost of encoding the message will go up by 1.58 bits on average. See [+marginal_message_cost] for more on this interpretation.
2. If you pack $n$ of independent and equally likely [3v9 3-messages] together into one giant $3^n$ message, then the cost (in bits) per individual 3-message drops as $n$ grows, ultimately converging to $\log_2(3)$ bits per 3-message as $n$ gets very large. For more on this, see [+average_message_cost] and the [3zn GalCom example of encoding trits using bits].
3. The infinite expansion of $\log_2(3) = 1.58496250072\ldots$ tells us not just how many bits it takes to send one 3-message $(\approx \lceil 1.585 \rceil = 2)$ but also how long it takes to send any number of 3-messages put together. For example, it costs 2 bits to send one 3-message; 16 bits to send 10; 159 bits to send 1000; 1585 to send 10,000; 15850 to send 100,000; 158497 to send 1,000,000; and so on. For more on this interpretation, see the [log_series_of_ceilings "series of ceilings"] interpretation of logarithms.


Trit

[summary: The trinary analog of the [3y2]. Depending on the context, a trit can mean any of the following things:

1. Abstract trit: one thing from a set of three
2. Trit of data: the amount of data carried by an object that can be put into three states
3. Trit of information: $\log_2(3) \approx 1.58$ [shannon Sh] of information
4. Trit of evidence: $3:1$ evidence in favor of a specific hypothesis]

A trit is the trinary analog of the [3y2]: Where a bit says 2, a trit says 3. "Trit," like "bit," is overloaded:

- An abstract trit (analog of the [3y3 abstract bit]) is an element of a [3jz set] with  three elements. Thus, there are three abstract trits. They are often denoted 0, 1, and 2.
- A trit of data (analog of a [3p0 bit of data]) is the amount of data that can be carried by a physical object that can be placed in any one of three states.
- A trit of information (analog of a [shannon]) is the amount of information someone receives when they start out maximally uncertain about which one of three outcomes happened, and then they learn which one happened. A trit of information is $\log_2(3)\approx 1.58$ bits of information.
- A trit of evidence (analog of a [evidence_bit bit of evidence]) in favor of a hypothesis is an observation that is three times as likely if the hypothesis is true (versus if the hypothesis is false).

"Trit" is a portmanteau of "trinary digit."


Intradependent encodings can be compressed

Given an encoding scheme $E$ which gives an [3x4] of a message $m,$ we can in principle design a more efficient coding $E^\prime$ that gives a shorter encoding of $m.$ For example, $E$ encodes 8-letter English words as a series of letters, $m$ is aardvark, then $E(m)$ is intradependent (because you already know what the message is by the time you've seen "aardv"), so you can define an alternative encoding $E^\prime$ which encodes "aardvark" as just "aardv", thereby saving three letters.

The practice of taking an intradependent encoding and finding a more efficient one is known as [compression].


Intradependent encoding

[summary: An encoding $E(m)$ of a message $m$ is intradependent if the fact that $E(m)$ encodes $m$ can be deduced without looking at the whole encoding. For example, if you know you're going to see an 8-letter English word, and you see "aardv", you know that the message is "aardvark", even without looking at the last three letters.

Formally, intradependence is defined in terms of the mutual information shared by symbols in the encoding. Given an encoding is intradependent, it's possible (in principle) to develop a more efficient encoding that is less intradependent.]

We call an [encoding] $E(m)$ of a message $m$ "intradependent" if the fact that $E(m)$ encodes $m$ can be deduced without looking at the whole encoding. For example, imagine that you know you're going to see an 8-letter English word, and you see the letters "aardv". You can then deduce that the message is  "aardvark" without looking at the last three letters, because that's the only valid message that starts with "aardv".

(Seeing "aard" is not enough, because [https://en.wikipedia.org/wiki/Aardwolf aardwolfs] exist.)

In an intradependent encoding, some parts of the encoding carry information about the other parts. For example, once you've seen "aard", there are $26^4 = 456976$ possible combinations of the next four letters, but "aard" cuts the space down to just two possibilities — "vark" and "wolf". This means that the first four letters carry $\log_2(26^4) - 1 \approx 17.8$ bits of information about the last four. (The fifth letter carries one final bit of information, in the choice between "v" and "w". The last three letters carry no new information.)

Formally, the intradependence of an encoding is defined to be the sum of [mutual_information mutual information] between each symbol in the encoding and all the previous symbols. Given an intradependent encoding, it is possible in principle to design a more efficient encoding; see [+intradependent_compression].


Non-standard terminology

A tag for terminology that is Arbital-specific, Arbital-originated, or just not very common outside of Arbital.


Dependent messages can be encoded cheaply

Say you want to transmit a [3v9 2-message], a 4-message, and a 256-message to somebody. For example, you might want to tell them which way a coin came up, a cardinal direction, and a letter (encoded as one byte of [https://en.wikipedia.org/wiki/ASCII ASCII] text). How many [3p0 bits] of information does it take to transmit those three messages?

At most, it takes 11 bits: One for the coin, two for the direction, and 8 for the ASCII byte. For example, [heads, north, A] might be encoded as 0001000001, where 0 means "heads", 00 means "north", and 10000001 means "A" in ASCII.

But what if the messages depend on each other? What if the way that the cardinal direction was picked was by looking at the coin (such that you always say north if the coin lands heads, and south if the coin lands tails), and then the letter is picked by looking at the direction (such that you always say A for north, B for east, Z for south, and Y for west). Then how many bits does it take to transmit the message [heads, north, A]? Only one! Why? Because there are now only two possible messages: [heads, north, A] and [tails, south, Z]. Given two people who know the links between the three messages, all you need to tell them is how the coin came up, and they can figure out the entire message.

Formally, if you want to send multiple messages, and those messages share [mutual_information mutual information], then the amount of [information] it takes to encode all three messages together is less than the amount of information it takes to encode each one separately. (In fact, the amount of information you can save by encoding them both together is at most the amount of mutual information between them).

Looking at the collection of multiple messages as a single message, this fact is an immediate consequence of the fact that [ some messages being more likely than others means you can develop more efficient encodings for them].

Alternatively, this fact can be seen as a corollary of the fact that [intradependent_compression intradependent encodings can be compressed]: Given three messages $m_1, m_2, m_3$ and an encoding scheme $E$, the encoding $E(m_1)E(m_2)E(m_3)$ made by simply putting all three encodings together can be interpreted as a single [3x4 intradependent encoding] of the triplet $(m_1, m_2, m_3)$.


Cartesian product

The Cartesian product of two sets $A$ and $B,$ denoted $A \times B,$ is the set of all [ordered_pair ordered pairs] $(a, b)$ such that $a \in A$ and $b \in B.$ For example, if $\mathbb B \times \mathbb N$ is the set of all pairs of a [boolean] with a [natural_number natural number], and it contains elements like (true, 0), (false, 17), (true, 17), (true, 100), and (false, 101).

Cartesian product are often referred to as just "products."

Cartesian products can be constructed from more than two sets, for example, $\mathbb B^3 = \mathbb B \times \mathbb B \times \mathbb B$ is the set of all [boolean] [tuple 3-tuples]. (The $\times$ operator is [3h4 associative], so we don't need to write parenthesis when using it on a whole chain of sets.) A product of $n$ sets is called an $n$-ary product.


Information

[summary: Information is a measure of how much a message grants an observer the ability to predict the world. Information is observer-dependent: If someone tells both of us your age, you'd learn nothing, while I'd learn how old you are. Information is measured in [shannon shannons]. If you are about to observe a coin that came up either "heads" or "tails," one shannon is the difference between utter uncertainty about which way the coin came up, and total certainty that it came up "heads." Information can be used to quantify both (a) how much uncertainty a person has; and (b) how much uncertainty a message resolves.]

[summary(Technical): Given a probability distribution $\mathrm P$ over a set $O$ of possible observations, an observation $o \in O$ is said to carry $\log \frac{1}{\mathrm P(o)}$ units of information (with respect to $\mathrm P$), where the base of the logarithm determines the unit of information. The standard choice is log base 2, in which case the information is measured in [shannon shannons].]

Information is a measure of how much a message grants an observer the ability to predict the world. For a formal description of what this means, see [ Information: Formalization]. Information is observer-dependent: If someone tells both of us your age, you'd learn nothing, while I'd learn how old you are. [+3qq] gives us tools for quantifying and studying information.

Information is measured in [shannon shannons], which are also the units used to measure [uncertainty] and [entropy]. Given that you're about to observe a coin that certainly came up either heads or tails, one shannon is the difference between utter uncertainty about which way the coin came up, and total certainty that the coin came up heads. Specifically, the amount of information in an observation is quantified as the [3nd] of the [reciprocal] of the [1rf] that the observer assigned to that observation.

For a version of the previous sentence written in English, see [measuring_information Measuring information]. For a discussion of why this quantity in particular is called "information," see [ Information: Intro].

# Information vs Data

The word "information" has a precise, technical meaning within the field of [3qq]. Information is not to be confused with [data], which is a measure of how many messages a communication medium can distinguish in principle. For example, a series of three ones and zeros is 3 [3p0 bits of data], but the amount of information those three bits carry depends on the observer. Unfortunately, in colloquial usage (and even in some texts on information theory!) the word "information" is used interchangeably with the word "data"; matters are not helped by the fact that the standard unit of information is sometimes called a "bit" (the name for the standard unit of data), despite the fact that these units are distinct. The proper name for a binary unit of information is a "[shannon]."

That said, there are many links between information and data (and between shannons and bits). For instance:

* An object with a [3y1] of $n$ bits can carry anywhere between 0 and $\infty$ [shannon shannons] of information (depending on the observer), but the maximum amount of information an observer can consistently [expectation _expect_] from observing the object is $n$ shannons. For details, see [+expected_info_capacity].
* The number of shannons an observer gets from an observation is equal to the number of bits in the [encoding encoding] for that observation in their [ideal_encoding]. In other words, shannons measure the number of bits of data you would use to communicate that observation to someone who knows everything you know (except for that one observation). For details, see [+ideal_encoding] and [ Information as encoding length].

For more on the difference between information and data, see [+info_vs_data].

# Information and Entropy

[+entropy] is a measure on probability distributions which, intuitively, measures the total uncertainty of that distribution. Specifically, entropy measures the number of shannons of information that the distribution [expectation expects] to gain by being told the actual state of the world. As such, it can be interpreted as a measure of how much information the distribution says it is missing about the world. See also [ Information and entropy].

# Formalization

The amount of information an observation carries to an observer is the [3nd] of the [reciprocal] of the [1rf] that they assigned to that observation. In other words, given a probability distribution $\mathrm P$ over a set $O$ of possible observations, an observation $o \in O$ is said to carry $\log_2\frac{1}{\mathrm P(o)}$ shannons of information with respect to $\mathrm P$. A different choice for the base of the logarithm corresponds to a different unit of information; see also [ Converting between units of information]. For a full formalization, see [ Information: Formalization]. For an understanding of why information is logarithmic, see [ Information is logarithmic]. For a full understanding of why we call this quantity in particular "information," see [ Information: Intro].


Data capacity

The data capacity of an object is defined to be the [-3nd] of the number of different distinguishable states the object can be placed into. For example, a coin that can be placed heads or tails has a data capacity of $\log(2)$ units of [-data]. The choice of [-logarithm_base base] for the logarithm determines the unit of data; common units include the [-3p0], [-nat], and [-3wn] (corresponding to base 2, e, and 10). For example, the coin has a data capacity of $\log_2(2)=1$ bit, and a pair of two dice (which can be placed into 36 distinguishable states) have a data capacity of $\log_2(36) \approx 5.17$ bits. Note that the data capacity of a channel depends on the ability of an observer to distinguish different states of the object: If the coin is a penny, and I'm able to tell whether you placed the image of Abraham Lincoln facing North, South, West, or East (regardless of whether the coin is heads or tails), then that coin has a data capacity of $\log_2(8) = 3$ bits when used to transmit a message from you to me.

The data capacity of an object is closely related to the [channel_capacity channel capacity] of a [communications_channel communications channel]. The difference is that the channel capacity is the amount of data the channel can transmit per unit time (measured, e.g., in bits per second), while the data capacity of an object is the amount of data that can be encoded by putting the object into a particular state (measured, e.g., in bits).

Why is data capacity defined as the logarithm of the number of states an object can be placed into? Intuitively, because a 2GB hard drive is supposed to carry twice as much data as a 1GB hard drive. More concretely, note that if you have $n$ copies of a physical object that can be placed into $b$ different states, then you can use those to encode $b^n$ different messages. For example for example, with three coins, you can encode eight different messages: HHH, HHT, HTH, HTT, THH, THT, TTH, and TTT. The number of messages that the objects can encode grows exponentially with the number of copies. Thus, if we want to define a unit of message-carrying-capacity that grows linearly in the number of copies of an object (such that 3 coins hold 3x as much data as 1 coin, and a 9GB hard drive holds 9x as much data as a 1GB hard drive, and so on) then data [47k must] grow logarithmically with the number of messages.

The data capacity of an object bounds the length of the message that you can send using that object. For example, it takes about 5 bits of data to encode a single letter A-Z, so if you want to transmit an 8-letter word to somebody, you need an object with a data capacity of $5 \cdot 8 = 40$ bits. In other words, if you have 40 coins in a coin jar on your desk, and if we worked out an [encoding_scheme encoding scheme] (such as [https://en.wikipedia.org/wiki/ASCII ASCII]) ahead of time, then you can tell me any 8-letter word using those coins.

What does it mean to say that an object "can" be placed into different states, and what does it mean for those states to be "distinguishable"? [+3qq] is largely agnostic about the answer to those questions. Rather, _given_ a set of states that you claim you could put an object into, which you claim I can distinguish, information theory can tell you how to use those objects in a clever way to send messages to me. For more on what it means for two physical subsystems to communicate with each other by encoding messages into their environment, see [communication Communication].

Note that the amount of [-3xd] carried by an object is not generally equal to the data capacity of the object. For example, say a trusted mutual friend of ours places a coin "heads" if your eyes are blue and "tails" otherwise. If I already know what color your eyes are, then the state of the coin doesn't carry any information for me. If instead I was very sure that your eyes are blue, but actually you've been wearing blue contact lenses and your eyes are really brown, then the coin may carry significantly more than 1 [-452] of information for me. See also [+3xd] and [ Information is subjective]. The amount of information carried by an object to me (measured in [452 shannons]) may be either more or less than its data capacity — More if the message is very surprising to me, less if I already knew the message in advance. The relationship between the data capacity of an object and the amount of information it carries to me is that the maximum amount of information I can [4b5 expect] to gain by observing the object is equal to the data capacity. For more, see [ Information and data capacity].


Bit

[summary: The term "bit" refers to different concepts in different fields. The common theme across all the uses is the number 2. Usually, if you see something measured in bits (such as a bit of data, a bit of evidence, or a bit of information) it means that someone is counting the number of 2-factors in something (the number of possibilities, a likelihood ratio, a probability) using [3nd $\log_2$].]

[summary(set): An element of the set [boolean $\mathbb B$], which has two elements. These elements are sometimes called 0 and 1, or true and false, or yes and no. See [3y3 Bit (abstract)].]

[summary(data): A unit of [-data], namely, the amount of data required to single out one message from a set of two, or, equivalently, the amount of data required to cut a set of possible messages in half. See [3p0 Bit of data].]

[summary(information): A unit of [subjective_information information], namely, the difference in certainty between having no idea which way a coin is going to come up, and being entirely certain that it's going to come up heads. While this unit of information is colloquially known as a "bit" (for historical reasons), it is more properly known as a [452].]

[summary(evidence): A unit of [-bayesian_evidence evidence], namely, a $2 : 1$ [-1rq] in favor of one outcome over another. See [evidence_bit Bit of evidence] and [1zh].]

The term "bit" is [-overloaded]. It can mean any of the following:

1. An element of the set [boolean $\mathbb B$], which has two elements. These elements are sometimes called 0 and 1, or true and false, or yes and no. See [3y3 Bit (abstract)].
2. A unit of [data], namely, the amount of data required to single out one message from a set of two, or, equivalently, the amount of data required to cut a set of possible messages in half. See [3p0 Bit of data].
3. A unit of [subjective_information (subjective) information], namely, the difference in certainty between having no idea which way a coin is going to come up, and being entirely certain that it's going to come up heads. While this unit of information is colloquially known as a "bit" (for historical reasons), it is more properly known as a [452].
4. A unit of [bayesian_evidence evidence], namely, a $2 : 1$ [-1rq] in favor of one outcome over another. See [evidence_bit Bit of evidence] and/or [1zh].

The common theme across all the uses listed above is the number 2. An abstract bit is one of two values. A bit of data is a portion of a representation of a message that cuts the set of possible messages in half (i.e., the $\log_2$ of the number of possible messages). A bit of information (aka a [452]) is the amount of information in the answer to a yes-or-no question about which the observer was maximally uncertain (i.e., the $\log_2$ of a probability). A bit of evidence in favor of A over B is an observation which provides twice as much support for A as it does for B (i.e., the $\log_2$ of a [-1rq]). Thus, if you see someone using bits as units (a bit _of data,_ or a bit _of evidence,_ etc.) you can bet that they took a $\log_2$ of something somewhere along the way.

Unfortunately, abstract bits break this pattern, so if you see someone talking about "bits" without disambiguating what sort of bit they mean, the most you can be sure of is that they're talking about something that has to do with the number 2. Unless they're just using the word "bit" to mean "small piece," in which case you're in a bit of trouble.


Bit (abstract)

An abstract bit is an element of the set [boolean $\mathbb B$], which has two elements. An abstract bit is to $\mathbb B$ as a number is to [natural_number $\mathbb N$]. The set $\mathbb N$ contains a bunch of "numbers", and these numbers sometimes have many different labels, such as "three" and "3". Similarly, the set $\mathbb B$ contains a pair of "bits", which sometimes have many different labels. The abstract bits are sometimes labeled 0 and 1; true and false; yes and no; or + and -. The labels don't matter; the point is that the set $\mathbb B$ contains two elements, and we call those elements (abstract) bits.


Compressing multiple messages

How many [3p0 bits of data] does it take to encode an [3v9 $n$-message]? Naively, the answer is $\lceil \log_2(n) \rceil$ ([n_message_bit_length why?]): For example, it takes 5 bits to encode a 21-message, because 4 bits are only enough to encode 16 different messages, but 5 bits are enough to encode 32. The use of the [3vc] function implies an inefficiency: 2 bits are required to encode a 3-message, but 2 bits are enough to distinguish between four different possibilities. One of those possibilities is being wasted. That inefficiency can be reduced by encoding multiple $n$-messages at the same time. For example, while an individual 3-message requires 2 bits to encode, a series of 10 3-messages requires at most 16 bits to encode: $3^{10} < 2^{16}.$

Why is it that encoding ten 3-messages together (using bits) is cheaper than encoding ten 3-messages separately? Naively, there are three different factors that allow the combined encoding to be shorter than the sum of the separate encodings: The messages could have different likelihoods ([expected_compression allowing the combined message to be compressed in expectation]); the messages could be dependent on each other ([compressing_dependent_messages meaning they can be compressed]); and the mismatch between bits and 3-messages gets washed out as we put more three-messages together (see [+3wv]).

In fact, the first two factors are equivalent: 10 3-messages are equivalent to one $3^{10}$ message, and in general, [n_k_messages $n$ $k$-messages are equivalent to one $n^k$-message]. If the individual n-messages are dependent on each other, then different $n^k$ messages have different likelihoods: For example, if message 3 never follows message 2, then in the combined message, "32" never appears as a substring.

Thus, there are two different ways that an encoding of $k$ $n$-messages can be shorter than $k$ times the encoding of an $n$-message: The various combined messages can have different likelihoods, and the efficiency of the coding might increase. To study the effect of different likelihoods on the encoding length in isolation, we can [assume_maximum_efficiency assume that the codings are maximally efficient] and see how much additional [compression] the different likelihoods get us. To study code efficiency in isolation, we can [assume_equal_likelihood_messages assume each message is equally likely] and see how much additional compression we get as we put more $n$-messages together. In practice, real compression involves using both techniques at once.


Communication: magician example

Imagine that you and I are both magicians, performing a trick where I think of a card from a deck of cards and then leave the room, at which point you enter the room and have to guess which card I picked. To perform this trick, I leave some [3xd] in the room that allows you to figure out which card I picked. I do this by [encoding] a [message] into the environment (using objects that can carry [data]). Let's say that our props table includes (among other things) a coin and two dice (one red, one blue). Assume I have the ability to (without the audience noticing) place the coin and the two dice however I like before leaving the room. Those three objects have a [3y1] of $\log_2(2 \times 6 \times 6) \approx 6.17$ bits. I can use those objects to [physical_encoding represent] the encoding of the message I want to send to you.

The messages that I want to send are messages saying which card I'm thinking of. Example messages include the message saying that I'm thinking of the ace of spades, or messages saying that I'm thinking of the king of hearts.

The usual way that humans encode messages like these is to use physical objects like pressure waves in air or pixels on a screen or ink on paper arranged to look like the following patterns: "my card is the ace of spades" or "my card is the king of hearts." ([ Don't confuse the message with the encoding.])

Those sentences are pretty long, though: 28 and 29 letters, respectively. A coin and two dice can't carry 29 characters. Fortunately, we don't need all those characters: I can easily encode a message about which card I'm thinking of using only two characters, a rank and a suit, such as $A♠$ or $K♡.$ and so on. However, the coin and dice don't naturally carry rank and suit symbols on them.

However, there are only 52 possible messages I might want to encode, and the coin and dice can be put into $2 \cdot 6 \cdot 6 = 72$ different configurations. Thus, if we're clever, and [ choose a decoding scheme ahead of time], then I can set those objects in such a way that when you see them, you know which card I'm thinking of. Let's say we've agreed on the following [ decoding scheme]:

1. If the coin is heads, the card is red. If the coin is tails, the card is black.
2. If the red die is a 1, 2, or 3 then the suit is either spades or hearts. Otherwise, the suit is either clubs or diamonds.
3. If the red die is a 1 or 4, the card is either A, 2, 3, 4, 5, or 6. If the red die is a 2 or 5, the card is either 7, 8, 9, 10, J, or Q. If the red die is a 3 or a 6, the card is a K.
4. If the blue die is 1, the card is either A or 7 or K; if the blue die is 2, the card is either 2 or 8 or K; If the blue die is 3, the card is either 3 or 9 or K; and so on.

When you walk in the room, you can glance at the coin and the two dice and know which card I picked. If you see that the coin is tails-up, the red die is 1-up, and the blue die is 2-up, then you know that the card is black, either spades or hearts, between A and 6, and either 2 or 8 — in other words, the card is the 2 of spades.


Encoding trits with GalCom bits

There are [3nd $\log_2(3) \approx 1.585$] [3p0 bits] to a [3ww]. Why is it that particular value? Consider the [+3tz] thought experiment, in which you make money by sending information about the stock market across the celestial reaches. Let's say that Company X is about to publish a big earnings report, and that you want to send a message to your stock trader in the Vega star system that says one of three things: "Performance was in the top 1/3 of what I expected," "Performance was in the middle 1/3 of what I expected," or "Performance was in the bottom 1/3 of what I expected." You obviously don't want to send whole sentences; it suffices to transmit a single [3ww] of information.

That is, if you had control of a communication channel that could hold one of three signals (say, a wire that could be positively charged, negatively charged, or uncharged) which was connected to your computer on Vega, then you could program the computer to recognize a positive charge as "top 1/3 performance", a neutral charge as "middle 1/3 performance", and a negative charge as "bottom 1/3 performance." You could then program it to buy, hold, or sell respectively, and make your money.

Unfortunately, you don't control a wire connected to your computer on Vega. The computer on Vega is in a different star system, and the only way to communicate with it is through GalCom, the galactic communications network. And GalCom doesn't sell trits (which let you transmit one of three messages) it sells [3y2 bits] (which let you transmit one of two messages).

So, how many bits on GalCom will it cost you to send your trit?

(You're invited to pause and find some method for encoding a trit using bits.)

You can easily transmit a trit via two bits. If you reserve two bits on GalCom, you get to control two of the pulses of light on the sender, by choosing wether they are present or absent. This affects whether the receiver writes down 0 or 1, and those two binary digits will be sent to your machine on Vega. Thus, if you purchase two bits on GalCom, you can send one of four messages to your machine on Vega: 00, 01, 10, or 11. This is more than enough to encode a trit: You can program the machine such that 00 causes it to sell, 01 causes it to hold, and 10 causes it to buy. Then you have one code left over, namely 11, that you'll never send.

![One trit encoded, inefficiently, using 2 bits](http://i.imgur.com/fKPRo2s.png)

_To encode a bit using trits, you could purchase two bits and then make use of only 3 of the 4 available codes._

However, this is inefficient: You purchased one code (11) that you are never using. Is there any way to repurpose this code to make some of your money back?

(You are invited to pause and find some method of encoding a trit using two bits such that, sometimes, you're able to sell one bit back to GalCom and recover some of your losses.)

Here's one thing you could do: Program your machine on Vega such that it recognizes both 10 and 11 as "top 1/3 performance."

![Encoding a trit using two bits, such that 1/3 of the time you can sell one bit back to GalCom.](http://i.imgur.com/AF49bHM.png)

Then, by the time you've transmitted a 1, the machine on Vega already knows to start purchasing stock: It doesn't need the next bit. This means that it doesn't matter (to you) whether you send 10 or 11; these are two different ways of representing the same message. Thus, if Company X has performance in the top 1/3 of your expectations, you can sell one bit back to GalCom: Send 10 if the next bit they were going to send was a 0, and 11 if the next bit they were going to send was a 1. You've just shaved one bit off of the next message that was going to come through GalCom!

(The above exposition implies that you need to know the first bit of the next message, and that your computer on Vega needs to be set up to forward the second bit to where it needs to go. In practice, this sort of thing would be standard operating procedure on GalCom, which could use [ prefix free codes] to avoid the need for seeing/forwarding bits of following messages.)

If you encode your trit this way, then you reserve 2 bits in advance, and 1/3 of the time you get to sell one back. Thus, if you send lots and lots of trits using this encoding, you only pay $2 - \frac{1}{3} \approx 1.67$ galcoins per trit on average (assuming, [assume_independent_equally_likely_messages as we should], that each message occurs with equal frequency).

Can we do better? Yes. (TODO)


Logarithm tutorial overview

[summary: The logarithm tutorial covers the following six subjects:

1. What are logarithms?
2. Logarithms as a measure of length.
3. Logarithms as a measure of data and information.
4. The main charactristics and properties of logarithm functions.
4. All logarithm functions are the same (up to a multiplicative constant)
5. Why do logarithms make some calculations more efficient?]

The following topics are covered in Arbital's [3wj introductory guide to logarithms]:

## 1. Definition of the logarithm

What is a logarithm function, anyway? You may have been told that it's a function with a graph that looks like this:

![](https://upload.wikimedia.org/wikipedia/commons/thumb/1/17/Binary_logarithm_plot_with_ticks.svg/408px-Binary_logarithm_plot_with_ticks.svg.png "By Krishnavedala. Own work, CC BY-SA 3.0, goo.gl/VzKMzu")

which is true, but not the whole story. This tutorial begins by asking: [40j].

## 2. Log as length

Logarithms measure how long a number is, for a specific notion of "length" where fractional lengths are allowed. I don't know what $\log_{10}(\text{2,310,426})$ is, but I can immediately tell you that it's between 6 and 7, because it takes 7 digits to write 2,310,426:

$$\underbrace{\text{2,310,426}}_\text{7 digits}$$

Why is it $\log_{10}$ that measures length? What does it mean for a measure of "length" to be fractional? Why do logarithms say that it is 316 (rather than 500) that is 2.5 digits long? These questions and others are answered in [416].

## 3. Logs measure data

Logarithms measure how hard it is to represent messages using physical objects in the world. Let's say that we're magicians, and I want to tell you what card I'm thinking of by using sleight-of-hand to arrange some dice. How many 6-sided dice do I need to set (without the audience noticing) to send you a message that tells you which card I'm thinking of, assuming there are 52 possibilities? The lower bound is $\log_6(52).$ Why? The answer reveals a link between logarithms and the "data capacity" of physical objects.

This idea is explored across three pages: [427], [44l], and [45q].

## 4. The characteristic of the logarithm

 Any time you see a function $f$ such that $f(x \cdot y) = f(x) + f(y)$ for all $x, y \in$ [positive_reals $\mathbb R^+$], $f$ is a logarithm — more or less. [47k] describes the main idea, and the optional [4bz] page takes you through the proofs and discusses some technicalities.

## 5. There is only one logarithm

While there are many logarithm functions (one for each positive number except 1), there is a sense in which they're all doing exactly the same thing: Tapping into an intricate "[4gp logarithm lattice]".

## 6. Life in log-space

Empirically, logarithms have proved quite a useful tool for people and machines who have to do lots and lots of multiplications. Scientists and engineers used to use giant pre-computed tables of the logs of common numbers, and use those to make their calculations. Today, many modern learning algorithms (such as [https://en.wikipedia.org/wiki/AlphaGo AlphaGo]) manipulate the logs of probabilities instead of manipulating probabilities directly. Why? This tutorial concludes by exploring when and why it is easier to deal with the logarithms of things than it is to deal with the things themselves, on the page [4h0].

---

These six concepts are nowhere near all there is to say on the topic of logarithms. Logarithms have applications to many domains, including physics, computer science, calculus, number theory, and psychology. Logarithms have many interesting properties, such as nice [47d derivatives], nice [integral_calculus integrals], and interesting [calculating_logs approximation algorithms]. One of the bases of the logarithm, $\log_e,$ is "special." The logarithm gets [complex_log quite a bit more interesting] when extended to the [complex_plane complex plane]. Those are all more advanced topics, which aren't covered in this tutorial. If you want to learn more about those sorts of topics, see Arbital's [advanced_log_tutorial advanced logarithm tutorial].

This guide focuses on building a solid intuition for what the logarithm does, and why it has its most important properties.


What is a logarithm?

[summary: Logarithms are a group of [3jy functions] that take a number as input and produce another number. There is a logarithm function for every positive number $b$ [41m except 1], and the logarithm function for $b$ (called "the logarithm base $b$" or just "log base $b$") takes an input $x$ and counts how many times you have to multiply $1$ by $b$ to get $x$. For example, the log base 10 of 1000 is 3, because you have to multiply 1 by 10 three times to get 1000. The log base $b$ of $x$ is written $\log_b(x).$]

Logarithms are a group of [3jy functions] that map numbers to numbers. There is a logarithm function for every positive number $b$ [41m except 1], and the logarithm function for $b$ (called "the logarithm base $b$" or just "log base $b$") takes an input $x$ and counts how many times you have to multiply $1$ by $b$ to get $x$.

For example, the log base 10 of 1000 is 3, because you have to multiply 1 by 10 three times to get 1000, and the log base 2 of 16 is 4, because you have to multiply 1 by 2 four times to get 16. The log base $b$ of $x$ is written $\log_b(x).$

The logarithm function grows slowly. For example, $\log_{10}(x)$ is less than 1 until $x$ reaches 10, and it's less than 2 until $x$ reaches 100, and it's less than 6 until $x$ reaches 1,000,000 and so on. You've got to put a lot of oomph (specifically, a whole factor of 10) into the input in order to make the output go up by 1. Here's a graph of $\log_{10}(x)$ inching its way towards a measly 2 as $x$ makes it all the way out to 100:

![log10](http://i.imgur.com/UhUBlTt.png)

[fixme: Make this graph interactive.]

Logarithms may grow slowly, but they never stop growing: For every number $n$, no matter how large, there is an input $x$ such that $\log_{10}(x) > n.$

## Examples

1. $\log_{10}(10000) = 4,$ because you have to multiply 1 by 10 four times to get ten thousand: $1 \cdot 10 \cdot 10 \cdot 10 \cdot 10 = 10000.$
3. $\log_2(8) = 3,$ because $1 \cdot 2 \cdot 2 \cdot 2 = 8.$
4. $\log_3(9) = 2,$ because $1 \cdot 3 \cdot 3 = 9.$
5. $\log_{b}(1) = 0$ for any $b$, because no matter what $b$ is, you don't need to multiply 1 by $b$ any times to get 1.
6. $\log_{b}(b) = 1$ for any $b$, because no matter what $b$ is, if you multiply 1 by $b$ once, you get $b.$
7. $\log_{1.5}(3.375) = 3,$ because $1 \cdot 1.5 \cdot 1.5 \cdot 1.5 = 3.375.$

Question: What's $\log_3(27)$?

%%hidden(Answer):
Answer: 3, because $1 \cdot 3 \cdot 3 \cdot 3 = 27$.
%%

Question: What's $\log_4(16)$?

%%hidden(Answer):
Answer: 2, because $1 \cdot 4 \cdot 4 = 16$.
%%

Question: What's $\log_{10}(\text{1,000,000})$?

%%hidden(Answer):
Answer: 6, because $1 \cdot 10 \cdot 10 \cdot 10 \cdot 10 \cdot 10 \cdot 10 = 1,000,000$.
%%

All of the examples above are cases where the answer is a whole number, because $x$ is a [-power] of $b.$ Logarithms also give answers in cases where the answer is not a whole number. For example, $\log_{10}(500) \approx 2.7$.

What's _that_ supposed to mean? That if you multiply 1 by 10 about 2.7 times, you get 500?

Yes, precisely! If you multiply 1 by ten once, and multiply the result by ten another time, and then multiply that by ten 0.7 times, the result is roughly 500. What does it mean to multiply a number by ten 0.7 times? Well, is there a number $x$ such that if we multiply 1 by $x$ ten times, it's the same as multiplying 1 by 10 seven times? Yes! That number is roughly 5:

$$\underbrace{5 \cdot 5 \cdot \ldots 5}_\text{10 times} \approx \underbrace{10 \cdot 10 \cdot \ldots 10}_\text{7 times}$$

Multiplying by 5 (ten times) is about the same as multiplying by 10 (seven times), so multiplying by five (once) is about the same as multiplying by 10 (seven-tenths of a time). Thus, multiplying 1 by ten 2.7 times means multiplying it by 10 twice, then multiplying it by approximately 5, for a result of approximately 500.

This is why $\log_{10}(500) \approx 2.7$: that's how many times you need to multiply 1 by 10 to get 500.

If that answer isn't satisfactory yet, don't worry: Throughout the tutorial, we'll explore both (a) what this means, and (b) why this is the "right way" to multiply a number by $b$ fractionally-many times.

You might be saying to yourself, "Ok, I can see how 500 is kinda sorta one times ten 2.7 times, but how do you _figure that out?_ As far as I can tell, you pulled that "$5^{10}$ is approximately $10^7$" thing out of thin air. Where did that come from? How is the logarithm actually calculated?"

The annoying answer is that I calculated $\log_{10}(500)$ by typing "log10(500)" into [http://www.wolframalpha.com/input/?i=log10(500) Wolfram Alpha]. In more seriousness, in the coming posts, you'll learn how to find things like $\log_{10}(500)$ on your own.

That said, the outputs of logarithms are generally difficult to find, but useful once you've found them. Historically, scientists and engineers would pay money for giant tables of pre-computed logarithm values, because those sped up their practical calculations considerably. Why are the outputs of logarithm functions so convenient to use? That question will be answered later in this tutorial.

You might also be saying to yourself, "ok, I buy that there's a way to see 500 as '1 times 10, 2.7 times,' but what does that _mean?_ What's the point of looking at 500 that way?"

It's hard to give a short answer to that question, but only because there are so many good answers. One answer goes something like, "if you think of 500 as 2.7 tens, and you think of 8000 as 3.9 tens, then you can multiply the numbers on the left by adding the representations on the right". For example, $500 \cdot 8000$ is approximately $2.7 + 3.9 = 6.6$ tens.

But that's getting ahead of ourselves a bit. The most intuitive interpretation to start with is probably this one: Logarithms are measuring how "long" a number is, in terms of how many digits it takes to write that number down, for a generalized notion of "length."

It may seem that numbers always take a whole number of digits to write down: 139 and 931 are both written using three digits ('9', '3', and '1'). However, as we will see, there's a sense in which 139 is barely using its third digit, while 931 is using all three of its digits to almost their full extent. Logarithms quantify this intuition, and the precise answer that they give reveals some interesting facts about how many digits it costs to write a given number down.


Log as generalized length

[summary: 139 and 981 are both three digits long, but 139 is "almost" a 2-digit number and 981 is "almost" a 3-digit number. In a sense, 139 is using its three digits to a lesser extent. Logarithms formalize and quantify this intuition. You can estimate the logarithm (base 10) of a number by counting how many digits it has, and logarithms give us a notion of how much a number is actually using those digits.]

Here are a handful of examples of how the [-3nd] base 10 behaves. Can you spot the pattern?
$$
\begin{align}
\log_{10}(2) &\ \approx 0.30 \\
\log_{10}(7) &\ \approx 0.85 \\
\log_{10}(22) &\ \approx 1.34 \\
\log_{10}(70) &\ \approx  1.85 \\
\log_{10}(139) &\ \approx 2.14 \\
\log_{10}(316) &\ \approx 2.50 \\
\log_{10}(123456) &\ \approx 5.09 \\
\log_{10}(654321) &\ \approx 5.82 \\
\log_{10}(123456789) &\ \approx 8.09 \\
\log_{10}(\underbrace{987654321}_\text{9 digits}) &\ \approx 8.99
\end{align}
$$

Every time the input gets one digit longer, the output goes up by one. In other words, the output of the logarithm is roughly the length — measured in digits — of the input. ([437 Why?])

Why is it the log base 10 (rather than, say, the log base 2) that roughly measures the length of a number? Because numbers are normally represented in [-4sl], where each new digit lets you write down ten times as many numbers. The logarithm base 2 would measure the length of a number if each digit only gave you the ability to write down twice as many numbers. In other words, the log base 2 of a number is roughly the length of that number when it's represented in [-56q] (where $13$ is written $\texttt{1101}$ and so on):

$$
\begin{align}
\log_2(3) = \log_2(\texttt{11}) &\ \approx 1.58 \\
\log_2(7) = \log_2(\texttt{111}) &\ \approx 2.81 \\
\log_2(13) = \log_2(\texttt{1101}) &\ \approx 3.70 \\
\log_2(22) = \log_2(\texttt{10110}) &\ \approx 4.46 \\
\log_2(70) = \log_2(\texttt{1010001}) &\ \approx 6.13 \\
\log_2(139) = \log_2(\texttt{10001011}) &\ \approx 7.12 \\
\log_2(316) = \log_2(\texttt{1100101010}) &\ \approx 8.30 \\
\log_2(1000) = \log_2(\underbrace{\texttt{1111101000}}_\text{10 digits}) &\ \approx 9.97
\end{align}
$$

If you aren't familiar with the idea of numbers represented in other number bases besides 10, and you want to learn more, see the [-number_base_tutorial].

Here's an interactive visualization which shows the link between the length of a number expressed in base $b$, and the logarithm base $b$ of that number:

[visualization(log-length-demo):]

As you can see, if $b$ is an [-48l] greater than 1, then the logarithm base $b$ of $x$ is pretty close to the number of digits it takes to write $x$ in base $b.$

Pretty close, but not exactly. The most obvious difference is that the outputs of logarithms generally have a fractional portion: the logarithm of $x$ always falls a little short of the length of $x.$ This is because, insofar as logarithms act like the "length" function, they generalize the notion of length, making it [continuity continuous].

What does this fractional portion mean? Roughly speaking, logarithms measure not only how long a number is, but also how much that number is _really_ using its digits. 12 and 99 are both two-digit numbers, but intuitively, 12 is "barely" two digits long, whereas 97 is "nearly" three digits. Logarithms formalize this intuition, and tell us that 12 is really only using about 1.08 digits, while 97 is using about 1.99.

Where are these fractions coming from? Also, looking at the examples above, notice that $\log_{10}(316) \approx 2.5.$ Why is it 316, rather than 500, that logarithms claim is "2.5 digits long"? What would it even _mean_ for a number to be 2.5 digits long? It very clearly takes 3 digits to write down "316," namely, '3', '1', and '6'. What would it mean for a number to use "half a digit"?

Well, here's one way to approach the notion of a "partial digit." Let's say that you work in a warehouse recording data using digit wheels like they used to have on old desktop computers.

![A digit wheel](http://www.cl.cam.ac.uk/~djg11/howcomputerswork/mechanical-counter3.jpg)

Let's say that one of your digit wheels is broken, and can't hold numbers greater than 4 — every notch 5-9 has been stripped off, so if you try to set it to a number between 5 and 9, it just slips down to 4. Let's call the resulting digit a [4sj 5-digit], because it can still be stably placed into 5 different states (0-4). We could easily call this 5-digit a "partial 10-digit."

The question is, how much of a partial 10-digit is it? Is it half a 10-digit, because it can store 5 out of 10 values that a "full 10-digit" can store? That would be a fine way to measure fractional digits, but it's not the method used by logarithms. Why? Well, consider a scenario where you have to record lots and lots of numbers on these digits (such that you can tell someone how to read off the right data later), and let's say also that you have to pay me one dollar for every digit that you use. Now let's say that I only charge you 50¢ per 5-digit. Then you should do all your work in 5-digits! Why? Because two 5-digits can be used to store 25 different values (00, 01, 02, 03, 04, 10, 11, ..., 44) for \$1, which is way more data-stored-per-dollar than you would have gotten by buying a 10-digit.%note:You may be wondering, are two 5-digits really worth more than one 10-digit? Sure, you can place them in 25 different configurations, but how do you encode "9" when none of the digits have a "9" symbol written on them? If so, see [ The symbols don't matter].%

In other words, there's a natural exchange rate between $n$-digits, and a 5-digit is worth more than half a 10-digit. (The actual price you'd be willing to pay is a bit short of 70¢ per 5-digit, for reasons that we'll explore shortly). A 4-digit is also worth a bit more than half a 10-digit (two 4-digits lets you store 16 different numbers), and a 3-digit is worth a bit less than half a 10-digit (two 3-digits let you store only 9 different numbers).

We now begin to see what the fractional answer that comes out of a logarithm actually means (and why 300 is closer to 2.5 digits long that 500 is). The logarithm base 10 of $x$ is not answering "how many 10-digits does it take to store $x$?" It's answering "how many digits-of-various-kinds does it take to store $x$, where as many digits as possible are 10-digits; and how big does the final digit have to be?" The fractional portion of the output describes how large the final digit has to be, using this natural exchange rate between digits of different sizes.

For example, the number 200 can be stored using only two 10-digits and one 2-digit. $\log_{10}(200) \approx 2.301,$ and a 2-digit is worth about 0.301 10-digits. In fact, a 2-digit is worth _exactly_ $(\log_{10}(200) - 2)$ 10-digits. As another example, $\log_{10}(500) \approx 2.7$ means "to record 500, you need two 10-digits, and also a digit worth at least $\approx$70¢", i.e., two 10-digits and a 5-digit.

This raises a number of additional questions:

---

__Question:__ Wait, there _is_ no digit that's worth 50¢. As you said, a 3-digit is worth less than half a 10-digit (because two 3-digits can only store 9 things), and a 4-digit is worth more than half a 10-digit (because two 4-digits store 16 things). If $\log_{10}(316) \approx 2.5$ means "you need two 10-digits and a digit worth at least 50¢," then why not just have the $\log_{10}$ of everything between 301 and 400 be 2.60? They're all going to need two 10-digits and a 4-digit, aren't they?

__Answer:__ The natural exchange rates between digits is actually _way more interesting_ than it first appears. If you're trying to store either "301" or "400", and you start with two 10-digits, then you have to purchase a 4-digit in both cases. But if you start with a 10-digit and an 8-digit, then the digit you need to buy is different in the two cases. In the "301" case you can still make do with a 4-digit, because the 10, 8, and 4-digits together give you the ability to store any number up to $10\cdot 8\cdot 4 = 320$. But in the "400" case you now need to purchase a 5-digit instead, because the 10, 8, and 4 digits together aren't enough. The logarithm of a number tells you about _every_ combination of $n$-digits that would work to encode the number (and more!). This is an idea that we'll explore over the next few pages, and it will lead us to a much better understanding of logarithms.

---

__Question:__ Hold on, where did the 2.60 number come from above? How did you know that a 5-digit costs 70¢? How are you calculating these exchange rates, and what do they mean?

__Answer:__ Good question. In [427], we'll explore what the natural exchange rate between digits is, and why.

---

__Question:__ $\log_{10}(100)=2,$ but clearly, 100 is 3 digits long. In fact, $\log_b(b^k)=k$ for any integers $b$ and $k$, but $k+1$ digits are required to represent $b^k$ in base $b$ (as a one followed by $k$ zeroes). Why is the logarithm making these off-by-one errors?

__Answer:__ Secretly, the logarithm of $x$ isn't answering the question "how hard is it to write $x$ down?", it's answering something more like "how many digits does it take to record a whole number less than $x$?" In other words, the $\log_{10}$ of 100 is the number of 10-digits you need to be able to name any one of a hundred numbers, and that's two digits (which can hold anything from 00 to 99).

---

__Question:__ Wait, but what about when the _input_ has a fractional portion? How long is the number 100.87? And also, $\log_{10}(100.87249072)$ is just a hair higher than 2, but 100.87249072 is way harder to write down that 100. How can you say that their "lengths" are almost the same?

__Answer:__ Great questions! The length interpretation on its own doesn't shed any light on how logarithm functions handle fractional inputs. We'll soon develop a second interpretation of logarithms which does explain the behavior on fractional inputs, but we aren't there yet.

Meanwhile, note that the question "how hard is it to write down an integer between 0 and $x$ using digits?" is _very different_ from the question of "how hard is it to write down $x$"? For example, 3 is easy to write down using digits, while [49r $\pi$] is very difficult to write down using digits. Nevertheless, the log of $\pi$ is very close to the log of 3. The concept for "how hard is this number to write down?" goes by the name of [-complexity]; see the [+Kolmogorov_complexity_tutorial] to learn more on this topic.

---

__Question:__ Speaking of fractional inputs, if $0 < x < 1$ then the logarithm of $x$ is _negative._ How does _that_ square with the length interpretation? What would it even mean for the length of the number $\frac{1}{10}$ to be $-1$?

__Answer:__ Nice catch! The length interpretation crashes and burns when the inputs are less than one.

---

The "logarithms measure length" interpretation is imperfect. The connection is still useful to understand, because you _already_ have an intuition for how slowly the length of a number grows as the number gets larger. The "length" interpretation is one of the easiest ways to get a gut-level intuition for what logarithmic growth _means._ If someone says "the amount of time it takes to search my database is logarithmic in the number of entries," you can get a sense for what this means by remembering that logarithmic growth is like how the length of a number grows with the magnitude of that number:

[visualization(log-length-animation):]

The interpretation doesn't explain what's going on when the input is fractional, but it's still one of the fastest ways to make logarithms start feeling like a natural property on numbers, rather than just some esoteric function that "[3wr inverts exponentials]." Length is the quick-and-dirty intuition behind logarithms.

For example, I don't know what the logarithm base 10 of 2,310,426 is, but I know it's between 6 and 7, because 2,310,426 is seven digits long.

$$\underbrace{\text{2,310,426}}_\text{7 digits}$$

In fact, I can also tell you that $\log_{10}(\text{2,310,426})$ is between 6.30 and 6.48. How? Well, I know it takes six 10-digits to get up to 1,000,000, and then we need something more than a 2-digit and less than a 3-digit to get to a number between 2 and 3 million. The natural exchange rates for 2-digits and 3-digits (in terms of 10-digits) are 30¢ and 48¢ respectively, so the cost of 2,310,426 in terms of 10-digits is between \$6.30 and \$6.48.

Next up, we'll be exploring this idea of an exchange rate between different types of digits, and building an even better interpretation of logarithms which helps us understand what they're doing on fractional inputs (and why).


Logarithm base 1

[summary: There is no [-3nd] base 1, because no matter how many times you multiply 1 by 1, you get 1. If there _were_ a log base 1, it would send 1 to 0 (because $\log_b(1)=0$ for every $b$), and it would also send 1 to 1 (because $\log_b(b)=1$ for every $b$), which demonstrates some of the difficulties with $\log_1.$ In fact, it would need to send 1 to every number, because $\log(1 \cdot 1) = \log(1) + \log(1)$ and so on. And it would need to send every $x > 1$ to $\infty$, and every $0 < x < 1$ to $-\infty,$ and those aren't numbers, so there's no logarithm base 1.

But _if there was,_ it would be a [-multifunction] with values in the [extended_reals extended real numbers]. This is actually a perfectly valid way to define $\log_1,$ though doing so is not necessarily a good idea.]

There is no [-3nd] base 1, because no matter how many times you multiply 1 by 1, you get 1. If there _were_ a log base 1, it would send 1 to 0 (because $\log_b(1)=0$ for every $b$), and it would also send 1 to 1 (because $\log_b(b)=1$ for every $b$), which demonstrates some of the difficulties with $\log_1.$ In fact, it would need to send 1 to every number, because $\log(1 \cdot 1) = \log(1) + \log(1)$ and so on. And it would need to send every $x > 1$ to $\infty$, and every $0 < x < 1$ to $-\infty,$ and those aren't numbers, so there's no logarithm base 1.

But if you _really_ want a logarithm base $1$, you can define $\log_1$ to be a multifunction [3js from] [positive_real_numebrs $\mathbb R^+$] [3lg to] $\mathbb R \cup \{ \infty, -\infty \}.$ On the input $1$ it outputs $\mathbb R$. On every input $x > 1$ it outputs $\{ \infty \}$. On every input $0 < x < 1$ it outputs $\{ -\infty \}$. This multifunction can be made to satisfy all the basic [4bz properties of the logarithm], if you interpret $=$ as $\in$, $1^{\{\infty\}}$ as the [interval_notation interval] $(1, \infty)$, and $1^{\{-\infty\}}$ as the interval $(0, 1)$. For example, $0 \in \log_1(1)$, $1 \in \log_1(1)$, and $\log_1(1) + \log_1(1) \in \log_1(1 \cdot 1)$. $7 \in log_1(1^7)$, and $15 \in 1^{\log_1(15)}$. This is not necessarily the best idea ever, but hey, the [complex_log final form] of the logarithm was already a multifunction, so whatever. See also [log_is_a_multifunction].

While you're thinking about weird logarithms, see also [4c8].


Exchange rates between digits

[summary: There is a natural exchange rate between types of digits. For example, a 100-digit is worth exactly twice as much as a 10-digit, because you can build a 100-digit out of two 10-digits. Furthermore, a 10-digit is worth between 3x and 4x as much as a 2-digit, because you can emulate a [-42d] using between 3 and 4 coins. The cost of an $n$-digit in terms of $b$-digits is $\log_b(n).$]

Let's say that you want to [storing_numbers store a lot of data], using physical objects such as coins, dice, or [42d digit wheels]. For concreteness, let's say that you're going to be given an arbitrary number between one and $2^\text{3,000,000,000,000}$ ([numbers_are_data a.k.a. a terabyte of data]), and you need to make sure you have enough physical objects to write that number down.

Let's say that a coin costs 1¢. Because [ $n$ coins can be used to encode any one of $2^n$ different possibilities], you could get the job done with three trillion coins, which would let you encode the number (in [binary_notation binary]) for the low low cost of thirty billion dollars. (At the time of writing this, three trillion bits of data will actually run you about \$60, but leave that aside for now. Pretend you can't use modern transistors, and you have to use comparatively large objects such as coins to store data.)

Instead of buying coins, you could also buy [42d digit wheels], and use those instead. Because digit wheels can store more values than coins, it takes fewer digit wheels than coins to do the job. If digit wheels also cost 1¢, you should just buy digit wheels; that will cost a mere $9B. If instead digit wheels cost a million dollars apiece, you should probably buy coins instead. Somewhere in between those two prices, digit wheels switch from being a good deal to being a bad deal. The question is, if coins are worth 1¢, then how much is a digit wheel worth? At what price should you buy wheels instead of coins, and at what price should you buy coins instead of wheels?

At a first glance, it may seem like the fair price is 5¢ per wheel. A coin can store two values (by being placed 'heads' or 'tails'), while a digit wheel can store ten. Ten is five times larger than 2, so perhaps the price of a digit wheel should be five times the price of a coin. However, if digit wheels cost 5¢, you should ignore digit wheels entirely and buy coins. Why? Because for only 4¢, you could buy four coins, which can hold $2^4=16$ different values. Four coins store a [4sj 16-digit], and a 16-digit is worth more than a 10-digit, so you would have no reason to even consider buying digit wheels at a price of 5¢ per wheel.

So what is the fair price for a digit wheel? 4¢ is still too high, because that makes a 10-digit cost the same as a 16-digit, and the 16-digit is always better at that price. What about 3¢? At that price, the answer is much less clear. On the one hand, spending 3¢ on coins gets you the ability to write down only 8 possibilities, while spending 3¢ on wheels lets you write down 10 different possibilities. On the other hand, if you're trying to store the number 101, you need either 7 coins (7¢, because $2^6 < 101 < 2^7$) or 3 wheels (one per digit, because a wheel stores a digit, for a total of 9¢), in which case the coins are better.

Given that digit wheels can store ten values and cost 3¢ each, and coins can store two values and cost 1¢ each:

__Question:__ When storing the number 8000, should you buy coins or wheels?

%%hidden(Answer):
Wheels. To store 8000, you need 13 coins (because $2^{12} < 8000 < 2^{13}$) for a cost of 13¢, but only 4 wheels (because 8000 is 4 digits long) for a cost of 12¢.
%%

__Question:__ When storing the number 15,000, should you buy coins or wheels?

%%hidden(Answer):
Coins. Storing 15,000 costs 14 coins ($2^{13} < 15,000 < 2^{14}$) or 5 wheels; given that wheels cost 3¢ and coins cost 1¢, the 14 coins are cheaper.
%%

__Question:__ When storing the number 230,000, should you buy coins or wheels?
%%hidden(Answer):
It doesn't matter. Storing 230,000 takes either 18 coins or 6 wheels, which costs 18¢ either way.
%%

At 3¢ per digit wheel, whether you should buy coins or wheels depends on which number you want to store.

But _once the number you're storing gets large enough,_ the digit wheels become a better deal, and stay that way. In this case, the final transition happens when the number you're storing is 11 digits long or longer. Why? Say that the number $x$ you want to store is $n$ digits long in decimal notation. $n$ digit wheels can always encode more possibilities than $3n$ coins (because $10^n > 2^{3n}$ for all $n$), but sometimes $x$ can be written using fewer than $3n$ coins (as in the case of 15,000). This stops happening once $n$ is so large than $10^n$ is $2^3$ times larger than $2^{3n}$, at which point no $n$-digit number can be encoded using $2^{3(n-1)}$ or fewer coins. This happens once $n \ge 11,$ i.e., once $x$ is at least 11 digits long. ([ Proof.])

This probably isn't too surprising: For 3¢ you can either buy 3 coins or one digit wheel. The coins store 8 possibilities, the wheel stores 10, and it's no surprise that a huge collection of 10-digits is unambiguously better than an equally sized collection of 8-digits.

So digit wheels are worth more than 3x as much as a coin, and less than 4x. How can we find the right price exactly? We could use exactly the same argument as above to check whether 3.5¢ is too high or too low, except that it's not clear what it would mean to buy "three and a half coins." We can get around that problem by considering ten-packs of coins vs ten-packs of digit wheels. If wheels cost 3.5¢ and coins cost 1¢, then with 35¢, you could either buy 10 wheels or 35 coins. Which encodes more possibilities? The coins, because $10^{10} < 2^{35}.$ Thus, by a [ generalized version of the argument above], for numbers sufficiently large, the coins are a better deal than the wheels — at that price, you should buy wheels.

As you can verify, $2^{33} < 10^{10} < 2^{34},$ so the fair price must be between 3.3¢ and 3.4¢. And $2^{332} < 10^{100} < 2^{333},$ so it must be between 3.32¢ and 3.33¢. We could keep going, and keep getting digits of the fair price. And we could keep going forever, because the fair price is [-irrational]. ([logs_are_usually_irrational Why?])

In general, if digit wheels cost $p$ times as much as coins, the [ general argument] we've been using says that if you're storing a large number and $2^p > 10$ then you should buy coins, whereas if $2^p < 10$ then you should buy digit wheels. Thus, the fair price must be the number $p$ such that $2^p = 10,$ and its first three digits are 3.32.

In other words, while a digit wheel holds 5 times as many values as a coin, it is only worth about 3.32 times as much as a coin (in terms of its ability to store data). Why? _Because it takes about 3.32 coins to emulate a digit-wheel._ What does that mean? Well, 3 coins is not enough to emulate a digit wheel (3 give you only 8 possibilities), 4 coins is more than enough (4 give you 16 possibilities). And 33 coins is not enough to emulate 10 digit wheels, but 34 coins are more than enough. And 332 coins are not enough to emulate 100 digit wheels, but 333 coins are more than enough.

A digit wheel isn't worth 5 times as much as a coin because, when you buy multiple coins, their data storage capacity _multiplies,_ rather than adding. Each new coin lets you say _twice as many things,_ not just two more things. With five coins you can store $2 \cdot 2 \cdot 2 \cdot 2 \cdot 2 = 32$ different values, not $2 + 2 + 2 + 2 + 2 = 10.$ When finding the exchange rate between coins and digit wheels, the question is not "How many more values does a digit wheel have written on it?" Rather, the question is "How many coins does it take to emulate a digit wheel?" The answer to _that_ question is "More than 332 coins per hundred digit wheels, less than 333 per hundred."

The fair price $p$ of digit wheels (in terms of coins) is the number $p$ such that $2^p = 10$ exactly. We have a name for this number, and it's $\log_2(10),$ and the argument described above is a [ simple algorithm for computing logarithms]. (Logarithm outputs are "difficult to calculate but useful to have," in general.)

Thus, we can look at $\log_b(x)$ as describing the fair price of $x$-digits in terms of $b$-digits. For example, if coins are worth 1¢, then dice are worth $\log_2(6) \approx 2.58$ cents, as you can verify by checking that $2^2 < 6 < 2^3$ and $2^{25} < 6^{10} < 2^{26}$ and $2^{258} < 6^{100} < 2^{259}.$ (Exercise: Find the next digit of $\log_2(6)$ using this method.)

This interpretation is a great intuition pump for what $\log_b(x)$ "really means" if $b$ and $x$ are whole-numbers. Imagine two sets of objects, one (say, weird dice) that can be placed in $b$ different states, and another (say, weird wheels) that can be placed in $x$ different states. Imagine you're storing a really really large number, and imagine that the $b$-objects cost 1¢. At what price of $x$-objects would you be indifferent between using $x$-objects versus $b$-objects to store data? In other words, how many $b$-objects does it take to [emulating_digits "emulate"] an $x$-object?

Probing this interpretation also explains many of the properties of the logarithm. For example, the fact that $\log_b(x)$ is the fair price of $x$-digits in terms of $b$-digits immediately implies that $\log_x(b) = \frac{1}{\log_b(x)}$, because if an $x$-digit is worth three $b$-digits then a $b$-digit must be worth a third of an $x$-digit.

This interpretation still doesn't shed light onto what logarithms are doing when their inputs are not whole numbers. For example, what's $\log_{1.5}(2.5)$? What the heck would an object that can be placed into "1.5 different states" even be? As we will see shortly, this notion of there being a "natural exchange rate" between digits reveals an interpretation of the logarithm that explains what it's doing with fractional inputs.


Digit wheel

[summary:
A mechanical device for storing a number from 0 to 9.

![](http://www.cl.cam.ac.uk/~djg11/howcomputerswork/mechanical-counter3.jpg)]

A mechanical device for storing a number from 0 to 9. Devices such as these were often attached to the desks of engineers, as part of historical "desktop computers."

![](http://www.cl.cam.ac.uk/~djg11/howcomputerswork/mechanical-counter3.jpg)

Each wheel can stably rest in any one of ten states. Digit wheels are often useful when it comes to thinking about how much data a physical object can store, as they were physical objects that were clearly designed for storing data. See also [3y1].


Graham's number

[summary:
Graham's number is a... rather large number. Tim Urban of _Wait but Why_ gives a [http://waitbutwhy.com/2014/11/1000000-grahams-number.html good introduction].

![](http://28oa9i1t08037ue3m1l0i861.wpengine.netdna-cdn.com/wp-content/uploads/2014/11/grahams-number.png)]

Graham's number is a... rather large number. Letting $f(x) = 3\uparrow^n 3$ (in [+knuth_up_arrow_notation]) and $f^n(x) = \underbrace{f(f(f(\cdots f(f(x)) \cdots ))}_{n\text{ applications of }f}$, Graham's number is defined to be $f^{64}(4).$

The result is sizable. For an explanation of how large this is and why, see Tim Urban's [http://waitbutwhy.com/2014/11/1000000-grahams-number.html explanation] at Wait but Why.


A googolplex

The number $10^{10^{100}}$ is named "a googolplex." That is, a googolplex is the number represented (in [-decimal_notation]) as a 1 followed by a [-42m] zeroes, i.e., $10^{googol}$, which is $ 10^{10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000}.$


A googol

$10^{100},$ i.e., 10000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000.

The search engine Google is named after the number googol. (The difference for the spelling, as the apocryphal story goes, comes from the fact that one of the early Google investors misspelled the name on a check.)


Needs summary

By default, a page's [1kl summary] (that shows up in the popover when a user hovers over a [17f greenlink]) is the page's first paragraph. Sometimes the first paragraph is not well-suited for a popover summary. Add this tag to pages that need to have a custom summary.

Some pages may require an [5cb accessible], [4pt technical], or [4q2 brief] summary.


Why is log like length?

If a number $x$ is $n$ digits long (in [-decimal_notation]), then its logarithm (base 10) is between $n-1$ and $n$. This follows directly from the [40j definition of the logarithm]: $\log_{10}(x)$ is the number of times you have to multiply 1 by 10 to get $x;$ and each new digit lets you write down ten times as many numbers. In other words, if you have one digit, you can write down any one of ten different things (0-9); if you have two digits you can write down any one of a hundred different things (00-99); if you have three digits, you can write down any one of a thousand different things (000-999); and in general, each new digit lets you write down ten times as many things. Thus, the number of digits you need to write $x$ is close to the number of times you have to multiply 1 by 10 to get $x$. The only difference is that, when computing logs, you multiply 1 by 10 exactly as many times as it takes to get $x$, which might require [fractional_exponent multiplying by 10 a fraction of a time] (if x is not a power of 10), whereas the number of digits in the base 10 representation of x is always a whole number.


Poset: Examples

The standard $\leq$ relation on integers, the $\subseteq$ relation on sets, and the $|$ (divisibility) relation on natural numbers are all examples of poset orders.

Integer Comparison
================

The set $\mathbb Z$ of integers, ordered by the standard "less than or equal to" operator $\leq$ forms a poset $\langle \mathbb Z, \leq \rangle$. This poset is somewhat boring however, because all pairs of elements are comparable; such posets are called chains or [540 totally ordered sets]. Here is its Hasse diagram.

![Truncated Hasse diagram](http://i.imgur.com/STsyMfJ.png)
%%%comment:
dot source (doctored in GIMP)

digraph G {
  node [width = 0.1, height = 0.1]
  edge [arrowhead = "none"]
  a [label = "-3"]
  b [label = "-2"]
  c [label = "-1"]
  d [label = "0"]
  e [label = "1"]
  f [label = "2"]
  g [label = "3"]
  rankdir = BT;
  a -> b
  b -> c
  c -> d
  d -> e
  e -> f
  f -> g
}
%%%
Power sets
=========

For any set $X$, the power set of $X$ ordered by the set inclusion relation $\subseteq$ forms a poset  $\langle \mathcal{P}(X), \subseteq \rangle$. $\subseteq$ is clearly [5dy reflexive], since any set is a subset of itself. For $A,B \in \mathcal{P}(X)$,  $A \subseteq B$ and $B \subseteq A$ combine to give $x \in A \Leftrightarrow x \in B$ which means $A = B$. Thus, $\subseteq$ is [5lt antisymmetric]. Finally, for $A, B, C \in \mathcal{P}(X)$, $A \subseteq B$ and $B \subseteq C$ give $x \in A \Rightarrow x \in B$ and $x \in B \Rightarrow x \in C$, and so the [573 transitivity] of $\subseteq$ follows from the transitivity of $\Rightarrow$.

Note that the strict subset relation $\subset$ is the strict ordering derived from the poset $\langle \mathcal{P}(X), \subseteq \rangle$.

Divisibility on the natural numbers
===========================

Let [45h $\mathbb N$] be the set of natural numbers including zero, and let $|$ be the divides relation, where $a|b$ whenever there exists an integer $k$ such that $ak=b$. Then $\langle \mathbb{N}, | \rangle$ is a poset. $|$ is reflexive because, letting k=1, any natural number divides itself. To see that $|$ is anti-symmetric, suppose $a|b$ and $b|a$. Then there exist integers $k_1$ and $k_2$ such that $a = k_1b$ and $b = k_2a$. By substitution, we have $a = k_1k_2a$. Thus, if either $k$ is $0$, then both $a$ and $b$ must be $0$. Otherwise, both $k$'s must equal $1$ so that $a = k_1k_2a$ holds. Either way, $a = b$, and so $|$ is anti-symmetric. To see that $|$ is transitive, suppose that $a|b$ and $b|c$. This implies the existence of integers $k_1$ and $k_2$ such that $a = k_1b$ and $b = k_2c$. Since by substitution $a = k_1k_2c$, we have $a|c$.


Ackermann function

The Ackermann function works as follows:

One may have noticed that addition, multiplication, and exponentiation seem to increase in "power", that is, when pitted against each other, it is easier to produce enormous numbers with exponentiation than with multiplication, and so on.

The Ackermann function produces a hierarchy of such growth operations, and ascends one step higher in the hierarchy each time.

Addition is the first operator in the hierarchy (though if we wanted, we could define [-addition_as_repeated_succession] and declare addition the second operator in the hierarchy). The next operator in the hierarchy is produced by iterating the previous element in the hierarchy. For instance, the next few operators in the hierarchy are multiplication, exponentiation, and tetration:

Multiplication is iterated addition: $A \cdot B = \underbrace{A + A + \ldots A}_{B \text{ copies of } A}$

Exponentiation is iterated multiplication: $A^B = \underbrace{A \times A \times \ldots A}_{B \text{ copies of } A}$.

($A ^ B$ is written $A \uparrow B$ in [knuth_up_arrow_notation].)

Tetration is iterated exponentiation. $A \uparrow\uparrow B = \underbrace{A^{A^{\ldots^A}}}_{B \text{ copies of } A}$ times.

$\uparrow^n$ just means $n$ up arrows, so we can also write tetration as: $A \uparrow^2 B = \underbrace{A \uparrow^1 (A \uparrow^1 (\ldots A))}_{B \text{ copies of } A}$

Now the pattern can be noticed and generalized, following the rule: $A \uparrow^n B = \underbrace{A \uparrow^{n-1} (A \uparrow^{n-1} (\ldots A))}_{B \text{ copies of } A}$

And now we can define the Ackermann function as $A(n) = n \uparrow^n n$.

This definition is relatively small, but the functions grow at incredible rates. [Wait But Why](http://waitbutwhy.com/2014/11/1000000-grahams-number.html) has a good intuitive description of how incredibly fast tetration, pentation, and hexation grow, but by the time we get to $A(6)$, the Ackermann function already grows far more quickly than that. $A(1)=1$, $A(2)=4$, and $A(3)$ cannot be written in the universe, as it has enormously more digits than our universe has subatomic particles.

Interestingly enough, the Ackermann function grows faster than all primitive recursive functions. This is related to a rather deep connection between the consistency strength of a mathematical theory (which sorts of results they are capable of proving) and the slowest-growing function for which the theory cannot prove "this function is defined on all integers."


Fractional digits

When $b$ and $x$ are [48l integers], $\log_b(x)$ has a few good interpretations. It's roughly the [416 length of $x$] when written in [number_bases base $b$]. More precisely, it's the [427 fair price of an $x$-digit] in terms of [4sj $b$-digits]. But what about when $x$ or $b$ is not a whole number? $\log_{3.16}(5.62) \approx 1.5$; how are we supposed to interpret this fact?

We could appeal to the definition of the logarithm, observe that $3.16^{1.5} \approx 5.62,$ and call it a day. But what does that mean? That 3.16 multiplied by itself one-and-a-half times is roughly 5.62? That 5.62, written in base 3.16, is about 1.5 digits long? That a 5.62-digit is worth about 1.5 times as much as a 3.16-digit? What does it mean to multiply a number by itself "half a time"? How would you even write a number down in "base 3.16"? What would a 3.16-digit even look like?

Let's say that you want to [storing_numbers store a lot of data], and [42d digit wheels] cost \$1. At that price, an object that can store 3 different values costs about 48¢, whereas an object that stores 4 different values costs about 60¢. What storage medium, then, costs exactly 50¢? I can think of two:

1. You and I could buy a digit wheel, split the costs, and share it.
2. You and I could both pay 50¢ each for a digit wheel, and then toss a coin to see who gets to use it.

How could we split the digit wheel? The digit wheel can be put into any one of ten different states, labeled 0, 1, 2, 3, 4, 5, 6, 7, 8, and 9. Here's one way to use it to encode two different messages: I choose a number $a$ between 0 and 1, then you choose a number $b$ between 0 and 4. Then we put the digit wheel into the state $5a + b.$ Thus, if you choose 0, the wheel will either say 0 or 5 (depending on which number I choose). If you choose 3, the wheel will either say 3 or 8 (depending on which number I chose). Each number you want to record (0-4) has two representations, and I choose whether we use the high one or the low one. Thus, the digit wheel records two messages: 7 means (high, 2) and so on.

Of course, this is not a fair way to split the digit wheel — you get to store a [3v9 5-message] whereas I only get to store a 2-message.%%note:Thus we see that a [4sj 10-digit] is worth a 5-digit plus a 2-digit, that is, $\log_{10}(5) + \log_{10}(2) = 1$.%% Can we do something fairer?

Well, the methodology can be used to share arbitrary $n$-digits between two people. If $x$ and $y$ are whole numbers such that $x \cdot y \le n$ then an $n$-digit can be used to store both an $x$-message and a $y$-message at once: You choose the [-quotient] of $n$ divided by $x$, I choose the [-remainder].

__Exercise:__ How would you use an $18$-digit to store both a $3$-message and a $6$-message? How do you store (2, 4) using that encoding?
%%hidden(Answer):
Here's one way: Let $a$ be the value of the 3-message (between 0 and 2) and $b$ be the value of the 6-message (between 0 and 5). Record $6a+b.$ Using this encoding, (2, 4) is stored as the number 16.

You can verify for yourself that there is exactly one way to store each (3-message, 6-message) pair in an 18-message under this encoding.
%%

(From this we can see that, when $n = x \cdot y,$ the cost of a $n$-digit is exactly equal to the cost of an $x$-digit plus the cost of a $y$-digit. In other words, $n = x \cdot y$ then $\log_b(x) + \log_b(y) = \log_b(n),$ no matter what base $b$ we're using.)

The fair split of the 10-digit, then, is to let us both store a 3-message: 3 is the highest integer $x$ such that $x \cdot x < 10.$ This isn't maximally efficient, though; that encoding only uses 9 of the 10 possibilities that we paid for, and therefore it's not worth our money: Why buy what is effectively a 3-message for 50¢ when the fair price of 3-messages is 48¢? This might be worth knowing if the market isn't selling objects that carry 3-messages, and both you and I just need a tiny bit more storage, but in general it's not worth it.

However, it becomes more worthwhile the more 10-digits we split. You might expect that, when splitting three digit wheels, we'd each get one digit wheel to ourselves plus a 3-message each from the third wheel, giving us 30 different possible messages each. However, we can do better than that if we're clever. If you pick a number $a$ between 0 and 30 (for a total of 31 different possibilities), and I also pick a number $b$ between 0 and 30, then the number $31a + b$ can always be stored on 3 digit wheels (because $31 \cdot 30 + 30 = 960 \le 999$), so if we're splitting three digit wheels, we can actually use them to store two 31-messages. ([ Why?]) This still isn't maximally efficient (the values from 961 to 999 are wasted), but it's a little better. And we can keep increasing the efficiency if we keep going: Five digit wheels can hold two 316-messages; 7 digit wheels can hold two 3162-messages, and so on. (You can verify those numbers yourself.)

As you can see, when splitting an $n$-digit, the fair split occurs at the largest whole number $x$ such that $x \cdot x \le n$, in which case we both get to record one $x$-message. This is why it is 316, and not 500, that can most naturally be seen as "about 2.5 decimal digits long:" The number that is 2.5 decimal digits long is the largest number $x$ such that, given five digits, both you and I can use those 5 digits to store independent $x$-messages. If you store a 500-message, that only leaves enough space for me to store a 200-message. If you only store a 300-message, that leaves enough space for me to store a 333-message. $x=316$ is the point where we can both store an $x$-message, because 316 is the largest whole number such that $x^2 \le 100000.$

(From this we can deduce that $\log_b(316) \approx \frac{5\log_b(10)}{2}$, no matter what the base: A 316-digit is worth about half as much as five 10-digits put together.)

The more 10-digits you split, the more efficient it gets — if we split 7 digit wheels, each of us gets to record a 3162-message; if we split 9 wheels, we each get to record a 31622-message, and so on. But the split is never maximally efficient, because 10 is not a [-square_number]: An $n$-digit can be split perfectly in half if and only if there is an $x$ such that $x \cdot x = n,$ and this can't happen if $n$ is an odd power of 10. Thus, splitting an odd number of digit wheels never makes the final digit wheel be worth _exactly_ 50¢.

You know what storage medium is worth exactly 50¢? A 50% chance of a 10-digit.

Let's say we pay 50¢ each for a digit wheel, and then toss a coin to see who gets to use it. How much does this decrease your storage costs? Half the time, you get to [ reduce the number you have left to store] by a factor of 10; the other half the time, you reduce the number you have left to store by a factor of 1 (i.e., not at all). How much does that reduce the number you have left to store, [4b5 on average]?

On average, your number gets reduced by the [-geometric_mean] of 1 and 10. Geometric means "average out" multiplications: Let's say that you have a thing and you're going to multiply it by 2, 12 and 9. At the end of the day, your thing is going to be 216 times larger, and you're going to have multiplied it by three numbers. Geometric means ask how big each multiplication was "on average" — instead of multiplying your thing by three different numbers, can we find one number $y$ such that $y \cdot y \cdot y = 216,$ such that multiplying your thing by $y$ three times is the same as multiplying by 2, 12, and 9 separately? The answer is yes, and $y = \sqrt[3]{2 \cdot 12 \cdot 9} = 6$, as you can verify. Thus, multiplying your thing by 6 is equivalent, [4b5 in expectation], to multiplying your thing by a value selected [uniform_random uniformly at random] from the list [2, 12, 9]. (For more on this, see [ Geometric mean: Intuition].)

Similarly, if we might reduce the size of the number we want to store by a factor of 1, and we might reduce it by a factor of 10, then "on average" we're reducing it by a factor of $\sqrt[2]{1 \cdot 10} \approx 3.16.$ Multiplying by $\sqrt[2]{10}$ twice is the same as multiplying by 1 once and 10 once, so it's equivalent [4b5 in expectation] to multiplying by values selected uniformly at random from the list [1, 10].

If _three_ people are sharing a 10-digit, then the amount of digits each one gets to use in expectation is $\sqrt[3]{1 \cdot 1 \cdot 10} \approx 2.15.$ So that's what a 2.15 digit looks like — a 10-digit shared between three people. This can be generalized to any probability, and any $n$-digit for $1 < n \le 10$ can be interpreted as a digit wheel that you have some probability of getting to use ([ How?]).

There are a few different things we can learn from this interpretation. It allows us to give a physical interpretation to facts like $\log_{3.16}(5.62) \approx 1.5$, which says that a 5.62-digit is worth one and a half times as much as a 3.16 digit. What's a 3.16 digit? It's (approximately) a 10-digit shared between two people, i.e., a 10-digit that you have a 50% chance of getting to use yourself. What's a 5.62-digit? It's a 10-digit that you have a 75% chance of getting to use yourself. It's 1.5x as valuable, because 75% is 1.5x as much as 50%. The upgrade from a 3.16-digit to a 5.62-digit constitutes 1.5x increase in my chance of getting to use the digit wheel.

The other thing that this interpretation tells us is that the digit worth exactly half an $n$-digit is the $\sqrt{n}$-digit, both because (1) if you want to use an $n$-digit to store two $x$-messages for some $x$, you need $x \cdot x$ to be at most $n$; and (2) if you have a 50% chance of getting to use an $n$-digit and a 50% chance of [1digit nothing], then that's equivalent [4b5 on average] to a certainty of a $\sqrt{n}$-digit.

In case you haven't noticed, we've been implicitly using the fact that doubling a digit squares the number of possibilities you have available to you: With one 10-digit you can store 10 possibilities, with two you can store $10^2 = 100.$ Because two $n$-digits [ emulate] an $n^2$-digit, it's no surprise that half an $n$-digit is a $\sqrt{n}$-digit.

You might be wondering how these square root signs weaseled into a discussion about logarithms. You also might be starting to suspect that roots, logarithms, and exponentials are secretly three sides of the same coin. If so, [exp_log_root you're right], though that's a topic for another day.

Lest you think that this gives us a complete interpretation of logarithms, note that it still doesn't explain what's going on when the inputs to the logarithm function are between 0 and 1. You might think, "well, now that I know what a 3.16-digit is, I can use that intuition to figure out what a 0.16-digit is" — but it's not quite as easy as that. A 3.16-digit can be interpreted as probabilistic access to a 10-digit, and the same is true for $x$-digits whenever $x > 1.$ But no matter how many people share a 10-digit, it's never worth _less_ than a 1-digit ([1digit which is worthless]); $\sqrt[n]{10} > 1$ no matter how large $n$ gets. To understand $x$-digits for $0 < x < 1,$ we need one final puzzle piece, covered in the next post.


Shannon

The shannon (Sh) is a unit of [-3xd]. One shannon is the difference in [info_entropy entropy] between a [-probability_distribution] on a [-binary_variable] that assigns 50% to each value, and one that assigns 100% to one of the two values.


Natural number

[summary: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, ...

The natural numbers are the numbers we use to count things.]

A **natural number** is a number like 0, 1, 2, 3, 4, 5, 6, ... which can be used to represent the count of an object. The set of natural numbers is $\mathbb N.$ Not all sources include 0 in $\mathbb N.$

Natural numbers are perhaps the simplest type of number. They don't include [48l negative numbers], [4zq fractional numbers], [4bc irrational numbers], [4zw imaginary numbers], or any of those complexities.

Thanks to their simplicity, natural numbers are often the first mathematical concept taught to children. Natural numbers are equipped with a notion of [-addition] ($2 + 3 = 5$ and so on) and [-multiplication] ($2 \cdot 3 = 6$ and so on), these are among the first mathematical operations taught to children.

Despite their simplicity, the natural numbers are a ubiquitous and useful mathematical object. They're quite useful for counting things. They represent all the possible [4w5 cardinalities] of finite [3jz sets]. They're also a useful [data_structure data structure], in that numbers can be used to [numeric_encoding encode] all sorts of data. Almost all of modern mathematics can be built out of natural numbers.

[todo: Add a "formalization" lens with the Peano axioms. I recommend at least one page with just the raw Peano axioms (and very little prose), and another gentler introduction sort of like http://bit.ly/29glDrR and http://bit.ly/29nKYAL, albeit more to-the-point and probably without going all the way up to non-standard number territory.]