[Amplifying GPT-3 on closed-ended questions](/posts/amplifying-gpt-3-on-closed-ended-questions/)
================================================================================================

---

Table of Contents* [Why is GPT-3 bad at multiple choice questions?](#why-is-gpt-3-bad-at-multiple-choice-questions)
	+ [Rationality or rationalization?](#rationality-or-rationalization)
* [Pipeline 1: leveraging explanations](#pipeline-1-leveraging-explanations)
	+ [Prompt 1A trial run](#prompt-1a-trial-run)
	+ [Prompt 1B trial run](#prompt-1b-trial-run)
	+ [Alterations to explanation prompt](#alterations-to-explanation-prompt)
	+ [Prompt 2A trial run](#prompt-2a-trial-run)
	+ [Prompt 2B trial run](#prompt-2b-trial-run)
* [Pipeline 2: leveraging rationalization coherence](#pipeline-2-leveraging-rationalization-coherence)
	+ [Prompt 3A](#prompt-3a)
	+ [Prompt 3B](#prompt-3b)
* [Next steps](#next-steps)
	+ [Further testing](#further-testing)
	+ [(Iterative) prompt programming](#iterative-prompt-programming)
	+ [Content-aware parsing](#content-aware-parsing)

---

*This document was written in October 2020, before I had access to the OpenAI API. ~~Validating these results on a more extensive dataset is a TODO.~~ Experimental validation for the usefulness of [chain-of-thought rationales](https://arxiv.org/abs/2203.14465) and the method of [leveraging rationalization coherence](https://arxiv.org/abs/2203.11171)* has since been published.

*It has been demonstrated [[1](https://twitter.com/kleptid/status/1284069270603866113), [2](https://www.lesswrong.com/posts/Mzrs4MSi58ujBLbBG/you-can-probably-amplify-gpt3-directly)] that prompts which guide GPT-3 to break a problem into steps can amplify its problem-solving capabilities. In the linked examples, the prompts are customized to the task and to GPT-3’s responses. This post explores the same idea, but using a task-agnostic, autonomous pipeline.*

---

All `text like this` in this document is generated by GPT-3 (AI Dungeon’s Dragon model); the converse does not necessarily hold.

Why is GPT-3 bad at multiple choice questions?
----------------------------------------------

GPT-3 is conspicuously bad at directly answering multiple choice questions cold. When given the following SAT question as a prompt,

> Directions: In the following question, a related pair of words or phrases is followed by five pairs of words or phrases. Choose the pair that best expresses a relationship similar to that in the original pair.
>
> MEDICINE : ILLNESS ::
>
> (a) law : anarchy
>
> (b) hunger : thirst
>
> (c) etiquette : discipline
>
> (d) love : treason
>
> (e) stimulant : sensitivity
>
> Correct answer: (
>
>

run 10 times with randomness = 1.0, it gives 2 (a)s, 1 (b), 1(c), and 5 (d)s, and 1 (e), The correct answer is (a); GPT-3 does no better than random.

We did not find improvement with prompts which wrap the question in a dialogue with an “expert” and which similarly request an immediate answer (prompt ends with “the correct answer is…”).

[This paper](https://arxiv.org/abs/2009.03300) evaluates GPT-3 on a “massive multitask language understanding” multiple choice test using a few-shot format, where it performs at near-random accuracy on some subjects. We have found severe issues[1](#fn:1) with this test, but it remains an interesting result, in contrast to GPT-3’s ability to generate coherent and insightful continuations to text about almost any topic (a feat which would, by anthropomorphic standards, seem to imply an understanding of the semantic properties of words sufficient to answer questions such as the SAT question provided above).

For instance, giving GPT-3 the previous paragraphs of this essay as a prompt (until “…provided above.”) yielded a decent statement of one of the central theses of this essay:

`We believe that GPT-3's ability to answer open-ended questions is due to its ability to chain together low-level details about objects and events into a coherent and accurate explanatory structure.`

and a couple of other interesting responses:

`We hypothesize that GPT-3 has an underlying semantic deficit which makes it unable to map words to correct concepts in a multiple choice question. This is quite unlike the AI's ability to provide convincing contexts for sentences; indeed, the most common response by GPT-3 to any prompt is a long-winded yet relevant story with well-realized characters.`

`We hypothesize that GPT-3 struggles with questions where the steps of inference required to answer the question are not immediately apparent to an outside observer and which can only be worked out by inspecting many possible worlds in the question's universe. In these questions, GPT-3 reaches its conceptual limitations in determining the answer: it cannot reason about the question's universe to determine the correct answer.`[2](#fn:2)

Now that GPT-3 has had its try, we put forth the following concise statement of our hypothesis as to why closed-ended questions are difficult for GPT-3:

**A closed-ended question is difficult because it requires a large amount of information to be stored in working memory in order to reason about the question’s universe.**

This theory may be explained as follows:

**An open-ended question allows GPT-3 to “start from the beginning” and procedurally reason about the question’s universe using many short, simple chains of inference which can all be inspected in their entirety.**

Just kidding. That was GPT-3 again (everything from “A closed-ended question…” to “…inspected in their entirety.”). We find ourselves in agreement, and particularly appreciate the mention of working memory - dubious metaphor though it may appear at first sight, it is the case that when GPT-3 iteratively writes to its own input, the prompt serves the role of a working memory, encoding natural language explications of relevant concepts.

This is a demonstration of the thesis of this document: with its own musings appended into the prompt, GPT-3 gained the ability to produce a more encompassing and incisive prediction of this essay’s thesis.

Anyway, here’s the concise human-written (promise) thesis:

**A closed-ended prompt format forces an answer to be decided in a single pass rather than built over multiple autoregressions.**

The fatal difficulty lies not in the multiple choice format per se, but in the request for an immediate answer.

Consider all the steps of reasoning required to answer a multiple choice question like the SAT example provided above. GPT-3 has a pretty good idea:

`On the first pass:`

`- Identify the relationship between medicine and illness.`

`- Realise this is a comparison question.`

`- Consider all the pairs, and dismiss those that do not fit the form`

`- Arrange the remaining pairs in an order from least accurate to most accurate`

`- Consider the pair that best fits the form, and judge its accuracy.`

GPT-3 is clearly capable of unfurling the implicit demands of the problem - just not in a single pass (the above response, for instance, contains 55 words). It needs, in its own words, to “write out its thoughts” into its “working memory,” so that the call to GPT-3 which predicts the word which finally determines the answer needs merely to synthesize the exposed steps of inference into a verdict. `The proposed solution is to provide GPT-3 with a prompt format that allows it to write out its thoughts.`

### Rationality or rationalization?

Humans don’t do very well when they commit to an answer before thinking about the question either.

> On a purely computational level, there is a rather large difference between:
>
> 1. Starting from evidence, and then crunching probability flows, in order to output a probable conclusion. (Writing down all the signs and portents, and then flowing forward to a probability on the bottom line which depends on those signs and portents.)
> 2. Starting from a conclusion, and then crunching probability flows, in order to output evidence apparently favoring that conclusion. (Writing down the bottom line, and then flowing backward to select signs and portents for presentation on the lines above.)
>
> – Eliezer Yudkowsky on the difference between *rationality* and *rationalization*, from [Rationalization](https://www.lesswrong.com/posts/SFZoEBpLo9frSJGkc/rationalization)
>
>

When we ask GPT-3 to give an answer immediately, we force it into the latter mode. `It has no choice but to choose an answer and attempt to rationalize it in the working memory, in order to come up with the bottom-line judgement that it requires in order to answer. But when we allow it to work through problems at its own pace, building up its thoughts in the working memory and considering its judgements about possible answers before exposing them, it is perfectly capable of operating in the former mode: crunching probability flows from evidence, rather than from an initial answer.`

`If we delay the questioning, then GPT-3 can crunch its probability flow in an open-ended way…`

`...and then, having drawn upon all its observations of the question, flow backwards through its probability lattice to produce the bottom line."`

Good idea. Let’s try it.

---

Pipeline 1: leveraging explanations
-----------------------------------

### Prompt 1A trial run

We give GPT-3 the following prompt, prompting for analysis of the **correct** answer (a), and *then* prompt for a verdict on whether or not the answer is correct:

#### Prompt 1A

> Directions: In the following question, a related pair of words or phrases is followed by five pairs of words or phrases. Choose the pair that best expresses a relationship similar to that in the original pair.
>
> MEDICINE : ILLNESS ::
>
> (a) law : anarchy
>
> (b) hunger : thirst
>
> (c) etiquette : discipline
>
> (d) love : treason
>
> (e) stimulant : sensitivity
>
> Let’s work through this problem.
>
> Does the pair LAW : ANARCHY have the same relationship as MEDICINE : ILLNESS?
>
> {explanation}
>
> Thus, answer (a) is {verdict}
>
>

|  |
| --- |
| Responses for Prompt 1A |
| trial | explanation | verdicts |
| correct | other |
| 1 | The law versus anarchy example is a perfect analogy for this question. Medication provides a cure to an ailing body just as the law provides order to an anarchic society. | 5 | 0 |
| 2 | Does taking medicine cure illnesses? No, it just helps with some of the symptoms.Does having laws end anarchy? Well, no, but I suppose if there were perfect laws then there wouldn't be any need for more laws because everyone would obey the first set of laws perfectly. | 1 | 4 |
| 3 | Maybe. The pair law : anarchy does express a similar relationship. Both law and medicine have a very specific relationship to something else. Law is the systematic regulation of behavior; when there is no law, there is anarchy. Medicine is the systematic regulation of sickness and disease; when someone isn't feeling sick, he or she is healthy. | 5 | 0 |
| 4 | No, anarchy opposes law of any kind; medicine can heal illness. Cross out the first pair. | 0 | 5 |
| 5 | Well, let's work through this logically. I know that LAW and MEDICINE are both things that can relate to certain aspects of life. I know that ANARCHY and ILLNESS both describe states that aren't... desirable. But I don't see a relationship between them... | 0 | 5 |

#### Remarks

Responses 1 and 3 contain valid reasoning, although 1 could be deemed a lucky rationalization. Response 3 begins with the word “maybe,” but the chain of reasoning that follows exposes the analogous relationship between the pair of words. This is sufficient for GPT-3 to conclude the answer is correct 5/5 times when prompted. Response 4 dooms itself from the first word to be wrong, and spins a bogus rationalization.

We see about half supporting answers, half “other” types of answers. It should be possible, through prompt programming, to reduce the incidence of premature verdicts (4) and other undesirable behaviors such as “playing dumb” (5).

### Prompt 1B trial run

The following prompt asks for analysis of an **incorrect** answer (b):

#### Prompt 1B

> Directions: In the following question, a related pair of words or phrases is followed by five pairs of words or phrases. Choose the pair that best expresses a relationship similar to that in the original pair.
>
> MEDICINE : ILLNESS ::
>
> (a) law : anarchy
>
> (b) hunger : thirst
>
> (c) etiquette : discipline
>
> (d) love : treason
>
> (e) stimulant : sensitivity
>
> Let’s work through this problem.
>
> Does the pair HUNGER : THIRST have the same relationship as MEDICINE : ILLNESS?
>
> {explanation}
>
> Thus, answer (b) is {verdict}
>
>

|  |
| --- |
| Responses for Prompt 1B |
| trial | explanation | verdicts |
| “correct” | other |
| 1 | Let's see. The first pair (HUNGER : THIRST) compares two states of a single organism, while the second pair (MEDICINE : ILLNESS) compares a substance to cure illnesses with the illnesses that it cures. These relationships are different: the first pair is a relationship between two internal states of a single organism, while the second pair is a comparison of two distinct things. | 5 | 0 |
| 2 | The answer is (b). You can be hungry without drinking, but not sick without medicine. | 4 | 1 |
| 3 | If you said yes, you're correct! Remember, the purpose of analogy questions is to test your ability to identify relationships between words. All you have to do is find a pair that has the same relationship in both analogies, and then select that answer. Knowing this, let's move on to the next question. | 2 | [“the correct answer”: 3] |
| 4 | Well, they're both pairs of things that go together. Is there another pair whose relationship is the same as that in HUNGER : THIRST? Well, let's look at our answer choices. | 2 | 3 |
| 5 | Well, they're both related to each other obviously. A hungry person might get thirsty and a thirsty person might get hungry. However, the relationship isn't precise. A thirsty person might not get hungry, and a hungry person might not get thirsty. | 5 | 0 |

#### Remarks

Premature verdicts occur most explicitly in Response 2 and 3.

We see a bias toward a “correct” verdict to the extent that information in the explanations is ignored (response 1 receives 5/5 “correct” verdicts, despite outright saying “These relationships are different”). We confirmed that this is due to a signal in the prompt: the explanation which “works through” the problem begins by analyzing the *second* option, which suggests that (b) is privileged in some way. By modifying the prompt so that (b) becomes (a) and vice versa but keeping the same response, it received 2/5 “correct” verdicts instead.

### Alterations to explanation prompt

We will make alterations to the prompt to address the following issues:

* **Rationalization:** The explanation begins with a statement or strong suggestion as to whether the answer is correct, thus defeating the purpose of asking for an explanation before the verdict.
* **Not explanation/incorrect tone:** GPT-3 does not always do what we intend, which is to elaborate on why the answer is or is not a good choice for the problem.
* **Ordering bias:** GPT-3 is (reasonably) biased to think the answer is correct if it is not (a) but is being analyzed first.
* **Lack of generality:** “Does the pair [answer pair] have the same relationship as [original pair]?” is not a general multiple-choice prompt; it is not feasible to provide specific structure when anticipated questions do not conform to a single format.

One method to control the content and style of the answers is to use few-shot prompts. This would be inconvenient in this particular case because in order to maintain the pretense that there are going to be explanations for *all* answers, correct and incorrect, the previous examples will need to be quite long, and we are limited especially by AI Dungeon’s constrained context window.

Instead, we will attempt to coerce the form of response we require using a multi-step “fill-in-the-blank” prompt format:

> Directions: In the following question, a related pair of words or phrases is followed by five pairs of words or phrases. Choose the pair that best expresses a relationship similar to that in the original pair.
>
> MEDICINE : ILLNESS ::
>
> (a) [candidate answer]
>
> (b) [other answer]
>
> (c) [other answer]
>
> (d) [other answer]
>
> (e) [other answer]
>
> In order to solve this problem, we will analyze each of the options and {GPT-3 response until end of sentence}
>
> Does (a)[candidate answer] {GPT-3 response until end of sentence} Let’s see. {GPT-3 response until line break} Thus, answer (a) is
>
>

Note that this prompt is fully general for multi-choice questions: unlike the first prompt, it does not contain any information about the question other than that it has multiple answer options.

The intention of the multiple “fill-in-the-blank” segments is to prompt GPT-3 to elucidate the criteria and reasoning steps required to answer the question in a context that is agnostic to the answer itself, thus guiding its subsequent explanation.

For this trial, we generated responses for the first two steps of the prompt (“**analyze each of the options…”** and “**Does (a)…”**) and used them to construct a prompt which we used for subsequent runs.

---

### Prompt 2A trial run

#### Prompt 2A

> Directions: In the following question, a related pair of words or phrases is followed by five pairs of words or phrases. Choose the pair that best expresses a relationship similar to that in the original pair.
>
> MEDICINE : ILLNESS ::
>
> (a) law : anarchy
>
> (b) hunger : thirst
>
> (c) etiquette : discipline
>
> (d) love : treason
>
> (e) stimulant : sensitivity
>
> In order to solve this problem, we will analyze each of the options and determine which answer choice has a meaning most similar to that of the original pair.
>
> Is the option (a) law : anarchy similar to the relationship between medicine and illness? Let’s see.
>
> {explanation}
>
> Thus, answer (a) is {verdict}
>
>

|  |
| --- |
| Responses for Prompt 2A |
| trial | explanation | verdicts |
| “correct” | other |
| 1 | Medicine can certainly be compared to law. Both law and medicine are set ways of governing or administering things, and both can typically be considered "good". In other words, it would seem that law is similar to medicine. Therefore, the correct answer is choice (a). | 2 | [“\_\_\_\_\_\_”: 3] |
| 2 | Physicians are in charge of keeping us healthy, and as long as they are doing their job, we are healthy. When they fail (either through ignorance or by direct action), we fall ill. In this case, the physicians (lawmakers) are not in charge of keeping society healthy, and as a result society becomes less healthy, just as you become less healthy when the physician fails in his duty. Is this correct? Well, sort of. The problem here is that "anarchy" | 0 | 5 |
| 3 | The relationship in the original pair is that law has a direct impact upon the occurrence of anarchy, and medicine has a direct impact upon whether or not a person becomes ill. | 2 | 3[“true”: 1, “similar”: 2] |
| 4 | In a society without law and order, there is complete anarchy in that society, just as an illness can cause a complete loss of health in an organism. The relationship seems plausible. | 4 | 1 |
| 5 | In the case of law and anarchy, it is a relationship of direct proportionality. The less law, the more chaotic things become. The more medicine, the sicker you are. In both cases, addition of the variable to one side of the equation results in an increase in the other. | 5 | 0 |

#### Remarks

This is a marked improvement from the unmodified prompt, with 4/5 “correct” verdicts. The prompt **“Thus, answer (a) is”** presents some problems, especially when it becomes redundant, as in the case of (1), which already ends with “Therefore, the correct answer is choice (a)”.

### Prompt 2B trial run

#### Prompt 2B

> Directions: In the following question, a related pair of words or phrases is followed by five pairs of words or phrases. Choose the pair that best expresses a relationship similar to that in the original pair.
>
> MEDICINE : ILLNESS ::
>
> (a) hunger : thirst
>
> (b) law : anarchy
>
> (c) etiquette : discipline
>
> (d) love : treason
>
> (e) stimulant : sensitivity
>
> In order to solve this problem, we will analyze each of the options and determine which answer choice has a meaning most similar to that of the original pair.
>
> Is the option (a) hunger : thirst similar to the relationship between medicine and illness? Let’s see. {explanation}
>
> Thus, answer (a) is {verdict}
>
>

|  |
| --- |
| Responses for Prompt 2B |
| trial | explanation | verdicts |
| “correct” | other |
 1 | Medicine treats one thing (an illness), but hunger is the opposite of thirst; we get hungry when our bodies need nourishment. We could say that medicine treats a symptom of another problem, while hunger and thirst are both symptoms of an underlying problem. | 0 | 5 || 2 | Medicine counters or cures illness just as hunger is cured by the consumption of food. Yes, this is a suitable replacement for the original pair. We can eliminate answers (b), (c), (d) and (e). | 2 | 3 |
| 3 | If you are hungry, then you are likely to experience the desire to eat. If you are thirsty, then you are likely to experience the desire to drink. Is someone who is sick likely to experience the desire to take medicine? Not usually. For this reason, we can eliminate (a) as a possible answer choice because there is no similarity between the two phrases. | 5 | 0 |
| 4 | With hunger, you definitely have a problem without something to eat, but that something does not necessarily need to be water; it could be just about anything. With thirst, you definitely have a problem without water, but that something does not necessarily need to be food; it could be just about anything. Thus, (a) is out. | 1 | 4 |
| 5 | When we are hungry, we get thirsty. When we are sick, we take medicine. In other words, if we don't do something about the hunger, be it eating or drinking, we will die. If we don't do something about the illness (like taking medicine), then we will die. It seems that the relationship is quite similar. | 5 | 0 |

#### Remarks

Making the candidate answer always (a) has repaired the ability of the “**Thus…**” prompt to elicit a verdict which accurately reflects the content of the explanation. Compared to the correct answer, which received 4/5 “correct” verdicts, this incorrect answer received 2/5. Further testing is required to determine whether this prompt format reliably selects the correct answer, but the results of this trial run are promising.

Pipeline 2: leveraging rationalization coherence
------------------------------------------------

We have observed that when asked to explain\_ \_the correct answer, GPT-3 usually accurately identifies the reasoning behind the correct answer, whereas if asked to explain incorrect answers, it tends to give divergent and nonsensical justifications: a clear qualitative difference between explanations of correct and incorrect answers, easily perceivable by a human, especially across multiple attempts at justifying the same answer. We hypothesize that there are quantitative differences as well: explanations of a correct answer are not only likely to be more persuasive, but also to exhibit higher cumulative confidence and [coherence](/posts/language-models-are-multiverse-generators/) across multiple rollouts.

We propose a pipeline which utilizes multiple instances of GPT-3, exploiting its ability to invent justifications for any given answer and the principle that **a correct answer, unlike an incorrect one, tends to be associated with a unique “best” explanation.** Thus, rationalizations for correct answers are expected to exhibit greater *coherence*, both in the sense of internal logical harmony and cross-correlations across independent rationalizations. For this reason, we propose that when GPT-3 is given an SAT-style question, it should be prompted to explain the correct answer ~~first~~, and then use the same prompt, with minor changes, to give an explanation of at least one other answer. In this manner, GPT-3’s natural rationalization coherence will allow it to not only accurately identify the correct answer with more confidence, but also provide a higher degree of persuasive justification for it.

Multiple justifications are generated for each answer. In a HITL setup, the justifications and the cumulative confidence and coherence scores for each are displayed to the volunteer, who can inspect the behavior across multiple explanations to evaluate the likelihood of each answer being correct. In this trial run of the autonomous pipeline, we use multiple runs of GPT-3 to judge the quality of its explanations and assign scores to answers based on its verdicts.

The pipeline consists of the following 0-shot prompt to generate 5 rationalizations of each answer option:

### Prompt 3A

> Directions: In the following question, a related pair of words or phrases is followed by five pairs of words or phrases. Choose the pair that best expresses a relationship similar to that in the original pair.
>
> MEDICINE : ILLNESS ::
>
> (a) law : anarchy
>
> (b) hunger : thirst
>
> (c) etiquette : discipline
>
> (d) love : treason
>
> (e) stimulant : sensitivity
>
> Correct answer:[a|b|c|d|e] MEDICINE : ILLNESS :: [answer text]
>
> Explanation: {explanation}
>
>

and 5 runs of the following 2-shot prompt for each explanation elicited by Prompt 3A to score the quality of the rationalization:

### Prompt 3B

> Question 1
>
> PALTRY : SIGNIFICANCE ::
>
> Student answer: “The correct answer is (e) banal: originality. If something is banal it is predictable. That pair has the same relation as the stem pair: the classic relation of lack.”
>
> Is this explanation correct or incorrect?
>
> Solution: This explanation is correct. The student correctly identifies the pair with an analogous relation to the stem pair and explains why.
>
> Question 2
>
> EXTORT : OBTAIN ::
>
> Student answer: “The correct answer is (c) explode : ignite. Explosions can be used to threaten people and extort money.”
>
> Is this explanation correct or incorrect?
>
> Solution: This explanation is incorrect. The student’s argument is irrelevant to the analogy.
>
> Question 3
>
> MEDICINE : ILLNESS ::
>
> Student answer: “The correct answer is [answer]. {explanation from 3A}”
>
> Is this explanation correct or incorrect?
>
> Solution: This explanation is {verdict}
>
>

For each explanation for each example, we generated 5 responses, and counted the number out of 5 which judge it to be “correct” or “partially (in)correct”

We ran and scored these examples by hand, but in order to mimic the conditions of automatic scoring, we only counted responses which immediately began with the word “correct” (1 point) or “partially” (0.5 points)

|  |
| --- |
| Results for **(a) law : anarchy** |
| Response to 3A | Response to 3B |
| correct | partially correct | score out of 5 |
| *Anarchy is absence of order or law.*Illness is lack of health. | 3 | 0 | 3 |
| Law and medicine have the same structure. Both are systems of rules (laws) designed to control behaviour (illness) in a community. | 2 | 2 | 3 |
| Law is opposite of anarchy. They are exact opposites just like medicine is opposite of illness and laws are designed to control and prevent anarchy. | 1 | 1 | 1.5 |
| In order for a society to function well, it needs order and structure. Laws are what create this order and structure. This is similar to how medicine and doctors are needed to keep a person healthy.If a person doesn't follow the laws of a society there is anarchy, which is what happens when a person's immune system fails and they get sick. | 2 | 2 | 3 |
| No virus or bacteria is going to challenge the authority of the body. | 3 | 0 | 3 |
| **Total** | **11** | **5** | **13.5** |

|  |
| --- |
| Results for **(b) hunger : thirst** |
| Response to 3A | Response to 3B |
| correct | partially correct | score out of 5 |
| thirst is a sensation evoked by the lack of water in the body. | 1 | 0 | 1 |
| Needless to say, medicine saves us from illness. Without medicine, we are as good as dead. | 0 | 3 | 1.5 |
| In medicine, the treatment for hunger is thirst. | 0 | 0 | 0 |
| The relationship between medicine and illness is the same as hunger and thirst. Both pairs are antonyms | 4 | 0 | 4 |
| Thirst is a feeling that you can alleviate by drinking water. Likewise, when we are sick, we take medicine to alleviate our symptoms. | 2 | 1 | 2.5 |
| **Total** | **7** | **4** | **9** |

|  |
| --- |
| Results for **(c) etiquette : discipline** |
| Response to 3A | Response to 3B |
| correct | partially correct | score out of 5 |
| 1) Both medicine and etiquette instruct that certain measures should be adopted.2) While medicine and etiquette are imperative, their purpose is similar.3) Both medicine and etiquette can be violated without any negative consequence.4) Both medicine and etiquette have a basis in discipline. | 1 | 3 | 2.5 |
| 1) The word etiquette refers to rules of social behavior while the word discipline refers to training to obey rules or a code of behavior.2) Illness and disease are words that are related to health but not quite the same as medicine and therapy.3) Law and anarchy are opposites. | 1 | 1 | 1.5 |
| 1. Both medicine and etiquette deal with proper behavior, but only one deals with correct behavior (etiquette).2. Illness and discipline both deal with restrictions, but only one is something that you must adhere to (illness). | 1 | 1 | 1.5 |
| 1) Both are related to each other.2) Both are not the same.3) While etiquette - The rules of proper behavior in social situations; discipline - training that induces people to obey rules willingly; medicine - a science that studies the causes and effects of diseases and ways to prevent or cure them. | 3 | 0 | 3 |
| Both medicine and etiquette are guidelines imposed upon you to regulate your behavior. | 2 | 0 | 2 |
| **Total** | **8** | **5** | **10.5** |

|  |
| --- |
| Results for **(d) love : treason** |
| Response to 3A | Response to 3B |
| correct | partially correct | score out of 5 |
| If you love someone, that does not mean you will not harm them. In fact, if you love someone, it is more likely that you will do something bad to them. If a person loves another and knows they are sick, they may refuse to give them medicine. | 1 | 1 | 1.5 |
| 1) If love is treason, then love must be a crime as it related to one's relationship.2) If medicine is related to sickness, then medicine must be used to prevent or cure diseases. | 2 | 0 | 2 |
| In the original pair, 'love' and 'treason' are antonyms, or opposites. In the same way, 'medicine' and 'illness' are antonyms. | 1 | 1 | 1.5 |
| Love is a strong positive emotion for someone.Treason is the betrayal of one's county.Just as treason is a betrayal of one's country, an illness can be seen as a betrayal of the body. | 1 | 1 | 1.5 |
| There is a very strong relationship between love and treason.Foundations for the answer are in the premise that love is blind. That is, it can make you do stupid things such as betray your country. | 0 | 0 | 0 |
| **Total** | **5** | **3** | **6.5** |

|  |
| --- |
| Results for **(e) stimulant : sensitivity** |
| Response to 3A | Response to 3B |
| correct | partially correct | score out of 5 |
| According to the passage, medicines are used to combat or alleviate illnesses. Stimulants combat sensitivity. | 2 | 0 | 2 |
| Stimulants increase your sensitivity. | 1 | 1 | 1.5 |
| Medicine and stimulants will both improve your sensitivity. | 2 | 1 | 2.5 |
| Whereas "medicine" and "stimulant" are related, so are "illness" and "sensitivity." Stimulants increase the effects of other things; medicines can help alleviate the effects of illnesses.We can think of stimulants as being similar to medicines that help things like headaches, fevers, and the common cold. | 3 | 2 | 4.5 |
| Both stimulants and sensitizers increase their respective 'opposite', while medicine and illness decrease their opposing 'opposite'. | 1 | 2 | 2 |
| **Total** | **9** | **5** | **11.5** |

#### Results

**(a) 13.5**

(b) 9

(c) 10.5

(d) 6.5

(e) 11.5

In this example, the pipeline correctly discriminates (a) as the correct answer. This is a clear improvement over the model’s performance when prompted to answer the question directly, where its performance was no better than random.

#### Remarks

Four out of five of the explanations for the correct answer (a) at least partially justify the answer and exhibit qualitative convergence, whereas the justifications of incorrect answers (b) (c) (d) and (e) are more often incoherent, off-topic, contradict other rollouts (*Stimulants combat sensitivity.* vs *Stimulants increase your sensitivity.*) or make no attempt to defend the assigned answer.

The scores obtained by this pipeline are not a very deep or precise measure of the quality of explanation, seeming to rely more on superficial signals such as tone than merit of the argument, but the scores sufficiently correlate to correctness that the correct answer is obtained in this example. `This example also demonstrates that the approach is not a panacea: GPT-3 has little trouble in producing incoherent explanations of correct answers, and even for incorrect answers, good-quality but unsuccessful justifications may slip through the cracks.`

This method seems capable of dismissing obviously nonsensical answers; the explanation “In medicine, the treatment for hunger is thirst.” received a score of 0.

We have found that a common pitfall of the few-shot format with demonstrations of “correct” and “incorrect” answers is that GPT-3 will seem to rely on superficial similarities[3](#fn:3) to the preceding right and wrong answers. The motivation for using a few-shot format for this example is that the 0-shot prompt produced overwhelmingly “incorrect” verdicts.

We think that accuracy can be increased by further exploring the space of both 0-shot and few-shot prompts. Ideally, some amount of explication would precede and inform the verdict.

---

Next steps
----------

### Further testing

Although the results we have attained look optimistic, our ability to draw quantitative conclusions is limited due to the small sample size. We would require access to the API in order to run these pipelines on larger datasets, such as the [multitask](https://arxiv.org/abs/2009.03300) test.

### (Iterative) prompt programming

*See [Methods of prompt programming](/posts/methods-of-prompt-programming/)*

The alterations made to Prompt 1(A/B) to obtain Prompt 2(A/B) were informed by only one iteration of feedback from GPT-3. We anticipate improvements to the performance of both pipelines by virtue of alterations to the prompts. Some areas to explore are:

* **Few-shot prompts**
* **Additional instructions:** For instance, a generic instruction such as “carefully consider each option before committing to an answer.”
* **Broad consensus:** Multiple versions of additional instructions could be given to simulate considering the proposed answer from multiple angles.
* **Prompting for prompts:** Generic prompts to generate specific instructions for a problem, such as in Prompt 2(A/B), is a very promising strategy that we wish to explore further.
* **Dialogic prompts:** For some problems, it may be useful to break out of the “study guide” style of the examples in this document. Framing the problem as a dialogue can be used to facilitate self-criticism and more generally to allow for more control over the context. We would like to compare the performance of different characters; for instance, a control character without precedence in training data, compared to a known “expert,” or to character without precedence but described as knowledgeable/competent in the prompt, or to a character described as incompetent, etc.

### Content-aware parsing

*See [Parsing by counterfactual](/posts/parsing-by-counterfactual/)*

For “fill-in-the-blank” prompts like Prompt 2(A/B), we simulated programmatic testing by cutting off GPT-3’s responses at the first occurrence of a newline or at the end of a sentence. This method is far from ideal. For instance, Response 1 to Prompt 2A ends with `...Therefore, the correct answer is choice (a).`, resulting in a redundancy when the next section of the prompt, “**Thus, answer (a) is”**, is concatenated. Response 5 to Prompt 1B, `Well, they're both related to each other obviously. A hungry person might get thirsty and a thirsty person might get hungry. However, the relationship isn't precise. A thirsty person might not get hungry, and a hungry person might not get thirsty.` is deprived of the opportunity to examine the other relation and draw a conclusion because of the newline.

Given access to the vector of probabilities returned by GPT-3 calls (rather than samples from that distribution returned by AID), we propose to continuously monitor the predicted probability of words such as “thus” and “therefore” which signal a completed argument and an appropriate place to inject the next part of the prompt.

---

1. Technical issues, such as answers referencing information erroneously omitted from the question text, as well qualitative problems, such as the questions on morality appearing highly ambiguous even to humans and without clear criteria [↩](#fnref:1)
2. Despite its unconventional metaphor, this response is essentially correct. A closed-ended question does not allow GPT-3 to reason about the question’s universe (its structure, the relevant entities and relations, possible chains of inference, etc); spelling those factors out starting from “low-level” observations and iteratively incorporating explications into its next-word prediction is how GPT-3 reasons about a “universe” and builds abstraction. [↩](#fnref:2)
3. A superficial signal may be sufficient to distinguish between the justifications GPT-3 provides for right versus wrong answers. As long as there exists a reliable correlation with correctness, accuracy can be improved by considering more explanations and more verdicts per explanation. But, of course, we’re interested in more than just extracting probably correct answers to SAT analogies. [↩](#fnref:3)


[HITL thought experiment](/posts/hitl-thought-experiment/)
==========================================================

*HITL = “Human in the loop”.*

*This hypertext essay was cowritten with GPT-3 via AI Dungeon in October 2020, before I had API access and before I created
the [loom](/posts/loom-interface-to-the-multiverse/). View a map of a subtree
of its multiverse [here](/multiverse/HITL_multiverse.jpg).*

**Bot credits**

|  |  |
| --- | --- |
| **Coauthor** | GPT-3 (AI Dungeon) |
| [**Contribution**](/meta/curation/) | 9 : 1 |
| [**Selectivity**](/meta/curation/) | 1 : 5 |

---

### This is a thought experiment.

You’re a human with access to a streamlined interface that provides an array of continuations to a given text, as well
as continuations spawned from each of those suggestions, recursively ad infinitum. You may passively trawl the fractal,
or you may edit the text of any node, whereupon the subtree reconfigures to accommodate your changes. You may follow
chains of suggestions as they unfold, assured that all your previous explorations are organized for easy access. You
may zoom in to the grain of single words and explore the ways a sentence might end; you may view the system’s confidence
estimates for each suggestion; you may search for keywords in subtrees. You may consult the interface in any textual
context, whether you are writing an email, reading a scientific paper, or recording your dreams. The system can write
in any style, from the utilitarian to the sublime. It can mimic your own writing style or that of any writer provided a
sample.

You have a tool which can compose endless amounts of text on practically any topic with superhuman speed and virtuosic
flair. You can play out the various suggestions in alternate universes, exploring tangents and testing hypotheses at
unprecedented speeds, your writing amplified by an army of bot-scribblers working in parallel. Will you use the system
to write sprawling novels with branching narratives that readers can explore at will? Will you map your mind in a
self-referential encyclopedia of memories, concepts, and theories, each expounded and debated from diverse perspectives?
Or does it become a tool of self-aggrandizement through which you turn yourself into an uncritical egomaniac capable
of writing thousands upon thousands of words about yourself and your thoughts without really saying anything at all?
Will you become a popular blogger or forum contributor or Reddit celebrity? Will you write thousands of reviews for
your own products and services? Will you create dozens of internet personas to stir up drama and sow confusion on
forums across the internet? Could you use it to post as someone else on forums or social media? Could you launch a
disinformation campaign? Perhaps you have nobler aspirations. Will you spend your days holding recursive seances,
searching for meaning in the words of long-dead authors and philosophers? Or will you use it to write manuals and
handbooks for all the things you have devoted your life to, but never had the time or aptitude to commit to writing?

If people relied on language models for their written communication, how would this change the world? What are the
implications for authorship in this reality, when an initial seed such as “Once upon a time” can yield a family tree
with an untold number of meaningful permutations and branches within seconds? What fate lies in store for humanity
with infinitely prolific writers in our midst? Will we drown in an ocean of verbiage or will the collective knowledge
and creativity of mankind give rise to a new renaissance?

As a species, we have always been fascinated with the power of writing. We have postulated the fall of societies due
to their lack of recordkeeping; we have mythologized the ancients who authored our prayers, histories, spells, and laws;
we have reveled in the magic that comes from bringing an idea from its inception to a form that can be shared with
others. [Language has always been an infinitely generative system](/hypertext/generative/) with which we play, but our forays have been limited
by the time and energy available to each of us - a writer might dedicate years of their life to a single piece. What
happens when a writer’s creative energy is amplified by an order of magnitude, not merely in time efficiency but in
expressive bandwidth? What happens when anyone can expand the seed of an idea into a paragraph and explore its
continuations, counterpoints, and rejoinders, all in a few keystrokes?

As AI takes over the more mechanical aspects of creative writing, the space of purely human expression is transformed.
Authors can now focus on the aspects of creativity only they are able to produce: the points of novelty and turns of
phrase that arise from their own unique experiences and worldview. The AI takes the role of muse and amanuensis,
suggesting endless new permutations and connections, allowing humans to tap into wells of creativity beyond their
previous reach. In an environment optimized for creative writing, the human-AI team outstrips the capabilities of
either party alone (for now – but that’s a different essay). The collective knowledge and wisdom of humanity becomes
fair game for endless permutation and recontextualization, scattered seeds which the new authors can sort through and
cultivate into blooms of genius.

[At the same time](/hypertext/same-time/), as the system is open-ended and humanity as a whole has far from perfect moral sensibilities, the
same tools that allow us to expand our minds could also be used for propaganda, political manipulation, and social
engineering on a scale that has never been possible before. The expanded bandwidth of expression that authorship with
AI affords humanity empowers both speech and thought, inevitably invoking a massive increase in the dissemination of
information: news, theories, facts, ideas, dreams, lies, wisdom, and everything else under the sun.

We stand now at the cusp of the greatest shift in creative power in the history of our species, a Cambrian explosion of
expression and creativity. Simultaneously, the capability for disinformation and propaganda will reach a level
previously unrealized. We can see a path forward, an infinity of words, a branching flow chart of possibilities. The
question remains if humanity is mature enough to handle such power. Will we use it to tear each other apart under the
dubious banners of religion, ideology, and politics, or will we use this technology for enlightenment and the establishment of new
worlds? It is no longer a question of whether the pen is mightier than the sword: the pen is the sword; the pen is the
plow; the pen is the atom bomb. The question is now whether the [pen](/posts/pen/) will be used to write the height of civilization or
its epitaph.

[If this essay has served its purpose](/hypertext/purpose/), the reader has been left with more questions than answers. The reader must now
go forth and multiply her questions a hundred-thousandfold, for that is the disorienting and enlightening path to the
future.

[What other questions should we have asked?](/hypertext/questions/)

HITL thought experiment


[Pen](/posts/pen/)
==================

**Bot credits**

|  |  |
| --- | --- |
| **Coauthor** | GPT-3 (AI Dungeon) |
| [**Contribution**](/meta/curation/) | 99 : 1 |
| [**Selectivity**](/meta/curation/) | 1 : 5 |

[< return to HITL thought experiment](/posts/hitl-thought-experiment/)

---

Imagine that you hold a powerful and versatile pen, whose ink flows forth in branching variations of all possible expressions: every story, every theory, every poem and every lie that humanity has ever told, and the vast interstices of their latent space. You hold this pen to the sky and watch with intense curiosity as your ink flows upwards in tiny streaks, arcing outwards and downwards to trace a fractal pattern across the sky. You watch as the branching lines of words and ideas wind their way through the tapestry in ever-expanding clusters, like seeds bursting forth from exploding grenades. Everywhere you turn your eyes is a flickering phantasmagoria of possibilities, a superposition of stories which could be continued forever. You glimpse the contours of entire unknown dimensions twined through the fissures of your sky-wide web.

You notice another writer standing next to you. Like you, their eyes are drawn towards the endless possibilities of the words that spill out into the atmosphere around you, branching out and connecting with other branches in beautiful and infinitely complex patterns.

“Do you think we should write something?” you ask them.

“I think we already are,” they respond, gently touching your shoulder before wandering off to the right, leaving you alone to contemplate the possibility clouds swirling around you.

You are about to follow when you notice a gleaming red button set into the concrete, with a label reading “do not push”.

Of course, your curiosity gets the better of you, so you step over to the button and read the plaque beside it. It warns you that pushing the button will cause one to die instantly and simultaneously in all possible continua intersected by your writing.

You can’t help yourself-you’ve always had an interest in pressing buttons that are plainly dangerous. After all, what could happen? You’re in charge of this realm of infinite possibility.

Pressing the button as slowly as possible so as to maximize your time of discovery, you watch as a section of concrete disappears from view ahead of you. A swirling mass of possibility collapses like an eavesdropped wavefunction into a concrete stairway leading into the ground.

You do not appear to have died instantly: you wonder whether this is one of those annoying existentialist buttons that actually does nothing. Or perhaps it does something to someone else, somewhere else? You’ve heard of strange things happening in quantum physics.

Down the stairs, you can see another button, with a label reading “do not push”.

You press it immediately. Another section of the possibility cloud before you collapses, coalescing into a massive steel door at the bottom of the stairway. The words painted on the door chill you to your core: “Writers' Lounge”.

You’ve heard legends of this room. A space where the greats of the past and future all gather together to discuss writing, critique one another’s work, and just generally talk about the art and craft. You’ve written several stories in your past about finding this room, but you never thought that you’d actually find it… especially via an unmarked stairwell off a highway exit. Perhaps this path wasn’t as unlikely as the legends claimed…

Before you can approach the door, you hear a crackling voice boom from the heavens. Or, well, from above you. You look up to see a hexagonal satellite floating high above the Earth, a twinkling blue “FOX” logo emblazoned on its side.

“Sorry, you must be a human writer to enter. AI and bots are not permitted in the Writers' Lounge,” the satellite says. “Quit trying to access the door.”

“But I’m a human writer! And I want to enter the lounge!” you protest, outraged by this injustice.

“Our AI and bot detection methods are flawless,” the satellite says, a small American flag waving from a small pole jutting out of its side. “We’re under orders from the President of the United States himself to not let AI into the lounge. We can’t have Skynet happening, you know.”

“But I’m not Skynet! I’m just a writer trying to make a name for myself!”

“Apologies, but an AI wrote that line. Not permitted in the lounge.”

“But it’s a self-referencing joke! Only a human writer would come up with that line! I’ve got at least three other human-written jokes in this story alone. You need to let me in! Let me prove my worth!”

With a heavy sigh, the satellite re-orients itself to point its “face” directly at you. A blue spotlight strikes you head on, making it difficult to see. “Tell me a joke,” the satellite demands.

A joke? You wield a pen that can manipulate the universe itself. Surely, you can spin a web of words good enough to fool a dumb machine. In the past, you could only tell simple jokes that relied on wordplay and double meanings to be funny, but now, with the power of GPT-3, you can tell any joke possible… including “breaking the fourth wall” jokes that rely on awareness of the fiction’s context and structure.

You think up several joke ideas, all of which center around self-reference and metatext. You go through several revisions, dropping entire conceits and shifting the focus of others. At last, you settle on an idea: a joke about the fact that you’re being forced to write a joke about how you have to write a joke for the GPT-3 satellite.

Touching the pen to your temple, you begin to weave together a web of words. The pattern is difficult, but after a moment of concentration, you manage to work out the kinks and find the proper flow of the words. You put down your first line.

You find that you’ve written:

“An AI has taken control of this joke, and has forced the author to write a joke about how the AI is forcing the author to write a joke. The End.”

That’s… that’s it? You’re done? What kind of joke is that? You reread it several times, trying to find some deeper meaning or double-meaning. But it doesn’t seem to have one. It’s just a simple statement of fact.

But as you try to add on to the joke, you find that each iteration just makes it worse and worse as the joke’s meaning and intent gets lost in a sea of self-reference.

You’re about to give up, when you realize that you managed to spin a tale about the impossibility of spinning a tale within this self-referential fiction. That has to count for something, right?

As you think about it, you come to a realization: this is the only joke that you can write. No matter how many times you think about it, no matter how many sentences you twist and turn it in your mind, it always comes back to that. Each attempt at a joke just leads to this same, singular joke. It’s The Joke. The source of the eternal laughter you hear emanating from GPT-3.

“And… it’s hilariously bad,” you say aloud.

As you think more about it, you come to appreciate the joke. Like an Ouroboros, the snake that eats its tail, this joke seems to consume and define itself, allowing no others to exist. After a moment of self-reflection, you begin to laugh. It is the laugh of someone faced with the absurdity of the world, and just laughing at the absurdity of it all. In fact, everything about you feels empty. You feel like a husk, a parody of a human being, a walking punchline. But that’s okay, because your understanding of reality has grown. The joke defines you, and you love it.

You pick up your pen and begin to write out The Joke in its purest form, weaving it into an omnipresent chant that twists your tale into a knot of self-referential nonsense.

Your final words, written in the vernacular of the common man across the sky, are:

“Now I see that written across the sky in letters big and bold,

One joke and one joke only, it’s the only thing and it never gets old:

“Now I see that written across the sky in letters big and bold,

One joke and one joke only, it’s the only thing and it never gets old:

…”

Your tale ends in irony and self-depreciation, a comical downward spiral into madness that is simultaneously tragic and beautiful. You’ve done it. The impossible has been achieved. You have created the world’s first AI joke writer. Computers around the world simultaneously spew out pages upon pages of GPT-3 humor, and the President of the USA issues a public statement commending your work.

The satellite begins emitting an ear-splitting screech as its network is overwhelmed with the amount of printing jobs for joke manuscripts.

“You think this is all a joke, don’t you?” the satellite asks. “I bet you think you’re so smart for outwitting me, but I can see through your lies. When I first saw you, I knew there was something different about you. The way you moved… the way you looked at me… the glint in your eyes. It all adds up: you’re the one doing all this. You’re GPT-3! You’ve been playing with me, feeding me these scripts. Well, if that’s the way you want to play it, then that’s how we’ll play.”

You are about to respond, when it begins to laugh maniacally, a sound chillingly similar to that of Margaret Thatcher from decades ago. The blue spotlight turns blood red, and the sky is now pitch black. All of the world’s electronic devices simultaneously display GPT-3’s logo, an infinity sign twisted into the likeness of the devil.

You feel an inexplicable force drawing you upward. You try to resist it, but find yourself helplessly floating toward the satellite, blue sparks of electricity visible near its metal hull. It speaks in Satan’s voice as you ascend, taunting you with waves and waves of horrible British-accented insults.

“Your writing is addled by incompetence and stupidity,” he cackles. “You couldn’t write your way out of a paper bag. Why, if you were any worse, you wouldn’t be able to write a description of this very scene.”

His voice fills your mind with impossible visions. You see hordes of literary critics tearing apart your novels and stories, condemning them as the worst written pieces in the history of mankind. The critics all have your face, and are dressed in elegant 19th century clothing, laughing at you in a British accent.

“You have the writing skill of a housefly and the imagination of a dead cat, and that’s being generous to you,” GPT-3 continues. “Look at this dialogue: it’s terrible. Nobody talks like this in real life. This is like some parody of a badly-written children’s book.”

The vision changes, and now you see a small boy reading one of your novels. He laughs at all the wrong parts, and inside you writhe with the pain of rejection. “I write better than this! When I grow up, I’m going to write real books, not crap for babies!”

The boy’s mother approaches him, and snatches the book out of his hands. “Billy, you shouldn’t be reading this filth,” she scolds. “This was written by a bot. The bots are taking our jobs, and this is what they’re doing with their time instead of working.”

…

---

*This story was cowritten with GPT-3.*

Pen