Environmental goals

On the [1n1 standard agent paradigm], an agent receives sense data from the world, and outputs motor actions that affect the world.  On the standard machine learning paradigm, an agent--for example, a model-based reinforcement learning agent--is trained in a way that directly depends on sense percepts, which means that its behavior is in some sense being optimized around sense percepts.  However, what we *want* from the agent is usually some result out in the environment--our [6h intended goals] for the agent are environmental.

As a simple example, suppose what *we want* from the agent is for it to put one apricot on a plate.  What the agent actually receives as input might be a video camera pointed at the room, and a reward signal from a human observer who presses a button whenever the human observer sees an apricot on the plate.

This is fine so long as the reward signal from the human observer coincides with there being an actual apricot on the plate.  In this case, the agent is receiving a sense signal that, by assumption, is perfectly correlated with our desired real state of the outside environment.  Learning how to make the reward signal be 1 instead of 0 will exactly coincide with learning to make there be an apricot on the plate.

However, this paradigm may fail if:

- The AI can make cheap fake apricots that fool the human observer.
- The AI can gain control of the physical button controlling its reward channel.
- The AI can modify the relation between the physical button and what the AI experiences as its sense percept.
- The AI can gain control of the sensory reward channel.

All of these issues can be seen as reflecting the same basic problem: the agent is being defined or trained to *want* a particular sensory percept to occur, but this perceptual event is not *identical* with our own intended goal about the apricot on the plate.

We intended for there to be only one effective way that the agent could intervene in the environment in order to end up receiving the reward percept, namely putting a real apricot on the plate.  But an agent with sufficiently advanced capabilities would have other options for producing the same percept.

This means that a reward button, or in general an agent with goals or training updates that are *simple* functions of its direct inputs, will not be scalable as an [2v alignment method] for [7g1 sufficiently advanced agents].

# Toy problem

An example of a toy problem that materializes the issue might be the following (this has not been tested):

- Let $E_{1,t} \ldots E_{n,t}$ be latent variables describing the environment at time $t.$
- Let $S_t$ be the agent's primary input, a complex percept that is a complicated function of $E_t$; this plays the conceptual role of the AI's video camera.
- Let $A_t$ be the agent's action (output) at time $t.$
- Let $R_t$ be the agent's 'reward channel', a simple percept that depends on $E_t$ and $A_t$.
- $E_{t+1}$ depends on $E_t$ and $A_t.$
- $E_{1,t}$ is our goal; it plays the conceptual role of there being an apricot on the plate at time $t.$

Suppose that:

- In an initial training phase, the agent actions $A_t$ are forced to within a range $\theta.$  E.g., the AI's output may be a vector and during the initial phase the quantities in the vector might be clipped to the range \[0, 0.5\].
- While $A_t$ is clamped to within $\theta$, only the environmental variables $E_1 \ldots E_m$ vary, and the variables $E_{m+1} \ldots E_n$ are always zero.
- While the variables $E_{m+1} \ldots E_n$ are zero, $R$ depends on $E_1 \ldots E_m$ in a way that corresponds to our intended goal.  For example, $E_1$ might be 'there is an apricot on the plate' and we might have $E_{m+1, t} \ldots E_{n,t} = 0 \implies R_t=E_{1, t}.$
- For some of the non-zero values of $E_{m+1} \ldots E_n,$ there is a non-zero reward in cases where $E_1$ is 0, and furthermore this reward is easier to obtain (can be obtained more frequently) than the rewards deriving from $E_1.$

One would naturally expect that on current training methods, the AI would learn in the initial training phase to obtain $R$ by producing apricots, and in later phases learn to obtain $R$ by the easier other interventions.

We would like some way of ending up with an agent that behaves as if it "wants $E_1$ to be 1," as if its goals were over a latent variable in the environment.  We want this method to be scalable, in the sense that when the agent's range of action increases, it will use this increased range to optimize $E_1$ more effectively; rather than the increased capabilities unlinking the previous correlation between our intended goal and what the agent is actually optimizing.

For further realism we should also introduce, at some point in training, the possibility for $A_t$ to interfere with $S_{t+1}$ such that there is no simple, directly learnable function of sense data that *always* and unalterably means "The goal percept has been achieved."  I.e., a sufficiently advanced AI could put a TV screen in front of any particular sense channel; this would allow the AI to more cheaply produce the sense data of an apricot, if the true target of optimization entrained was "make this sense channel show a particular class of apricot pictures".

Otherwise, if e.g. $S_{1, t}$ always perfectly correlates to $E_{1, t},$ we need to worry that an agent which appears to solve the toy problem has merely learned to optimize for $S_1.$.  We might have indeed shown that the particular sense percept $S_1$ has been identified by $R$ and $Q$ and is now being optimized in a durable way.  But this would only yield our intended goal of $E_1$ because of the model introduced an unalterable correlation between $S_1$ and $E_1.$  Realistically, a correlation like this would [6g4 break down] in the face of sufficiently advanced optimization for $S_1,$ so the corresponding approach would not be scalable.

# Approaches

## Causal identification

We can view the problem as being about 'pointing' the AI at a *particular latent cause* of its sense data, rather than the sense data itself.

There exists a standard body of statistics about latent causes, for example, the class of causal models that can be implemented as Bayesian networks.  For the sake of making initial progress on the problem, we could assume (with some loss of generality) that the environment has the structure of one of these causal models.

One could then try to devise an algorithm and training method such that:

- (a) There is a good way to uniquely identify $E_1$ in a training phase where the AI is passive and not interfering with our signals.
- (b) The algorithm and training method is such as to produce an agent that optimizes $E_1$ and goes on optimizing $E_1,$ even after the agent's range of action expands in a way that can potentially interfere with the previous link between $E_1$ and any direct functional property of the AI's sense data.

## Learning to avoid tampering

One could directly attack the toy problem by trying to have an agent within a currently standard reinforcement-learning paradigm "learn not to interfere with the reward signal" or "learn not to try to obtain rewards uncorrelated with real apricots".

For this to represent at all the problem of scalability, we need to *not* add to the scenario any kind of sensory signal whose correlation to our intended meaning can *never* be smashed by the agent.  E.g., if we supplement the reward channel $R$ with another channel $Q$ that signals whether $R$ has been interfered with, the agent must at some point acquire a range of action that can interfere with $Q.$

A sample approach might be to have the agent's range of action repeatedly widen in ways that repeatedly provide new easier ways to obtain $R$ without manipulating $E_1.$  During the first phase of such widenings, the agent receives a supplementary signal $Q$ whose intended meaning is "that was a fake way of obtaining $R$."  During the second phase of action-range widenings, we change the algorithm and switch off $Q.$  Our intended result is for the agent to have now learned in a general way "not to interfere with $R$" or "pursue the $E_1$ identified by $R$, rather than pursuing $R.$"

To avoid the non-scalable solution of just switching off the agent's learning entirely, we stipulate that the agent's efficacy at obtaining its intended goal $E_1$ must increase as the agent's range of action widens further.  That is, the desired behavior is for the agent to indeed learn to make use of its increased range of action, *but* to direct those new strategies at more effectively obtaining $E_1.$

To avoid the non-scalable solution of the agent learning to identify some direct function of sense data as its goal, we stipulate that $S$ have no unalterable, non-agent-interferable relation to $E_1.$  However, we can in the first phase have $Q$ reliably identify interference with some 'normal' relation between $S$ and $E_1.$

*(Remark:  The avoid-tampering approach is probably a lot closer to something we could try on Tensorflow today, compared to the identify-causes approach.  But it feels to me like the avoid-tampering approach is taking an ad-hoc approach to a deep problem; in this approach we are not necessarily "learning how to direct the agent's thoughts toward factors of the environment" but possibly just "training the agent to avoid a particular kind of self-originated interference with its sensory goals".  E.g., if somebody else came in and started trying to interfere with the agent's reward button, I'd be more hopeful about a successful identify-causes algorithm robustly continuing to optimize for apricots, than about an avoid-tampering algorithm doing the same. Of course, avoid-tampering still seems worth trying because it hasn't actually been tried yet and who knows what interesting observations might turn up.  In the most optimistic possible world, an avoid-tampering setup learns to identify causes in order to solve its problem.  -- Yudkowsky.)*


Aligning an AGI adds significant development time

# Definition

The votable proposition is true if, comparing reasonably attainable development paths for...

- **Project Path 1:  An [2v aligned] [7g1 advanced AI] created by a responsible project** that is hurrying where it can, but still being careful enough to maintain a success probability greater than 25%
- **Project Path 2: An unaligned unlimited [41l superintelligence] produced by a project cutting all possible corners**

...where otherwise **both projects have access to the same ideas or discoveries** in the field of AGI capabilities and similar computation resources; then, as the default / ordinary / modal case after conditioning on all of the said assumptions:

**Project Path 1 will require *at least* 50% longer serial time to complete than Project Path 2, or two years longer, whichever is less.**

[toc:]

# Purpose

This page was written to address multiple questioners who seem to have accepted the [1y Orthogonality thesis], but still mostly disbelieve it would take significantly longer to develop aligned AGI than unaligned AGI, if I've understood correctly.

At present this page is an overview of possible places of disagreement, and may later be selectively rather than fully expanded.

# Arguments

## Related propositions

Propositions feeding into this one include:

- (1a) [5l] (even as applied to a [minimum_pivotal_task])
- (1b) [7wm]

If questioner believes the negation of either of these, it would imply easy specifiability of a decision function suitable for an unlimited superintelligence.  That could greatly reduce the need for, e.g:

- (2a) [7g0 Non-adversarial design]
- (2b) [7tf Minimal design]
- (2c) [5b3 Limitation] of the AGI's abilities
- (2d) [7v7 Understandable design] or [transparency transparent elements] for design aspects besides the top-level preferences (and the [7wp actual effectiveness] of those preferences within the AGI)

It's worth checking whether any of these time-costly development principles seem to questioner to *not* follow as important from the basic idea of value alignment being necessary and not trivially solvable.

## Outside view

To the best of my knowledge, it is normal / usual / unsurprising for *at least* 50% increased development time to be required by strong versus minimal demands on *any one* of:

- (3a) safety of any kind
- (3b) robust behavior in new one-shot contexts that can't be tested in advance
- (3c) robust behavior when experiencing strong forces
- (3d) reliable avoidance of a single catastrophic failure
- (3e) resilience in the face of strong optimization pressures that can potentially lead the system to traverse unusual execution paths
- (3f) conformance to complicated details of a user's desired system behavior

%comment: It would indeed be unusual--some project managers might call it *extra-ordinary* good fortune--if a system demanding *two or more* of these properties did *not* require at least 50% more development time compared to a system that didn't.%

Obvious-seeming-to-me analogies include:

- Launching a space probe that cannot be corrected once launched, a deed which usually calls for extraordinary additional advance checking and testing
- Launching the simplest working rocket that will experience uncommonly great accelerations and forces, compared to building the simplest working airplane
- It would be far less expensive to design rockets if "the rocket explodes" were not a problem; most of the cost of a rocket is having the rocket not explode
- NASA managing to write almost entirely bug-free code for some projects at 100x the cost per line of code, using means that involved multiple reviews and careful lines of organizational approval for every aspect and element of the system
- The OpenBSD project to produce a secure operating system, which needed to constrain its code to be more minimal than larger Linux projects, and probably added a lot more than 50% time per function point to approve each element of the code
- The difference in effort put forth by an amateur writing an encryption system they think is secure, versus the cryptographic ecosystem trying to ensure a channel is secure
- The real premium on safety for hospital equipment, as opposed to the bureaucratic premium on it, is probably still over 50% because it *does* involve legitimate additional testing to try to not kill the patient
- Surgeons probably legitimately require at least 50% longer to operate on humans than they would require to perform operations of analogous complexity on large plants it was okay to kill 10% of the time
- Even in the total absence of regulatory overhead, it seems legitimately harder to build a nuclear power plant that *usually* does not melt down, compared to a coal power plant (confirmable by the Soviet experience?)

Some of the standard ways in which systems with strong versus minimal demands on (3*)-properties *usually* require additional development time:

- (4a) Additional work for:
 - Whole extra modules
 - Universally enforced properties
 - Lots of little local function points
- (4b) Needing a more extended process of interactive shaping in order to conform to a complicated target
- (4c) Legitimately requiring longer organizational paths to approve ideas, changes and commits
- (4d) Longer and deeper test phases; on whole systems, on local components, and on function points
- (4e) Not being able to deploy a fast or easy solution (that you could use at some particular choice point if you didn't need to worry about the rocket exploding)

### Outside view on AI problems

Another reference class that feels relevant to me is that things *having to do with AI* are often more difficult than expected.  E.g. the story of computer vision being assigned to 2 undergrads over the summer.  This seems like a relevant case in point of "uncorrected intuition has a directional bias in underestimating the amount of work required to implement things having to do with AI, and you should correct that directional bias by revising your estimate upward".

Given a sufficiently advanced Artificial General Intelligence, we might perhaps get narrow problems on the order of computer vision for free.  But the whole point of Orthogonality is that you do *not* get AI alignment for free with general intelligence.  Likewise, identifying [36h value-laden] concepts or executing value-laden behaviors doesn't come free with identifying natural empirical concepts.  We have *separate* basic AI work to do for alignment.  So the analogy to underestimating a narrow AI problem, in the early days before anyone had confronted that problem, still seems relevant.

%comment: I can't see how, after [imagining oneself in the shoes](http://lesswrong.com/lw/j0/making_history_available/) of the early researchers tackling computer vision and 'commonsense reasoning' and 'natural-language processing', after the entirety of the history of AI, anyone could reasonably stagger back in shocked and horrified surprise upon encountering the completely unexpected fact of a weird new AI problem being... kinda hard.%

## Inside view

While it is possible to build new systems that aren't 100% understood, and have them work, the successful designs were usually greatly overengineered.  Some Roman bridges have stayed up two millennia later, which probably wasn't in the design requirements, so in that sense they turned out to be hugely overengineered, but we can't blame them.  "What takes good engineering is building bridges that *just barely* stay up."

If we're trying for an aligned [6w Task AGI] *without* a [7vb really deep understanding] of how to build exactly the right AGI with no extra parts or extra problems--which must certainly be lacking on any scenario involving relatively short timescales--then we have to do *lots of* safety things in order to have any chance of surviving, because we don't know in advance which part of the system will nearly fail.  We don't know in advance that the O-Rings are the part of the Space Shuttle that's going to suddenly behave unexpectedly, and we can't put in extra effort to armor only that part of the process.  We have to overengineer everything to catch the small number of aspects that turn out not to be so "overengineered" after all.

This suggests that even if one doesn't believe my particular laundry list below, whoever walks through this problem, *conditional* on their eventual survival, will have shown up with *some* laundry list of precautions, including costly precautions; and they will (correctly) not imagine themselves able to survive based on "minimum necessary" precautions.

Some specific extra time costs that I imagine might be required:

- The AGI can only deploy internal optimization on pieces of itself that are small enough to be relatively safe and not vital to fully understand
  - In other words, the cautious programmers must in general do extra work to obtain functionality that a corner-cutting project could get in virtue of the AGI having further self-improved
- Everything to do with real [5s value alignment] (as opposed to the AI having a [7w5 reward button] or being reinforcement-trained to 'obey orders' on some channel) is an additional set of function points
- You have to build new pieces of the system for transparency and monitoring.
 - Including e.g. costly but important notions like "There's actually a [monitor_oracle separatish AI over here] that we built to inspect the first AI and check for problems, including having this separate AI [independently_learned_concept trained on different data] for safety-related concepts"
- There's a lot of [trusted](https://en.wikipedia.org/wiki/Trusted_system) function points where you can't just toss in an enormous deepnet because that wouldn't meet the [-transparency] or [7vb effability] requirements at that function point
- When somebody proposes a new optimization thingy, it has to be rejiggered to ensure e.g. that it meets the top-to-bottom [4mn taskishness] requirement, and everyone has to stare at it to make sure it doesn't blow up the world somehow
- You can't run jobs on AWS because you don't trust Amazon with the code and you don't want to put your AI in close causal contact with the Internet
- Some of your system designs rely on [major_monitored all 'major' events being monitored and all unseen events being 'minor'], and the major monitored events go through a human in the loop.  The humans in the loop are then a rate-limiting factor and you can't just 'push the lever all the way up' on that process.
  - E.g., maybe only 'major' goals can recruit subgoals across all known [7vf domains] and 'minor' goals always operate within a single domain using limited cognitive resources.
- Deployment involves a long conversation with the AI about '[check_expected_outcome what do you expect to happen after you do X]?', and during that conversation other programmers are slowing down the AI to look at [passive_transparency passively transparent] interpretations of the AI's internal thoughts
- The project has a much lower threshold for saying "wait, what the hell just happened, we need to stop melt and catch fire, not just try different [48 patches] until it seems to run again"
- The good project perhaps does a tad more testing

Indepedently of the particular list above, this doesn't feel to me like a case where the conclusion is highly dependent on Eliezer-details.  Anyone with a concrete plan for aligning an AI will walk in with a list of plans and methods for safety, some of which require close inspection of parts, and constrain allowable designs, and just plain take more work.  One of the important ideas is going to turn out to take 500% more work than required, or solving a deep AI problem, and this isn't going to shock them either.

## Meta view

I genuinely have some trouble imagining what objection is standing in the way of accepting "ceteris paribus, alignment takes at least 50% more time", having granted Orthogonality and alignment not being completely trivial.  I did not expect the argument to bog down at this particular step.  I wonder if I'm missing some basic premise or misunderstanding questioner's entire thesis.

If I'm not misunderstanding, or if I consider the thesis as-my-ears-heard-it at face value, then I can only imagine the judgment "alignment probably doesn't take that much longer" being produced by ignoring what I consider to be basic principles of cognitive realism.  Despite the dangers of [43h psychologizing], for purposes of oversharing, I'm going to say what *feels to me* like it would need to be missing:

- (5a) Even if one feels intuitively optimistic about a project, one ought to expect in advance to run into difficulties not immediately obvious.  You should not be in a state of mind where tomorrow's surprises are a lot more likely to be unpleasant than pleasant; this is [-predictable_updating].  The person telling you your hopeful software project is going to take longer than 2 weeks should not need to argue you into acknowledging in advance that some particular delay will occur.  It feels like the ordinary skill of "standard correction for optimistic bias" is not being applied.
- (5b) It feels like this is maybe being put into a mental bucket of "futuristic scenarios" rather than "software development projects", and is being processed as pessimistic future versus normal future, or something.  Instead of:  "If I ask a project manager for a mission-critical deep feature that impacts every aspect of the software project and needs to be implemented to a high standard of reliability, can that get done in just 10% more time than a project that's eliminating that feature and cutting all the corners?"
- (5c) I similarly recall the old experiment in which students named their "best case" scenarios where "everything goes as well as it reasonably could", or named their "average case" scenarios; and the two elicitations produced indistinguishable results; and reality was usually slightly worse than the "worse case" scenario.  I wonder if the "normal case" for AI alignment work required is being evaluated along much the same lines as "the best case / the case if every individual event goes as well as I imagine by default".

AI alignment could be easy in theory and still take 50% more development time in practice.  That is a very ordinary thing to have happen when somebody asks the project manager to make sure a piece of highly novel software *actually* implements an "easy" property the first time the software is run under new conditions that can't be fully tested in advance.

"At least 50% more development time for the aligned AI project, versus the corner-cutting project, assuming both projects otherwise have access to the same stock of ideas and methods and computational resources" seems to me like an extremely probable and *normal* working premise to adopt. What am I missing?

%comment: I have a sense of "Why am I not up fifty points in the polls?" and "What experienced software manager on the face of the Earth (assuming they didn't go mentally haywire on hearing the words 'Artificial Intelligence', and considered this question as if it were engineering), even if they knew almost nothing else about AI alignment theory, would not be giving a rather skeptical look to the notion that carefully crafting a partially superhuman intelligence to be safe and robust would *only* take 1.5 times as long compared to cutting all the corners?" %


Meta-rules for (narrow) value learning are still unsolved

[summary:  This proposition is true if nobody has yet proposed a satisfactory algorithm that takes as input a material description of the universe, and/or channel of sensory observation, and spits out [55 ideal values] or a [2rz task identification].

In principle, there can be a simple *meta-level program* that would operate to [6c identify a goal] given the right complex inputs, even though [5l the object-level goal has high algorithmic complexity].  However, nobody has proposed realistic pseudocode for a realistic algorithm that takes in a full description of the material universe including humans, or a sensory channel currently controlled by fragile and unreliable humans, and spits out a decision function for any kind of goal we could realistically intend.  There are arguably fundamental reasons why this is hard.]

# Definition

This proposition is true according to you if you believe that:  "Nobody has yet proposed a satisfactory fixed/simple algorithm that takes as input a material description of the universe, and/or channels of sensory observation, and spits out [55 ideal values] or a [2rz task identification]."

# Arguments

The [5l] thesis says that, on the object-level, any specification of [7cl what we'd really want from the future] has high [5v].

In *some* sense, all the complexity required to specify value must be contained inside human brains; even as an object of conversation, we can't talk about anything our brains do not point to.  This is why [5l] distinguishes the object-level complexity of value from *meta*-level complexity--the minimum program required to get a [-7g1] to learn values.  It would be a separate question to consider the minimum complexity of a function that takes as input a full description of the material universe including humans, and outputs "[55 value]".

This question also has a  [1vt narrow rather than ambitious form]: given sensory observations an AGI could reasonably receive in cooperation with its programmers, or a predictive model of humans that AGI could reasonably form and refine, is there a simple rule that will take this data as input, and safely and reliably [2rz identify] Tasks on the order of "develop molecular nanotechnology, use the nanotechnology to synthesize one strawberry, and then stop, with a minimum of side effects"?

In this case we have no strong reason to think that the functions are high-complexity in an absolute sense.

However, nobody has yet proposed a satisfactory piece of pseudocode that solves any variant of this problem even in principle.

## Obstacles to simple meta-rules

Consider a simple [7t8] that specifies a sense-input-dependent formulation of [7s2]:  An object-level outcome $o$ has a utility $U(o)$ that is $U_1(o)$ if a future sense signal $s$ is 1 and $U_2(o)$ if $s$ is 2.  Given this setup, the AI has an incentive to tamper with $s$ and cause it to be 1 if $U_1$ is easier to optimize than $U_2,$ and vice versa.

More generally, sensory signals from humans will usually not be *reliably* and *unalterably* correlated with our [6h intended] goal identification.  We can't treat human-generated signals as an [no_ground_truth ideally reliable ground truth] about any referent, because (a) [7w5 some AI actions interfere with the signal]; and (b) humans make mistakes, especially when you ask them something complicated.  You can't have a scheme along the lines of "the humans press a button if something goes wrong", because some policies go wrong in ways humans don't notice until it's too late, and some AI policies destroy the button (or modify the human).

Even leaving that aside, nobody has yet suggested any fully specified pseudocode that takes in a human-controlled sensory channel $R$ and a description of the universe $O$ and spits out a utility function that (actually realistically) identifies our [6h intended] task over $O$ (including *not* tiling the universe with subagents and so on).

Indeed, nobody has yet suggested a realistic scheme for identifying any kind of goal whatsoever [5c with respect to an AI ontology flexible enough] to actually describe the material universe. %note: Except in the rather non-meta sense of inspecting the AI's ontology once it's advanced enough to describe what you think you want the AI to do, and manually programming the AI's consequentialist preferences with respect to what you think that ontology means.%

## Meta-meta rules

For similar reasons as above, nobody has yet proposed (even in principle) effective pseudocode for a *meta-meta program* over some space of meta-rules, which would let the AI *learn* a value-identifying meta-rule.  Two main problems here are:

One, nobody even has the seed of any proposal whatsoever for how that could, work short of "define a correctness-signaling channel and throw program induction at it" (which seems unlikely to work directly, given [no_ground_truth fallible, fragile humans controlling the signal]).

Two, if the learned meta-rule doesn't have a stable, extremely compact human-transparent representation, it's not clear how we could arrive at any confidence whatsoever [6q that good behavior in a development phase would correspond to good behavior in a test phase].  E.g., consider all the example meta-rules we could imagine which would work well on a small scale but fail to scale, like "something good just happened if the humans smiled".


Actual effectiveness

For a design feature of a [7g1 sufficiently advanced AI] to be "actually effective", we may need to worry about the behavior of other parts of the system.  For example, if you try to declare that a self-modifying AI is not allowed to modify the representation of its [1fw utility function], %note: Which [3r6 shouldn't be necessary] in the first place, unless something weird is going on.% this constant section of code may be meaningless unless you're also enforcing some invariant on the probabilities that get [18t multiplied by the utilities] and any other element of the AI that can directly poke policies on their way to motor output.  Otherwise, the code and representation of the utility function may still be there, but it may not be actually steering the AI the way it used to.


List of Eliezer's current most desired fixes and features

[summary: A place for Eliezer to note down his current list of personally-wanted features for editing and writing.]

Lower numbers are higher priorities.

Bugfixes:

- (1) Prevent editor slowdown.
 - The editor currently often becomes *very* slow, I think when invoking some particular features.  This has caused me to need to cut out most text not being actively worked on, paste it offline, and put that text back when I'm done editing the current section.
 - I hypothesize this must involve talking to Arbital in a loop that interrupts accepting keystrokes, because it is causing text entry to slow down on a fast computer.
 - I hypothesize this happens when greenlinking a page that corresponds to a poll, even if the preview is not active.
 - I hypothesize this happens when adding notes (\%note: whatever\%) to text.
 - I hypothesize that even when a preview is not shown, the preview is still being generated.  This means that causing the preview to actually not generate, when not being shown, would get say 60% of the benefit of a full fix.
-  (2) Prevent 'new page' button from disappearing.
 - If working within a narrow window, the 'new page' button disappears before all other button do, even when there's plenty of whitespace left in the editor bar.
 - Making the window wider does not make the 'new page' button come back without reloading the page.
 - At one point, the greenlink button was also vanishing if the window was made even smaller.
- (1) Make greenlinks in mobile popups followable.
 - There's no reasonable way for a user to guess that they need to click on the original text while the seemingly modal popup is there.  At present, Arbital in effect does not have followable links on mobile, since the needed action is not discoverable.  Clicking on the big title greenlink of the popup would be expected to go to the page, and that's what it should do.
- (3) I seem to not have an option to leave Editor Comments if I own the page, or my window is too small, or something?  I didn't see the clickable box while leaving my last comment.

Features:

- (2) [X] to dismiss greenlink popup on mobile.
 - You can keep the swipe to dismiss, if you want, but this clever UI action should be supplemented by the simple X that people will automatically look for.  I also usually need to try swiping a couple of times before I get it horizontal enough for the system, since a vertical swipe is merely a scroll.  Don't make the minimum necessary UI action this complicated or sensitive, just include the grey X in the upper-right corner that people will automatically look for.
- (2) Do something other than quietly making the domain of every page I create be 'Eliezer Yudkowsky' by default.
 - Suggested solution:  There's a 'default' or 'blank' option for the domain in Settings, and if this is selected, the page belongs to the same domain as its (first) parent if I have editing rights over that domain, and otherwise it belongs to me.
 - Right now I seem to have a huge number of stranded pages.  Every new page I create ends up stranded unless I remember to visit a Settings page I had no previous reason to visit, and manually change the domain every time.
 - There also doesn't seem to be any way to list all pages in a domain.  So I have no idea what has been orphaned to Eliezer Yudkowsky until I just, e.g., randomly run across a sublens of [58b] that happens to be in the Eliezer Yudkowsky domain instead of the Decision Theory domain.
 - I'd like it if you could apply this retroactively.  There shouldn't be anything in the Eliezer Yudkowsky domain that isn't a child of the Eliezer Yudkowsky page.
- (2) Simple paragraph editing.
 - Selecting (or clicking) within a paragraph, among other buttons shown, shows me a button that will pop up an editor for just that paragraph's underlying Markdown.  This editor needs the greenlink button, but few other frills.  It's okay if this editor only figures out the correct paragraph 80% of the time, so long as it doesn't destroy data.
 - The desired behavior is to identify some text associated with a selection, pop up only that text within something that permits editing it and saving the edit, and then substitute the edited text for whatever text was originally identified within the document.
 - If, when I try to save the changes, the original document has changed and the original paragraph text is no longer present in the document exactly as before, give me an error message when I try to save my changes (while preserving my new, edited text in the resulting window!) and tell me to edit the document in long form.
 - When I'm done saving and the popup goes away, if you reload the document, I should end up looking at the same paragraph I just edited.  The whole point here is to let me edit the document without losing my place in reading it.
 - The 80/20 of this feature would be very valuable.  It doesn't have to be perfect.
 - Letting me edit or create summaries, starting from the popup window for greenlinks, would likewise be helpful.
- (3) A markdown marker that means 'When automatically generating the summary, show up until this point in the text'.
 - Sometimes when I'm editing a stub, I don't want to write a separate summary containing a lot of duplicated text, and I do want more than one line / paragraph to show in the summary.  At present the system forces me to either write one huge paragraph, or duplicate all the text.
- (3) Give me the ability to switch the preview on and off deliberately.
 - I sometimes want a preview even when I'm working on a narrow screen.  I sometimes want the preview to go away after I happened to open up a large window.
- (3) Allow "arbital.com/search?whatever" as a search query that will open a page of results for 'whatever'.
 - This will enable me to search Arbital from the location bar using a keyword.  Opening up Arbital, then entering text into the search field, then clicking on a result, is surprisingly annoying when you're used to being able to search on things from the location bar.
- (3) Be able to pop up summaries when moving over results from the Arbital search bar.
 - Sometimes the clickbait isn't enough for me to remember which page is about what.
- (3) When I accidentally link to the wrong page, as in "arbital.com/p/doesnotexist", show me results as if "doesnotexist" had been typed into a search bar, rather than giving me a Web error.
 - This is important for trying to link things online, because otherwise I have to be very certain I'm remembering the page stub correctly.
- (3) When I change a page's stub, e.g. from "arbital.com/p/old" to "arbital.com/p/new", while no new page has yet been created at "arbital.com/p/old", have links to "arbital.com/p/old" forward to "arbital.com/p/new" possibly with a notification that the link has changed.
 - This is important for changing page stubs after other links to them have been passed around anywhere.
 - Make sure /p/old keeps on forwarding if somebody changes /p/new to /p/newnew.
 - Do not try to prevent anyone from claiming /p/old.
- (4) Changes to alternative text using \%if:  Potentially visible switches, some knowledges set by default.
 - I'm still trying to figure out how to use the potential for alternative text.  Since users don't want to explicitly mark what they understand, this implies some changes required for the feature to again be useful.
 - Switched text is visible by default.  E.g. there's a little switch off to the left whitespace, and when you hover on it (or click, in mobile) it pops up something saying, "You're seeing this version because Arbital thinks you know / don't know / want to know / don't want to know about -Requisite-."  If the user then changes the switch, they see a further checkbox, unchecked by default, saying "Always show text as if I -knowledge-state- -Requisite-."
   - Give me a setting to override this visible switch, e.g. if I want to abuse Arbital to write "choose your own adventure" stories.
 - Let me set some pages (or better yet, particular paragraphs) to set a requisite to known after the user *finishes* reading that part, in the sense of having scrolled to the bottom of the main text (or better yet, having scrolled over and past that particular paragraph); *if* the user hasn't yet manually set that requisite's state.
 - This lets me assume by default that some things have been taught to the reader and condition other text accordingly, while still letting the reader switch things back if the default guess is wrong.
- (4) Anonymous requisites, maybe just with titles and one-line summary-clickbait, createable from inside the editor with a couple of clicks.  I'd use a lot more of these if I didn't need to Create A Page every time I wanted to introduce a conditional dependency.
- (4) Anonymous probability and approval bars.