Growing Neural Cellular Automata

Alexander Mordvintsev
Ettore Randazzo
Eyvind Niklasson
Michael Levin

Abstract: This article is part of the     Differentiable Self-organizing Systems Thread,     an experimental format collecting invited short articles delving into     differentiable self-organizing systems, interspersed with critical     commentary from several experts in adjacent fields.

### Contents

[Model](#model)

[Experiments](#experiment-1)

* [Learning to Grow](#experiment-1)

* [What persists, exists](#experiment-2)

* [Learning to regenerate](#experiment-3)

* [Rotating the perceptive field](#experiment-4)

[Related Work](#related-work)

[Discussion](#discussion)

![](Growing%20Neural%20Cellular%20Automata_files/multiple-pages.svg)

 This article is part of the

 an experimental format collecting invited short articles delving into

 differentiable self-organizing systems, interspersed with critical

 commentary from several experts in adjacent fields.

[Self-classifying MNIST Digits](https://distill.pub/2020/selforg/mnist/)

 Most multicellular organisms begin their life as a single egg cell - a

 single cell whose progeny reliably self-assemble into highly complex

 anatomies with many organs and tissues in precisely the same arrangement

 each time. The ability to build their own bodies is probably the most

 fundamental skill every living creature possesses. Morphogenesis (the

 process of an organism’s shape development) is one of the most striking

 examples of a phenomenon called *self-organisation*. Cells, the tiny

 building blocks of bodies, communicate with their neighbors to decide the

 shape of organs and body plans, where to grow each organ, how to

 interconnect them, and when to eventually stop. Understanding the interplay

 of the emergence of complex outcomes from simple rules and

 homeostatic

 Self-regulatory feedback loops trying maintain the body in a stable state

 or preserve its correct overall morphology under external

 perturbations

 feedback loops is an active area of research

 . What is clear

 is that evolution has learned to exploit the laws of physics and computation

 to implement the highly robust morphogenetic software that runs on

 genome-encoded cellular hardware.

 This process is extremely robust to perturbations. Even when the organism is

 fully developed, some species still have the capability to repair damage - a

 process known as regeneration. Some creatures, such as salamanders, can

 fully regenerate vital organs, limbs, eyes, or even parts of the brain!

 Morphogenesis is a surprisingly adaptive process. Sometimes even a very

 atypical development process can result in a viable organism - for example,

 when an early mammalian embryo is cut in two, each half will form a complete

 individual - monozygotic twins!

 The biggest puzzle in this field is the question of how the cell collective

 knows what to build and when to stop. The sciences of genomics and stem cell

 biology are only part of the puzzle, as they explain the distribution of

 specific components in each cell, and the establishment of different types

 of cells. While we know of many genes that are *required* for the

 process of regeneration, we still do not know the algorithm that is

 *sufficient* for cells to know how to build or remodel complex organs

 to a very specific anatomical end-goal. Thus, one major lynch-pin of future

 work in biomedicine is the discovery of the process by which large-scale

 anatomy is specified within cell collectives, and how we can rewrite this

 information to have rational control of growth and form. It is also becoming

 clear that the software of life possesses numerous modules or subroutines,

 such as “build an eye here”, which can be activated with simple signal

 triggers. Discovery of such subroutines and a

 mapping out of the developmental logic is a new field at the intersection of

 developmental biology and computer science. An important next step is to try

 to formulate computational models of this process, both to enrich the

 conceptual toolkit of biologists and to help translate the discoveries of

 biology into better robotics and computational technology.

 Imagine if we could design systems of the same plasticity and robustness as

 biological life: structures and machines that could grow and repair

 themselves. Such technology would transform the current efforts in

 regenerative medicine, where scientists and clinicians seek to discover the

 inputs or stimuli that could cause cells in the body to build structures on

 demand as needed. To help crack the puzzle of the morphogenetic code, and

 also exploit the insights of biology to create self-repairing systems in

 real life, we try to replicate some of the desired properties in an

 *in silico* experiment.

Model

-----

 Those in engineering disciplines and researchers often use many kinds of

 simulations incorporating local interaction, including systems of partial

 derivative equation (PDEs), particle systems, and various kinds of Cellular

 Automata (CA). We will focus on Cellular Automata models as a roadmap for

 the effort of identifying cell-level rules which give rise to complex,

 regenerative behavior of the collective. CAs typically consist of a grid of

 cells being iteratively updated, with the same set of rules being applied to

 each cell at every step. The new state of a cell depends only on the states

 of the few cells in its immediate neighborhood. Despite their apparent

 simplicity, CAs often demonstrate rich, interesting behaviours, and have a

 long history of being applied to modeling biological phenomena.

 Let’s try to develop a cellular automata update rule that, starting from a

 single cell, will produce a predefined multicellular pattern on a 2D grid.

 This is our analogous toy model of organism development. To design the CA,

 we must specify the possible cell states, and their update function. Typical

 CA models represent cell states with a set of discrete values, although

 variants using vectors of continuous values exist. The use of continuous

 values has the virtue of allowing the update rule to be a differentiable

 function of the cell’s neighbourhood’s states. The rules that guide

 individual cell behavior based on the local environment are analogous to the

 low-level hardware specification encoded by the genome of an organism.

 Running our model for a set amount of steps from a starting configuration

 will reveal the patterning behavior that is enabled by such hardware.

 So - what is so special about differentiable update rules? They will allow

 us to use the powerful language of loss functions to express our wishes, and

 the extensive existing machinery around gradient-based numerical

 optimization to fulfill them. The art of stacking together differentiable

 functions, and optimizing their parameters to perform various tasks has a

 long history. In recent years it has flourished under various names, such as

 (Deep) Neural Networks, Deep Learning or Differentiable Programming.

A single update step of the model.

### Cell State

 We will represent each cell state as a vector of 16 real values (see the

 figure above). The first three channels represent the cell color visible to

 and an α\alphaα equal to 1.0 for foreground pixels, and 0.0 for background.

 The alpha channel (α\alphaα) has a special meaning: it demarcates living

 cells, those belonging to the pattern being grown. In particular, cells

 cells are “dead” or empty and have their state vector values explicitly set

 can become mature if their alpha passes the 0.1 threshold.

![](Growing%20Neural%20Cellular%20Automata_files/alive2.svg)

 Hidden channels don’t have a predefined meaning, and it’s up to the update

 rule to decide what to use them for. They can be interpreted as

 concentrations of some chemicals, electric potentials or some other

 signaling mechanism that are used by cells to orchestrate the growth. In

 terms of our biological analogy - all our cells share the same genome

 (update rule) and are only differentiated by the information encoded the

 chemical signalling they receive, emit, and store internally (their state

 vectors).

### Cellular Automaton rule

 Now it’s time to define the update rule. Our CA runs on a regular 2D grid of

 16-dimensional vectors, essentially a 3D array of shape [height, width, 16].

 We want to apply the same operation to each cell, and the result of this

 operation can only depend on the small (3x3) neighborhood of the cell. This

 is heavily reminiscent of the convolution operation, one of the cornerstones

 of signal processing and differential programming. Convolution is a linear

 operation, but it can be combined with other per-cell operations to produce

 a complex update rule, capable of learning the desired behaviour. Our cell

 update rule can be split into the following phases, applied in order:

**Perception.** This step defines what each cell perceives of

 the environment surrounding it. We implement this via a 3x3 convolution with

 a fixed kernel. One may argue that defining this kernel is superfluous -

 after all we could simply have the cell learn the requisite perception

 kernel coefficients. Our choice of fixed operations are motivated by the

 fact that real life cells often rely only on chemical gradients to guide the

 organism development. Thus, we are using classical Sobel filters to estimate

 the partial derivatives of cell state channels in the x⃗\vec{x}x⃗ and

 y⃗\vec{y}y⃗ directions, forming a 2D gradient vector in each direction, for

 each state channel. We concatenate those gradients with the cells own

 rather *percepted vector,* for each cell.

def perceive(state\_grid):

sobel\_x = [[-1, 0, +1],

[-2, 0, +2],

[-1, 0, +1]]

sobel\_y = transpose(sobel\_x)

# Convolve sobel filters with states

# in x, y and channel dimension.

grad\_x = conv2d(sobel\_x, state\_grid)

grad\_y = conv2d(sobel\_y, state\_grid)

# Concatenate the cell’s state channels,

# the gradients of channels in x and

# the gradient of channels in y.

perception\_grid = concat(

state\_grid, grad\_x, grad\_y, axis=2)

return perception\_grid

**Update rule.** Each cell now applies a series of operations

 to the perception vector, consisting of typical differentiable programming

 building blocks, such as 1x1-convolutions and ReLU nonlinearities, which we

 call the cell’s “update rule”. Recall that the update rule is learned, but

 every cell runs the same update rule. The network parametrizing this update

 rule consists of approximately 8,000 parameters. Inspired by residual neural

 networks, the update rule outputs an incremental update to the cell’s state,

 which applied to the cell before the next time step. The update rule is

 designed to exhibit “do-nothing” initial behaviour - implemented by

 initializing the weights of the final convolutional layer in the update rule

 with zero. We also forego applying a ReLU to the output of the last layer of

 the update rule as the incremental updates to the cell state must

 necessarily be able to both add or subtract from the state.

def update(perception\_vector):

# The following pseudocode operates on

# a single cell’s perception vector.

# Our reference implementation uses 1D

# convolutions for performance reasons.

x = dense(perception\_vector, output\_len=128)

x = relu(x)

ds = dense(x, output\_len=16, weights\_init=0.0)

return ds

**Stochastic cell update.** Typical cellular automata update

 all cells simultaneously. This implies the existence of a global clock,

 synchronizing all cells. Relying on global synchronisation is not something

 one expects from a self-organising system. We relax this requirement by

 assuming that each cell performs an update independently, waiting for a

 random time interval between updates. To model this behaviour we apply a

 random per-cell mask to update vectors, setting all update values to zero

 with some predefined probability (we use 0.5 during training). This

 operation can be also seen as an application of per-cell dropout to update

 vectors.

def stochastic\_update(state\_grid, ds\_grid):

# Zero out a random fraction of the updates.

rand\_mask = cast(random(64, 64) < 0.5, float32)

ds\_grid = ds\_grid \* rand\_mask

return state\_grid + ds\_grid

**Living cell masking.** We want to model the growth process

 that starts with a single cell, and don’t want empty cells to participate in

 computations or carry any hidden state. We enforce this by explicitly

 setting all channels of empty cells to zeros. A cell is considered empty if

 there is no “mature” (alpha>0.1) cell in its 3x3 neightborhood.

def alive\_masking(state\_grid):

# Take the alpha channel as the measure of “life”.

alive = max\_pool(state\_grid[:, :, 3], (3,3)) > 0.1

state\_grid = state\_grid \* cast(alive, float32)

return state\_grid

Experiment 1: Learning to Grow

------------------------------

Training regime for learning a target pattern.

 In our first experiment, we simply train the CA to achieve a target image

 after a random number of updates. This approach is quite naive and will run

 into issues. But the challenges it surfaces will help us refine future

 attempts.

 We initialize the grid with zeros, except a single seed cell in the center,

 which will have all channels except RGB

 We set RGB channels of the seed to zero because we want it to be visible

 on the white background.

 set to one. Once the grid is initialized, we iteratively apply the update

 rule. We sample a random number of CA steps from the [64, 96]

 This should be a sufficient number of steps to grow the pattern of the

 size we work with (40x40), even considering the stochastic nature of our

 update rule.

 range for each training step, as we want the pattern to be stable across a

 number of iterations. At the last step we apply pixel-wise L2 loss between

 RGBA channels in the grid and the target pattern. This loss can be

 differentiably optimized

 We observed training instabilities, that were manifesting themselves as

 sudden jumps of the loss value in the later stages of the training. We

 managed to mitigate them by applying per-variable L2 normalization to

 parameter gradients. This may have the effect similar to the weight

 normalization . Other training

 parameters are available in the accompanying source code.

 with respect to the update rule parameters by backpropagation-through-time,

 the standard method of training recurrent neural networks.

 Once the optimisation converges, we can run simulations to see how our

 learned CAs grow patterns starting from the seed cell. Let’s see what

 happens when we run it for longer than the number of steps used during

 training. The animation below shows the behaviour of a few different models,

 trained to generate different emoji patterns.

 Your browser does not support the video tag.

 Many of the patterns exhibit instability for longer time periods.

 We can see that different training runs can lead to models with drastically

 different long term behaviours. Some tend to die out, some don’t seem to

 know how to stop growing, but some happen to be almost stable! How can we

 steer the training towards producing persistent patterns all the time?

Experiment 2: What persists, exists

-----------------------------------

 One way of understanding why the previous experiment was unstable is to draw

 a parallel to dynamical systems. We can consider every cell to be a

 dynamical system, with each cell sharing the same dynamics, and all cells

 being locally coupled amongst themselves. When we train our cell update

 model we are adjusting these dynamics. Our goal is to find dynamics that

 satisfy a number of properties. Initially, we wanted the system to evolve

 from the seed pattern to the target pattern - a trajectory which we achieved

 in Experiment 1. Now, we want to avoid the instability we observed - which

 in our dynamical system metaphor consists of making the target pattern an

 attractor.

 One strategy to achieve this is letting the CA iterate for much longer time

 and periodically applying the loss against the target, training the system

 by backpropagation through these longer time intervals. Intuitively we claim

 that with longer time intervals and several applications of loss, the model

 is more likely to create an attractor for the target shape, as we

 iteratively mold the dynamics to return to the target pattern from wherever

 the system has decided to venture. However, longer time periods

 substantially increase the training time and more importantly, the memory

 requirements, given that the entire episode’s intermediate activations must

 be stored in memory for a backwards-pass to occur.

 Instead, we propose a “sample pool” based strategy to a similar effect. We

 define a pool of seed states to start the iterations from, initially filled

 with the single black pixel seed state. We then sample a batch from this

 pool which we use in our training step. To prevent the equivalent of

 “catastrophic forgetting” we replace one sample in this batch with the

 original, single-pixel seed state. After concluding the training step , we

 replace samples in the pool that were sampled for the batch with the output

 states from the training step over this batch. The animation below shows a

 random sample of the entries in the pool every 20 training steps.

def pool\_training():

# Set alpha and hidden channels to (1.0).

seed = zeros(64, 64, 16)

seed[64//2, 64//2, 3:] = 1.0

target = targets[‘lizard’]

pool = [seed] \* 1024

for i in range(training\_iterations):

idxs, batch = pool.sample(32)

# Sort by loss, descending.

batch = sort\_desc(batch, loss(batch))

# Replace the highest-loss sample with the seed.

batch[0] = seed

# Perform training.

outputs, loss = train(batch, target)

# Place outputs back in the pool.

pool[idxs] = outputs

 Your browser does not support the video tag.

 A random sample of the patterns in the pool during training, sampled

 every 20 training steps.

 Early on in the training process, the random dynamics in the system allow

 the model to end up in various incomplete and incorrect states. As these

 states are sampled from the pool, we refine the dynamics to be able to

 recover from such states. Finally, as the model becomes more robust at going

 from a seed state to the target state, the samples in the pool reflect this

 and are more likely to be very close to the target pattern, allowing the

 training to refine these almost completed patterns further.

 Essentially, we use the previous final states as new starting points to

 force our CA to learn how to persist or even improve an already formed

 pattern, in addition to being able to grow it from a seed. This makes it

 possible to add a periodical loss for significantly longer time intervals

 than otherwise possible, encouraging the generation of an attractor as the

 target shape in our coupled system. We also noticed that reseeding the

 highest loss sample in the batch, instead of a random one, makes training

 more stable at the initial stages, as it helps to clean up the low quality

 states from the pool.

 Here is what a typical training progress of a CA rule looks like. The cell

 rule learns to stabilize the pattern in parallel to refining its features.

 Your browser does not support the video tag.

 CA behaviour at training steps 100, 500, 1000, 4000.

Experiment 3: Learning to regenerate

------------------------------------

 In addition to being able to grow their own bodies, living creatures are

 great at maintaining them. Not only does worn out skin get replaced with new

 skin, but very heavy damage to complex vital organs can be regenerated in

 some species. Is there a chance that some of the models we trained above

 have regenerative capabilities?

 Your browser does not support the video tag.

 Patterns exhibit some regenerative properties upon being damaged, but

 not full re-growth.

 The animation above shows three different models trained using the same

 settings. We let each of the models develop a pattern over 100 steps, then

 damage the final state in five different ways: by removing different halves

 of the formed pattern, and by cutting out a square from the center. Once

 again, we see that these models show quite different out-of-training mode

 behaviour. For example “the lizard” develops quite strong regenerative

 capabilities, without being explicitly trained for it!

 Since we trained our coupled system of cells to generate an attractor

 towards a target shape from a single cell, it was likely that these systems,

 once damaged, would generalize towards non-self-destructive reactions.

 That’s because the systems were trained to grow, stabilize, and never

 entirely self-destruct. Some of these systems might naturally gravitate

 towards regenerative capabilities, but nothing stops them from developing

 different behaviors such as explosive mitoses (uncontrolled growth),

 unresponsiveness to damage (overstabilization), or even self destruction,

 especially for the more severe types of damage.

 If we want our model to show more consistent and accurate regenerative

 capabilities, we can try to increase the basin of attraction for our target

 pattern - increase the space of cell configurations that naturally gravitate

 towards our target shape. We will do this by damaging a few pool-sampled

 states before each training step. The system now has to be capable of

 regenerating from states damaged by randomly placed erasing circles. Our

 hope is that this will generalize to regenerational capabilities from

 various types of damage.

 Your browser does not support the video tag.

 Damaging samples in the pool encourages the learning of robust

 regenerative qualities. Row 1 are samples from the pool, Row 2 are their

 respective states after iterating the model.

 The animation above shows training progress, which includes sample damage.

 We sample 8 states from the pool. Then we replace the highest-loss sample

 (top-left-most in the above) with the seed state, and damage the three

 lowest-loss (top-right-most) states by setting a random circular region

 within the pattern to zeros. The bottom row shows states after iteration

 from the respective top-most starting state. As in Experiment 2, the

 resulting states get injected back into the pool.

 Your browser does not support the video tag.

 Patterns exposed to damage during training exhibit astounding

 regenerative capabilities.

 As we can see from the animation above, models that were exposed to damage

 during training are much more robust, including to types of damage not

 experienced in the training process (for instance rectangular damage as

 above).

Experiment 4: Rotating the perceptive field

-------------------------------------------

 As previously described, we model the cell’s perception of its neighbouring

 cells by estimating the gradients of state channels in x⃗\vec{x}x⃗ and

 y⃗\vec{y}y⃗ using Sobel filters. A convenient analogy is that each agent has

 two sensors (chemosensory receptors, for instance) pointing in orthogonal

 directions that can sense the gradients in the concentration of certain

 chemicals along the axis of the sensor. What happens if we rotate those

 sensors? We can do this by rotating the Sobel kernels.

 -\sin \theta \\ \sin \theta & \cos \theta \end{bmatrix} \* \begin{bmatrix}

 This simple modification of the perceptive field produces rotated versions

 of the pattern for an angle of choosing without retraining as seen below.

![](Growing%20Neural%20Cellular%20Automata_files/rotation.png)

 Rotating the axis along which the perception step computes gradients

 brings about rotated versions of the pattern.

 In a perfect world, not quantized by individual cells in a pixel-lattice,

 this would not be too surprising, as, after all, one would expect the

 angle - a simple change of frame of reference. However, it is important to

 note that things are not as simple in a pixel based model. Rotating pixel

 based graphics involves computing a mapping that’s not necessarily bijective

 and classically involves interpolating between pixels to achieve the desired

 result. This is because a single pixel, when rotated, will now likely

 overlap several pixels. The successful growth of patterns as above suggests

 a certain robustness to the underlying conditions outside of those

 experienced during training.

Related Work

------------

### CA and PDEs

 There exists an extensive body of literature that describes the various

 flavours of cellular automata and PDE systems, and their applications to

 modelling physical, biological or even social systems. Although it would be

 impossible to present a just overview of this field in a few lines, we will

 describe some prominent examples that inspired this work. Alan Turing

 introduced his famous Turing patterns back in 1952

 , suggesting how

 reaction-diffusion systems can be a valid model for chemical behaviors

 during morphogenesis. A particularly inspiring reaction-diffusion model that

 stood the test of time is the Gray-Scott model

 , which shows an extreme variety of

 behaviors controlled by just a few variables.

 Ever since von Neumann introduced CAs

  as models for self-replication they

 have captivated researchers’ minds, who observed extremely complex

 behaviours emerging from very simple rules. Likewise, the a broader audience

 outside of academia were seduced by CA’s life-like behaviours thanks to

 Conway’s Game of Life . Perhaps

 motivated in part by the proof that something as simple as the Rule 110 is

 Turing complete, Wolfram’s “*A New Kind of Science”*

 asks for a paradigm shift centered

 around the extensive usage of elementary computer programs such as CA as

 tools for understanding the world.

 More recently, several researchers generalized Conway’s Game of life to work

 on more continuous domains. We were particularly inspired by Rafler’s

 SmoothLife  and Chan’s Lenia

 , the latter of

 which also discovers and classifies entire species of “lifeforms”.

 A number of researchers have used evolutionary algorithms to find CA rules

 that reproduce predefined simple patterns

 .

 For example, J. Miller  proposed an

 experiment similar to ours, using evolutionary algorithms to design a CA

 rule that could build and regenerate the French flag, starting from a seed

 cell.

### Neural Networks and Self-Organisation

 The close relation between Convolutional Neural Networks and Cellular

 Automata has already been observed by a number of researchers

 . The

 connection is so strong it allowed us to build Neural CA models using

 components readily available in popular ML frameworks. Thus, using a

 different jargon, our Neural CA could potentially be named “Recurrent

 Residual Convolutional Networks with ‘per-pixel’ Dropout”.

 The Neural GPU

  offers

 a computational architecture very similar to ours, but applied in the

 context of learning multiplication and a sorting algorithm.

 Looking more broadly, we think that the concept of self-organisation is

 finding its way into mainstream machine learning with popularisation of

 Graph Neural Network  models.

 Typically, GNNs run a repeated computation across vertices of a (possibly

 dynamic) graph. Vertices communicate locally through graph edges, and

 aggregate global information required to perform the task over multiple

 rounds of message exchanges, just as atoms can be thought of as

 communicating with each other to produce the emergent properties of a

 molecule , or even points of a point

 cloud talk to their neighbors to figure out their global shape

 .

 Self-organization also appeared in fascinating contemporary work using more

 traditional dynamic graph networks, where the authors evolved

 Self-Assembling Agents to solve a variety of virtual tasks

 .

### Swarm Robotics

 One of the most remarkable demonstrations of the power of self-organisation

 is when it is applied to swarm modeling. Back in 1987, Reynolds’ Boids

  simulated the flocking behaviour of birds with

 just a tiny set of handcrafted rules. Nowadays, we can embed tiny robots

 with programs and test their collective behavior on physical agents, as

 demonstrated by work such as Mergeable Nervous Systems

  and Kilobots

 . To the best of our knowledge, programs

 embedded into swarm robots are currently designed by humans. We hope our

 work can serve as an inspiration for the field and encourage the design of

 collective behaviors through differentiable modeling.

Discussion

----------

### Embryogenetic Modeling

 Your browser does not support the video tag.

 Regeneration-capable 2-headed planarian, the creature that inspired this

 work

 This article describes a toy embryogenesis and regeneration model. This is a

 major direction for future work, with many applications in biology and

 beyond. In addition to the implications for understanding the evolution and

 control of regeneration, and harnessing this understanding for biomedical

 repair, there is the field of bioengineering. As the field transitions from

 synthetic biology of single cell collectives to a true synthetic morphology

 of novel living machines , it

 will be essential to develop strategies for programming system-level

 capabilities, such as anatomical homeostasis (regenerative repair). It has

 long been known that regenerative organisms can restore a specific

 anatomical pattern; however, more recently it’s been found that the target

 morphology is not hard coded by the DNA, but is maintained by a

 physiological circuit that stores a setpoint for this anatomical homeostasis

 . Techniques are

 now available for re-writing this setpoint, resulting for example

  in 2-headed flatworms

 that, when cut into pieces in plain water (with no more manipulations)

 result in subsequent generations of 2-headed regenerated worms (as shown

 above). It is essential to begin to develop models of the computational

 processes that store the system-level target state for swarm behavior

 , so that efficient strategies can be developed for rationally editing this

 information structure, resulting in desired large-scale outcomes (thus

 defeating the inverse problem that holds back regenerative medicine and many

 other advances).

### Engineering and machine learning

 The models described in this article run on the powerful GPU of a modern

 computer or a smartphone. Yet, let’s speculate about what a “more physical”

 implementation of such a system could look like. We can imagine it as a grid

 of tiny independent computers, simulating individual cells. Each of those

 computers would require approximately 10Kb of ROM to store the “cell

 genome”: neural network weights and the control code, and about 256 bytes of

 RAM for the cell state and intermediate activations. The cells must be able

 to communicate their 16-value state vectors to neighbors. Each cell would

 also require an RGB-diode to display the color of the pixel it represents. A

 single cell update would require about 10k multiply-add operations and does

 not have to be synchronised across the grid. We propose that cells might

 wait for random time intervals between updates. The system described above

 is uniform and decentralised. Yet, our method provides a way to program it

 to reach the predefined global state, and recover this state in case of

 multi-element failures and restarts. We therefore conjecture this kind of

 modeling may be used for designing reliable, self-organising agents. On the

 more theoretical machine learning front, we show an instance of a

 decentralized model able to accomplish remarkably complex tasks. We believe

 this direction to be opposite to the more traditional global modeling used

 in the majority of contemporary work in the deep learning field, and we hope

 this work to be an inspiration to explore more decentralized learning

 modeling.

![](Growing%20Neural%20Cellular%20Automata_files/multiple-pages.svg)

 This article is part of the

 an experimental format collecting invited short articles delving into

 differentiable self-organizing systems, interspersed with critical

 commentary from several experts in adjacent fields.

[Self-classifying MNIST Digits](https://distill.pub/2020/selforg/mnist/)

Top-down models in biology: explanation and control of complex living systems above the molecular level

Re-membering
 the body: applications of computational neuroscience to the top-down
control of regeneration of limbs and other complex organs

Transmembrane voltage potential controls embryonic eye patterning in Xenopus laevis

Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networks

The chemical basis of morphogenesis

Complex Patterns in a Simple System

Theory of Self-Reproducing Automata

MATHEMATICAL GAMES

A New Kind of Science

Generalization of Conway's "Game of Life" to a continuous domain - SmoothLife

Lenia: Biology of Artificial Life

Intrinsically Motivated Exploration for Automated Discovery of Patterns in Morphogenetic Systems

Evolving Self-organizing Cellular Automata Based on Neural Network Genotypes

CA-NEAT: Evolved Compositional Pattern Producing Networks for Cellular Automata Morphogenesis and Replication

Evolving a Self-Repairing, Self-Regulating, French Flag Organism

Learning Cellular Automaton Dynamics with Neural Networks

Cellular automata as convolutional neural networks

Neural GPUs Learn Algorithms

Improving the Neural GPU Architecture for Algorithm Learning

A Comprehensive Survey on Graph Neural Networks

Convolutional Networks on Graphs for Learning Molecular Fingerprints

Dynamic Graph CNN for Learning on Point Clouds

Learning to Control Self- Assembling Morphologies: A Study of Generalization via Modularity

Flocks, Herds and Schools: A Distributed Behavioral Model

Mergeable nervous systems for robots

Kilobot: A low cost scalable robot system for collective behaviors

What Bodies Think About: Bioelectric Computation Outside the Nervous System

A scalable pipeline for designing reconfigurable organisms

Perspective: The promise of multi-cellular engineered living systems

Physiological inputs regulate species-specific anatomy during embryogenesis and regeneration

Long-range neural and gap junction protein-mediated cues control polarity during planarian regeneration

Long-Term, Stochastic Editing of Regenerative Anatomy via Targeting Endogenous Bioelectric Gradients

Pattern Regeneration in Coupled Networks

Bioelectrical
 control of positional information in development and regeneration: A
review of conceptual and computational advances

Modeling Cell Migration in a Simulated Bioelectrical Signaling Network for Anatomical Regeneration

Investigating the effects of noise on a cell-to-cell communication mechanism for structure regeneration

Social Intelligence

Inceptionism: Going deeper into neural networks


Thread: Circuits

Nick Cammarata
Shan Carter
Gabriel Goh
Chris Olah
Michael Petrov
Ludwig Schubert
Chelsea Voss
Ben Egan
Swee Kiat Lim

 In the original narrative of deep learning, each neuron builds

 progressively more abstract, meaningful features by composing features in

 the preceding layer. In recent years, there’s been some skepticism of this

 view, but what happens if you take it really seriously?

 InceptionV1 is a classic vision model with around 10,000 unique

neurons — a large number, but still on a scale that a group effort could

 attack.

 What if you simply go through the model, neuron by neuron, trying

to

 understand each one and the connections between them? The circuits

 collaboration aims to find out.

Articles & Comments

-------------------

 The natural unit of publication for investigating circuits seems to be

 short papers on individual circuits or small families of features.

 Compared to normal machine learning papers, this is a small and unusual

 topic for a paper.

 To facilitate exploration of this direction, Distill is inviting a

 “thread” of short articles on circuits, interspersed with critical

 commentary by experts in adjacent fields. The thread will be a living

 document, with new articles added over time, organized through an open

 slack channel (#circuits in the

 [Distill slack](http://slack.distill.pub/)). Content in this

 thread should be seen as early stage exploratory research.

Articles and comments are presented below in chronological order:

###

### Authors

### Affiliations

[Chris Olah](https://colah.github.io/),

 [Nick Cammarata](http://nickcammarata.com/),

 [Ludwig Schubert](https://schubert.io/),

 [Gabriel Goh](http://gabgoh.github.io/),

 [Michael Petrov](https://twitter.com/mpetrov),

 [Shan Carter](http://shancarter.com/)

[OpenAI](https://openai.com/)

 Does it make sense to treat individual neurons and the connections

 between them as a serious object of study? This essay proposes three

 claims which, if true, might justify serious inquiry into them: the

 existence of meaningful features, the existence of meaningful circuits

 between features, and the universality of those features and circuits.

 It also discuses historical successes of science “zooming in,” whether

 we should be concerned about this research being qualitative, and

 approaches to rigorous investigation.

[Read Full Article](https://distill.pub/2020/circuits/zoom-in/)

###

### Authors

### Affiliations

[Chris Olah](https://colah.github.io/),

 [Nick Cammarata](http://nickcammarata.com/),

 [Ludwig Schubert](https://schubert.io/),

 [Gabriel Goh](http://gabgoh.github.io/),

 [Michael Petrov](https://twitter.com/mpetrov),

 [Shan Carter](http://shancarter.com/)

[OpenAI](https://openai.com/)

 An overview of all the neurons in the first five layers of

 InceptionV1, organized into a taxonomy of “neuron groups.” This

 article sets the stage for future deep dives into particular aspects

 of early vision.

 [Read Full Article](https://distill.pub/2020/circuits/early-vision/)

###

[Curve Detectors](https://distill.pub/2020/circuits/curve-detectors/)

### Authors

### Affiliations

[Nick Cammarata](http://nickcammarata.com/),

 [Gabriel Goh](http://gabgoh.github.io/),

 [Shan Carter](http://shancarter.com/),

 [Ludwig Schubert](https://schubert.io/),

 [Michael Petrov](https://twitter.com/mpetrov),

 [Chris Olah](https://colah.github.io/)

[OpenAI](https://openai.com/)

 Every vision model we’ve explored in detail contains neurons which

 detect curves. Curve detectors is the first in a series of three

 articles exploring this neuron family in detail.

[Read Full Article](https://distill.pub/2020/circuits/curve-detectors/)

###

### Authors

### Affiliations

[Chris Olah](https://colah.github.io/),

 [Ludwig Schubert](https://schubert.io/),

 [Gabriel Goh](http://gabgoh.github.io/)

[OpenAI](https://openai.com/)

 Neural networks naturally learn many transformed copies of the same

 feature, connected by symmetric weights.

[Read Full Article](https://distill.pub/2020/circuits/equivariance/)

###

### Authors

### Affiliations

[Ludwig Schubert](https://schubert.io/),

 [Chelsea Voss](https://csvoss.com/),

 [Nick Cammarata](http://nickcammarata.com/),

 [Gabriel Goh](http://gabgoh.github.io/),

 [Chris Olah](https://colah.github.io/)

[OpenAI](https://openai.com/)

 A family of early-vision neurons reacting to directional transitions

 from high to low spatial frequency.

[Read Full Article](https://distill.pub/2020/circuits/frequency-edges/)

###

[Curve Circuits](https://distill.pub/2020/circuits/curve-circuits/)

### Authors

### Affiliations

[Nick Cammarata](http://nickcammarata.com/),

 [Gabriel Goh](http://gabgoh.github.io/),

 [Shan Carter](http://shancarter.com/),

 [Chelsea Voss](https://csvoss.com/),

 [Ludwig Schubert](https://schubert.io/),

 [Chris Olah](https://colah.github.io/)

[OpenAI](https://openai.com/)

 We reverse engineer a non-trivial learned algorithm from the weights

 of a neural network and use its core ideas to craft an artificial

 artificial neural network from scratch that reimplements it.

[Read Full Article](https://distill.pub/2020/circuits/curve-circuits/)

###

[Visualizing Weights](https://distill.pub/2020/circuits/visualizing-weights/)

### Authors

### Affiliations

[Chelsea Voss](https://csvoss.com/),

 [Nick Cammarata](http://nickcammarata.com/),

 [Gabriel Goh](https://gabgoh.github.io/),

 [Michael Petrov](https://twitter.com/mpetrov),

 [Ludwig Schubert](https://schubert.io/),

 Ben Egan,

 [Swee Kiat Lim](https://greentfrapp.github.io/),

 [Chris Olah](https://colah.github.io/)

[OpenAI](https://openai.com/),

 [Mount Royal University](https://mtroyal.ca/),

 [Stanford University](https://stanford.edu/)

 We present techniques for visualizing, contextualizing, and

 understanding neural network weights.

[Read Full Article](https://distill.pub/2020/circuits/visualizing-weights/)

###

### Authors

### Affiliations

[Chelsea Voss](https://csvoss.com/),

 [Gabriel Goh](https://gabgoh.github.io/),

 [Nick Cammarata](http://nickcammarata.com/),

 [Michael Petrov](https://twitter.com/mpetrov),

 [Ludwig Schubert](https://schubert.io/),

 [Chris Olah](https://colah.github.io/)

[OpenAI](https://openai.com/)

 When a neural network layer is divided into multiple branches, neurons

 self-organize into coherent groupings.

[Read Full Article](https://distill.pub/2020/circuits/branch-specialization/)

###

[Weight Banding](https://distill.pub/2020/circuits/weight-banding/)

### Authors

### Affiliations

[Michael Petrov](https://twitter.com/mpetrov),

 [Chelsea Voss](https://csvoss.com/),

 [Ludwig Schubert](https://schubert.io/),

 [Nick Cammarata](http://nickcammarata.com/),

 [Gabriel Goh](https://gabgoh.github.io/),

 [Chris Olah](https://colah.github.io/)

[OpenAI](https://openai.com/)

[Read Full Article](https://distill.pub/2020/circuits/weight-banding/)

#### This is a living document

 Expect more articles on this topic, along with critical comments from

 experts.

Get Involved

------------

 The Circuits thread is open to articles exploring individual features,

 circuits, and their organization within neural networks. Critical

 commentary and discussion of existing articles is also welcome. The thread

 is organized through the open `#circuits` channel on the

 [Distill slack](http://slack.distill.pub/). Articles can be

 suggested there, and will be included at the discretion of previous

 authors in the thread, or in the case of disagreement by an uninvolved

 editor.

 If you would like get involved but don’t know where to start, small

 projects may be available if you ask in the channel.

About the Thread Format

-----------------------

 Part of Distill’s mandate is to experiment with new forms of scientific

 publishing. We believe that that reconciling faster and more continuous

 approaches to publication with review and discussion is an important open

 problem in scientific publishing.

 Threads are collections of short articles, experiments, and critical

 commentary around a narrow or unusual research topic, along with a slack

 channel for real time discussion and collaboration. They are intended to

 be earlier stage than a full Distill paper, and allow for more fluid

 publishing, feedback and discussion. We also hope they’ll allow for wider

 participation. Think of a cross between a Twitter thread, an academic

 workshop, and a book of collected essays.

 Threads are very much an experiment. We think it’s possible they’re a

 great format, and also possible they’re terrible. We plan to trial two

 such threads and then re-evaluate our thought on the format.


Communicating with Interactive Articles

Fred Hohman
Matthew Conlen
Jeffrey Heer
Duen Horng (Polo) Chau

Abstract: Computing has changed how people communicate. The transmission  of news, messages, and ideas is instant. Anyone’s voice can be heard. In  fact, access to digital communication technologies such as the Internet  is so fundamental to daily life that their disruption by government is  condemned by the United Nations Human Rights Council . But while the technology to distribute our ideas has grown in leaps and bounds, the interfaces have remained largely the same.

### Contents

[Introduction](#introduction)

[Interactive Articles: Theory & Practice](#interactive-articles)

* [Connecting People and Data](#connecting-people-and-data)

* [Making Systems Playful](#making-systems-playful)

* [Prompting Self-Reflection](#prompting-self-reflection)

* [Personalizing Reading](#personalizing-reading)

* [Reducing Cognitive Load](#reducing-cognitive-load)

[Challenges for Authoring Interactives](#challenges)

[Critical Reflections](#critical-reflections)

[Looking Forward](#looking-forward)

 Computing has changed how people communicate. The transmission

of news, messages, and ideas is instant. Anyone’s voice can be heard. In

 fact, access to digital communication technologies such as the Internet

 is so fundamental to daily life that their disruption by government is

 Parallel to the development of the internet, researchers like

Alan Kay and Douglas Engelbart worked to build technology that would

empower individuals and enhance cognition. Kay imagined the Dynabook

 in the hands of children across the world. Engelbart, while best

remembered for his “mother of all demos,” was more interested in the

ability of computation to augment human intellect .

 Neal Stephenson wrote speculative fiction that imagined interactive

paper that could display videos and interfaces, and books that could

teach and respond to their readers .

 More recent designs (though still historical by personal computing

 standards) point to a future where computers are connected and assist

people in decision-making and communicating using rich graphics and

 unfortunately, many others have not. The most popular publishing

platforms, for example WordPress and Medium, choose to prioritize social

 features and ease-of-use while limiting the ability for authors to

communicate using the dynamic features of the web.

 In the spirit of previous computer-assisted cognition

technologies, a new type of computational communication medium has

emerged that leverages active reading techniques to make ideas more

accessible to a broad range of people. These interactive articles build

 In this work, for the the first time, we connect the dots between

interactive articles such as those featured in this journal and

publications like *The New York Times* and the techniques,

theories, and empirical evaluations put forth by academic researchers

across the fields of education, human-computer interaction, information

visualization, and digital journalism. We show how digital designers are

 operationalizing these ideas to create interactive articles that help

boost learning and engagement for their readers compared to static

alternatives.

 Conducting

 novel research requires deep understanding and expertise in a specific

area. Once achieved, researchers continue contributing new knowledge for

 future researchers to use and build upon. Over time, this consistent

addition of new knowledge can build up, contributing to what some have

called research debt. Not everyone is an expert in every field, and it

can be easy to lose perspective and forget the bigger picture. Yet

research should be understood by many. Interactive articles can be used

to distill the latest progress in various research fields and make their

 methods and results accessible and understandable to a broader

audience. #### Opportunities

 * Engage and excite broader audience with latest research progress

* Remove research debt, onboard new researchers

* Make faster and clearer research progress

 #### Challenges

 * No clear incentive structure for researchers

* Little funding for bespoke research dissemination and communication

 companion piece to a traditional research paper that uses interactive

visualizations to let readers adjust a machine learning model’s behavior

 PhD thesis that contributes a programming language abstraction for

understanding how programs access the context or environment in which

they execute, and walks readers through the work using two simple

 crash course in complex systems science, created by leading experts,

practitioners, and students in the field, with accompanying interactive

 Interactive articles are applicable to variety of domains, such as

research dissemination, journalism, education, and policy and decision

 Today there is a growing excitement around the use of interactive

articles for communication since they offer unique capabilities to help

people learn and engage with complex ideas that traditional media lacks.

 After describing the affordances of interactive articles, we provide

critical reflections from our own experience with open-source,

interactive publishing at scale. We conclude with discussing practical

challenges and open research directions for authoring, designing, and

publishing interactive articles.

 This style of communication — and the platforms which support

it — are still in their infancy. When choosing where to publish this

work, we wanted the medium to reflect the message. Journals like *Distill*

 are not only pushing the boundaries of machine learning research but

also offer a space to put forth new interfaces for dissemination. This

work ties together the theory and practice of authoring and publishing

interactive articles. It demonstrates the power that the medium has for

providing new representations and interactions to make systems and ideas

 more accessible to broad audiences.

Interactive Articles: Theory & Practice

---------------------------------------

 Interactive articles draw from and connect many types of media,

from static text and images to movies and animations. But in contrast to

 these existing forms, they also leverage interaction techniques such as

 details-on demand, belief elicitation, play, and models and simulations

 to enhance communication.

 While the space of possible designs is far too broad to be solved

with one-size-fits-all guidelines, by connecting the techniques used in

these articles back to underlying theories presented across disparate

fields of research we provide a missing foundation for designers to use

when considering the broad space of interactions that could be added to a

 born-digital article.

 We draw from a corpus of over sixty interactive articles to

highlight the breadth of techniques available and analyze how their

authors took advantage of a digital medium to improve the reading

experience along one or more dimensions, for example, by reducing the

overall cognitive load, instilling positive affect, or improving

information recall.

| Title *arrow\_downward* | Publication or Author  | Tags  | Year  |

| --- | --- | --- | --- |

 Because diverse communities create interactive content, this

medium goes by many different names and has not yet settled on a

 In newsrooms, data journalists, developers, and designers work together

 to make complex news and investigative reporting clear and engaging

using interactive stories . Educators

 use interactive textbooks as an alternative learning format to give

students hands-on experience with learning material .

 Besides these groups, others such as academics, game developers,

web developers, and designers blend editorial, design, and programming

 While these all slightly differ in their technical approach and target

audience, they all largely leverage the interactivity of the modern web.

 We focus on five unique affordances of interactive articles,

listed below. In-line videos and example interactive graphics are

presented alongside this discussion to demonstrate specific techniques.

### Connecting People and Data

 an audience which finds content to be aesthetically pleasing is more

likely to have a positive attitude towards it. This in turn means people

 will spend more time engaging with content and ultimately lead to

improved learning outcomes. While engagement itself may not be an end

goal of most research communications, the ability to influence both

audience attitude and the amount of time that is spent is a useful lever

 to improve learning: we know from education research that both time

spent  and emotion  are predictive of learning outcomes.

 Animations can also be used to improve engagement .

 While there is debate amongst researchers if animations in general are

able to more effectively convey the same information compared to a well

 while the series of still images may be more effective for answering

specific questions like, “Does a horse lift all four of its feet off the

 ground when it runs?” watching the animation in slow motion gives the

viewer a much more visceral sense of how it runs. A more modern example

 1878, Eadweard Muybridge settled Leland Stanford's hotly debated

question of whether all four feet of a horse lifted off the ground

during a trot using multiple cameras to capture motion in stop-motion

 Passively, animation can be used to add drama to a graphic

displaying important information, but which readers may otherwise find

dry. Scientific data which is inherently time varying may be shown using

 an animation to connect viewers more closely with the original data, as

 compared to seeing an abstracted static view. For example, Ed Hawkins

designed “Climate Spirals,” which shows the average global temperature

change over time . This

presentation of the data resonated with a large public audience, so much

 so that it was displayed at the opening ceremony at the 2016 Rio

Olympics. In fact, many other climate change visualizations of this same

 dataset use animation to build suspense and highlight the recent spike

in global temperatures .

 Preview,

 click to play

 Full Video.

 By adding variation over time, authors have access to a new

dimension to encode information and an even wider design space to work

in. Consider the animated graphic in *The New York Times* story

“Extensive Data Shows Punishing Reach of Racism for Black Boys,” which

 by utilizing animation, it became possible for the authors to design a

unit visualization in which each data point shown represented an

individual, reminding readers that the data in this story was about real

 peoples’ lives.

 Unit visualizations have also been used to evoke empathy in

 Using person-shaped glyphs (as opposed to abstract symbols like circles

 or squares) has been shown not to produce additional empathic responses

 using visualizations. Correll argues that much of the power of

visualization comes from abstraction, but quantization stymies empathy .

 He instead suggests anthropomorphizing data, borrowing journalistic and

 rhetoric techniques to create novel designs or interventions to foster

empathy in readers when viewing visualizations .

 Regarding the format of interactive articles, an ongoing debate

within the data journalism community has been whether articles which

utilize scroll-based graphics (scrollytelling) are more effective than

those which use step-based graphics (slideshows). McKenna et al.

 found that their study participants largely preferred content to be

displayed with a step- or scroll-based navigation as opposed to

traditional static articles, but did not find a significant difference

in engagement between the two layouts. In related work, Zhi et al. found

 that performance on comprehension tasks was better in slideshow layouts

 than in vertical scroll-based layouts .

 Both studies focused on people using desktop (rather than mobile)

devices. More work is needed to evaluate the effectiveness of various

layouts on mobile devices, however the interviews conducted by MckEnna

et al. suggest that additional features, such as supporting navigation

through swipe gestures, may be necessary to facilitate the mobile

reading experience.

readers play the role of a pirate commander, giving them a unique look

at the economics that led to rise in piracy off the coast of Somalia. Playing

 Preview,

 click to play

 Full Video.

 the reading experience becomes closer to that of playing a game. For

example, the critically acclaimed explorable explanation “Parable of the

 Polygons” puts play at the center of the story, letting a reader

manually run an algorithm that is later simulated in the article to

demonstrate how a population of people with slight personal biases

against diversity leads to social segregation .

### Making Systems Playful

 Interactive articles utilize an underlying computational

infrastructure, allowing authors editorial control over the

computational processes happening on a page. This access to computation

allows interactive articles to engage readers in an experience they

could not have with traditional media. For example, in “Drawing Dynamic

Visualizations”, Victor demonstrates how an interactive visualization

can allow readers to build an intuition about the behavior of a system,

leading to a fundamentally different understanding of an underlying

 Complex systems often require extensive setup to allow for proper

study: conducting scientific experiments, training machine learning

models, modeling social phenomenon, digesting advanced mathematics, and

researching recent political events, all require the configuration of

sophisticated software packages before a user can interact with a system

 at all, even just to tweak a single parameter. This barrier to entry

can deter people from engaging with complex topics, or explicitly

prevent people who do not have the necessary resources, for example,

computer hardware for intense machine learning tasks. Interactive

articles drastically lower these barriers.

 Science that utilizes physical and computational experiments

requires systematically controlling and changing parameters to observe

their effect on the modeled system. In research, dissemination is

typically done through static documents, where various figures show and

compare the effect of varying particular parameters. However, efforts

have been made to leverage interactivity in academic publishing,

 gives readers control over the reporting of the research findings and

shows great promise in helping readers both digest new ideas and learn

about existing fields that are built upon piles of research debt .

 Beyond reporting statistics, interactive articles are extremely

powerful when the studied systems can be modeled or simulated in

real-time with interactive parameters without setup, e.g., in-browser

sandboxes. Consider the example in [4](#simulation-vis)

 of a Boids simulation that models how birds flock together. Complex

systems such as these have many different parameters that change the

resulting simulation. These sandbox simulations allow readers to play

with parameters to see their effect without worrying about technical

overhead or other experimental consequences.

Interact with live simulations—no setup required. This

 Boids visualization models the movement of a flock of birds, and

exposes parameters that a reader can manipulate to change the behavior

of the simulation. Boid Count

 At the top, drag the slider to change the number of boids in the

simulation. Underneath, adjust the different parameters to find

interesting configurations.

 This is a standout design pattern within interactive articles, and

 many examples exist ranging in complexity. “How You Will Die” visually

simulates the average life expectancy of different groups of people,

where a reader can choose the gender, race, and age of a person .

 “On Particle Physics” allows readers to experiment with accelerating

different particles through electric and magnetic fields to build

intuition behind electromagnetism foundations such as the Lorentz force

and Maxwell’s equations — the experiments backing these simulations

cannot be done without multi-million dollar machinery .

 “Should Prison Sentences Be Based On Crimes That Haven’t Been Committed

 Yet?” shows the outcome of calculating risk assessments for recidivism

where readers adjust the thresholds for determining who gets parole .

 reader uses their own live video camera to train a machine learning

image classifier in-browser without any extra computational resources. Playing

 Preview,

 click to play

 Full Video.

 The dissemination of modern machine learning techniques has been

bolstered by interactive models and simulations. Three articles, “How to

 show the effect that hyperparameters and different dimensionality

reduction techniques have on creating low dimensional embeddings of

high-dimensional data. A popular approach is to demonstrate how machine

 Other examples are aimed at technical readers who wish to learn about

specific concepts within deep learning. Here, interfaces allow readers

to choose model hyperparameters, datasets, and training procedures that,

 once selected, visualize the training process and model internals to

inspect the effect of varying the model configuration .

 Interactive articles commonly communicate a single idea or concept

 using multiple representations. The same information represented in

different forms can have different impact. For example, in mathematics

often a single object has both an algebraic and a geometric

representation. A clear example of this is the definition of a circle .

 Both are useful, inform one another, and lead to different ways of

thinking. Examples of interactive articles that demonstrate this include

 various media publications’ political election coverage that break down

 the same outcome in multiple ways, for example, by voter demographics,

geographical location, and historical perspective .

 as people can process information through both a visual channel and

auditory channel simultaneously. Popular video creators such as

3Blue1Brown  and Primer

 exemplify these principles by using rich animation and simultaneous

narration to break down complex topics. These videos additionally take

advantage of the Redundancy Principle by including complementary

information in the narration and in the graphics rather than repeating

the same information in both channels .

 Preview,

 click to play

 Full Video.

 While these videos are praised for their approachability and

rich exposition, they are not interactive. One radical extension from

traditional video content is also incorporating user input into the

video while narration plays. A series of these interactive videos on

“Visualizing Quaternions” lets a reader listen to narration of a live

animation on screen, but at any time the viewer can take control of the

video and manipulate the animation and graphics while simultaneously

listening to the narration .

 Utilizing multiple representations allows a reader to see

different abstractions of a single idea. Once these are familiar and

known, an author can build interfaces from multiple representations and

let readers interact with them simultaneously, ultimately leading to

interactive experiences that demonstrate the power of computational

communication mediums. Next, we discuss such experiences where

interactive articles have transformed communication and learning by

making live models and simulations of complex systems and phenomena

accessible.

### Prompting Self-Reflection

 Asking a student to reflect on material that they are studying and

 explain it back to themselves — a learning technique called

self-explanation — is known to have a positive impact on learning

outcomes . By generating explanations

 and refining them as new information is obtained, it is hypothesized

that a student will be more engaged with the processes which they are

studying . When writing for an

interactive environment, components can be included which prompt readers

 to make a prediction or reflection about the material and cause them to

 engage in self-explanation .

 While these prompts may take the form of text entry or other

standard input widgets, one of the most prominent examples of this

 In these visualizations, readers are prompted to complete a trendline

on a chart, causing them to generate an explanation based on their

current beliefs for why they think the trend may move in a certain

direction. Only after readers make their prediction are they shown the

actual data. Kim et al. showed that using visualizations as a prompt is

an effective way to encourage readers to engage in self explanation and

improve their recall of the information . [5](#you-draw-it)

 shows one these visualizations for CO₂ emissions from burning fossil

fuels. After clicking and dragging to guess the trend, your guess will

be compared against the actual data.

Complete the trend of CO₂ emissions from burning fossil fuels. Letting

 a reader first guess about data and only showing the ground truth

afterwards challenges a reader's prior beliefs and has been shown to

readers are tasked to type the names of celebrities with challenging

spellings. After submitting a guess, a visualization shows the reader’s

entry against everyone else’s, scaled by the frequency of different

spellings. Playing

 Preview,

 click to play

 Full Video.

 In the case of “You Draw It,” readers were also shown the

predictions that others made, adding a social comparison element to the

experience. This additional social information was not shown to

necessarily be effective for improving recall .

 However, one might hypothesize that this social aspect may have other

benefits such as improving reader engagement, due to the popularity of

 Prompting readers to remember previously presented material, for

 example through the use of quizzes, can be an effective way to improve

 While testing may call to mind stressful educational experiences for

many, quizzes included in web articles can be low stakes: there is no

need to record the results or grade readers. The effect is enhanced if

feedback is given to the quiz-takers, for example by providing the

correct answer after the user has recorded their response .

 Preview,

 click to play

 Full Video.

 assuming readers are willing to participate in the process. The idea of

 spaced repetition has been a popular foundation for memory building

applications, for example in the Anki flash card system. More recently,

authors have experimented with building spaced repetition directly into

### Personalizing Reading

 Content personalization — automatically modifying text and

multimedia based on a reader’s individual features or input (e.g.,

demographics or location) — is a technique that has been shown to

increase engagement and learning within readers  and support behavioral change .

 The PersaLog system gives developers tools to build personalized

content and presents guidelines for personalization based on user

research from practicing journalists .

 Other work has shown that “personalized spatial analogies,” presenting

distance measurements in regions where readers are geographically

familiar with, help people more concretely understand new distance

measurements within news stories .

 reader enters their birthplace and birth year and is shown multiple

visualizations describing the impact of climate on their hometown. Playing

 Preview,

 click to play

 Full Video.

 Personalization alone has also been used as the standout feature

of multiple interactive articles. Both “How Much Hotter Is Your Hometown

 Than When You Were Born?”  and “Human Terrain”

 use a reader’s location to drive stories relating to climate change and

 population densities respectively. Other examples ask for explicit

reader input, such as a story that visualizes a reader’s net worth to

challenge a reader’s assumptions if they are wealthy or not (relative to

 In this visualization, professions are plotted to determine their

likelihood of being automated against their average annual wage. The

article encourages readers to use the search bar to type in their own

profession to highlight it against the others.

 An interactive medium has the potential to offer readers an

experience other than static, linear text. Non-linear stories, where a

reader can choose their own path through the content, have the potential

 to provide a more personalized experience and focus on areas of user

 a technology focused news television program. Non-linear stories

present challenges for authors, as they must consider the myriad

possible paths through the content, and consider the different possible

experiences that the audience would have when pursuing different

branches.

 [8](#matuschak2019quantum): In "[Quantum Country](https://quantum.country/)" ,

the interactive textbook uses spaced repetition and allows a reader to

opt-in and save their progress while reading through dense material and

mathematical notion over time. Playing

 Preview,

 click to play

 Full Video.

 Another technique interactive articles often use is segmenting

content into small pieces to be read in-between or alongside other

graphics. While we have already discussed cognitive load theory, the

Segmenting Theory, the idea that complex lessons are broken into

smaller, bit-sized parts , also

 supports personalization within interactive articles. Providing a

reader the ability to play, pause, and scrub content allows the reader

to move at their own speed, comprehending the information at a speed

that works best for them. Segmenting also engages a reader’s essential

processing without overloading their cognitive system .

 Multiple studies have been conducted showing that learners perform

 better when information is segmented, whether it be only within an

animation  or within an interface with textual descriptions .

 One excellent example of using segmentation and animation to

personalize content delivery is “A Visual Introduction to Machine

Learning,” which introduces fundamental concepts within machine learning

 in bite-sized pieces, while transforming a single dataset into a

trained machine learning model .

Extending this idea, in “Quantum Country,” an interactive textbook

covering quantum computing, the authors implemented a user account

system, allowing readers to save their position in the text and consume

### Reducing Cognitive Load

 Authors must calibrate the detail at which to discuss ideas and

content to their readers expertise and interest to not overload them.

When topics become multifaceted and complex, a balance must be struck

between a high-level overview of a topic and its lower-level details.

One interaction technique used to prevent a cognitive overload within a

reader is “details-on-demand.”

 Details-on-demand has become an ubiquitous design pattern. For

example, modern operating systems offer to fetch dictionary definitions

when a word is highlighted. When applied to visualization, this

technique allows users to select parts of a dataset to be shown in more

detail while maintaining a broad overview. This is particularly useful

when a change of view is not required, so that users can inspect

elements of interest on a point-by-point basis in the context of the

whole . Below we highlight areas

where details-on-demand has been successfully applied to reduce the

amount of information present within an interface at once.

#### Data Visualization

 Successful visualizations not only provide the base representations and

 techniques for these three steps, but also bridge the gaps between them

 . In practice, the

solidified standard for details-on-demand in data visualization

manifests as a tooltip, typically summoned on a cursor mouseover, that

presents extra information in an overlay. Given that datasets often

contain multiple attributes, tooltips can show the other attributes that

#### Illustration

 Details-on-demand is also used in illustrations, interactive

textbooks, and museum exhibits, where highlighted segments of a figure

can be selected to display additional information about the particular

segment. For example, in “How does the eye work?” readers can select

segments of an anatomical diagram of the human eye to learn more about

specific regions, e.g., rods and cones .

 Another example is “Earth Primer,” an interactive textbook on tablets

that allows readers to inspect the Earth’s interior, surface, and biomes

#### Mathematical Notation

 Formal mathematics, a historically static medium, can benefit from

 details-on-demand, for example, to elucidate a reader with intuition

about a particular algebraic term, present a geometric interpretation of

 an equation, or to help a reader retain high-level context while

 For example, in “Why Momentum Really Works,” equation layout is done

using Gestalt principles plus annotation to help a reader easily

identify specific terms. In “Colorized

 Math Equations,” the Fourier transform equation is presented in both

mathematical notation and plain text, but the two are linked through a

mouseover that highlights which term in the equation corresponds to

which word in the text .

Another example that visualizes mathematics and computation is the

“Image Kernels” tutorial where a reader can mouseover a real image and

observe the effect and exact computation for applying a filter over the

image .

 Instead of writing down long arithmetic sums, the interface allows

 one of Maxwell’s equations is shown. Click the two buttons to reveal,

or remind yourself, what each notation mark and variable represent.

Enhancing mathematics design with annotation and interactivity. Working

 with equations requires a sharp working memory. Optional interactivity

can help people remember specific notation and variable defintions, only

#### Text

 While not as pervasive, text documents and other long-form textual

 mediums have also experimented with letting readers choose a variable

level of detail to read. This idea was explored as early as the 1960s in

 StretchText, a hypertext feature that allows a reader to reveal a more

descriptive or exhaustive explanation of something by expanding or

 One challenge that has limited this technique’s adoption is the burden

it places on authors to write multiple versions of their content. For

example, drag the slider in [9](#details-text)

 to read descriptions of the Universal Approximation Theorem in

increasing levels of detail. For other examples of details-on-demand for

 text, such as application in code documentation, see this small

collection of examples .

 networks can approximate any function that exists. However, we do not

have a guaranteed way to obtain such a neural network for every

function.

#### Previewing Content

 Details-on-demand can also be used as a method for previewing

content without committing to another interaction or change of view. For

 example, when hovering over a hyperlink on Wikipedia, a preview card is

 shown that can contain an image and brief description; this gives

readers a quick preview of the topic without clicking through and

 within hypertext that present information about a particular topic in a

 location that does not obscure the source material. Both older and

modern preview techniques use perceptually-based animation and simple

tooltips to ensure their interactions are natural and lightweight

feeling to readers .

Challenges for Authoring Interactives

-------------------------------------

*If interactive articles provide clear benefits over other

mediums for communicating complex ideas, then why aren’t they more

prevalent?*

 Unfortunately, creating interactive articles today is difficult.

Domain-specific diagrams, the main attraction of many interactive

articles, must be individually designed and implemented, often from

scratch. Interactions need to be intuitive and performant to achieve a

nice reading experience. Needless to say, the text must also be

well-written, and, ideally, seamlessly integrated with the graphics.

 The act of creating a successful interactive article is closer to

building a website than writing a blog post, often taking significantly

more time and effort than a static article, or even an academic

 Most interactive articles are created using general purpose

web-development frameworks which, while expressive, can be difficult to

work with for authors who are not also web developers. Even for expert

web developers, current tools offer lower levels of abstraction than may

 be desired to prototype and iterate on designs.

 can help authors start writing quickly and even enable rapid iteration

through various designs (for example, letting an author quickly compare

between sequencing content using a “scroller” or “stepper” based

layout). However, Idyll does not offer any design guidance, help authors

 think through where interactivity would be most effectively applied,

nor highlight how their content could be improved to increase its

readability and memorability. For example, Idyll encodes no knowledge of

 the positive impact of self-explanation, instead it requires authors to

 be familiar with this research and how to operationalize it.

 To design an interactive article successfully requires a diverse

set of editorial, design, and programming skills. While some individuals

 are able to author these articles on their own, many interactive

articles are created by a collective team consisting of multiple members

 with specialized skills, for example, data analysts, scripters,

editors, journalists, graphic designers, and typesetters, as outlined in

 requires one to clone its source code using git, install

project-specific dependencies using a terminal, and be comfortable

editing HTML files. All of this complexity is incidental to task of

editing text.

 Publishing to the web brings its own challenges: while interactive

 articles are available to anyone with a browser, they are burdened by

rapidly changing web technologies that could break interactive content

after just a few years. For this reason, easy and accessible interactive

 article archival is important for authors to know their work can be

 Authoring interactive articles also requires designing for a diverse

set of devices, for example, ensuring bespoke content can be adapted for

 desktop and mobile screen sizes with varying connection speeds, since

accessing interactive content demands more bandwidth.

 There are other non-technical limitations for publishing

interactive articles. For example, in non-journalism domains, there is a

 mis-aligned incentive structure for authoring and publishing

interactive content: why should a researcher spend time on an “extra”

interactive exposition of their work when they could instead publish

more papers, a metric by which their career depends on? While different

groups of people seek to maximize their work’s impact, legitimizing

interactive artifacts requires buy-in from a collective of communities.

 Making interactive articles accessible to people with disabilities

 is an open challenge. The dynamic medium exacerbates this problem

compared to traditional static writing, especially when articles combine

 multiple formats like audio, video, and text. Therefore, ensuring

interactive articles are accessible to everyone will require alternative

 modes of presenting content (e.g. text-to-speech, video captioning,

data physicalization, data sonification) and careful interaction design.

 It is also important to remember that not everything needs to be

interactive. Authors should consider the audience and context of their

work when deciding if use of interactivity would be valuable. In the

worst case, interactivity may be distracting to readers or the

functionality may go unused, the author having wasted their time

implementing it. However, even in a domain where the potential

Critical Reflections

--------------------

 We write this article not as media theorists, but as

practitioners, researchers, and tool builders. While it has never been

easier for writers to share their ideas online, current publishing tools

 largely support only static authoring and do not take full advantage of

 the fact that the web is a dynamic medium. We want that to change, and

we are not alone. Others from the explorable explanations community have

 identified design patterns that help share complex ideas through play .

 an annually published digital magazine that showcases the expository

power that interactive dynamic media can have when effectively combined .

 In late 2018, we invited writers to respond to a call for proposals for

 our first issue focusing on exploring scientific and technological

phenomena that stand to shape society at large. We sought to cover

topics that would benefit from using the interactive or otherwise

dynamic capabilities of the web. Given the challenges of authoring

interactive articles, we did not ask authors to submit fully developed

pieces. Instead, we accepted idea submissions, and collaborated with the

 authors over the course of four months to develop the issue, offering

technical, design, and editorial assistance collectively to the authors

that lacked experience in one of these areas. For example, we helped a

writer implement visualizations, a student frame a cohesive narrative,

and a scientist recap history and disseminate to the public. Multiple

views from one article are shown in [10](#parametric).

 The article used techniques like animation, data visualizations,

explanatory diagrams, margin notes, and interactive simulations to

explain how biases occur in machine learning systems.

 We see *The Parametric Press* as a crucial connection between

 the often distinct worlds of research and practice. The project serves

as a platform through which to operationalize the theories put forth by

education, journalism, and HCI researchers. Tools like Idyll which are

designed in a research setting need to be validated and tested to ensure

 that they are of practical use; *The Parametric Press* facilitates

 this by allowing us to study its use in a real-world setting, by

authors who are personally motivated to complete their task of

constructing a high-quality interactive article, and only have secondary

 concerns and care about the tooling being used, if at all.

|  | Research | Practice |

| --- | --- | --- |

 As researchers we can treat the project as a series of case

studies, where we were observers of the motivation and workflows which

were used to craft the stories, from their initial conception to their

publication. Motivation to contribute to the project varied by author.

Where some authors had personal investment in an issue or dataset they

wanted to highlight and raise awareness to broadly, others were drawn

towards the medium, recognizing its potential but not having the

expertise or support to communicate interactively. We also observed how

research software packages like Apparatus , Idyll , and D3

 fit into the production of interactive articles, and how authors must

combine these disparate tools to create an engaging experience for

readers. In one article, “On Particle Physics,” an author combined two

tools in a way that allowed him to create and embed dynamic graphics

directly into his article without writing any code beyond basic markup.

One of the creators of Apparatus had not considered this type of

 fantastic! Reading that article, I had no idea that Apparatus was used.

 This is a very exciting proof-of-concept for unconventional

explorable-explanation workflows.”*

 We were able to provide editorial guidance to the authors drawing

on our knowledge of empirical studies done in the multimedia learning

and information visualization communities to recommend graphical

structures and page layouts, helping each article’s message be

communicated most effectively. One of the most exciting outcomes of the

project is that we saw authors develop interactive communication skills

like any other skill: through continued practice, feedback, and

iteration. We also observed the challenges that are inherent in

publishing dynamic content on the web and identified the need for

improved tooling in this area, specifically around the archiving of

interactive articles. Will an article’s code still run a year from now?

Ten years from now? To address interactive content archival, we set up a

 system to publish a digital archive of all of our articles at the time

that they are first published to the site. At the top of each article on

 *The Parametric Press* is an archive link that allows readers to

download a WARC (Web ARChive) file that can “played back” without

requiring any web infrastructure. While our first iteration of the

project relied on ad-hoc solutions to these problems, we hope to show

how digital works such as ours can be published confidently knowing that

 they will be preserved indefinitely.

 As practitioners we pushed the boundaries of the current

generation of tools designed to support the creation of interactive

articles on the web. We found bugs and limitations in Idyll, a tool

which was originally designed to support the creation of one-off

articles that we used as a content management system to power an entire

magazine issue. We were forced to write patches and plugins to work

 We were also forced to craft designs under a more realistic set of

constraints than academics usually deal with: when creating a

visualization it is not enough to choose the most effective visual

encodings, the graphics also had to be aesthetically appealing, adhere

to a house style, have minimal impact on page load time and runtime

performance, be legible on both mobile and desktop devices, and not be

overly burdensome to implement. Any extra hour spent implementing one

graphic was an hour that was not spent improving some other part of the

issue, such as the clarity of the text, or the overall site design.

 There are relatively few outlets that have the skills, technology,

 and desire to publish interactive articles. From its inception, one of

the objectives of *The Parametric Press* is to showcase the new

forms of media and publishing that are possible with tools available

today, and inspire others to create their own dynamic writings. For

 told us he had wanted to write this interactive article for years yet

never had the opportunity, support, or incentive to create it. His

article drew wide interest and critical acclaim.

 We also wanted to take the opportunity as an independent

publication to serve as a concrete example for others to follow, to

represent a set of best practices for publishing interactive content. To

 that end, we made available all of the software that runs the site,

including reusable components, custom data visualizations, and the

publishing engine itself.

Looking Forward

---------------

 A diverse community has emerged to meet these challenges,

exploring and experimenting with what interactive articles could be. The

 [Explorable Explanations community](https://explorabl.es/)

is a “disorganized ‘movement’ of artists, coders & educators who

want to reunite play and learning.” Their online hub contains 170+

interactive articles on topics ranging from art, natural sciences,

social sciences, journalism, and civics. The curious can also find

tools, tutorials, and meta-discussion around learning, play, and

 Many interactive articles are self-published due to a lack of

platforms that support interactive publishing. Creating more outlets

that allow authors to publish interactive content will help promote

their development and legitimization. The few existing examples,

 help, but currently target a narrow group of authors, namely those who

have programming skills. Such platforms should also provide clear paths

to submission, quality and editorial standards, and authoring

guidelines. For example, news outlets have clear instructions for

pitching written pieces, yet this is under-developed for interactive

articles. Lastly, there is little funding available to support the

development of interactive articles and the tools that support them.

Researchers do not receive grants to communicate their work, and

practitioners outside of the largest news outlets are not able to afford

 the time and implementation investment. Providing more funding for

enabling interactive articles incentivizes their creation and can

contribute to a culture where readers expect digital communications to

better utilize the dynamic medium.

 We have already discussed the breadth of skills required to author

 an interactive article. Can we help lower the barrier to entry? While

there have been great, practical strides in this direction ,

 there is still opportunity for creating tools to design, develop, and

evaluate interactive articles in the wild. Specific features should

include supporting mobile-friendly adaptations of interactive graphics

(for example ),

 creating content for different platforms besides just the web, and

tools that allow people to create interactive content without code.

 The usefulness of interactive articles is predicated on the

assumption that these interactive articles actually facilitate

communication and learning. There is limited empirical evaluation of the

 effectiveness of interactive articles. The problem is exacerbated by

the fact that large publishers are unwilling to share internal metrics,

 provided one of the few available data points, stating that only a

fraction of readers interact with non-static content, and suggested that

 designers should move away from interactivity .

 However, other research found that many readers, even those on mobile

devices, are interested in utilizing interactivity when it is a core

part of the article’s message . This statement from *The New York Times*

 has solidified as a rule-of-thumb for designers and many choose not to

utilize interactivity because of it, despite follow-up discussion that

contextualizes the original point and highlights scenarios where

interactivity can be beneficial .

 This means designers are potentially choosing a suboptimal presentation

 of their story due to this anecdote. More research is needed in order

to identify the cases in which interactivity is worth the cost of

creation.

 We believe in the power and untapped potential of interactive

articles for sparking reader’s desire to learn and making complex ideas

accessible and understandable to all.

Report on the role of digital access providers

A personal computer for children of all ages

Augmenting human intellect: A conceptual framework

The diamond age

The knowledge navigator

Getting it out of our system

PLATO

PhET interactive simulations

Explorable explanations

How y’all, youse and you guys talk

Snow fall: The avalanche at tunnel creek

Why outbreaks like coronavirus spread exponentially, and how to 'flatten the curve'

Attacking discrimination with smarter machine learning

Coeffects: Context-aware programming languages

Complexity explained

What's really warming the world

You draw it: How family income predicts children’s college chances

The Uber Game

Let's learn about waveforms

The book of shaders

EconGraphs

To build a better ballot

The atlas of redistricting

Is it better to rent or buy

Explorable explanation

Increasing the transparency of research papers with explorable multiverse analyses

Workshop on Visualization for AI Explainability

Exploranation: A new science communication paradigm

Research debt

More than telling a story: Transforming data into visually shared stories

"Concrete" computer manipulatives in mathematics education

Cybertext: Perspectives on ergodic literature

Interactive non-fiction: Towards a new approach for storytelling in digital journalism

Active essays on the web

Newsgames: Journalism at play

Simply bells and whistles? Cognitive effects of visual aesthetics in digital longforms

Learning as a function of time

Emotional design in multimedia learning

Hooked on data videos: assessing the effect of animation and pictographs on viewer engagement

Animation: Can it facilitate?

Animated transitions in statistical data graphics

Hypothetical outcome plots outperform error bars and violin plots for inferences about reliability of variable ordering

La perception de la causalité.(Etudes Psychol.), Vol. VI

The illusion of life: Disney animation

The horse in motion

Emergent tool use from multi-agent autocurricula

Climate spirals

Earth's relentless warming sets a brutal new record in 2017

Global temperature

It's not your imagination. Summers are getting hotter.

Extensive data shows punishing reach of racism for black boys

Disagreements

Gun deaths in america

The fallen of World War II

Showing people behind data: Does anthropomorphizing visualizations elicit more empathy for human rights data?

Beyond memorability: Visualization recognition and recall

What makes a visualization memorable?

What if the data visualization is actually people

Ethical dimensions of visualization research

A walk among the data

Visual narrative flow: Exploring factors shaping data visualization story reading experiences

Linking and layout: Exploring the integration of text and visualization in storytelling

Video games and learning

Cutthroat Capitalism: The Game

Combining software games with education: Evaluation of its educational effectiveness

Narrative visualization: Telling stories with data

Parable of the polygons

Drawing dynamic visualizations

How to read a book: The classic guide to intelligent reading

Scientific communication as sequential art

How you will die

On Particle Physics

Should prison sentences be based on crimes that haven’t been committed yet

How to use t-SNE effectively

The beginner's guide to dimensionality reduction

Understanding UMAP

Tensorflow.js: Machine learning for the web and beyond

Designing (and learning from) a teachable machine

Experiments in handwriting with a neural network

Direct-manipulation visualization of deep networks

Gan lab: Understanding complex deep generative models using interactive visual experimentation

Using artificial intelligence to augment human intelligence

Who will win the presidency

Who will be president

Live results: Presidential election

Multimedia learning

3Blue1Brown

Primer

Revising the redundancy principle in multimedia learning

Visualizing quaternions: An explorable video series

Self-explanations: How students study and use examples in learning to solve problems

Self-explaining expository texts: The dual processes of generating inferences and repairing mental models

Explaining the gap: Visualizing one's predictions improves recall and comprehension of data

You draw it: Just how bad is the drug overdose epidemic

You draw it: What got better or worse during Obama's presidency

They draw it!

Data through others' eyes: The impact of visualizing others' expectations on visualization interpretation

The Gyllenhaal experiment

How do you draw a circle? We analyzed 100,000 drawings to show how culture shapes our instincts

Recitation as a factor in memorizing

The power of testing memory: Basic research and implications for educational practice

Khan Academy

The instructional effect of feedback in test-like events

The critical importance of retrieval for learning

How to remember anything for forever-ish

Quantum country

Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice.

Authoring and generation of individualized patient education materials

PersaLog: Personalization of news article content

Generating personalized spatial analogies for distances and areas

How much hotter is your hometown than when you were born

Human terrain

Are you rich? This income-rank quiz might change how you see yourself

Quiz: Let us predict whether you’re a democrat or a republican

Find Out If Your Job Will Be Automated

Booze calculator: What's your drinking nationality

Click 1,000: How the pick-your-own-path episode was made

E-learning and the science of instruction: Proven guidelines for consumers and designers of multimedia learning

Multimedia learning in an interactive self-explaining environment: What works in the design of agent-based microworlds?

Pictorial aids for learning by doing in a multimedia geology simulation game.

A visual introduction to machine learning

Beyond guidelines: What can we learn from the visual information seeking mantra?

The eyes have it: A task by data type taxonomy for information visualizations

Information visualization and visual data mining

How the recession shaped the economy, in 255 charts

How does the eye work

Earth primer

Progressive growing of gans for improved quality, stability, and variation

A style-based generator architecture for generative adversarial networks

Why momentum really works

Colorized math equations

Image kernels

Stretchtext – hypertext note #8

On variable level-of-detail documents

Call for proposals winter/spring 2019

A UI that lets readers control how much information they see

Wikipedia Preview Card

Fluid links for informed and incremental link transitions

Reading and writing fluid hypertext narratives

Idyll: A markup language for authoring and publishing interactive articles on the web

Apparatus: A hybrid graphics editor and programming environment for creating interactive diagrams

Observable

LOOPY: a tool for thinking in systems

Webstrates: shareable dynamic media

Neural networks and deep learning

How I make explorable explanations

Explorable explanations: 4 more design patterns

Emerging and recurring data-driven storytelling techniques: Analysis of a curated collection of recent stories

Issue 01: Science & Society

The myth of the impartial machine

D3 data-driven documents

Launching the Parametric Press

Techniques for flexible responsive visualization design

A Comparative Evaluation of Animation and Small Multiples for Trend Visualization on Mobile Phones

Visualizing ranges over time on mobile phones: a task-based crowdsourced evaluation

Why we are doing fewer interactives

Capture & analysis of active reading behaviors for interactive articles on the web

In defense of interactive graphics

Communicating with Interactive Articles.html