Curve Detectors

Nick Cammarata
Gabriel Goh
Shan Carter
Ludwig Schubert
Michael Petrov
Chris Olah

### Contents

 an experimental format collecting invited short articles and critical

commentary delving into the inner workings of neural networks.

 vision model we've explored in detail contains neurons which detect

curves. Curve detectors in vision models have been hinted at in the

 curve in our earlier overview of early vision, but wanted to examine

them in more depth. This article is the first part of a three article

deep dive into curve detectors: their behavior, how they're built from

earlier neurons, and their prevalence across models.

We're doing

this because we believe that the interpretability community disagrees on

 several crucial questions. In particular, are neural network

representations composed of meaningful features — that is, features

tracking articulable properties of images? On the one hand, there are a

number of papers reporting on seemingly meaningful features, such as eye

 detectors, head detectors, car detectors, and so forth .

 At the same time, there's a significant amount of skepticism, only

partially reflected in the literature. One concern is that features

which seem superficially to be meaningful may in fact not be what they

 rather than the kind of meaningful features described earlier. Finally,

 even if some meaningful features exist, it's possible they don't play

an especially important role in the network.

 Some reconcile these results by concluding that if one observes, for

example, what appears to be a dog head detector, it is actually a

detector for special textures correlated with dog heads.

This

disagreement really matters. If every neuron was meaningful, and their

connections formed meaningful circuits, we believe it would open a path

to completely reverse engineering and interpreting neural networks. Of

course, we know not every neuron is meaningful, As

 discussed in Zoom In, the main issue we see is what we call

polysemantic neurons which respond to multiple different features,

seemingly as a way to compress many features into a smaller number of

neurons. We're hopeful this can be worked around. but we

think it's close enough for this path to be tractable. However, our

position is definitely not the consensus view. Moreover, it seems too

good to be true, and rings of the similar failed promises in other

fieldsFor example, genetics seems to have

been optimistic in the past that genes had individual functions and that

 the human genome project would allow us to “mine miracles," a position

We

 believe that curve detectors are a good vehicle for making progress on

this disagreement. Curve detectors seem like a modest step from

edge-detecting Gabor filters, which the community widely agrees often

form in the first convolutional layer. Furthermore, artificial curves

are simple to generate, opening up lots of possibilities for rigorous

investigation. And the fact that they're only a couple convolutional

layers deep means we can follow every string of neurons back to the

input. At the same time, the underlying algorithm the model has

implemented for curve detection is quite sophisticated. If this paper

persuades skeptics that at least curve detectors exist, that seems like a

 substantial step forward. Similarly, if it surfaces a more precise

point of disagreement, that would also advance the dialogue.

[A Simplified Story of Curve Neurons

-----------------------------------](#a-simplified-story-of-curve-neurons)Before

 running detailed experiments, let's look at a high level and slightly

simplified story of how the curve 10 neurons in 3b work.

 curve detector implements a variant of the same algorithm: it responds

to a wide variety of curves, preferring curves of a particular

orientation and gradually firing less as the orientation changes. Curve

neurons are invariant to cosmetic properties such as brightness,

texture, and color.

 are normalized by the neuron's maximum activation. We'll examine

 across ImageNet, and usually activating weakly when they do fire. When

they activate strongly, it's in response to curves with similar

orientation and curvature to their feature visualization.

---

It’s

 worth stepping back and reflecting on how surprising the existence of

seemingly meaningful features like curve detectors is. There’s no

explicit incentive for the network to form meaningful neurons. It’s not

like we optimized these neurons to be curve detectors! Rather,

InceptionV1 is trained to classify images into categories many levels of

 abstraction removed from curves and somehow curve detectors fell out of

 gradient descent.

Moreover, detecting curves across a wide

variety of natural images is a difficult and arguably unsolved problem

in classical computer visionThis is our

sense from trying to implement programmatic curve detection to compare

them to curve neurons. We found that practitioners generally had to

choose between several algorithms, each with significant trade-offs such

 as robustness to different kinds of visual “noise” (for instance,

texture), even in images much less complex than the natural images in

 in general, is a very challenging one and, except for toy examples,

there are no good solutions.” Additionally, many classical curve

detection algorithms are too slow to run in real-time, or require often

intractable amounts of memory.. InceptionV1 seems to learn a

 flexible and general solution to this problem, implemented using five

convolutional layers. We’ll see in the next article that the algorithm

used is straightforward and understandable, and we’ve since

reimplemented it by hand.

What exactly are we claiming when we say

 these neurons detect curves? We think part of the reason there is

sometimes disagreement about whether neurons detect particular stimuli

is that there are a variety of claims one may be making. It’s pretty

easy to show that, empirically, when a curve detector fires strongly the

 stimulus is reliably a curve. But there are several other claims which

might be more contentious:

* **Causality**

Curve detectors genuinely detect a curve feature, rather than another

stimulus correlated with curves. We believe our feature visualization

and visualizing attribution experiments establish a causal link, since

“running it in reverse” produces a curve.

* **Generality:**

 Curve detectors respond to a wide variety of curve stimuli. They

tolerate a wide range of radii and are largely invariant to cosmetic

attributes like color, brightness, and texture. We believe that our

experiments explicitly testing these invariances with synthetic stimuli

are the most compelling evidence of this.

* **Purity:**

 Curve detectors are not polysemantic and they have no meaningful

secondary function. Images that cause curve detectors to activate

weakly, such as edges or angles, are a natural extension of the

algorithm that InceptionV1 uses to implement curve detection. We believe

 our experiments classifying dataset examples at different activation

magnitudes and visualizing their attributions show that any secondary

function would need to be rare. In the next article, exploring the

mechanics of the algorithm implementing curve detectors, we’ll provide

further evidence for this claim.

* **Family:** Curve neurons collectively span all orientations of curves.

[Feature Visualization

 uses optimization to find the input to a neural network that maximizes a

 given objective. The objective we often use is to make the neuron fire

as strongly as possible, but we'll use other objectives throughout in

this article. One reason feature visualization is powerful is that it

tells us about causality. Since the input starts with random noise and

optimizes pixels rather than a generative prior, we can be confident

that any property in the resulting image contributed to the objective.

 feature visualizations is a bit of a skill, and these images might feel

 disorienting if you haven't spent much time with them before. The most

important thing to take away is the curve shape. You may also notice

that there are bright, opposite hue colors on each side of the curve:

this reflects a preference to see a change in color at the boundary of

the curve. Finally, if you look carefully, you will notice small lines

perpendicular to the boundary of the curve. We call this weak preference

 for small perpendicular lines “combing" and will discuss it later.

Every

 time we use feature visualization to make curve neurons fire as

strongly as possible we get images of curves, even when we explicitly

Feature

 visualization finds images that maximally cause a neuron to fire, but

are these superstimuli representative of the neuron's behavior? When we

see a feature visualization, we often imagine that the neuron fires

strongly for stimuli qualitatively similar to it, and gradually becomes

weaker as the stimuli exhibit those visual features less. But one could

imagine cases where the neuron's behavior is completely different in the

 non-extreme activations, or cases where it does fire weakly for messy

versions of the extreme stimulus, but also has a secondary class of

stimulus to which it responds weakly.

If we want to understand how

 a neuron behaves in practice, there's no substitute to simply looking

at how it actually responds to images from the dataset.

[Dataset Analysis

 because some experiments required non-trivial manual labor. However,

the core ideas in this section will apply to all curve detectors in 3b.

 fire? When it fires, how often does it fire strongly? And when it

doesn't fire, is it often strongly inhibited, or just on the verge of

firing? We can answer these questions by visualizing the distribution of

 activations across the dataset.

When studying ReLU networks, we

find it helpful to look at the distribution of pre-activation values.

Since ReLU just truncates the left hand side, it's easy to reason about

the post-activation values, but it also shows us how close the neuron

 has a pre-activation mean of about -200. It fires in just 11% of cases

across the dataset, since negative values will be set to zero by the

ReLU activation function.

 observation that neural network activations generally follow an

exponential distribution was first made to us by Brice Ménard, who

observed it to be the case over all but the first layer of several

networks. This is mildly surprising both because of how perfectly they

seem to follow an exponential distribution, and also because one often

 looking at pre-ReLU values for 3b:379 activations, we see that both

positive and negative values follow an exponential distribution. Since

all negative values will be lifted to zero by the ReLU, 3b:379

activations are sparse, with only 11% of stimuli across the dataset

causing activations.To understand different parts

of this distribution qualitatively we can render a quilt of images by

 to activate different amounts. The quilt shows a pattern. Images that

cause the strongest activations have curves that are similar to the

neuron's feature visualization. Images that cause weakly positive

activations are imperfect curves, either too flat, off-orientation, or

with some other defect. Images that cause pre-ReLU activations near zero

 tend to be straight lines or images with no arcs, although some images

are of curves about 45 degrees off orientation. Finally, the images that

 cause the strongest negative activations have curves with an

orientation more than 45 degrees away from the neuron's ideal curve.

 of images reveal patterns across a wide range of activations, but they

can be misleading. Since a neuron's activation to a receptive-field

sized crop of an image is just a single number, we can't be sure which

part of the image caused it. As a result, we could be fooled by spurious

[Visualizing Attribution

 These methods attempt to describe which pixels or earlier neurons are

responsible for causing neurons to fire. In the general case of complex

non-linear functions, there is a lot of disagreement over which

attribution methods are principled and whether they are reliable .

 But in the linear-case, attribution is generally agreed upon, with most

Since

 a neuron’s pre-activation function and bias value is a linear function

of neurons in the layer before it, we can use this generally agreed upon

 attribution method. In particular, curve detectors in 3b’s

pre-activation value is a linear function of 3a. The attribution tensor

describing how all neurons in the previous layer influenced a given

neuron is the activations pointwise multiplied by the weights.

We

normally use feature visualization to create a superstimulus that

activates a single neuron, but we can also use it to activate linear

combinations of neurons. By applying feature visualization to the

attribution tensor, we are creating the stimulus that maximally

 to fire. Additionally, we will use the absolute value of the

attribution tensor, which shows us features that caused the neuron to

fire and also features that inhibited it. This can be useful for seeing

curve-related visual properties that influenced our cure neuron, even if

 that influence was to make it fire less.

 is the activations of the previous hidden layer. In practice, we find

it helpful to parameterize these attribution visualizations to be

grayscale and transparent, making the visualization easier to read for

non-experts . Example code can be found in the notebook.

 the above experiment visualizes every neuron in 3a, attribution is a

powerful and flexible tool that could be used to apply to studies of

circuits in a variety of ways. For instance, we could visualize how an

 before 3b, visualizing the image's activation vector and attribution

vector to curve neurons at each family along the way. Each activation

vector would show what a family saw in the image, and each attribution

In

 the next section we'll look at a less sophisticated technique for

extracting information from dataset images: blindfolding yourself from

seeing neuron activations and classifying images by hand.

[Human Comparison

----------------](#human-comparison)Nick

 Cammarata, an author of this paper, manually labelled over 800 images

into four groups: curve, imperfect curve, unrelated image, or opposing

 While labeling, Nick could only see the image's pixels, not additional

information such as the neuron's activations or attribution

visualizations. He used the following rubric in labeling:

* **Curves**:

 The Image has a curve with a similar orientation to the neuron's

feature visualization. The curve goes across most of the width of the

image.

* **Imperfect Curve**: The image has a

curve that is similar to the neuron's feature visualization, but has at

least one significant defect. Perhaps it is too flat, has an angle

interrupting the arc, or the orientation is slightly off.

* **Unrelated**: The image doesn't have a curve.

 during labeling Nick felt it was often difficult to place samples into

groups, as many images seemed to fall within the boundaries of the

rubric. We were surprised when we saw that activations clearly separate

into different levels of activation..

 probability of each group by 3b:379 activation across our hand-labelled

 dataset of around 850 images. We see that the different human labels

separate into different ranges of activations.Still,

 there are many images that cause the neuron to activate but aren't

classified as curves or imperfect curves. When we visualize attribution

 examples that activate 3b:379 but were labelled "unrelated" by humans

often contain subtle curves that are revealed by visualizing the image's

 that occurs when looking at one kind of stimulus for a long time. As a

result, he found it hard to tell whether subtle curves were simply

perceptual illusions. By visualizing attribution, we can reveal the

curves that the neuron sees in the image, showing us curves that our

**How important are different points on the activation spectrum?**

 seems to be highly selective for curve stimuli when it fires strongly,

this is only a tiny fraction of cases where it fires. Most of the time,

it doesn't fire at all, and when it does it's usually very weakly.

To

 see this, we can look at the probability density over activation

magnitudes from all ImageNet examples, split into the same

per-activation-magnitude (x-axis) ratio of classes as our hand labelled

dataset.

 our hand-labelled dataset uniformly samples from activations, images of

 curves are rare within the dataset and 3b:379 activations follow an

exponential distribution. In this plot we show 3b:379 activations split

into the conditional probabilities of different groups at a given

activation level within our hand-labelled dataset.From

 this perspective, we can't even see the cases where our neuron fires

strongly! Probability density exponentially decays as we move right, and

 so these activations are rare. To some extent, this is what we should

expect if these neurons really detect curves, since clear-cut curves

rarely occur in images.

 only weakly fire or didn't fire, this graph seems to show that the

majority of stimuli classified as curves also fall in these cases, as a

result of neurons firing strongly being many orders of magnitude rarer.

This seems to be at least partly due to labeling error and the rarity of

 curves (see discussion later). But it makes things a bit hard to reason

 about. This is why we haven't provided a precision-recall curve: recall

 would be dominated by the cases where the neuron didn't fire strongly

and be dominated by potential labeling error as a result.

It's not

 clear that probability density is really the right way to think about

the behavior of a neuron. The vast majority of cases are cases where the

 neuron didn't fire: are those actually important to think about? And if

 a neuron frequently barely fires, how important is that for

understanding the role of a neuron in the network?

 This measure can be thought of as giving an approximation at how much

that activation value influences the output of the neuron, and by

extension network behavior. There's still reason to think that high

activation cases may be disproportionately important beyond this (for

example, in max pooling only the highest value matters), but

contribution to expected value seems like a reasonable estimate.If

 one wanted to push further on exploring the importance of different

parts of the activation spectrum, they might take some notion of

attribution (methods for estimating the influence of one neuron on later

 neurons in a particular case) and estimate the contribution to the

expected value of the attribution to the logit. A simple version of this

 to the expected value of different activations, which shows how much

each activation value influences the output of a neuron. Since curves

are rare within the dataset, weak neuron activations contribute most to

 was really a curve detector in a meaningful sense. Even if it's highly

selective when it fires strongly, how can that be what matters when it

isn't even visible on a probability density plot? Contribution to

expected value shows us that even by a conservative measure, curves and

imperfect curves form 55%. This seems consistent with the hypothesis

that it really is a curve detector, and the other stimuli causing it to

fire are labeling errors or cases where noisy images cause the neuron to

 misfire.

 activations seem to correspond roughly to a human labelled judgement of

 whether images contain curves. Additionally, visualizing the

attribution vector of these images tells us that the reason these images

 fire is because of the curves in the images, and we're not being fooled

 by spurious correlations. But these experiments are not enough to

defend the claim that curve neurons detect images of curves. Since

images of curves appear infrequently in the dataset, using it to

systematically study curve images is difficult. Our next few experiments

 will focus on this directly, studying how curve neurons respond to the

space of reasonable curve images.

[Joint Tuning Curves

-------------------](#joint-tuning-curves)Our

 first two experiments suggest that each curve detector responds to

curves at a different orientation. Our next experiment will help verify

that they really do detect rotated versions of the same feature, and

characterize how sensitive each unit is to changes in orientation.

We do this by creating a **joint tuning curve**In

 neuroscience, tuning curves — charts of neural response to a continuous

 stimulus parameter — came to prominence in the early days of vision

research. Observation of receptive fields and orientation-specific

responses in neurons gave rise to some of the earliest theories about

how low-level visual features might combine to create higher-level

representations. Since then they have been a mainstay technique in the

 collect dataset examples that maximally activate neuron. We rotate them

 by increments of 1 degree from 0 to 360 degrees and record activations.The

 activations are shifted so that the points where each neuron responds

are aligned. The curves are then averaged to create a typical response

 tuning curves are useful for measuring neuron activations across

perturbations in natural images, we're limited by the kinds of

perturbations we can do on these images. In our next experiment we'll

get access to a larger range of perturbations by rendering artificial

stimuli from scratch.

[Synthetic Curves

----------------](#synthetic-curves)While

 the dataset gives us almost every imaginable curve, they don't come

labelled with data such as orientation or radius, making it hard to

answer questions that require systematically measuring responses to

visual properties. How sensitive are curve detectors to curvature? What

orientations do they respond to? Does it matter what colors are

involved? One way to get more insight into these questions is to draw

our own curves. Using synthetic stimuli like this is a common method in

visual neuroscience, and we've found it to also be very helpful in the

study of artificial neural networks. The experiments in this section are

 specifically inspired by similar experiments probing for curve

detecting biological neurons .

Since

 dataset examples suggest curve detectors are most sensitive to

orientation and curvature, we'll use them as parameters in our curve

renderer. We can use this to measure how changes in each property causes

 to fire. We find it helpful to present this as a heatmap, in order to

get a higher resolution perspective on what causes the neuron to fire.

 find that simple drawings can be extraordinarily exciting. The curve

images that cause the strongest excitations — up to 24 standard

deviations above the average dataset activation! — have similar

orientation and curvature to the neuron's feature visualization.

The

 triangular geometry shows that curve detectors respond to a wider range

 of orientations in curves with higher curvature. This is because curves

 with more curvature contain more orientations. Consider that a line

contains no curve orientations and a circle contains every curve

orientation. Since the synthetic images closer to the top are closer to

lines, their activations are more narrow.

The wisps show that tiny

 changes in orientation or curvature can cause dramatic changes in

activations, which indicate that curve detectors are fragile and

non-robust. Sadly, this is a more general problem across neuron

Varying

 curves along just two variables reveals barely-perceptible

perturbations that sway activations several standard deviations. This

suggests that the higher dimensional pixel-space contains more

pernicious exploits. We're excited about the research direction of

carefully studying neuron-specific adversarial attacks, particularly in

early vision. One benefit of studying early vision families is that it's

 tractible to follow the whole circuit back to the input, and this could

 be made simpler by extracting the important parts of a circuit and

studying it in isolation. Perhaps this simplified environment could give

 us clues into how to make neurons more robust or even protect whole

models against adversarial attacks.

In addition to testing

orientation and curvature, we can also test other variants like whether

the curve shapes are filled, or if they have color. Dataset analyses

hints that curve detectors are invariant to cosmetic properties like

lighting, and color, and we can confirm this with synthetic stimuli.

----------------](#synthetic-angles)Both

 our synthetic curve experiments and dataset analysis show that although

 curves are sensitive to orientation, they have a wide tolerance for the

 radius of curves. At the extreme, curve neurons partially respond to

edges in a narrow band of orientations, which can be seen as a curve

with infinite radius. This may cause us to think curve neurons actually

respond to lots of shapes with the right orientation, rather than curves

 specifically. While we cannot systematically render all possible

shapes, we think angles are a good test case for studying this

hypothesis.

In the following experiment we vary synthetic angles

similarly to our synthetic curves, with radius on the y axis and

orientation across the x axis.

 activations form two distinct lines, with the strongest activations

where they touch. Each line is where one of the two lines in the angle

aligns with the tangent of the curve. The two lines touch where the

angle most similar to a curve with an orientation that matches the

neuron's feature visualization. The weaker activations on the right side

 of the activations have the same cause, but with the inhibitory half of

 the angle stimulus facing outwards instead of inwards.

![](Curve%20Detectors_files/figure-f4e54f9da80ab5da3ac54a096ca11be4.svg)The

 first stimuli we looked at were synthetic curves and the second stimuli

 was synthetic angles. In the next examples we show a series of stimuli

that transition from angles to curves. Each column's strongest

activation is stronger than the column before it since rounder stimuli

are closer to curves, causing curve neurons to fire more strongly.

Additionally, as each stimulus becomes rounder, their “triangles of

activation" become increasingly filled as the two lines from the

original angle stimuli transition into a smooth arc.

 transition from angles on the left to curves on the right, making the

stimuli rounder at each step. Each step we see the maximum activation

for each neuron increase, and the activation "triangle" fill in as the

 interface is useful for seeing how different curve neurons respond to

changes in multiple stimuli properties, but it's bulky. In the next

section we'll be exploring curve families across different layers, and

it will be helpful to have a more compact way to view activations of a

curve neuron family. For this, we'll introduce a *radial tuning curve*.

[Radial Tuning Curve

---------------------------------](#the-curve-families-of-inceptionv1)So

 far we've been looking at a set of curve neurons in 3b. But InceptionV1

 actually contains curve neurons in four contiguous layers, with 3b

being the third of these layers.

 which we sometimes shorten to "2", is the third convolutional layer in

InceptionV1. It contains two types of curve detectors: concentric curves

 and combed edges.

Concentric curves are small curve detectors

that have a preference for multiple curves at the same orientation with

increasing radii. We believe this feature has a role in the development

of curve detectors in 3a and 3b that are tolerant of a wide range of

radii.

![](Curve%20Detectors_files/imgsappnick-personal1XjsJAYh6F.png)Combed

 edges detect several lines protruding perpendicularly from a larger

line. These protruding lines also detect curves, making them a type of

curve detector. These neurons are used to construct later curve

detectors and play a part in the [combing effect](#combing-effect).

Looking

 at conv2d2 activations we see that curves respond to one contiguous

range like the ones in 3b, but also weakly activate to a range on the

opposite side, 180 degrees away. We call this secondary range **echoes**.

 3a non-concentric curve detectors have formed. In many ways they

resemble the curve detectors in 3b, and in the next article we'll see

how they're used to build 3b curves. One difference is that the 3a

curves have echoes.

You

 may notice that there are two large angular gaps at the top of the

radial tuning curve for 3b, and smaller ones at the bottom. Why is that?

 4a the network constructs many complex shapes such as spirals and

boundary detectors, and it is also the first layer to construct 3d

geometry. It has several curve detectors, but we believe they are better

 thought of as corresponding to specific worldly objects rather than

 appears to be a upwards facing curve detector with confusing secondary

behavior at two other angles. But dataset examples reveal its secret:

it's detecting the tops of cups and pans viewed from an angle. In this

sense, it is better viewed as a tilted 3d circle detector.

 is a good example of how neural network interpretability can be

subjective. We usually think of abstract concepts like curves and

worldly objects like coffee cups as belonging as different kinds of

things — and for most of the network they are separate. But there's a

transition period where we have to make a judgement call, and 4a is that

 transition.

---------------------------](#repurposing-curve-detectors)We

 started studying curve neurons to better understand neural networks,

not because we were intrinsically interested in curves. But during our

investigation we became aware that curve detection is important for

fields like aerial imaging, self-driving cars, and medical research, and

 there's a breadth of literature from classical computer vision on curve

 detection in each domain. We've prototyped a technique that leverages

the curve neuron family to do a couple different curve related computer

vision tasks.

One task is *curve extraction*

 , the task of highlighting the pixels of the image that are part of

curves. Visualizing attribution to curve neurons, as we've been doing in

 this article, can be seen as a form of curve extraction. Here we

compare it to the commonly used Canny edge detection algorithm on an

x-ray of blood vessels known as an angiogram, taken from  , Figure 2.1.

 attribution visualization clearly separates and illuminates the lines

and curves, and displays less visual artifacts. However, it displays a

strong [combing effect](#combing-effect) — unwanted

perpendicular lines emanating from the edge being traced. We're unsure

how harmful these lines are in practice for this application, but we

think it's possible to remove them by editing the circuits of curve

neurons.

We don't mean to suggest we've created a competitive

curve tracing algorithm. We haven't done a detailed comparison to state

of the art curve detection algorithms, and believe it's likely that

classical algorithms tuned for precisely this goal outperform our

approach. Instead, our goal here is to explore how leveraging internal

neural network representations opens a vast space of visual operations,

of which curve extraction is just one point.

[### Spline Parameterization](#spline-parameterization)We

 can access more parts of this space by changing what we optimize. So

far we've been optimizing pixels, but we can also create a

Occlusion![](Curve%20Detectors_files/source_006.png)Our

 splines can trace curves even if they have significant occlusion.

Furthermore, we can use attribution to construct complex occlusion

rules. For instance, we can strongly penalize our spline for overlapping

 with a particular object or texture, disincentivizing the spline from

 curve neurons are robust to a wide variety of natural visual features,

 seemingly unrelated visual operation is image segmentation. This can be

 done in an unsupervised way using non-negative matrix factorization

(NMF) . We can

visualize attribution to each of these factors with our spline

parameterization to trace the curves of different objects in the image.

 of factoring the activations of a single image, we can jointly

factorize lots of butterflies to find the neurons in the network that

respond to butterflies in general. One big difference between factoring

activations and normal image segmentation is that we get groups of

neurons rather than pixels. These neuron groups can be applied to find

butterflies in images in general, and by composing this with

differentiable spline parameterization we get a single optimization we

can apply to any image that automatically finds butterflies and gives us

 equations to splines that fit them.

 this above example we manipulated butterflies and curves without having

 to worry about the details of either. We delegated the intricacies of

recognizing butterflies of many species and orientations to the neurons,

 letting us work with the abstract concept of butterflies.

We

think this is one exciting way to fuse classical computer vision with

deep learning. There is plenty of low hanging fruit in extending the

technique shown above, as our spline parameterization is just an early

prototype and our optimizations are using a neural network that's half a

 decade old. However, we're more excited by investigations of how users

can explore the space between tasks than improvements in any particular

task. Once a task is set

in stone, training a neural network for exactly that job will likely

give the best results. But real world tasks are rarely specified with

precision, and the harder challenge is to explore the space of tasks to

find which to commit to.

For instance, a more developed version of

 our algorithm that automatically finds the splines of butterflies in an

 image could be used as a basis for turning video footage of butterflies

 But an animator may wish to add texture neurons and change to a soft

brush parameterization to add a rotoscoping style to their animation.

Since they have full access to every neuron in the render, they could

manipulate attribution to fur neuron families and specific dog breeds,

changing how fur is rendered on specific species of dogs across the

entire movie. Since none of these algorithms require retraining a neural

 network or any training data, in-principle an animator could explore

this space of algorithms in real time, which is important because tight

feedback loops can be crucial in unlocking creative potential.

[The Combing Phenomenon

----------------------](#the-combing-phenomenon)One

 curious aspect of curve detectors is that they seem to be excited by

small lines perpendicular to the curve, both inwards and outwards. You

can see this most easily by inspecting feature visualizations. We call

this phenomenon "combing."

Combing seems to occur across curve

detectors from many models, including models trained on Places365

instead of ImageNet. In fact, there's some weak evidence it occurs in

biological neural networks as well: a team that ran a process similar to

 feature visualization on a biological neuron in a Macaque monkey's V4

region of the visual cortex found a circular shape with outwardly

protruding lines to be one of the highest activating stimuli.

One

 hypothesis is that many important curves in the modern world have

perpendicular lines, such as the spokes of a wheel or the markings on

the rim of a clock.

 related hypothesis is that combing might allow curve detectors to be

used for fur detection in some contexts. Another hypothesis is that a

curve has higher "contrast" with perpendicular lines running towards.

Recall that in the dataset examples, the strongest negative pre-ReLU

activations were curves at opposite orientations. If a curve detector

wants to see a strong change in orientation between the curve and the

space around it, it may consider perpendicular lines to be more contrast

 than a solid color.

Finally, we think it's possible that combing

is really just a convenient way to implement curve detectors — a side

effect of a shortcut in circuit construction rather than an

intrinsically useful feature. In conv2d1, edge detectors are inhibited

by perpendicular lines in conv2d0. One of the things a line or curve

detector needs to do is check that the image is not just a single

repeating texture, but that it has a strong line surrounded by contrast.

 It seems to do this by weakly inhibiting parallel lines alongside the

tangent. Being excited by a perpendicular line may be an easy way to

implement a "inhibit an excitatory neuron" pattern which allows for

capped inhibition, without creating dedicated neurons at the previous

layer.

Combing is not unique to curves. We also observe it in

lines, and basically any shape feature like curves that is derivative of

 lines. A lot more work could be done exploring the combing phenomenon.

Why does combing form? Does it persist in adversarially robust models?

Is it an example of what Ilyas et al  call a "non-robust feature"?

[Conclusion

----------](#conclusion)Compared

 to fields like neuroscience, artificial neural networks make careful

investigation easy. We can read and write to every weight in the neural

network, use gradients to optimize stimuli, and analyze billions of

realistic activations across a dataset. Composing these tools lets us

run a wide range of experiments that show us different perspectives on a

 neuron. If every perspective shows the same story, it's unlikely we're

missing something big.

Given this, it may seem odd to invest so

much energy into just a handful of neurons. We agree. We first estimated

 it would take a week to understand the curve family. Instead, we spent

months exploring the fractal of beauty and structure we found.

Many

 paths led to new techniques for studying neurons in general, like

synthetic stimuli or using circuit editing to ablate neurons behavior.

Others are only relevant for some families, such as the equivariance

motif or our hand-trained “artificial artificial neural network" that

reimplements curve detectors. A couple were curve-specific, like

exploring curve detectors as a type of curve analysis algorithms.

If

 our broader goal is fully reverse-engineer neural networks it may seem

concerning that studying just one family took so much effort. However,

from our experience studying neuron families at a variety of depths,

 shows you feature visualizations, dataset examples, and soon weights in

 just a few seconds. Since feature visualization shows strong evidence

of causal behavior and dataset examples show what neurons respond to in

practice, these are collectively strong evidence of what a neuron does.

In fact, we understood the basics of curves at our first glance at them.

While it's usually possible to understand the main function of a

 neuron family at a glance, researchers engaging in closer inquiry of

neuron families will be rewarded with deeper beauty.

When we started, we were nervous that 10 neurons was too narrow a topic

for a paper, but now we realize a complete investigation would take a

book.

 an experimental format collecting invited short articles and critical

commentary delving into the inner workings of neural networks.

Visualizing and understanding convolutional networks

Complex pattern selectivity in macaque primary visual cortex revealed by large-scale two-photon imaging

Distributed representations of words and phrases and their compositionality

Visualizing and understanding recurrent networks

Learning to generate reviews and discovering sentiment

Object detectors emerge in deep scene cnns

Feature Visualization

Network Dissection: Quantifying Interpretability of Deep Visual Representations

On Interpretability and Feature Representations: An Analysis of the Sentiment Neuron

Measuring the tendency of CNNs to Learn Surface Statistical Regularities

ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness

Approximating cnns with bag-of-local-features models works surprisingly well on imagenet

On the importance of single directions for generalization

Visualizing higher-layer features of a deep network

Deep inside convolutional networks: Visualising image classification models and saliency maps

Deep neural networks are easily fooled: High confidence predictions for unrecognizable images

Inceptionism: Going deeper into neural networks

Plug & play generative networks: Conditional iterative generation of images in latent space

Axiomatic attribution for deep networks

Grad-cam: Why did you say that? visual explanations from deep networks via gradient-based localization

PatternNet and PatternLRP--Improving the interpretability of neural networks

The (Un)reliability of saliency methods

Sanity checks for saliency maps

Differentiable Image Parameterizations

An overview of early vision in inceptionv1

Responses to contour features in macaque area V4

Discrete neural clusters encode orientation, curvature and corners in macaque V4

Curve tracing and curve detection in images

Synthetic Abstractions

Neural painters: A learned differentiable constraint for generating brushstroke paintings

The Building Blocks of Interpretability

Using Artificial Intelligence to Augment Human Intelligence


Exploring Bayesian Optimization

 Many modern machine learning algorithms have a large number of

hyperparameters. To effectively use these algorithms, we need to pick

good hyperparameter values.

 In this article, we talk about Bayesian Optimization, a suite of

techniques often used to tune hyperparameters. More generally, Bayesian

Optimization can be used to optimize any black-box function.

Mining Gold!

============

 For now, we assume that the gold is distributed about a line. We

want to find the location along this line with the maximum gold while

only drilling a few times (as drilling is expensive).

![](Exploring%20Bayesian%20Optimization_files/GT.svg)

 Initially, we have no idea about the gold distribution. We can

learn the gold distribution by drilling at different locations. However,

 We now discuss two common objectives for the gold mining problem.

* **Problem 1: Best Estimate of Gold Distribution (Active Learning)**

 **Active Learning**.

* **Problem 2: Location of Maximum Gold (Bayesian Optimization)**

 In this problem, we want to find the location of the maximum

gold content. We, again, can not drill at every location. Instead, we

 **Bayesian Optimization**.

 We will soon see how these two problems are related, but not the same.

Active Learning

---------------

 For many machine learning problems, unlabeled data is readily

available. However, labeling (or querying) is often expensive. As an

example, for a speech-to-text task, the annotation requires expert(s) to

 label words and sentences manually. Similarly, in our gold mining

problem, drilling (akin to labeling) is expensive.

 Active learning minimizes labeling costs while maximizing modeling

 accuracy. While there are various methods in active learning

literature, we look at **uncertainty reduction**. This

method proposes labeling the point whose model uncertainty is the

highest. Often, the variance acts as a measure of uncertainty.

 for the values our function takes elsewhere. This surrogate should be

flexible enough to model the true function. Using a Gaussian Process

(GP) is a common choice, both because of its flexibility and its ability

 to give us uncertainty estimates

 Gaussian Process supports setting of priors by using specific

kernels and mean functions. One might want to look at this excellent

Distill article on Gaussian Processes to learn more.

 .

 .

![](Exploring%20Bayesian%20Optimization_files/prior2posterior.png)

 Each new data point updates our surrogate model, moving it closer

to the ground truth. The black line and the grey shaded region indicate

 However, we want to minimize the number of evaluations. Thus, we

should choose the next query point “smartly” using active learning.

Although there are many ways to pick smart points, we will be picking

the most uncertain one.

 This gives us the following procedure for Active Learning:

2. Train on the new training set

3. Go to #1 till convergence or budget elapsed

![](Exploring%20Bayesian%20Optimization_files/0_004.png)

The visualization shows that one can estimate the true

distribution in a few iterations. Furthermore, the most uncertain

positions are often the farthest points from the current evaluation

Bayesian Optimization

---------------------

 In the previous section, we picked points in order to determine an

 accurate model of the gold content. But what if our goal is simply to

find the location of maximum gold content? Of course, we could do active

 learning to estimate the true function accurately and then find its

maximum. But that seems pretty wasteful — why should we use evaluations

improving our estimates of regions where the function expects low gold

content when we only care about the maximum?

 This is the core question in Bayesian Optimization: “Based on what

 we know so far, which point should we evaluate next?” Remember that

evaluating each point is expensive, so we want to pick carefully! In the

 active learning case, we picked the most uncertain point, exploring the

 function. But in Bayesian Optimization, we need to balance exploring

uncertain regions, which might unexpectedly have high gold content,

against focusing on regions we already know have higher gold content (a

kind of exploitation).

 We make this decision with something called an acquisition

function. Acquisition functions are heuristics for how desirable it is

 This brings us to how Bayesian Optimization works. At every step,

we determine what the best point to evaluate next is according to the

acquisition function by optimizing it. We then update our model and

repeat this process to determine the next point to evaluate.

 You may be wondering what’s “Bayesian” about Bayesian Optimization

 if we’re just optimizing these acquisition functions. Well, at every

step we maintain a model describing our estimates and uncertainty at

### Formalizing Bayesian Optimization

 We present the general constraints in Bayesian Optimization and

 * [Youtube talk](https://www.youtube.com/watch?v=c4KKvyWW_Xk),

.

| General Constraints | Constraints in Gold Mining example |

| --- | --- |

| fff’s feasible set AAA is simple,

| fff is continuous but lacks special structure,

| fff is derivative-free:

| fff is expensive to evaluate:

 the number of times we can evaluate it

 is severely limited. | Drilling is costly. |

it is easy to incorporate normally distributed noise for GP regression). |

 To solve this problem, we will follow the following algorithm:

### Acquisition Functions

#### Probability of Improvement (PI)

xt+1=argmax(αPI(x))=argmax(P(f(x)≥(f(x+)+ϵ)))

 x\_{t+1} = argmax(\alpha\_{PI}(x)) = argmax(P(f(x) \geq (f(x^+) +\epsilon)))

 xt+1=argmax(αPI(x))=argmax(P(f(x)≥(f(x+)+ϵ)))

xt+1=argmax(αPI(x))=argmax(P(f(x)≥(f(x+)+ϵ)))

 \begin{aligned}

 x\_{t+1} & = argmax(\alpha\_{PI}(x))\\

 & = argmax(P(f(x) \geq (f(x^+) +\epsilon)))

 \end{aligned}

 xt+1=argmax(αPI(x))=argmax(P(f(x)≥(f(x+)+ϵ)))

 where,

* P(⋅)P(\cdot)P(⋅) indicates probability

* ϵ\epsilonϵ is a small positive number

 Looking closely, we are just finding the upper-tail probability

(or the CDF) of the surrogate posterior. Moreover, if we are using a GP

as a surrogate the expression above converts to,

 where,

* Φ(⋅)\Phi(\cdot)Φ(⋅) indicates the CDF

 The violet region shows the probability density at each point. The grey

 regions show the probability density below the current max. The “area”

of the violet region at each point represents the “probability of

improvement over current maximum”. The next point to evaluate via the PI

 criteria (shown in dashed blue line) is x=6x = 6x=6.

![](Exploring%20Bayesian%20Optimization_files/density_pi.png)

##### Intuition behind ϵ\epsilonϵ in PI

 PI uses ϵ\epsilonϵ to strike a balance between exploration and exploitation.

![](Exploring%20Bayesian%20Optimization_files/0_009.png)

 This observation also shows that we do not need to construct an

accurate estimate of the black-box function to find its maximum.

![](Exploring%20Bayesian%20Optimization_files/0_013.png)

 What happens if we increase ϵ\epsilonϵ a bit more?

![](Exploring%20Bayesian%20Optimization_files/0_012.png)

 We see that we made things worse! Our model now uses ϵ=3\epsilon = 3ϵ=3,

 and we are unable to exploit when we land near the global maximum.

Moreover, with high exploration, the setting becomes similar to active

learning.

#### Expected Improvement (EI)

 The idea is fairly simple — choose the next query point as the one

xt+1=argminxE(∣∣ht+1(x)−f(x⋆)∣∣ ∣ Dt)

 xt+1=argminxE(∣∣ht+1(x)−f(x⋆)∣∣ ∣ Dt)

 In essence, we are trying to select the point that minimizes the

distance to the objective evaluated at the maximum. Unfortunately, we do

 not know the ground truth function, fff. Mockus proposed

 the following acquisition function to overcome the issue.

xt+1=argmaxxE(max{0, ht+1(x)−f(x+)} ∣ Dt)

 xt+1=argmaxxE(max{0, ht+1(x)−f(x+)} ∣ Dt)

xt+1= argmaxxE(max{0, ht+1(x)−f(x+)} ∣ Dt)

 \begin{aligned}

 x\_{t+1} = \ & argmax\_x \mathbb{E} \\

 & \left( {max} \{ 0, \ h\_{t+1}(x) - f(x^+) \} \ | \ \mathcal{D}\_t \right)

 \end{aligned}

 xt+1= argmaxxE(max{0, ht+1(x)−f(x+)} ∣ Dt)

EI(x)={(μt(x)−f(x+)−ϵ)Φ(Z)+σt(x)ϕ(Z),if σt(x)>00,if σt(x)=0

 EI(x)=

 \begin{cases}

 0, & \text{if}\ \sigma\_t(x) = 0

 \end{cases}

 EI(x)={(μt(x)−f(x+)−ϵ)Φ(Z)+σt(x)ϕ(Z),0,if σt(x)>0if σt(x)=0

EI(x)={[(μt(x)−f(x+)−ϵ) σt(x)>0∗Φ(Z)]+σt(x)ϕ(Z),0, σt(x)=0

 EI(x)= \begin{cases}

 [(\mu\_t(x) - f(x^+) - \epsilon) & \ \sigma\_t(x) > 0 \\

 \quad \* \Phi(Z)] + \sigma\_t(x)\phi(Z),\\

 0, & \ \sigma\_t(x) = 0

 \end{cases}

 EI(x)=⎩⎪⎨⎪⎧[(μt(x)−f(x+)−ϵ)∗Φ(Z)]+σt(x)ϕ(Z),0, σt(x)>0 σt(x)=0

 where Φ(⋅)\Phi(\cdot)Φ(⋅) indicates CDF and ϕ(⋅)\phi(\cdot)ϕ(⋅) indicates pdf.

![](Exploring%20Bayesian%20Optimization_files/0_002.png)

 We now increase ϵ\epsilonϵ to explore more.

![](Exploring%20Bayesian%20Optimization_files/0_010.png)

 As we expected, increasing the value to ϵ=0.3\epsilon = 0.3ϵ=0.3

 makes the acquisition function explore more. Compared to the earlier

evaluations, we see less exploitation. We see that it evaluates only two

 points near the global maxima.

 Let us increase ϵ\epsilonϵ even more.

![](Exploring%20Bayesian%20Optimization_files/0_005.png)

#### PI vs Ei

![](Exploring%20Bayesian%20Optimization_files/0.svg)

 dot is a point in the search space. Additionally, the training set used

### Thompson Sampling

 Another common acquisition function is Thompson Sampling .

 At every step, we sample a function from the surrogate’s posterior and

optimize it. For example, in the case of gold mining, we would sample a

plausible distribution of the gold given the evidence and evaluate

(drill) wherever it peaks.

 Below we have an image showing three sampled functions from the

learned surrogate posterior for our gold mining problem. The training

![](Exploring%20Bayesian%20Optimization_files/thompson.svg)

 We can understand the intuition behind Thompson sampling by two observations:

* Locations with high uncertainty (σ(x) \sigma(x) σ(x))

 will show a large variance in the functional values sampled from the

surrogate posterior. Thus, there is a non-trivial probability that a

sample can take high value in a highly uncertain region. Optimizing such

 samples can aid **exploration**.

* The sampled functions must pass through the current max

value, as there is no uncertainty at the evaluated locations. Thus,

![](Exploring%20Bayesian%20Optimization_files/0.png)

The visualization above uses Thompson sampling for optimization. Again,

we can reach the global optimum in relatively few iterations.

### Random

 We have been using intelligent acquisition functions until now.

 We can create a random acquisition function by sampling xxx

 randomly.

![](Exploring%20Bayesian%20Optimization_files/0_014.png)

 The visualization above shows that the performance of the random

acquisition function is not that bad! However, if our optimization was

more complex (more dimensions), then the random acquisition might

perform poorly.

### Summary of Acquisition Functions

 Let us now summarize the core ideas associated with acquisition

functions: i) they are heuristics for evaluating the utility of a point;

 ii) they are a function of the surrogate posterior; iii) they combine

exploration and exploitation; and iv) they are inexpensive to evaluate.

#### Other Acquisition Functions

We have seen various acquisition functions until now. One

trivial way to come up with acquisition functions is to have a

explore/exploit combination.

###  Upper Confidence Bound (UCB)

 One such trivial acquisition function that combines the

exploration/exploitation tradeoff is a linear combination of the mean

and uncertainty of our surrogate model. The model mean signifies

exploitation (of our model’s knowledge) and model uncertainty signifies

exploration (due to our model’s lack of observations).

 α(x)=μ(x)+λ×σ(x)\alpha(x) = \mu(x) + \lambda \times \sigma(x)α(x)=μ(x)+λ×σ(x)

 The intuition behind the UCB acquisition function is weighing

of the importance between the surrogate’s mean vs. the surrogate’s

 We can further form acquisition functions by combining the

existing acquisition functions though the physical interpretability of

such combinations might not be so straightforward. One reason we might

want to combine two methods is to overcome the limitations of the

individual methods.

 One such combination can be a linear combination of PI and EI.

 We know PI focuses on the probability of improvement, whereas

EI focuses on the expected improvement. Such a combination could help in

 having a tradeoff between the two based on the value of λ\lambdaλ.

### Gaussian Process Upper Confidence Bound (GP-UCB)

 a−ba -

 GP-UCB’s formulation is given by:

αGP−UCB(x)=μt(x)+βtσt(x)

 \alpha\_{GP-UCB}(x) = \mu\_t(x) + \sqrt{\beta\_t}\sigma\_t(x)

 αGP−UCB(x)=μt(x)+βt

σt(x)

 Where ttt is the timestep.

### Comparison

 slides from Nando De Freitas. We have used the

optimum hyperparameters for each acquisition function.

 We ran the random acquisition function several times with

different seeds and plotted the mean gold sensed at every iteration.

![](Exploring%20Bayesian%20Optimization_files/comp.svg)

 strategy grows slowly. In comparison, the other acquisition functions

can find a good solution in a small number of iterations. In fact, most

acquisition functions reach fairly close to the global maxima in as few

as three iterations.

Hyperparameter Tuning

---------------------

Before we talk about Bayesian optimization for hyperparameter tuning,

 we will quickly differentiate between hyperparameters and parameters:

hyperparameters are set before learning and the parameters are learned

from the data. To illustrate the difference, we take the example of

Ridge regression.

θ^ridge=argminθ ∈ Rp∑i=1n(yi−xiTθ)2+λ∑j=1pθj2

 \hat{\theta}\_{ridge} = argmin\_{\theta\ \in \ \mathbb{R}^p}

\sum\limits\_{i=1}^{n} \left(y\_i - x\_i^T\theta \right)^2 + \lambda

\sum\limits\_{j=1}^{p} \theta^2\_j

 θ^ridge=argminθ ∈ Rpi=1∑n(yi−xiTθ)2+λj=1∑pθj2

θ^ridge=argminθ ∈ Rp∑i=1n(yi−xiTθ)2+λ∑j=1pθj2

 \begin{aligned}

 \hat{\theta}\_{ridge} = & argmin\_{\theta\ \in \ \mathbb{R}^p}

 \sum\limits\_{i=1}^{n} \left(y\_i - x\_i^T\theta \right)^2 \\

 & + \lambda \sum\limits\_{j=1}^{p} \theta^2\_j

 \end{aligned}

 θ^ridge=argminθ ∈ Rpi=1∑n(yi−xiTθ)2+λj=1∑pθj2

 If we solve the above regression problem via gradient descent

optimization, we further introduce another optimization parameter, the

learning rate α\alphaα.

When training a model is not expensive and time-consuming, we

can do a grid search to find the optimum hyperparameters. However, grid

search is not feasible if function evaluations are costly, as in the

case of a large neural network that takes days to train. Further, grid

search scales poorly in terms of the number of hyperparameters.

### Example 1 — Support Vector Machine (SVM)

In this example, we use an SVM to classify on sklearn’s moons

dataset and use Bayesian Optimization to optimize SVM hyperparameters.

 Let us have a look at the dataset now, which has two classes and two features.

![](Exploring%20Bayesian%20Optimization_files/moons.svg)

 the surface plots you see for the Ground Truth Accuracies below were

calculated for each possible hyperparameter for showcasing purposes

only. We do not have these values in real applications.

![](Exploring%20Bayesian%20Optimization_files/0_003.png)

![](Exploring%20Bayesian%20Optimization_files/0_006.png)

### Comparison

![](Exploring%20Bayesian%20Optimization_files/comp3d.svg)

 method seemed to perform much better initially, but it could not reach

the global optimum, whereas Bayesian Optimization was able to get fairly

 close. The initial subpar performance of Bayesian Optimization can be

attributed to the initial exploration.

#### Other Examples

### Example 2 — Random Forest

 Using Bayesian Optimization in a Random Forest Classifier.

 We will continue now to train a Random Forest on the moons

dataset we had used previously to learn the Support Vector Machine

model. The primary hyperparameters of Random Forests we would like to

optimize our accuracy are the  **number** of

 We will be again using Gaussian Processes with Matern kernel

to estimate and predict the accuracy function over the two

hyperparameters.

![](Exploring%20Bayesian%20Optimization_files/0_007.png)

![](Exploring%20Bayesian%20Optimization_files/0_015.png)

![](Exploring%20Bayesian%20Optimization_files/0_008.png)

![](Exploring%20Bayesian%20Optimization_files/0_011.png)

Let us now use the Random acquisition function.

![](Exploring%20Bayesian%20Optimization_files/RFcomp3d.svg)

 The optimization strategies seemed to struggle in this

example. This can be attributed to the non-smooth ground truth. This

shows that the effectiveness of Bayesian Optimization depends on the

surrogate’s efficiency to model the actual black-box function. It is

interesting to notice that the Bayesian Optimization framework still

beats the *random* strategy using various acquisition functions.

### Example 3 — Neural Networks

 Let us take this example to get an idea of how to apply

 which also provides us support for optimizing function with a search

space of categorical, integral, and real variables. We will not be

plotting the ground truth here, as it is extremely costly to do so.

Below are some code snippets that show the ease of using Bayesian

Optimization packages for hyperparameter tuning.

 The code initially declares a search space for the

optimization problem. We limit the search space to be the following:

* batch\_size — This hyperparameter sets the number of

training examples to combine to find the gradients for a single step in

gradient descent.

* learning rate — This hyperparameter sets the stepsize with

 which we will perform gradient descent in the neural network.

* activation — We will have one categorical variable, i.e.

the activation to apply to our neural network layers. This variable can

take on values in the set {relu, sigmoid}\{ relu, \ sigmoid \}{relu, sigmoid}.

 log\_batch\_size = Integer(

 low=2,

 high=7,

 name='log\_batch\_size'

 )

 lr = Real(

 low=1e-6,

 high=1e0,

 prior='log-uniform',

 name='lr'

 )

 activation = Categorical(

 categories=['relu', 'sigmoid'],

 name='activation'

 )

 dimensions = [

 dim\_num\_batch\_size\_to\_base,

 dim\_learning\_rate,

 dim\_activation

 ]

 # initial parameters (1st point)

 default\_parameters =

 [4, 1e-1, 'relu']

 # bayesian optimization

 search\_result = gp\_minimize(

 func=train,

 dimensions=dimensions,

 acq\_func='EI', # Expctd Imprv.

 n\_calls=11,

 x0=default\_parameters

 )

![](Exploring%20Bayesian%20Optimization_files/conv.svg)

 Looking at the above example, we can see that incorporating

Bayesian Optimization is not difficult and can save a lot of time.

Optimizing to get an accuracy of nearly one in around seven iterations

 Let us get the numbers into perspective. If we had run this

 iterations. Whereas Bayesian Optimization only took seven iterations.

Each iteration took around fifteen minutes; this sets the time required

for the grid search to complete around seventeen hours!

Conclusion and Summary

======================

 In this article, we looked at Bayesian Optimization for

optimizing a black-box function. Bayesian Optimization is well suited

when the function evaluations are expensive, making grid or exhaustive

search impractical. We looked at the key components of Bayesian

Optimization. First, we looked at the notion of using a surrogate

function (with a prior over the space of objective functions) to model

our black-box function. Next, we looked at the “Bayes” in Bayesian

Optimization — the function evaluations are used as data to obtain the

surrogate posterior. We look at acquisition functions, which are

functions of the surrogate posterior and are optimized sequentially.

This new sequential optimization is in-expensive and thus of utility of

us. We also looked at a few acquisition functions and showed how these

different functions balance exploration and exploitation. Finally, we

looked at some practical examples of Bayesian Optimization for

optimizing hyper-parameters for machine learning models.

A statistical approach to some basic mine valuation problems on the Witwatersrand

Active Learning Literature Survey

Active learning: theory and applications

Taking the Human Out of the Loop: A Review of Bayesian Optimization

A
 Tutorial on Bayesian Optimization of Expensive Cost Functions, with
Application to Active User Modeling and Hierarchical Reinforcement
Learning

A Visual Exploration of Gaussian Processes

Gaussian Processes in Machine Learning

Bayesian approach to global optimization and application to multiobjective and constrained problems

On The Likelihood That One Unknown Probability Exceeds Another In View Of The Evidence Of Two Samples

Using Confidence Bounds for Exploitation-Exploration Trade-Offs

Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design

Practical Bayesian Optimization of Machine Learning Algorithms

Algorithms for Hyper-Parameter Optimization

Making a Science of Model Search: Hyperparameter Optimization in Hundreds of Dimensions for Vision Architectures

Scikit-learn: Machine Learning in {P}ython

Bayesian Optimization with Gradients

Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimization

Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets

Safe Exploration for Optimization with Gaussian Processes

Scalable Bayesian Optimization Using Deep Neural Networks

Portfolio Allocation for Bayesian Optimization

Bayesian Optimization for Sensor Set Selection

Constrained Bayesian Optimization with Noisy Experiments

Parallel Bayesian Global Optimization of Expensive Functions