![](images/multiple-pages.svg)

 This article is part of the [Circuits thread](/2020/circuits/), a collection of short articles and commentary by an open scientific collaboration delving into the inner workings of neural networks.

[Zoom In: An Introduction to Circuits](/2020/circuits/zoom-in/)
[Curve Detectors](/2020/circuits/curve-detectors/)

 The first few articles of the Circuits project will be focused on early vision in InceptionV1 — for our purposes, the five convolutional layers leading up to the third pooling layer.

input

softmax

[0](#conv2d0)

[1](#conv2d1)
[2](#conv2d2)

5a

4d

5b

[3b](#mixed3b)

4e

4c

4b

[3a](#mixed3a)

4a

For our purposes, we’ll consider early vision to be the first five layers. Click on a layer to jump to section.

 Over the course of these layers, we see the network go from raw pixels
 up to sophisticated [boundary detection](#group_mixed3b_boundary), basic shape detection
 (eg. [curves](#group_mixed3b_curves), [circles](#group_mixed3b_circles_loops), [spirals](#group_mixed3b_curve_shapes), [triangles](#group_mixed3a_angles)),
 [eye detectors](#group_mixed3b_eyes), and even crude detectors for [very small heads](#group_mixed3b_proto_head).
 Along the way, we see a variety of interesting intermediate features,
 including [Complex Gabor detectors](#conv2d1_discussion_complex_gabor) (similar to some classic “complex cells” of neuroscience),
 [black and white vs color detectors](#mixed3a_discussion_BW), and [small circle formation from curves](#mixed3a_discussion_small_circle).

 Studying early vision has two major advantages as a starting point in our investigation.
 Firstly, it’s particularly easy to study:
 it’s close to the input, the circuits are only a few layers deep, there aren’t that many different neurons,It’s common for vision models to have on the order of 64 channels in their initial convolutional layers, which are applied at many spatial positions. So while there are many neurons, the number of unique neurons is orders of magnitude smaller. and the features seem quite simple.
 Secondly, early vision seems most likely to be universal: to have the same features and circuits form across different architectures and tasks.

 Before we dive into detailed explorations of different parts of early vision, we wanted to give a broader overview of how we presently understand it.
 This article sketches out our understanding, as an annotated collection of what we call “neuron groups.”
 We also provide illustrations of selected circuits at each layer.

 By limiting ourselves to early vision, this article “only” considers the first 1,056 neurons of InceptionV1.We will not discuss the “bottleneck” neurons in mixed3a/mixed3b, which we generally think of as low-rank connections to the previous layer.
 But our experience is that a thousand neurons is more than enough to be disorienting when one begins studying a model.
 Our hope is that this article will help readers avoid this disorientation by providing some structure and handholds for thinking about them.

### Playing Cards with Neurons

 Dmitri Mendeleev is often accounted to have discovered the Periodic Table by playing “chemical solitaire,” writing the details of each element on a card and patiently fiddling with different ways of classifying and organizing them.
 Some modern historians are skeptical about the cards, but Mendeleev’s story is a compelling demonstration of that there can be a lot of value in simply organizing phenomena, even when you don’t have a theory or firm justification for that organization yet.
 Mendeleev is far from unique in this.
 For example, in biology, taxonomies of species preceded genetics and the theory of evolution giving them a theoretical foundation.

 Our experience is that many neurons in vision models seem to fall into families of similar features.
 For example, it’s not unusual to see a dozen neurons detecting the same feature in different orientations or colors.
 Perhaps even more strikingly, the same “neuron families” seem to recur across models!
 Of course, it’s well known that Gabor filters and color contrast detectors reliably comprise neurons in the first layer of convolutional neural networks, but we were quite surprised to see this generalize to later layers.

 This article shares our working categorization of units in the first five layers of InceptionV1 into neuron families.
 These families are ad-hoc, human defined collections of features that seem to be similar in some way.
 We’ve found these helpful for communicating among ourselves and breaking the problem of understanding InceptionV1 into smaller chunks.
 While there are some families we suspect are “real”, many others are categories of convenience, or categories we have low-confidence about.
 The main goal of these families is to help researchers orient themselves.

 In constructing this categorization, our understanding of individual neurons was developed by looking at feature visualizations, dataset examples, how a feature is built from the previous layer, how it is used by the next layer, and other analysis.
 It’s worth noting that the level of attention we’ve given to individual neurons varies greatly: we’ve dedicated entire forthcoming articles to detailed analysis some of these units, while many others have only received a few minutes of cursory investigation.

 In some ways, our categorization of units is similar to [Net Dissect](http://netdissect.csail.mit.edu/) , which correlates neurons with a pre-defined set of features and groups them into categories like color, texture, and object.
 This has the advantage of removing subjectivity and being much more scalable.
 At the same time, it also has downsides: correlation can be misleading and the pre-defined taxonomy may miss the true feature types.
[Net Dissect](http://netdissect.csail.mit.edu/)  was very elegant work which advanced our ability to systematically talk about network features.
 However, to understand the differences between correlating features with a pre-defined taxonomy and individually studying them, it may be illustrative to consider how it classifies some features.
 Net Dissect doesn’t include the canonical InceptionV1, but it does include a variant of it.
 Glancing through their version of layer [`mixed3b`](http://netdissect.csail.mit.edu/dissect/googlenet_imagenet/html/inception_3b-output.html) we see many units which appear from dataset examples likely to be familiar feature types like curve detectors, divot detectors, boundary detectors, eye detector, and so forth, but are classified as weakly correlated with another feature — often objects that it seems unlikely could be detected at such an early layer.
 Or in another fun case, there is a feature (372) which is most correlated with a cat detector, but appears to be detecting left-oriented whiskers!

 In particular, if we expect models to have novel, unanticipated features — for example, high-low frequency detectors — the fact that they are unanticipated makes them impossible to include in a set of pre-defined features.
 The only way to discover them is the laborious process of manually investigating each feature.
 In the future, you could imagine hybrid approaches, where a human investigator is saved time by having many features sorted into a (continually growing) set of known features, especially if the [universality hypothesis](https://drafts.distill.pub/circuits-zoom-in/#claim-3) holds.

#### Caveats

* This is a broad overview and our understanding of many of these units is low-confidence. We fully expect, in retrospect, to realize we misunderstood some units and categories.
* Many neuron groups are catch-all categories or convenient organizational categories that we don’t think reflect fundamental structure.
* Even for neuron groups we suspect do reflect a fundamental structure (eg. some can be recovered from factorizing the layer’s weight matrices) the boundaries of these groups can be blurry and some neurons inclusion involve judgement calls.

### Presentation of Neurons

 In order to talk about neurons, we need to somehow represent them.
 While we could use neuron indices, it’s very hard to keep hundreds of numbers straight in one’s head.
 Instead, we use [feature visualizations](https://distill.pub/2017/feature-visualization/), optimized images which highly stimulate a neuron.
 Our feature visualization is done with the [lucid library](https://github.com/tensorflow/lucid).
 We use small amounts of transformation robustness when visualizing the first few layers, because it has a larger proportional affect on their small receptive fields, and increase as we move to higher layers.
 For low layers, we use L2 regularization to push pixels towards gray.
 For the first layer, we follow the convention of other papers and just show the weights, which for the special case of the first layer are equivalent to feature visualization with the right L2 penalty.

 When we represent a neuron with a feature visualization, we don’t intend to claim that the feature visualization captures the entirety of the neuron’s behavior.
 Rather, the role of a feature visualization is like a variable name in understanding a program.
 It replaces an arbitrary number with a more meaningful symbol .

### Presentation of Circuits

 Although this article is focused on giving an overview of the features which exist in early vision, we’re also interested in understanding how they’re computed from earlier features.
 To do this, we present [circuits](https://distill.pub/2020/circuits/zoom-in/#claim-2) consisting of a neuron, the units it has the strongest (L2 norm) weights to in the previous layer, and the weights between them.
 Some neurons in `mixed3a` and `mixed3b` are in branches consisting of a “bottleneck” 1x1 conv that greatly reduces the number of channels followed by a 5x5 conv. Although there is a ReLU between them, we generally think of them as a low rank factorization of a single weight matrix and visualize the product of the two weights. Additionally, some neurons in these layers are in a branch consisting of maxpooling followed by a 1x1 conv; we present these units as their weights replicated over the region of their maxpooling.

 In some cases, we’ve also included a few neurons that have weaker connections if they seem to have particular pedagogical value; in these cases, we’ve mentioned doing so in the caption.
 Neurons are visually displayed by their feature visualizations, as discussed above.
 Weights are represented using a color map with red as positive and blue as negative.

 For example, here is a circuit of a circle detecting unit in `mixed3a` being assembled from earlier curves and a primitive circle detector.
 We’ll discuss this example [in more depth](#mixed3a_discussion_small_circle) later.

positive (excitation)
positive (excitation)
negative (inhibition)
negative (inhibition)

Click on the feature visualization of any neuron to see more weights!

 At any point, you can click on a neuron’s feature visualization to see its weights to the 50 neurons in the previous layer it is most connected to (that is, how it assembled from the previous layer, and also the 50 neurons in the next layer it is most connected to (that is, how it is used going forward).
 This allows further investigation, and gives you an unbiased view of the weights if you’re concerned about cherry-picking.

---

`conv2d0`
---------

 The first conv layer of every vision model we’ve looked at is mostly comprised of two kinds of features: color-contrast detectors and Gabor filters.
 InceptionV1′s `conv2d0` is no exception to this rule, and most of its units fall into these categories.

 In contrast to other models, however, the features aren’t perfect color contrast detectors and Gabor filters.
 For lack of a better word, they’re messy.
 We have no way of knowing, but it seems likely this is a result of the gradient not reaching the early layers very well during training.
 Note that InceptionV1 predated the adoption of modern techniques like batch norm and Adam, which make it much easier to train deep models well.
 If we compare to the TF-Slim rewrite of InceptionV1, which does use BatchNorm, we see crisper features.
 The weights for the units in the first layer of the TF-Slim version of InceptionV1, which adds BatchNorm. (Units are sorted by the first principal component of the adjacency matrix between the first and second layers.) These features are typical of a well trained conv net. Note how, unlike the canonical InceptionV1, these units have a crisp division between black and white Gabors, color Gabors, color-contrast units and color center-surround units.

![](images/slim_weights.png)

 One subtlety that’s worth noting here is that Gabor filters almost always come in pairs of weights which are negative versions of each other, both in InceptionV1 and other vision models.
 A single Gabor filter can only detect edges at some offsets, but the negative version fills in holes, allowing for the formation of complex Gabor filters in the next layer.

### [**Gabor Filters** 44%](#group_conv2d0_gabor_filters)

[![](images/neuron/conv2d0_9.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_9.html)9[![](images/neuron/conv2d0_53.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_53.html)53[![](images/neuron/conv2d0_54.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_54.html)54[![](images/neuron/conv2d0_15.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_15.html)15[![](images/neuron/conv2d0_22.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_22.html)22[![](images/neuron/conv2d0_39.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_39.html)39[![](images/neuron/conv2d0_30.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_30.html)30[![](images/neuron/conv2d0_1.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_1.html)1[![](images/neuron/conv2d0_3.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_3.html)3[![](images/neuron/conv2d0_49.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_49.html)49[![](images/neuron/conv2d0_14.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_14.html)14[![](images/neuron/conv2d0_17.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_17.html)17[![](images/neuron/conv2d0_62.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_62.html)62[![](images/neuron/conv2d0_20.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_20.html)20[![](images/neuron/conv2d0_27.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_27.html)27[![](images/neuron/conv2d0_0.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_0.html)0[![](images/neuron/conv2d0_10.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_10.html)10[![](images/neuron/conv2d0_28.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_28.html)28[![](images/neuron/conv2d0_21.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_21.html)21[![](images/neuron/conv2d0_63.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_63.html)63[![](images/neuron/conv2d0_45.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_45.html)45[![](images/neuron/conv2d0_18.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_18.html)18[![](images/neuron/conv2d0_6.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_6.html)6[![](images/neuron/conv2d0_57.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_57.html)57[![](images/neuron/conv2d0_41.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_41.html)41[![](images/neuron/conv2d0_43.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_43.html)43[![](images/neuron/conv2d0_46.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_46.html)46[![](images/neuron/conv2d0_8.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_8.html)8

Show all 28 neurons.
Collapse neurons.
Gabor filters are a simple edge detector, highly sensitive to the alignment of the edge. They’re almost universally found in the fist layer of vision models. Note that Gabor filters almost always come in pairs of negative reciprocals.

### [**Color Contrast** 42%](#group_conv2d0_color_contrast)

[![](images/neuron/conv2d0_59.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_59.html)59[![](images/neuron/conv2d0_23.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_23.html)23[![](images/neuron/conv2d0_5.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_5.html)5[![](images/neuron/conv2d0_7.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_7.html)7[![](images/neuron/conv2d0_48.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_48.html)48[![](images/neuron/conv2d0_29.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_29.html)29[![](images/neuron/conv2d0_12.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_12.html)12[![](images/neuron/conv2d0_24.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_24.html)24[![](images/neuron/conv2d0_55.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_55.html)55[![](images/neuron/conv2d0_13.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_13.html)13[![](images/neuron/conv2d0_32.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_32.html)32[![](images/neuron/conv2d0_50.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_50.html)50[![](images/neuron/conv2d0_36.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_36.html)36[![](images/neuron/conv2d0_37.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_37.html)37[![](images/neuron/conv2d0_16.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_16.html)16[![](images/neuron/conv2d0_11.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_11.html)11[![](images/neuron/conv2d0_33.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_33.html)33[![](images/neuron/conv2d0_34.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_34.html)34[![](images/neuron/conv2d0_47.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_47.html)47[![](images/neuron/conv2d0_38.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_38.html)38[![](images/neuron/conv2d0_58.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_58.html)58[![](images/neuron/conv2d0_2.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_2.html)2[![](images/neuron/conv2d0_60.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_60.html)60[![](images/neuron/conv2d0_61.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_61.html)61[![](images/neuron/conv2d0_4.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_4.html)4[![](images/neuron/conv2d0_51.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_51.html)51[![](images/neuron/conv2d0_25.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_25.html)25

Show all 27 neurons.
Collapse neurons.
These units detect a color one side of their receptive field, and the opposite color on the other side. Compare to later color contrast ([`conv2d1`](#group_conv2d1_color_contrast), [`conv2d2`](#group_conv2d2_color_contrast), [`mixed3a`](#group_mixed3a_color_contrast), [`mixed3b`](#group_mixed3b_color_contrast_gradient)).

### [**Other Units** 14%](#group_conv2d0_other_units)

[![](images/neuron/conv2d0_19.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_19.html)19[![](images/neuron/conv2d0_26.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_26.html)26[![](images/neuron/conv2d0_31.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_31.html)31[![](images/neuron/conv2d0_35.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_35.html)35[![](images/neuron/conv2d0_40.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_40.html)40[![](images/neuron/conv2d0_42.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_42.html)42[![](images/neuron/conv2d0_44.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_44.html)44[![](images/neuron/conv2d0_52.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_52.html)52[![](images/neuron/conv2d0_56.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d0_56.html)56

Units that don’t fit in another category.

---

`conv2d1`
---------

 In `conv2d1`, we begin to see some of the classic [complex cell](https://en.wikipedia.org/wiki/Complex_cell) features of visual neuroscience.
 These neurons respond to similar patterns to units in `conv2d0`, but are invariant to some changes in position and orientation.

**Complex Gabors:**
 A nice example of this is the “Complex Gabor” feature family.
 Like simple Gabor filters, complex Gabors detect edges.
 But unlike simple Gabors, they are relatively invariant to the exact position of the edge or which side is dark or light.
 This is achieved by being excited by multiple Gabor filters in similar orientations — and most critically, by being excited by “reciprocal Gabor filters” that detect the same pattern with dark and light switched.
 This can be seen as an early example of the “union over cases” motif.

All neurons in the previous layer with at least 30% of the max weight magnitude are shown,
 both positive (excitation) and negative (inhibition).
 Click on a neuron to see its forwards and backwards weights.

 Note that `conv2d1` is a 1x1 convolution, so there’s only a single weight — a single line, in this diagram — between each channel in the previous and this one.
 There is a pooling layer between them, so the features it connects to are pooled versions of the previous layer rather than original features.
 This plays an important role in determining the features we observe: in models with larger convolutions in their second layer, we often see a jump to crude versions of the larger more complex features we’ll see in the following layers.

 In addition to Complex Gabors, we see a variety of other features, including
 more invariant color contrast detectors, Gabor-like features that are less selective for a single orientation, and lower-frequency features.

### [**Low Frequency** 27%](#group_conv2d1_low_frequency)

[![](images/neuron/conv2d1_1.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_1.html)1[![](images/neuron/conv2d1_13.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_13.html)13[![](images/neuron/conv2d1_27.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_27.html)27[![](images/neuron/conv2d1_47.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_47.html)47[![](images/neuron/conv2d1_56.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_56.html)56[![](images/neuron/conv2d1_60.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_60.html)60[![](images/neuron/conv2d1_23.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_23.html)23[![](images/neuron/conv2d1_49.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_49.html)49[![](images/neuron/conv2d1_0.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_0.html)0[![](images/neuron/conv2d1_43.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_43.html)43[![](images/neuron/conv2d1_28.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_28.html)28[![](images/neuron/conv2d1_29.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_29.html)29[![](images/neuron/conv2d1_34.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_34.html)34[![](images/neuron/conv2d1_8.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_8.html)8[![](images/neuron/conv2d1_37.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_37.html)37[![](images/neuron/conv2d1_19.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_19.html)19[![](images/neuron/conv2d1_15.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_15.html)15

Show all 17 neurons.
Collapse neurons.
These units seem to respond to lower-frequency edge patterns, but we haven’t studied them very carefully.

### [**Gabor Like** 17%](#group_conv2d1_gabor_like)

[![](images/neuron/conv2d1_4.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_4.html)4[![](images/neuron/conv2d1_6.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_6.html)6[![](images/neuron/conv2d1_32.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_32.html)32[![](images/neuron/conv2d1_38.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_38.html)38[![](images/neuron/conv2d1_41.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_41.html)41[![](images/neuron/conv2d1_63.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_63.html)63[![](images/neuron/conv2d1_7.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_7.html)7[![](images/neuron/conv2d1_11.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_11.html)11[![](images/neuron/conv2d1_18.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_18.html)18[![](images/neuron/conv2d1_24.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_24.html)24[![](images/neuron/conv2d1_39.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_39.html)39

Show all 11 neurons.
Collapse neurons.
These units respond to edges stimuli, but seem to respond to a wider range of orientations, and also respond to color contrasts that align with the edge. We haven’t studied them very carefully.

### [**Color Contrast** 16%](#group_conv2d1_color_contrast)

[![](images/neuron/conv2d1_5.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_5.html)5[![](images/neuron/conv2d1_9.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_9.html)9[![](images/neuron/conv2d1_10.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_10.html)10[![](images/neuron/conv2d1_12.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_12.html)12[![](images/neuron/conv2d1_20.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_20.html)20[![](images/neuron/conv2d1_21.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_21.html)21[![](images/neuron/conv2d1_42.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_42.html)42[![](images/neuron/conv2d1_45.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_45.html)45[![](images/neuron/conv2d1_50.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_50.html)50[![](images/neuron/conv2d1_53.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_53.html)53

These units detect a color on one side of the receptive field, and a different color on the opposite side. Composed of lower-level color contrast detectors, they often respond to color transitions in a range of translation and orientation variations. Compare to earlier [color contrast (`conv2d0`)](#group_conv2d0_color_contrast) and later color contrast ([`conv2d2`](#group_conv2d2_color_contrast), [`mixed3a`](#group_mixed3a_color_contrast), [`mixed3b`](#group_mixed3b_color_contrast_gradient)).

### [**Multicolor** 14%](#group_conv2d1_multicolor)

[![](images/neuron/conv2d1_3.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_3.html)3[![](images/neuron/conv2d1_33.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_33.html)33[![](images/neuron/conv2d1_35.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_35.html)35[![](images/neuron/conv2d1_40.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_40.html)40[![](images/neuron/conv2d1_17.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_17.html)17[![](images/neuron/conv2d1_16.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_16.html)16[![](images/neuron/conv2d1_57.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_57.html)57[![](images/neuron/conv2d1_26.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_26.html)26[![](images/neuron/conv2d1_31.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_31.html)31

These units respond to mixtures of colors without an obvious strong spatial structure preference.

### [**Complex Gabor** 14%](#group_conv2d1_complex_gabor)

[![](images/neuron/conv2d1_51.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_51.html)51[![](images/neuron/conv2d1_58.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_58.html)58[![](images/neuron/conv2d1_30.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_30.html)30[![](images/neuron/conv2d1_25.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_25.html)25[![](images/neuron/conv2d1_52.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_52.html)52[![](images/neuron/conv2d1_54.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_54.html)54[![](images/neuron/conv2d1_22.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_22.html)22[![](images/neuron/conv2d1_61.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_61.html)61[![](images/neuron/conv2d1_55.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_55.html)55

Like Gabor Filters, but fairly invariant to the exact position, formed by adding together multiple Gabor detectors in the same orientation but different phases. We call these ‘Complex’ after [complex cells](https://en.wikipedia.org/wiki/Complex_cell) in neuroscience.

### [**Color** 6%](#group_conv2d1_color)

[![](images/neuron/conv2d1_2.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_2.html)2[![](images/neuron/conv2d1_48.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_48.html)48[![](images/neuron/conv2d1_36.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_36.html)36[![](images/neuron/conv2d1_46.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_46.html)46

Two of these units seem to track brightness (bright vs dark), while the other two units seem to mostly track hue, dividing the space of hues between them. One responds to red/orange/yellow, while the other responds to purple/blue/turqoise. Unfortunately, their circuits seem to heavily rely on the existence of a Local Response Normalization layer after `conv2d0`, which makes it hard to reason about.

### [**Other Units** 5%](#group_conv2d1_other_units)

[![](images/neuron/conv2d1_44.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_44.html)44[![](images/neuron/conv2d1_62.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_62.html)62[![](images/neuron/conv2d1_59.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_59.html)59

Units that don’t fit in another category.

### [**hatch** 2%](#group_conv2d1_hatch)

[![](images/neuron/conv2d1_14.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d1_14.html)14

This unit detects Gabor patterns in two orthogonal directions, selecting for a “hatch” pattern.

---

`conv2d2`
---------

In `conv2d2` we see the emergence of very simple shape predecessors.
This layer sees the first units that might be described as “line detectors”, preferring a single longer line to a Gabor pattern and accounting for about 25% of units.
We also see tiny curve detectors, corner detectors, divergence detectors, and a single very tiny circle detector.
One fun aspect of these features is that you can see that they are assembled from Gabor detectors in the feature visualizations, with curves being built from small piecewise Gabor segments.
All of these units still moderately fire in response to incomplete versions of their feature, such as a small curve running tangent to the edge detector.

Since `conv2d2` is a 3x3 convolution, our understanding of these shape precursor features (and some texture features) maps to particular ways Gabor and lower-frequency edges are being spatially assembled into new features.
At a high-level, we see a few primary patterns:

Many line-like features are weakly excited by perpendicular lines beside the primary line, a phenomenon we call “combing”.
Line
Curve
Shifted Line
Gabor Texture
Corner / Lisp
Hatch Texture
Divergence

 We also begin to see various kinds of texture and color detectors start to become a major constituent of the layer, including color-contrast and color center surround features, as well as Gabor-like, hatch, low-frequency and high-frequency textures.
 A handful of units look for different textures on different sides of their receptive field.

### [**Color Contrast** 21%](#group_conv2d2_color_contrast)

[![](images/neuron/conv2d2_10.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_10.html)10[![](images/neuron/conv2d2_36.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_36.html)36[![](images/neuron/conv2d2_172.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_172.html)172[![](images/neuron/conv2d2_56.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_56.html)56[![](images/neuron/conv2d2_131.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_131.html)131[![](images/neuron/conv2d2_176.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_176.html)176[![](images/neuron/conv2d2_35.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_35.html)35[![](images/neuron/conv2d2_41.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_41.html)41[![](images/neuron/conv2d2_45.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_45.html)45[![](images/neuron/conv2d2_126.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_126.html)126[![](images/neuron/conv2d2_77.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_77.html)77[![](images/neuron/conv2d2_120.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_120.html)120[![](images/neuron/conv2d2_101.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_101.html)101[![](images/neuron/conv2d2_0.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_0.html)0[![](images/neuron/conv2d2_43.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_43.html)43[![](images/neuron/conv2d2_62.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_62.html)62[![](images/neuron/conv2d2_106.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_106.html)106[![](images/neuron/conv2d2_169.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_169.html)169[![](images/neuron/conv2d2_127.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_127.html)127[![](images/neuron/conv2d2_6.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_6.html)6[![](images/neuron/conv2d2_68.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_68.html)68[![](images/neuron/conv2d2_60.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_60.html)60[![](images/neuron/conv2d2_134.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_134.html)134[![](images/neuron/conv2d2_51.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_51.html)51[![](images/neuron/conv2d2_74.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_74.html)74[![](images/neuron/conv2d2_85.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_85.html)85[![](images/neuron/conv2d2_115.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_115.html)115[![](images/neuron/conv2d2_24.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_24.html)24[![](images/neuron/conv2d2_14.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_14.html)14[![](images/neuron/conv2d2_16.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_16.html)16[![](images/neuron/conv2d2_151.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_151.html)151[![](images/neuron/conv2d2_168.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_168.html)168[![](images/neuron/conv2d2_19.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_19.html)19[![](images/neuron/conv2d2_7.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_7.html)7[![](images/neuron/conv2d2_88.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_88.html)88[![](images/neuron/conv2d2_177.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_177.html)177[![](images/neuron/conv2d2_183.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_183.html)183[![](images/neuron/conv2d2_76.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_76.html)76[![](images/neuron/conv2d2_70.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_70.html)70[![](images/neuron/conv2d2_122.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_122.html)122

Show all 40 neurons.
Collapse neurons.
These units detect a color on one side of the receptive field, and a different color on the opposite side. Composed of lower-level color contrast detectors, they often respond to color transitions in a range of translation and orientation variations. Compare to earlier color contrast ([`conv2d0`](#group_conv2d0_color_contrast), [`conv2d1`](#group_conv2d1_color_contrast)) and later color contrast ([`mixed3a`](#group_mixed3a_color_contrast), [`mixed3b`](#group_mixed3b_color_contrast_gradient)).

### [**Line** 17%](#group_conv2d2_line)

[![](images/neuron/conv2d2_107.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_107.html)107[![](images/neuron/conv2d2_31.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_31.html)31[![](images/neuron/conv2d2_9.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_9.html)9[![](images/neuron/conv2d2_112.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_112.html)112[![](images/neuron/conv2d2_133.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_133.html)133[![](images/neuron/conv2d2_103.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_103.html)103[![](images/neuron/conv2d2_97.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_97.html)97[![](images/neuron/conv2d2_125.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_125.html)125[![](images/neuron/conv2d2_20.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_20.html)20[![](images/neuron/conv2d2_33.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_33.html)33[![](images/neuron/conv2d2_113.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_113.html)113[![](images/neuron/conv2d2_185.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_185.html)185[![](images/neuron/conv2d2_150.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_150.html)150[![](images/neuron/conv2d2_166.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_166.html)166[![](images/neuron/conv2d2_157.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_157.html)157[![](images/neuron/conv2d2_57.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_57.html)57[![](images/neuron/conv2d2_145.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_145.html)145[![](images/neuron/conv2d2_48.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_48.html)48[![](images/neuron/conv2d2_55.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_55.html)55[![](images/neuron/conv2d2_15.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_15.html)15[![](images/neuron/conv2d2_152.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_152.html)152[![](images/neuron/conv2d2_11.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_11.html)11[![](images/neuron/conv2d2_141.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_141.html)141[![](images/neuron/conv2d2_174.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_174.html)174[![](images/neuron/conv2d2_44.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_44.html)44[![](images/neuron/conv2d2_170.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_170.html)170[![](images/neuron/conv2d2_27.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_27.html)27[![](images/neuron/conv2d2_100.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_100.html)100[![](images/neuron/conv2d2_30.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_30.html)30[![](images/neuron/conv2d2_82.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_82.html)82[![](images/neuron/conv2d2_65.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_65.html)65[![](images/neuron/conv2d2_3.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_3.html)3[![](images/neuron/conv2d2_108.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_108.html)108

Show all 33 neurons.
Collapse neurons.
These units are beginning to look for a single primary line. Some look for different colors on each side. Many exhibit “combing” (small perpendicular lines along the main one), a very common but not presently understood phenomenon in line-like features across vision models. Compare to [shifted lines](#group_conv2d2_shifted_line) and later [lines (`mixed3a`)](#group_mixed3a_lines).

### [**Shifted Line** 8%](#group_conv2d2_shifted_line)

[![](images/neuron/conv2d2_116.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_116.html)116[![](images/neuron/conv2d2_61.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_61.html)61[![](images/neuron/conv2d2_8.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_8.html)8[![](images/neuron/conv2d2_22.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_22.html)22[![](images/neuron/conv2d2_69.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_69.html)69[![](images/neuron/conv2d2_96.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_96.html)96[![](images/neuron/conv2d2_78.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_78.html)78[![](images/neuron/conv2d2_18.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_18.html)18[![](images/neuron/conv2d2_17.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_17.html)17[![](images/neuron/conv2d2_132.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_132.html)132[![](images/neuron/conv2d2_190.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_190.html)190[![](images/neuron/conv2d2_64.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_64.html)64[![](images/neuron/conv2d2_81.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_81.html)81[![](images/neuron/conv2d2_154.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_154.html)154[![](images/neuron/conv2d2_179.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_179.html)179[![](images/neuron/conv2d2_136.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_136.html)136

Show all 16 neurons.
Collapse neurons.
These units look for edges “shifted” to the side of the receptive field instead of the middle. This may be linked to the many 1x1 convs in the next layer. Compare to [lines](#group_conv2d2_line) (non-shifted) and later [lines (`mixed3a`)](#group_mixed3a_lines).

### [**Textures** 8%](#group_conv2d2_textures)

[![](images/neuron/conv2d2_21.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_21.html)21[![](images/neuron/conv2d2_52.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_52.html)52[![](images/neuron/conv2d2_59.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_59.html)59[![](images/neuron/conv2d2_119.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_119.html)119[![](images/neuron/conv2d2_148.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_148.html)148[![](images/neuron/conv2d2_161.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_161.html)161[![](images/neuron/conv2d2_162.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_162.html)162[![](images/neuron/conv2d2_186.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_186.html)186[![](images/neuron/conv2d2_189.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_189.html)189[![](images/neuron/conv2d2_191.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_191.html)191[![](images/neuron/conv2d2_72.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_72.html)72[![](images/neuron/conv2d2_46.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_46.html)46[![](images/neuron/conv2d2_67.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_67.html)67[![](images/neuron/conv2d2_40.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_40.html)40[![](images/neuron/conv2d2_38.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_38.html)38

Show all 15 neurons.
Collapse neurons.
A broad category of units detecting repeating local structure.

### [**Other Units** 7%](#group_conv2d2_other_units)

[![](images/neuron/conv2d2_2.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_2.html)2[![](images/neuron/conv2d2_50.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_50.html)50[![](images/neuron/conv2d2_54.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_54.html)54[![](images/neuron/conv2d2_66.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_66.html)66[![](images/neuron/conv2d2_84.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_84.html)84[![](images/neuron/conv2d2_90.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_90.html)90[![](images/neuron/conv2d2_93.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_93.html)93[![](images/neuron/conv2d2_109.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_109.html)109[![](images/neuron/conv2d2_129.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_129.html)129[![](images/neuron/conv2d2_149.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_149.html)149[![](images/neuron/conv2d2_153.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_153.html)153[![](images/neuron/conv2d2_158.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_158.html)158[![](images/neuron/conv2d2_175.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_175.html)175[![](images/neuron/conv2d2_187.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_187.html)187

Show all 14 neurons.
Collapse neurons.
Catch-all category for all other units.

### [**Color Center-Surround** 7%](#group_conv2d2_color_center_surround)

[![](images/neuron/conv2d2_155.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_155.html)155[![](images/neuron/conv2d2_34.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_34.html)34[![](images/neuron/conv2d2_156.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_156.html)156[![](images/neuron/conv2d2_13.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_13.html)13[![](images/neuron/conv2d2_49.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_49.html)49[![](images/neuron/conv2d2_138.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_138.html)138[![](images/neuron/conv2d2_23.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_23.html)23[![](images/neuron/conv2d2_160.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_160.html)160[![](images/neuron/conv2d2_86.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_86.html)86[![](images/neuron/conv2d2_124.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_124.html)124[![](images/neuron/conv2d2_25.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_25.html)25[![](images/neuron/conv2d2_4.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_4.html)4[![](images/neuron/conv2d2_29.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_29.html)29

Show all 13 neurons.
Collapse neurons.
These units look for one color in the middle and another (typically opposite) on the boundary. Genereally more sensitive to the center than boundary. Compare to later  [Color Center-Surround (`mixed3a`)](#group_mixed3a_color_center_surround) and [Color Center-Surround (`mixed3b`)](#group_mixed3b_color_center_surround).

### [**Tiny Curves** 6%](#group_conv2d2_tiny_curves)

[![](images/neuron/conv2d2_182.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_182.html)182[![](images/neuron/conv2d2_117.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_117.html)117[![](images/neuron/conv2d2_171.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_171.html)171[![](images/neuron/conv2d2_111.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_111.html)111[![](images/neuron/conv2d2_63.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_63.html)63[![](images/neuron/conv2d2_130.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_130.html)130[![](images/neuron/conv2d2_146.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_146.html)146[![](images/neuron/conv2d2_180.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_180.html)180[![](images/neuron/conv2d2_39.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_39.html)39[![](images/neuron/conv2d2_140.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_140.html)140[![](images/neuron/conv2d2_32.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_32.html)32[![](images/neuron/conv2d2_80.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_80.html)80

Show all 12 neurons.
Collapse neurons.
Very small curve (and one circle) detectors. Many of these units respond to a range of curvatures all the way from a flat line to a curve. Compare to later [curves (`mixed3a`)](#group_mixed3a_curves) and [curves (`mixed3b`)](#group_mixed3b_curves). See also [circuit example and discussion](#mixed3a_discussion_small_circle) of use in forming [small circles/eyes (`mixed3a`)](#group_mixed3a_eyes_small_circles).

### [**Early Brightness Gradient** 6%](#group_conv2d2_early_brightness_gradient)

[![](images/neuron/conv2d2_94.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_94.html)94[![](images/neuron/conv2d2_142.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_142.html)142[![](images/neuron/conv2d2_99.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_99.html)99[![](images/neuron/conv2d2_104.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_104.html)104[![](images/neuron/conv2d2_163.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_163.html)163[![](images/neuron/conv2d2_188.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_188.html)188[![](images/neuron/conv2d2_98.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_98.html)98[![](images/neuron/conv2d2_165.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_165.html)165[![](images/neuron/conv2d2_83.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_83.html)83[![](images/neuron/conv2d2_79.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_79.html)79[![](images/neuron/conv2d2_128.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_128.html)128[![](images/neuron/conv2d2_5.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_5.html)5

Show all 12 neurons.
Collapse neurons.
These units detect oriented gradients in brightness. They support a variety of similar units in the next layer. Compare to later [brightness gradients (`mixed3a`)](#group_mixed3a_brightness_gradient) and [brightness gradients (`mixed3b`)](#group_mixed3b_brightness_gradients).

### [**Gabor Textures** 6%](#group_conv2d2_gabor_textures)

[![](images/neuron/conv2d2_89.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_89.html)89[![](images/neuron/conv2d2_91.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_91.html)91[![](images/neuron/conv2d2_110.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_110.html)110[![](images/neuron/conv2d2_114.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_114.html)114[![](images/neuron/conv2d2_123.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_123.html)123[![](images/neuron/conv2d2_135.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_135.html)135[![](images/neuron/conv2d2_139.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_139.html)139[![](images/neuron/conv2d2_143.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_143.html)143[![](images/neuron/conv2d2_144.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_144.html)144[![](images/neuron/conv2d2_167.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_167.html)167[![](images/neuron/conv2d2_173.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_173.html)173[![](images/neuron/conv2d2_118.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_118.html)118

Show all 12 neurons.
Collapse neurons.
Like complex Gabor units from the previous layer, but larger. They’re probably starting to be better described as a texture.

### [**Texture Contrast** 4%](#group_conv2d2_texture_contrast)

[![](images/neuron/conv2d2_1.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_1.html)1[![](images/neuron/conv2d2_37.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_37.html)37[![](images/neuron/conv2d2_71.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_71.html)71[![](images/neuron/conv2d2_73.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_73.html)73[![](images/neuron/conv2d2_105.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_105.html)105[![](images/neuron/conv2d2_147.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_147.html)147[![](images/neuron/conv2d2_178.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_178.html)178[![](images/neuron/conv2d2_181.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_181.html)181

These units look for different textures on opposite sides of their receptive field. One side is typically a Gabor pattern.

### [**Hatch Textures** 3%](#group_conv2d2_hatch_textures)

[![](images/neuron/conv2d2_164.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_164.html)164[![](images/neuron/conv2d2_184.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_184.html)184[![](images/neuron/conv2d2_28.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_28.html)28[![](images/neuron/conv2d2_121.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_121.html)121[![](images/neuron/conv2d2_159.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_159.html)159[![](images/neuron/conv2d2_102.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_102.html)102

These units detect Gabor patterns in two orthogonal directions, selecting for a “hatch” pattern.

### [**Color/Multicolor** 3%](#group_conv2d2_color_multicolor)

[![](images/neuron/conv2d2_58.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_58.html)58[![](images/neuron/conv2d2_75.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_75.html)75[![](images/neuron/conv2d2_92.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_92.html)92[![](images/neuron/conv2d2_42.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_42.html)42[![](images/neuron/conv2d2_12.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_12.html)12

Several units look for mixtures of colors but seem indifferent to their organization.

### [**Corners** 2%](#group_conv2d2_corners)

[![](images/neuron/conv2d2_47.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_47.html)47[![](images/neuron/conv2d2_87.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_87.html)87[![](images/neuron/conv2d2_95.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_95.html)95[![](images/neuron/conv2d2_26.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_26.html)26

These units detect two Gabor patterns which meet at apprixmately 90 degrees, causing them to respond to corners.

### [**Line Divergence** 1%](#group_conv2d2_line_divergence)

[![](images/neuron/conv2d2_137.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_137.html)137[![](images/neuron/conv2d2_53.png)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/conv2d2_53.html)53

These units detect lines diverging from a point.

---

`mixed3a`
---------

`mixed3a` has a significant increase in the diversity of features we observe.
Some of them — [curve detectors](#group_mixed3a_curves) and [high-low frequency detectors](#group_mixed3a_high_low_frequency) — were discussed in [Zoom In](https://distill.pub/2020/circuits/zoom-in/)
and will be discussed again in later articles in great detail.
But there are some really interesting circuits in `mixed3a` which we haven’t discussed before,
and we’ll go through a couple selected ones to give a flavor of what happens at this layer.

**Black & White Detectors:** One interesting property of `mixed3a` is the emergence of [“black and white” detectors](#group_mixed3a_bw_vs_color), which detect the absence of color.
Prior to `mixed3a`, color contrast detectors look for transitions of a color to near complementary colors (eg. blue vs yellow).
From this layer on, however, we’ll often see color detectors which compare a color to the absence of color.
Additionally, black and white detectors can allow the detection of greyscale images, which may be correlated with ImageNet categories (see
 [4a:479](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed4a_479.html)  which detects black and white portraits).

The circuit for our black and white detector is quite simple:
almost all of its large weights are negative, detecting the absence of colors.
 Roughly, it computes `**NOT(**color_feature_1 **OR** color_feature_2 **OR** ...**)**`.

Black and white detectors are created by  against a wide variety of color detectors.

inhibiting

positive (excitation)
negative (inhibition)

 The sixteen strongest magnitude weights to the previous layer are shown.
 For simplicity, only one spatial weight for positive and negative have been shown, but they all have almost identical structure.
 Click on a neuron to see its forwards and backwards weights.

**Small Circle Detector:** We also see somewhat more complex shapes in `mixed3a`. Of course, curves (which we discussed in [Zoom In](https://distill.pub/2020/circuits/zoom-in/)) are a prominent example of this.
But there’s lots of other interesting examples.
For instance, we see a variety of [small circle and eye detectors](#group_mixed3a_eyes_small_circles) form
by piecing together [early curves and circle detectors (`conv2d2`)](#group_conv2d2_tiny_curves):

positive (excitation)
positive (excitation)
negative (inhibition)
negative (inhibition)

**Triangle Detectors:** While on the matter of basic shapes, we also see [triangle detectors](#group_mixed3a_angles)
 form from earlier [line (`conv2d2`)](#group_conv2d2_line) and [shifted line (`conv2d2`)](#group_conv2d2_shifted_line) detectors.

positive (excitation)
negative (inhibition)

 The circuit constructing a triangle detector.
 The choice of which neurons in the previous layer to show is slightly cherrypicked for pedagogy.
 The six neurons with the highest magnitude weights to the triangle are shown, plus one other neuron with slightly weaker weights.
 (Left leaning edges have slightly higher weights than right ones, but it seemed more illustrative to show two of both.) Click on neurons to see the full weights.

 However, in practice, these triangle detectors (and other angle units) seem to often just be used as multi-edge detectors downstream,
 or in conjunction with many other units to detect convex boundaries.

 The selected circuits discussed above only scratch the surface of the intricate structure in `mixed3a`.
 Below, we provide a taxonomized overview of all of them:

### [**Texture** 25%](#group_mixed3a_texture)

[![](images/neuron/mixed3a_246.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_246.html)246[![](images/neuron/mixed3a_242.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_242.html)242[![](images/neuron/mixed3a_253.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_253.html)253[![](images/neuron/mixed3a_232.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_232.html)232[![](images/neuron/mixed3a_233.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_233.html)233[![](images/neuron/mixed3a_209.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_209.html)209[![](images/neuron/mixed3a_139.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_139.html)139[![](images/neuron/mixed3a_65.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_65.html)65[![](images/neuron/mixed3a_44.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_44.html)44[![](images/neuron/mixed3a_51.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_51.html)51[![](images/neuron/mixed3a_194.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_194.html)194[![](images/neuron/mixed3a_207.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_207.html)207[![](images/neuron/mixed3a_111.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_111.html)111[![](images/neuron/mixed3a_218.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_218.html)218[![](images/neuron/mixed3a_224.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_224.html)224[![](images/neuron/mixed3a_225.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_225.html)225[![](images/neuron/mixed3a_215.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_215.html)215[![](images/neuron/mixed3a_198.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_198.html)198[![](images/neuron/mixed3a_62.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_62.html)62[![](images/neuron/mixed3a_21.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_21.html)21[![](images/neuron/mixed3a_254.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_254.html)254[![](images/neuron/mixed3a_255.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_255.html)255[![](images/neuron/mixed3a_61.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_61.html)61[![](images/neuron/mixed3a_2.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_2.html)2[![](images/neuron/mixed3a_3.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_3.html)3[![](images/neuron/mixed3a_8.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_8.html)8[![](images/neuron/mixed3a_12.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_12.html)12[![](images/neuron/mixed3a_53.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_53.html)53[![](images/neuron/mixed3a_56.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_56.html)56[![](images/neuron/mixed3a_102.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_102.html)102[![](images/neuron/mixed3a_148.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_148.html)148[![](images/neuron/mixed3a_244.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_244.html)244[![](images/neuron/mixed3a_250.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_250.html)250[![](images/neuron/mixed3a_11.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_11.html)11[![](images/neuron/mixed3a_238.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_238.html)238[![](images/neuron/mixed3a_248.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_248.html)248[![](images/neuron/mixed3a_9.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_9.html)9[![](images/neuron/mixed3a_219.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_219.html)219[![](images/neuron/mixed3a_234.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_234.html)234[![](images/neuron/mixed3a_252.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_252.html)252[![](images/neuron/mixed3a_236.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_236.html)236[![](images/neuron/mixed3a_5.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_5.html)5[![](images/neuron/mixed3a_183.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_183.html)183[![](images/neuron/mixed3a_241.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_241.html)241[![](images/neuron/mixed3a_229.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_229.html)229[![](images/neuron/mixed3a_93.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_93.html)93[![](images/neuron/mixed3a_243.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_243.html)243[![](images/neuron/mixed3a_99.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_99.html)99[![](images/neuron/mixed3a_45.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_45.html)45[![](images/neuron/mixed3a_33.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_33.html)33[![](images/neuron/mixed3a_135.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_135.html)135[![](images/neuron/mixed3a_231.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_231.html)231[![](images/neuron/mixed3a_60.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_60.html)60[![](images/neuron/mixed3a_235.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_235.html)235[![](images/neuron/mixed3a_48.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_48.html)48[![](images/neuron/mixed3a_55.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_55.html)55[![](images/neuron/mixed3a_42.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_42.html)42[![](images/neuron/mixed3a_151.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_151.html)151[![](images/neuron/mixed3a_54.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_54.html)54[![](images/neuron/mixed3a_72.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_72.html)72[![](images/neuron/mixed3a_6.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_6.html)6[![](images/neuron/mixed3a_239.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_239.html)239[![](images/neuron/mixed3a_66.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_66.html)66[![](images/neuron/mixed3a_129.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_129.html)129[![](images/neuron/mixed3a_245.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_245.html)245

Show all 65 neurons.
Collapse neurons.
This is a broad, not very well defined category for units that seem to look for simple local structures over a wide receptive field, including mixtures of colors. Many live in a branch consisting of a maxpool followed by a 1x1 conv, which structurally encourages this.Maxpool branches (ie. maxpool 5x5 stride 1 -> conv 1x1) have large receptive fields, but can’t control where in in their receptive field each feature they detect is, nor the relative position of these features. In early vision, this unstructured of feature detection makes them a good fit for textures.

### [**Color Center-Surround** 12%](#group_mixed3a_color_center_surround)

[![](images/neuron/mixed3a_119.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_119.html)119[![](images/neuron/mixed3a_34.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_34.html)34[![](images/neuron/mixed3a_167.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_167.html)167[![](images/neuron/mixed3a_76.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_76.html)76[![](images/neuron/mixed3a_19.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_19.html)19[![](images/neuron/mixed3a_30.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_30.html)30[![](images/neuron/mixed3a_131.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_131.html)131[![](images/neuron/mixed3a_251.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_251.html)251[![](images/neuron/mixed3a_226.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_226.html)226[![](images/neuron/mixed3a_13.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_13.html)13[![](images/neuron/mixed3a_7.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_7.html)7[![](images/neuron/mixed3a_50.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_50.html)50[![](images/neuron/mixed3a_1.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_1.html)1[![](images/neuron/mixed3a_4.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_4.html)4[![](images/neuron/mixed3a_41.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_41.html)41[![](images/neuron/mixed3a_192.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_192.html)192[![](images/neuron/mixed3a_36.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_36.html)36[![](images/neuron/mixed3a_40.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_40.html)40[![](images/neuron/mixed3a_103.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_103.html)103[![](images/neuron/mixed3a_213.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_213.html)213[![](images/neuron/mixed3a_10.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_10.html)10[![](images/neuron/mixed3a_35.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_35.html)35[![](images/neuron/mixed3a_221.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_221.html)221[![](images/neuron/mixed3a_193.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_193.html)193[![](images/neuron/mixed3a_158.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_158.html)158[![](images/neuron/mixed3a_73.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_73.html)73[![](images/neuron/mixed3a_74.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_74.html)74[![](images/neuron/mixed3a_177.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_177.html)177[![](images/neuron/mixed3a_97.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_97.html)97[![](images/neuron/mixed3a_141.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_141.html)141

Show all 30 neurons.
Collapse neurons.
These units look for one color in the center, and another (usually opposite) color surrounding it. They are typically much more sensitive to the center color than the surrounding one. In visual neuroscience, center-surround units are classically an extremely low-level feature, but we see them in the later parts of early vision. Compare to earlier  [Color Center-Surround (`conv2d2`)](#group_conv2d2_color_center_surround) and later [Color Center-Surround (`mixed3b`)](#group_mixed3b_color_center_surround).

### [**High-Low Frequency** 6%](#group_mixed3a_high_low_frequency)

[![](images/neuron/mixed3a_110.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_110.html)110[![](images/neuron/mixed3a_180.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_180.html)180[![](images/neuron/mixed3a_153.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_153.html)153[![](images/neuron/mixed3a_106.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_106.html)106[![](images/neuron/mixed3a_112.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_112.html)112[![](images/neuron/mixed3a_186.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_186.html)186[![](images/neuron/mixed3a_132.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_132.html)132[![](images/neuron/mixed3a_136.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_136.html)136[![](images/neuron/mixed3a_117.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_117.html)117[![](images/neuron/mixed3a_113.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_113.html)113[![](images/neuron/mixed3a_108.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_108.html)108[![](images/neuron/mixed3a_70.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_70.html)70[![](images/neuron/mixed3a_86.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_86.html)86[![](images/neuron/mixed3a_88.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_88.html)88[![](images/neuron/mixed3a_160.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_160.html)160

Show all 15 neurons.
Collapse neurons.
These units look for transitions from high-frequency texture to low-frequency. They are primarily used by [boundary detectors (`mixed3b`)](#group_mixed3b_boundary) as an additional cue for a boundary between objects. (Larger scale high-low frequency detectors can be found in `mixed4a` (245, 93, 392, 301), but are not discussed in this article.)

 A detailed article on these is forthcoming.

### [**Brightness Gradient** 6%](#group_mixed3a_brightness_gradient)

[![](images/neuron/mixed3a_216.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_216.html)216[![](images/neuron/mixed3a_127.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_127.html)127[![](images/neuron/mixed3a_22.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_22.html)22[![](images/neuron/mixed3a_182.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_182.html)182[![](images/neuron/mixed3a_162.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_162.html)162[![](images/neuron/mixed3a_25.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_25.html)25[![](images/neuron/mixed3a_249.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_249.html)249[![](images/neuron/mixed3a_15.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_15.html)15[![](images/neuron/mixed3a_28.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_28.html)28[![](images/neuron/mixed3a_59.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_59.html)59[![](images/neuron/mixed3a_29.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_29.html)29[![](images/neuron/mixed3a_196.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_196.html)196[![](images/neuron/mixed3a_206.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_206.html)206[![](images/neuron/mixed3a_18.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_18.html)18[![](images/neuron/mixed3a_247.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_247.html)247

Show all 15 neurons.
Collapse neurons.
These units detect brightness gradients. Among other things they will help detect specularity (shininess), curved surfaces, and the boundary of objects. Compare to earlier [brightness gradients (`conv2d2`)](#group_conv2d2_early_brightness_gradient) and later [brightness gradients (`mixed3b`)](#group_mixed3b_brightness_gradients).

### [**Color Contrast** 5%](#group_mixed3a_color_contrast)

[![](images/neuron/mixed3a_195.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_195.html)195[![](images/neuron/mixed3a_84.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_84.html)84[![](images/neuron/mixed3a_85.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_85.html)85[![](images/neuron/mixed3a_123.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_123.html)123[![](images/neuron/mixed3a_203.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_203.html)203[![](images/neuron/mixed3a_217.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_217.html)217[![](images/neuron/mixed3a_199.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_199.html)199[![](images/neuron/mixed3a_211.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_211.html)211[![](images/neuron/mixed3a_205.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_205.html)205[![](images/neuron/mixed3a_212.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_212.html)212[![](images/neuron/mixed3a_202.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_202.html)202[![](images/neuron/mixed3a_200.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_200.html)200[![](images/neuron/mixed3a_138.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_138.html)138[![](images/neuron/mixed3a_32.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_32.html)32

Show all 14 neurons.
Collapse neurons.
These units look for one color on one side of their receptive field, and another (usually opposite) color on the opposing side. They typically don’t care about the exact position or orientation of the transition. Compare to earlier color contrast ([`conv2d0`](#group_conv2d0_color_contrast), [`conv2d1`](#group_conv2d1_color_contrast), [`conv2d2`](#group_conv2d2_color_contrast)) and later color contrast ([`mixed3b`](#group_mixed3b_color_contrast_gradient)).

### [**Complex Center-Surround** 5%](#group_mixed3a_complex_center_surround)

[![](images/neuron/mixed3a_178.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_178.html)178[![](images/neuron/mixed3a_181.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_181.html)181[![](images/neuron/mixed3a_161.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_161.html)161[![](images/neuron/mixed3a_166.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_166.html)166[![](images/neuron/mixed3a_172.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_172.html)172[![](images/neuron/mixed3a_68.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_68.html)68[![](images/neuron/mixed3a_130.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_130.html)130[![](images/neuron/mixed3a_49.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_49.html)49[![](images/neuron/mixed3a_52.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_52.html)52[![](images/neuron/mixed3a_114.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_114.html)114[![](images/neuron/mixed3a_115.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_115.html)115[![](images/neuron/mixed3a_120.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_120.html)120[![](images/neuron/mixed3a_144.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_144.html)144[![](images/neuron/mixed3a_37.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_37.html)37

Show all 14 neurons.
Collapse neurons.
This is a broad, not very well defined category for center-surround units that detect a pattern or complex texture in their center.

### [**Line Misc.** 5%](#group_mixed3a_line_misc.)

[![](images/neuron/mixed3a_191.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_191.html)191[![](images/neuron/mixed3a_121.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_121.html)121[![](images/neuron/mixed3a_116.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_116.html)116[![](images/neuron/mixed3a_14.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_14.html)14[![](images/neuron/mixed3a_24.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_24.html)24[![](images/neuron/mixed3a_0.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_0.html)0[![](images/neuron/mixed3a_159.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_159.html)159[![](images/neuron/mixed3a_152.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_152.html)152[![](images/neuron/mixed3a_165.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_165.html)165[![](images/neuron/mixed3a_83.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_83.html)83[![](images/neuron/mixed3a_173.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_173.html)173[![](images/neuron/mixed3a_87.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_87.html)87[![](images/neuron/mixed3a_90.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_90.html)90[![](images/neuron/mixed3a_82.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_82.html)82

Show all 14 neurons.
Collapse neurons.
Broad, low confidence organizational category.

### [**Lines** 5%](#group_mixed3a_lines)

[![](images/neuron/mixed3a_227.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_227.html)227[![](images/neuron/mixed3a_75.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_75.html)75[![](images/neuron/mixed3a_146.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_146.html)146[![](images/neuron/mixed3a_69.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_69.html)69[![](images/neuron/mixed3a_169.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_169.html)169[![](images/neuron/mixed3a_57.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_57.html)57[![](images/neuron/mixed3a_154.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_154.html)154[![](images/neuron/mixed3a_187.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_187.html)187[![](images/neuron/mixed3a_27.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_27.html)27[![](images/neuron/mixed3a_134.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_134.html)134[![](images/neuron/mixed3a_150.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_150.html)150[![](images/neuron/mixed3a_240.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_240.html)240[![](images/neuron/mixed3a_101.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_101.html)101[![](images/neuron/mixed3a_176.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_176.html)176

Show all 14 neurons.
Collapse neurons.
Units used to detect extended lines, often further excited by different colors on each side. A few are highly combed line detectors that aren’t obviously such at first glance. The decision to include a unit was often decided by whether it seems to be used by downstream client units as a line detector.

### [**Other Units** 5%](#group_mixed3a_other_units)

[![](images/neuron/mixed3a_38.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_38.html)38[![](images/neuron/mixed3a_43.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_43.html)43[![](images/neuron/mixed3a_58.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_58.html)58[![](images/neuron/mixed3a_67.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_67.html)67[![](images/neuron/mixed3a_190.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_190.html)190[![](images/neuron/mixed3a_109.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_109.html)109[![](images/neuron/mixed3a_122.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_122.html)122[![](images/neuron/mixed3a_128.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_128.html)128[![](images/neuron/mixed3a_142.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_142.html)142[![](images/neuron/mixed3a_143.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_143.html)143[![](images/neuron/mixed3a_155.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_155.html)155[![](images/neuron/mixed3a_170.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_170.html)170[![](images/neuron/mixed3a_179.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_179.html)179[![](images/neuron/mixed3a_184.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_184.html)184

Show all 14 neurons.
Collapse neurons.
Catch-all category for all other units.

### [**Repeating patterns** 5%](#group_mixed3a_repeating_patterns)

[![](images/neuron/mixed3a_237.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_237.html)237[![](images/neuron/mixed3a_31.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_31.html)31[![](images/neuron/mixed3a_17.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_17.html)17[![](images/neuron/mixed3a_20.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_20.html)20[![](images/neuron/mixed3a_39.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_39.html)39[![](images/neuron/mixed3a_126.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_126.html)126[![](images/neuron/mixed3a_124.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_124.html)124[![](images/neuron/mixed3a_156.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_156.html)156[![](images/neuron/mixed3a_98.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_98.html)98[![](images/neuron/mixed3a_105.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_105.html)105[![](images/neuron/mixed3a_230.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_230.html)230[![](images/neuron/mixed3a_228.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_228.html)228

Show all 12 neurons.
Collapse neurons.
This is broad, catch-all category for units that seem to look for repeating local patterns that seem more complex than textures.

### [**Curves** 4%](#group_mixed3a_curves)

[![](images/neuron/mixed3a_81.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_81.html)81[![](images/neuron/mixed3a_104.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_104.html)104[![](images/neuron/mixed3a_92.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_92.html)92[![](images/neuron/mixed3a_145.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_145.html)145[![](images/neuron/mixed3a_95.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_95.html)95[![](images/neuron/mixed3a_163.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_163.html)163[![](images/neuron/mixed3a_171.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_171.html)171[![](images/neuron/mixed3a_71.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_71.html)71[![](images/neuron/mixed3a_147.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_147.html)147[![](images/neuron/mixed3a_189.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_189.html)189[![](images/neuron/mixed3a_137.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_137.html)137

Show all 11 neurons.
Collapse neurons.
These curve detectors detect significantly larger radii curves than their predecessors. They will be refined into more specific, larger curve detectors in the next layer. Compare to earlier [curves (`conv2d2`)](#group_conv2d2_tiny_curves) and later [curves (`mixed3b`)](#group_mixed3b_curves).

See the [full paper on curve detectors](https://distill.pub/2020/circuits/curve-detectors/).

### [**BW vs Color** 4%](#group_mixed3a_bw_vs_color)

[![](images/neuron/mixed3a_214.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_214.html)214[![](images/neuron/mixed3a_208.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_208.html)208[![](images/neuron/mixed3a_201.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_201.html)201[![](images/neuron/mixed3a_223.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_223.html)223[![](images/neuron/mixed3a_210.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_210.html)210[![](images/neuron/mixed3a_197.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_197.html)197[![](images/neuron/mixed3a_222.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_222.html)222[![](images/neuron/mixed3a_204.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_204.html)204[![](images/neuron/mixed3a_220.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_220.html)220

These “black and white” detectors respond to absences of color. Prior to this, color detectors contrast to the opposite hue, but from this point on we’ll see many compare to the absence of color. See also [BW circuit example and discussion](#mixed3a_discussion_BW).

### [**Angles** 3%](#group_mixed3a_angles)

[![](images/neuron/mixed3a_188.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_188.html)188[![](images/neuron/mixed3a_94.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_94.html)94[![](images/neuron/mixed3a_164.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_164.html)164[![](images/neuron/mixed3a_107.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_107.html)107[![](images/neuron/mixed3a_77.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_77.html)77[![](images/neuron/mixed3a_157.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_157.html)157[![](images/neuron/mixed3a_149.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_149.html)149[![](images/neuron/mixed3a_100.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_100.html)100

Units that detect multiple lines, forming angles, triangles and squares. They generally respond to any of the individual lines, and more strongly to them together.

### [**Fur Precursors** 3%](#group_mixed3a_fur_precursors)

[![](images/neuron/mixed3a_46.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_46.html)46[![](images/neuron/mixed3a_47.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_47.html)47[![](images/neuron/mixed3a_26.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_26.html)26[![](images/neuron/mixed3a_63.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_63.html)63[![](images/neuron/mixed3a_80.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_80.html)80[![](images/neuron/mixed3a_23.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_23.html)23[![](images/neuron/mixed3a_16.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_16.html)16

These units are not yet highly selective for fur (they also fire for other high-frequency patterns), but their primary use in the next layer is supporting [fur detection](#group_mixed3b_generic_oriented_fur). At the 224x224 image resolution, individual fur hairs are generally not detectable, but tufts of fur are. These units use Gabor textures to detect those tufts in different orientations. The also detect lower frequency edges or changes in lighting perpendicular to the tufts.

### [**Eyes / Small Circles** 2%](#group_mixed3a_eyes_small_circles)

[![](images/neuron/mixed3a_174.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_174.html)174[![](images/neuron/mixed3a_168.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_168.html)168[![](images/neuron/mixed3a_79.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_79.html)79[![](images/neuron/mixed3a_125.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_125.html)125[![](images/neuron/mixed3a_175.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_175.html)175

We think of eyes as high-level features, but small eye detectors actually form very early. Compare to later [eye detectors (`mixed3b`)](#group_mixed3b_eyes). See also [circuit example and discussion](#mixed3a_discussion_small_circle).

### [**Crosses / Diverging Lines** 2%](#group_mixed3a_crosses_diverging_lines)

[![](images/neuron/mixed3a_91.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_91.html)91[![](images/neuron/mixed3a_185.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_185.html)185[![](images/neuron/mixed3a_64.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_64.html)64[![](images/neuron/mixed3a_118.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_118.html)118

These units seem to respond to lines crossing or to lines diverging from a central point.

### [**Thick Lines** 1%](#group_mixed3a_thick_lines)

[![](images/neuron/mixed3a_140.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_140.html)140[![](images/neuron/mixed3a_78.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_78.html)78[![](images/neuron/mixed3a_89.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_89.html)89

Low confidence organizational category.

### [**Line Ends** 1%](#group_mixed3a_line_ends)

[![](images/neuron/mixed3a_133.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_133.html)133[![](images/neuron/mixed3a_96.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3a_96.html)96

These units detect a line ending or sharply turning. Often used in boundary detection and more complex shape detectors.

---

`mixed3b`
---------

`mixed3b` straddles two levels of abstraction.
On the one hand, it has some quite sophisticated features that don’t really seem like they should be characterized as “early” or “low-level”: object boundary detectors, early head detectors, and more sophisticated part of shape detectors.
On the other hand, it also has many units that still feel quite low-level, such as color center-surround units.

**Boundary detectors:** One of the most striking transitions in `mixed3b` is the formation of boundary detectors.
 When you first look at the feature visualizations and dataset examples,
 you might think these are just another iteration of edge or curve detectors.
 But they are in fact combining a variety of cues to detect boundaries and transitions between objects.
 Perhaps the most important one is the high-low frequency detectors we saw develop at the previous layer.
 Notice that it largely doesn’t care which direction the change in color or frequency is, just that there’s a change.

positive (excitation)
negative (inhibition)

High-low frequency detectors
These detectors vary in orientation, preferring concave vs convex boundaries, and type of foreground.
mixed3b creates boundary detectors that rely on many cues, including changes in frequency, changes in color, and actual edges.
Edges
End of Line
Color Contrasts

 We sometimes find it useful to think about the “goal” of early vision.
 Gradient descent will only create features if they are useful for features in later layers.
 Which later features incentivized the creation of the features we see in early vision?
 These boundary detectors seem to be the “goal” of the [high-low frequency detectors (`mixed3a`)](#group_mixed3a_high_low_frequency) we saw in the previous layer.

**Curve-based Features:**
 Another major theme in this layer is the emergence of more complex and specific shape detectors based on curves.
 These include more sophisticated curves,
 [circles](#group_mixed3b_circles_loops), [S-shapes](#group_mixed3b_curve_shapes), [spirals](#group_mixed3b_curve_shapes),
 [divots](#group_mixed3b_divots), and [“evolutes”](#group_mixed3b_evolute)
 (a term we’ve repurposed to describe units detecting curves facing away from the middle).
 We’ll discuss these in detail in a forthcoming article on curve circuits, but they warrant mention here.

 Conceptually, you can think of the weights as piecing together curve detectors as something like this:

Curve
Circle
Spiral
Evolute

**Fur detectors:** Another interesting (albeit, probably quite specific to the dog focus of ImageNet)
 circuit is the implementation of [“oriented fur detectors”](#group_mixed3b_generic_oriented_fur) which detect fur parting, like hair on one’s head.
 They’re implemented by piecing together [fur precursors (`mixed3a`)](#group_mixed3a_fur_precursors) so that they converge in a particular way.

positive (excitation)
negative (inhibition)

Oriented fur detectors detect fur parting like hair by assembling early fur detectors which detect fur at different angles to coverge at one point. These two are primairly used to create head detectors in the next layer.

 Again, these circuits only scratch the surface of `mixed3b`.
 Since it’s a larger layer with lots of families, we’ll go through a couple particularly interesting and well understood families first:

### [**Boundary** 8%](#group_mixed3b_boundary)

[![](images/neuron/mixed3b_220.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_220.html)220[![](images/neuron/mixed3b_402.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_402.html)402[![](images/neuron/mixed3b_364.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_364.html)364[![](images/neuron/mixed3b_293.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_293.html)293[![](images/neuron/mixed3b_356.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_356.html)356[![](images/neuron/mixed3b_151.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_151.html)151[![](images/neuron/mixed3b_203.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_203.html)203[![](images/neuron/mixed3b_394.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_394.html)394[![](images/neuron/mixed3b_376.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_376.html)376[![](images/neuron/mixed3b_400.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_400.html)400[![](images/neuron/mixed3b_328.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_328.html)328[![](images/neuron/mixed3b_219.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_219.html)219[![](images/neuron/mixed3b_320.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_320.html)320[![](images/neuron/mixed3b_313.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_313.html)313[![](images/neuron/mixed3b_329.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_329.html)329[![](images/neuron/mixed3b_321.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_321.html)321[![](images/neuron/mixed3b_251.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_251.html)251[![](images/neuron/mixed3b_298.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_298.html)298[![](images/neuron/mixed3b_257.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_257.html)257[![](images/neuron/mixed3b_143.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_143.html)143[![](images/neuron/mixed3b_366.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_366.html)366[![](images/neuron/mixed3b_345.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_345.html)345[![](images/neuron/mixed3b_405.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_405.html)405[![](images/neuron/mixed3b_414.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_414.html)414[![](images/neuron/mixed3b_301.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_301.html)301[![](images/neuron/mixed3b_368.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_368.html)368[![](images/neuron/mixed3b_398.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_398.html)398[![](images/neuron/mixed3b_383.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_383.html)383[![](images/neuron/mixed3b_396.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_396.html)396[![](images/neuron/mixed3b_261.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_261.html)261[![](images/neuron/mixed3b_184.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_184.html)184[![](images/neuron/mixed3b_144.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_144.html)144[![](images/neuron/mixed3b_360.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_360.html)360[![](images/neuron/mixed3b_183.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_183.html)183[![](images/neuron/mixed3b_239.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_239.html)239[![](images/neuron/mixed3b_386.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_386.html)386

Show all 36 neurons.
Collapse neurons.
These units use multiple cues to detect the boundaries of objects. They vary in orientation, detecting convex/concave/straight boundaries, and detecting artificial vs fur foregrounds. Cues they rely on include line detectors, high-low frequency detectors, and color contrast.

### [**Proto-Head** 3%](#group_mixed3b_proto_head)

[![](images/neuron/mixed3b_362.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_362.html)362[![](images/neuron/mixed3b_413.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_413.html)413[![](images/neuron/mixed3b_334.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_334.html)334[![](images/neuron/mixed3b_331.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_331.html)331[![](images/neuron/mixed3b_174.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_174.html)174[![](images/neuron/mixed3b_225.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_225.html)225[![](images/neuron/mixed3b_393.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_393.html)393[![](images/neuron/mixed3b_185.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_185.html)185[![](images/neuron/mixed3b_435.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_435.html)435[![](images/neuron/mixed3b_180.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_180.html)180[![](images/neuron/mixed3b_441.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_441.html)441[![](images/neuron/mixed3b_163.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_163.html)163

Show all 12 neurons.
Collapse neurons.
The tiny eye detectors, along with texture detectors for fur, hair and skin developed at the previous layer enable these early head detectors, which will continue to be refined in the next layer.

### [**Generic, Oriented Fur** 2%](#group_mixed3b_generic_oriented_fur)

[![](images/neuron/mixed3b_57.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_57.html)57[![](images/neuron/mixed3b_387.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_387.html)387[![](images/neuron/mixed3b_404.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_404.html)404[![](images/neuron/mixed3b_333.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_333.html)333[![](images/neuron/mixed3b_375.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_375.html)375[![](images/neuron/mixed3b_381.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_381.html)381[![](images/neuron/mixed3b_335.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_335.html)335[![](images/neuron/mixed3b_378.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_378.html)378[![](images/neuron/mixed3b_62.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_62.html)62[![](images/neuron/mixed3b_52.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_52.html)52

We don’t typically think of fur as an oriented feature, but it is. These units detect fur parting in various ways, much like how hair on your head parts.

### [**Curves** 2%](#group_mixed3b_curves)

[![](images/neuron/mixed3b_379.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_379.html)379[![](images/neuron/mixed3b_406.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_406.html)406[![](images/neuron/mixed3b_385.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_385.html)385[![](images/neuron/mixed3b_343.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_343.html)343[![](images/neuron/mixed3b_342.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_342.html)342[![](images/neuron/mixed3b_388.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_388.html)388[![](images/neuron/mixed3b_340.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_340.html)340[![](images/neuron/mixed3b_330.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_330.html)330[![](images/neuron/mixed3b_349.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_349.html)349[![](images/neuron/mixed3b_324.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_324.html)324

The third iteration of curve detectors. They detect larger radii curves than their predecessors, and are the first to not slightly fire for curves rotated 180 degrees. Compare to the earlier [curves (conv2d2)](#group_conv2d2_tiny_curves) and [curves (mixed3a)](#group_mixed3a_curves).

See the [full paper on curve detectors](https://distill.pub/2020/circuits/curve-detectors/).

### [**Divots** 2%](#group_mixed3b_divots)

[![](images/neuron/mixed3b_395.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_395.html)395[![](images/neuron/mixed3b_159.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_159.html)159[![](images/neuron/mixed3b_237.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_237.html)237[![](images/neuron/mixed3b_409.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_409.html)409[![](images/neuron/mixed3b_357.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_357.html)357[![](images/neuron/mixed3b_190.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_190.html)190[![](images/neuron/mixed3b_212.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_212.html)212[![](images/neuron/mixed3b_211.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_211.html)211[![](images/neuron/mixed3b_198.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_198.html)198[![](images/neuron/mixed3b_218.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_218.html)218

Curve-like detectors for sharp corners or bumps.

### [**Square / Grid** 2%](#group_mixed3b_square_grid)

[![](images/neuron/mixed3b_392.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_392.html)392[![](images/neuron/mixed3b_361.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_361.html)361[![](images/neuron/mixed3b_401.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_401.html)401[![](images/neuron/mixed3b_68.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_68.html)68[![](images/neuron/mixed3b_341.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_341.html)341[![](images/neuron/mixed3b_382.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_382.html)382[![](images/neuron/mixed3b_397.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_397.html)397[![](images/neuron/mixed3b_66.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_66.html)66[![](images/neuron/mixed3b_125.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_125.html)125

Units detecting grid patterns.

### [**Brightness Gradients** 1%](#group_mixed3b_brightness_gradients)

[![](images/neuron/mixed3b_0.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_0.html)0[![](images/neuron/mixed3b_317.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_317.html)317[![](images/neuron/mixed3b_136.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_136.html)136[![](images/neuron/mixed3b_455.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_455.html)455[![](images/neuron/mixed3b_417.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_417.html)417[![](images/neuron/mixed3b_469.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_469.html)469

These units detect brightness gradients. This is their third iteration; compare to earlier [brightness gradients (`conv2d2`)](#group_conv2d2_early_brightness_gradient) and [brightness gradients (`mixed3a`)](#group_mixed3a_brightness_gradient).

### [**Eyes** 1%](#group_mixed3b_eyes)

[![](images/neuron/mixed3b_370.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_370.html)370[![](images/neuron/mixed3b_352.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_352.html)352[![](images/neuron/mixed3b_363.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_363.html)363[![](images/neuron/mixed3b_322.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_322.html)322[![](images/neuron/mixed3b_83.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_83.html)83[![](images/neuron/mixed3b_199.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_199.html)199

Again, we continue to see eye detectors quite early in vision. Note that several of these detect larger eyes than the earlier [eye detectors (mixed3a)](#group_mixed3a_eyes_small_circles). In the next layer, we see much larger scale eye detectors again.

### [**Shallow Curves** 1%](#group_mixed3b_shallow_curves)

[![](images/neuron/mixed3b_403.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_403.html)403[![](images/neuron/mixed3b_353.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_353.html)353[![](images/neuron/mixed3b_355.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_355.html)355[![](images/neuron/mixed3b_336.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_336.html)336

Detectors for curves with wider radii than [regular curve detectors](#group_mixed3b_curves).

### [**Curve Shapes** 1%](#group_mixed3b_curve_shapes)

[![](images/neuron/mixed3b_325.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_325.html)325[![](images/neuron/mixed3b_338.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_338.html)338[![](images/neuron/mixed3b_327.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_327.html)327[![](images/neuron/mixed3b_347.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_347.html)347

Simple shapes created by composing curves, such as spirals and S-curves.

### [**Circles / Loops** 1%](#group_mixed3b_circles_loops)

[![](images/neuron/mixed3b_389.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_389.html)389[![](images/neuron/mixed3b_384.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_384.html)384[![](images/neuron/mixed3b_346.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_346.html)346[![](images/neuron/mixed3b_323.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_323.html)323

Piece together curves in a circle or partial circle. Opposite of [evolute](#group_mixed3b_evolute).

### [**Circle Cluster** 1%](#group_mixed3b_circle_cluster)

[![](images/neuron/mixed3b_446.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_446.html)446[![](images/neuron/mixed3b_462.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_462.html)462[![](images/neuron/mixed3b_82.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_82.html)82

Units detecting circles and curves without necesarily requiring spatial coherrence.

### [**Double Curves** 1%](#group_mixed3b_double_curves)

[![](images/neuron/mixed3b_359.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_359.html)359[![](images/neuron/mixed3b_337.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_337.html)337[![](images/neuron/mixed3b_380.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_380.html)380

Weights appear to be two curve detectors added together. Likely best thought of as a polysemantic neuron.

### [**Evolute** 0.2%](#group_mixed3b_evolute)

[![](images/neuron/mixed3b_373.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_373.html)373

Detects curves facing away from the middle. Opposite of [circles](#group_mixed3b_circles_loops). Term repurposed from [mathematical evolutes](https://en.wikipedia.org/wiki/Evolute) which can sometimes be visually similar.

 In addition to the above features, are also a lot of other features which don’t fall into such a neat categorization.
 One frustrating issue is that `mixed3b` has many units that don’t have a simple low-level articulation, but also are not yet very specific to a high-level feature.
 For example, there are units which seem to be developing towards detecting certain animal body parts, but still respond to many other stimuli as well and so are difficult to describe.

### [**Color Center-Surround** 16%](#group_mixed3b_color_center_surround)

[![](images/neuron/mixed3b_285.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_285.html)285[![](images/neuron/mixed3b_451.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_451.html)451[![](images/neuron/mixed3b_208.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_208.html)208[![](images/neuron/mixed3b_122.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_122.html)122[![](images/neuron/mixed3b_93.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_93.html)93[![](images/neuron/mixed3b_75.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_75.html)75[![](images/neuron/mixed3b_46.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_46.html)46[![](images/neuron/mixed3b_294.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_294.html)294[![](images/neuron/mixed3b_44.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_44.html)44[![](images/neuron/mixed3b_247.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_247.html)247[![](images/neuron/mixed3b_91.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_91.html)91[![](images/neuron/mixed3b_14.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_14.html)14[![](images/neuron/mixed3b_10.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_10.html)10[![](images/neuron/mixed3b_271.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_271.html)271[![](images/neuron/mixed3b_60.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_60.html)60[![](images/neuron/mixed3b_80.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_80.html)80[![](images/neuron/mixed3b_84.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_84.html)84[![](images/neuron/mixed3b_70.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_70.html)70[![](images/neuron/mixed3b_202.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_202.html)202[![](images/neuron/mixed3b_422.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_422.html)422[![](images/neuron/mixed3b_48.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_48.html)48[![](images/neuron/mixed3b_436.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_436.html)436[![](images/neuron/mixed3b_65.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_65.html)65[![](images/neuron/mixed3b_300.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_300.html)300[![](images/neuron/mixed3b_105.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_105.html)105[![](images/neuron/mixed3b_34.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_34.html)34[![](images/neuron/mixed3b_121.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_121.html)121[![](images/neuron/mixed3b_424.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_424.html)424[![](images/neuron/mixed3b_457.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_457.html)457[![](images/neuron/mixed3b_186.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_186.html)186[![](images/neuron/mixed3b_23.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_23.html)23[![](images/neuron/mixed3b_479.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_479.html)479[![](images/neuron/mixed3b_89.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_89.html)89[![](images/neuron/mixed3b_283.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_283.html)283[![](images/neuron/mixed3b_22.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_22.html)22[![](images/neuron/mixed3b_124.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_124.html)124[![](images/neuron/mixed3b_6.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_6.html)6[![](images/neuron/mixed3b_9.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_9.html)9[![](images/neuron/mixed3b_50.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_50.html)50[![](images/neuron/mixed3b_5.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_5.html)5[![](images/neuron/mixed3b_71.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_71.html)71[![](images/neuron/mixed3b_59.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_59.html)59[![](images/neuron/mixed3b_182.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_182.html)182[![](images/neuron/mixed3b_87.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_87.html)87[![](images/neuron/mixed3b_308.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_308.html)308[![](images/neuron/mixed3b_428.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_428.html)428[![](images/neuron/mixed3b_109.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_109.html)109[![](images/neuron/mixed3b_141.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_141.html)141[![](images/neuron/mixed3b_12.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_12.html)12[![](images/neuron/mixed3b_474.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_474.html)474[![](images/neuron/mixed3b_112.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_112.html)112[![](images/neuron/mixed3b_192.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_192.html)192[![](images/neuron/mixed3b_2.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_2.html)2[![](images/neuron/mixed3b_177.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_177.html)177[![](images/neuron/mixed3b_249.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_249.html)249[![](images/neuron/mixed3b_281.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_281.html)281[![](images/neuron/mixed3b_284.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_284.html)284[![](images/neuron/mixed3b_30.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_30.html)30[![](images/neuron/mixed3b_27.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_27.html)27[![](images/neuron/mixed3b_255.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_255.html)255[![](images/neuron/mixed3b_53.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_53.html)53[![](images/neuron/mixed3b_432.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_432.html)432[![](images/neuron/mixed3b_475.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_475.html)475[![](images/neuron/mixed3b_79.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_79.html)79[![](images/neuron/mixed3b_67.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_67.html)67[![](images/neuron/mixed3b_25.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_25.html)25[![](images/neuron/mixed3b_351.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_351.html)351[![](images/neuron/mixed3b_420.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_420.html)420[![](images/neuron/mixed3b_152.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_152.html)152[![](images/neuron/mixed3b_26.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_26.html)26[![](images/neuron/mixed3b_193.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_193.html)193[![](images/neuron/mixed3b_448.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_448.html)448[![](images/neuron/mixed3b_153.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_153.html)153[![](images/neuron/mixed3b_164.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_164.html)164[![](images/neuron/mixed3b_113.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_113.html)113[![](images/neuron/mixed3b_216.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_216.html)216[![](images/neuron/mixed3b_259.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_259.html)259

Show all 77 neurons.
Collapse neurons.
These units look for one color in the center, and another color surrounding it. These units likely have many subtleties about the range of hues, texture preferences, and interactions that similar neurons in earlier layers may not have. Note how many units detect the absence (or generic presence) of color, building off of the [black and white detectors](#group_mixed3a_bw_vs_color) in `mixed3a`. Compare to earlier  [Color Center-Surround (`conv2d2`)](#group_conv2d2_color_center_surround) and  [(Color Center-Surround `mixed3a`)](#group_mixed3a_color_center_surround).

### [**Complex Center-Surround** 15%](#group_mixed3b_complex_center_surround)

[![](images/neuron/mixed3b_299.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_299.html)299[![](images/neuron/mixed3b_139.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_139.html)139[![](images/neuron/mixed3b_7.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_7.html)7[![](images/neuron/mixed3b_170.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_170.html)170[![](images/neuron/mixed3b_16.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_16.html)16[![](images/neuron/mixed3b_28.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_28.html)28[![](images/neuron/mixed3b_291.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_291.html)291[![](images/neuron/mixed3b_439.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_439.html)439[![](images/neuron/mixed3b_443.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_443.html)443[![](images/neuron/mixed3b_69.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_69.html)69[![](images/neuron/mixed3b_11.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_11.html)11[![](images/neuron/mixed3b_13.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_13.html)13[![](images/neuron/mixed3b_56.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_56.html)56[![](images/neuron/mixed3b_116.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_116.html)116[![](images/neuron/mixed3b_117.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_117.html)117[![](images/neuron/mixed3b_72.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_72.html)72[![](images/neuron/mixed3b_36.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_36.html)36[![](images/neuron/mixed3b_35.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_35.html)35[![](images/neuron/mixed3b_41.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_41.html)41[![](images/neuron/mixed3b_51.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_51.html)51[![](images/neuron/mixed3b_55.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_55.html)55[![](images/neuron/mixed3b_88.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_88.html)88[![](images/neuron/mixed3b_101.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_101.html)101[![](images/neuron/mixed3b_110.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_110.html)110[![](images/neuron/mixed3b_114.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_114.html)114[![](images/neuron/mixed3b_158.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_158.html)158[![](images/neuron/mixed3b_161.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_161.html)161[![](images/neuron/mixed3b_169.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_169.html)169[![](images/neuron/mixed3b_176.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_176.html)176[![](images/neuron/mixed3b_215.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_215.html)215[![](images/neuron/mixed3b_228.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_228.html)228[![](images/neuron/mixed3b_230.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_230.html)230[![](images/neuron/mixed3b_232.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_232.html)232[![](images/neuron/mixed3b_233.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_233.html)233[![](images/neuron/mixed3b_234.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_234.html)234[![](images/neuron/mixed3b_238.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_238.html)238[![](images/neuron/mixed3b_242.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_242.html)242[![](images/neuron/mixed3b_244.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_244.html)244[![](images/neuron/mixed3b_245.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_245.html)245[![](images/neuron/mixed3b_252.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_252.html)252[![](images/neuron/mixed3b_256.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_256.html)256[![](images/neuron/mixed3b_275.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_275.html)275[![](images/neuron/mixed3b_280.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_280.html)280[![](images/neuron/mixed3b_290.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_290.html)290[![](images/neuron/mixed3b_296.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_296.html)296[![](images/neuron/mixed3b_297.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_297.html)297[![](images/neuron/mixed3b_302.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_302.html)302[![](images/neuron/mixed3b_310.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_310.html)310[![](images/neuron/mixed3b_410.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_410.html)410[![](images/neuron/mixed3b_442.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_442.html)442[![](images/neuron/mixed3b_17.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_17.html)17[![](images/neuron/mixed3b_8.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_8.html)8[![](images/neuron/mixed3b_15.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_15.html)15[![](images/neuron/mixed3b_18.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_18.html)18[![](images/neuron/mixed3b_20.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_20.html)20[![](images/neuron/mixed3b_24.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_24.html)24[![](images/neuron/mixed3b_31.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_31.html)31[![](images/neuron/mixed3b_37.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_37.html)37[![](images/neuron/mixed3b_42.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_42.html)42[![](images/neuron/mixed3b_61.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_61.html)61[![](images/neuron/mixed3b_73.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_73.html)73[![](images/neuron/mixed3b_92.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_92.html)92[![](images/neuron/mixed3b_315.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_315.html)315[![](images/neuron/mixed3b_103.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_103.html)103[![](images/neuron/mixed3b_104.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_104.html)104[![](images/neuron/mixed3b_118.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_118.html)118[![](images/neuron/mixed3b_119.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_119.html)119[![](images/neuron/mixed3b_131.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_131.html)131[![](images/neuron/mixed3b_274.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_274.html)274[![](images/neuron/mixed3b_278.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_278.html)278[![](images/neuron/mixed3b_289.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_289.html)289[![](images/neuron/mixed3b_29.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_29.html)29[![](images/neuron/mixed3b_147.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_147.html)147

Show all 73 neurons.
Collapse neurons.
This is a broad, not very well defined category for center-surround units that detect a pattern or complex texture in their center.

### [**Texture** 9%](#group_mixed3b_texture)

[![](images/neuron/mixed3b_3.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_3.html)3[![](images/neuron/mixed3b_40.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_40.html)40[![](images/neuron/mixed3b_32.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_32.html)32[![](images/neuron/mixed3b_54.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_54.html)54[![](images/neuron/mixed3b_74.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_74.html)74[![](images/neuron/mixed3b_309.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_309.html)309[![](images/neuron/mixed3b_267.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_267.html)267[![](images/neuron/mixed3b_438.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_438.html)438[![](images/neuron/mixed3b_416.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_416.html)416[![](images/neuron/mixed3b_440.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_440.html)440[![](images/neuron/mixed3b_460.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_460.html)460[![](images/neuron/mixed3b_276.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_276.html)276[![](images/neuron/mixed3b_458.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_458.html)458[![](images/neuron/mixed3b_132.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_132.html)132[![](images/neuron/mixed3b_133.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_133.html)133[![](images/neuron/mixed3b_106.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_106.html)106[![](images/neuron/mixed3b_120.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_120.html)120[![](images/neuron/mixed3b_123.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_123.html)123[![](images/neuron/mixed3b_426.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_426.html)426[![](images/neuron/mixed3b_434.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_434.html)434[![](images/neuron/mixed3b_429.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_429.html)429[![](images/neuron/mixed3b_445.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_445.html)445[![](images/neuron/mixed3b_452.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_452.html)452[![](images/neuron/mixed3b_456.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_456.html)456[![](images/neuron/mixed3b_459.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_459.html)459[![](images/neuron/mixed3b_464.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_464.html)464[![](images/neuron/mixed3b_465.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_465.html)465[![](images/neuron/mixed3b_421.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_421.html)421[![](images/neuron/mixed3b_437.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_437.html)437[![](images/neuron/mixed3b_418.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_418.html)418[![](images/neuron/mixed3b_425.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_425.html)425[![](images/neuron/mixed3b_221.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_221.html)221[![](images/neuron/mixed3b_195.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_195.html)195[![](images/neuron/mixed3b_204.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_204.html)204[![](images/neuron/mixed3b_39.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_39.html)39[![](images/neuron/mixed3b_76.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_76.html)76[![](images/neuron/mixed3b_77.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_77.html)77[![](images/neuron/mixed3b_468.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_468.html)468[![](images/neuron/mixed3b_471.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_471.html)471[![](images/neuron/mixed3b_227.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_227.html)227[![](images/neuron/mixed3b_415.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_415.html)415[![](images/neuron/mixed3b_126.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_126.html)126[![](images/neuron/mixed3b_128.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_128.html)128[![](images/neuron/mixed3b_172.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_172.html)172

Show all 44 neurons.
Collapse neurons.
This is a broad, not very well defined category for units that seem to look for simple local structures over a wide receptive field, including mixtures of colors.

### [**Other Units** 9%](#group_mixed3b_other_units)

[![](images/neuron/mixed3b_43.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_43.html)43[![](images/neuron/mixed3b_45.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_45.html)45[![](images/neuron/mixed3b_58.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_58.html)58[![](images/neuron/mixed3b_78.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_78.html)78[![](images/neuron/mixed3b_85.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_85.html)85[![](images/neuron/mixed3b_86.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_86.html)86[![](images/neuron/mixed3b_96.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_96.html)96[![](images/neuron/mixed3b_100.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_100.html)100[![](images/neuron/mixed3b_127.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_127.html)127[![](images/neuron/mixed3b_134.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_134.html)134[![](images/neuron/mixed3b_137.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_137.html)137[![](images/neuron/mixed3b_142.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_142.html)142[![](images/neuron/mixed3b_145.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_145.html)145[![](images/neuron/mixed3b_146.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_146.html)146[![](images/neuron/mixed3b_148.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_148.html)148[![](images/neuron/mixed3b_150.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_150.html)150[![](images/neuron/mixed3b_154.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_154.html)154[![](images/neuron/mixed3b_179.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_179.html)179[![](images/neuron/mixed3b_181.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_181.html)181[![](images/neuron/mixed3b_187.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_187.html)187[![](images/neuron/mixed3b_188.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_188.html)188[![](images/neuron/mixed3b_207.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_207.html)207[![](images/neuron/mixed3b_213.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_213.html)213[![](images/neuron/mixed3b_214.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_214.html)214[![](images/neuron/mixed3b_222.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_222.html)222[![](images/neuron/mixed3b_231.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_231.html)231[![](images/neuron/mixed3b_235.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_235.html)235[![](images/neuron/mixed3b_240.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_240.html)240[![](images/neuron/mixed3b_253.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_253.html)253[![](images/neuron/mixed3b_265.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_265.html)265[![](images/neuron/mixed3b_266.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_266.html)266[![](images/neuron/mixed3b_268.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_268.html)268[![](images/neuron/mixed3b_273.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_273.html)273[![](images/neuron/mixed3b_306.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_306.html)306[![](images/neuron/mixed3b_350.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_350.html)350[![](images/neuron/mixed3b_354.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_354.html)354[![](images/neuron/mixed3b_358.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_358.html)358[![](images/neuron/mixed3b_371.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_371.html)371[![](images/neuron/mixed3b_391.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_391.html)391[![](images/neuron/mixed3b_399.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_399.html)399[![](images/neuron/mixed3b_411.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_411.html)411[![](images/neuron/mixed3b_433.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_433.html)433

Show all 42 neurons.
Collapse neurons.
Units that don’t fall in any other category.

### [**Color Contrast/Gradient** 5%](#group_mixed3b_color_contrast_gradient)

[![](images/neuron/mixed3b_4.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_4.html)4[![](images/neuron/mixed3b_217.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_217.html)217[![](images/neuron/mixed3b_450.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_450.html)450[![](images/neuron/mixed3b_191.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_191.html)191[![](images/neuron/mixed3b_287.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_287.html)287[![](images/neuron/mixed3b_49.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_49.html)49[![](images/neuron/mixed3b_196.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_196.html)196[![](images/neuron/mixed3b_473.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_473.html)473[![](images/neuron/mixed3b_430.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_430.html)430[![](images/neuron/mixed3b_305.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_305.html)305[![](images/neuron/mixed3b_447.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_447.html)447[![](images/neuron/mixed3b_277.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_277.html)277[![](images/neuron/mixed3b_165.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_165.html)165[![](images/neuron/mixed3b_279.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_279.html)279[![](images/neuron/mixed3b_226.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_226.html)226[![](images/neuron/mixed3b_303.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_303.html)303[![](images/neuron/mixed3b_224.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_224.html)224[![](images/neuron/mixed3b_269.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_269.html)269[![](images/neuron/mixed3b_264.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_264.html)264[![](images/neuron/mixed3b_189.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_189.html)189[![](images/neuron/mixed3b_156.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_156.html)156[![](images/neuron/mixed3b_463.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_463.html)463[![](images/neuron/mixed3b_270.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_270.html)270[![](images/neuron/mixed3b_272.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_272.html)272

Show all 24 neurons.
Collapse neurons.
Units which respond to different colors on each side. These units look for one color in the center, and another color surrounding it. These units likely have many subtleties about the range of hues, texture preferences, and interactions that similar neurons in earlier layers may not have. Compare to earlier color contrast ([`conv2d0`](#group_conv2d0_color_contrast), [`conv2d1`](#group_conv2d1_color_contrast), [`conv2d2`](#group_conv2d2_color_contrast), [`mixed3a`](group_mixed3a_color_contrast)).

### [**Texture Contrast** 3%](#group_mixed3b_texture_contrast)

[![](images/neuron/mixed3b_319.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_319.html)319[![](images/neuron/mixed3b_155.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_155.html)155[![](images/neuron/mixed3b_201.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_201.html)201[![](images/neuron/mixed3b_171.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_171.html)171[![](images/neuron/mixed3b_178.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_178.html)178[![](images/neuron/mixed3b_197.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_197.html)197[![](images/neuron/mixed3b_260.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_260.html)260[![](images/neuron/mixed3b_412.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_412.html)412[![](images/neuron/mixed3b_248.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_248.html)248[![](images/neuron/mixed3b_250.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_250.html)250[![](images/neuron/mixed3b_241.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_241.html)241[![](images/neuron/mixed3b_390.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_390.html)390

Show all 12 neurons.
Collapse neurons.
Units that detect one texture on one side and a different texture on the other.

### [**Other Fur** 2%](#group_mixed3b_other_fur)

[![](images/neuron/mixed3b_472.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_472.html)472[![](images/neuron/mixed3b_476.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_476.html)476[![](images/neuron/mixed3b_477.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_477.html)477[![](images/neuron/mixed3b_453.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_453.html)453[![](images/neuron/mixed3b_454.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_454.html)454[![](images/neuron/mixed3b_427.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_427.html)427[![](images/neuron/mixed3b_449.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_449.html)449[![](images/neuron/mixed3b_467.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_467.html)467[![](images/neuron/mixed3b_64.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_64.html)64[![](images/neuron/mixed3b_129.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_129.html)129

Units which seem to detect fur but, unlike the oriented fur detectors, don’t seem to detect it parting in a particular way. Many of these seem to prefer a particular fur pattern.

### [**Lines** 2%](#group_mixed3b_lines)

[![](images/neuron/mixed3b_377.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_377.html)377[![](images/neuron/mixed3b_326.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_326.html)326[![](images/neuron/mixed3b_95.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_95.html)95[![](images/neuron/mixed3b_38.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_38.html)38[![](images/neuron/mixed3b_307.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_307.html)307[![](images/neuron/mixed3b_1.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_1.html)1[![](images/neuron/mixed3b_19.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_19.html)19[![](images/neuron/mixed3b_209.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_209.html)209[![](images/neuron/mixed3b_210.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_210.html)210

Units which seem, to a significant extent, to detect a line. Many seem to have additional, more complex behavior.

### [**Cross / Corner Divergence** 2%](#group_mixed3b_cross_corner_divergence)

[![](images/neuron/mixed3b_108.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_108.html)108[![](images/neuron/mixed3b_47.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_47.html)47[![](images/neuron/mixed3b_339.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_339.html)339[![](images/neuron/mixed3b_344.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_344.html)344[![](images/neuron/mixed3b_374.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_374.html)374[![](images/neuron/mixed3b_99.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_99.html)99[![](images/neuron/mixed3b_369.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_369.html)369[![](images/neuron/mixed3b_236.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_236.html)236[![](images/neuron/mixed3b_408.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_408.html)408

Units detecting lines crossing or diverging from a center point. Some are early predecessors for 3D corner detection.

### [**Pattern** 2%](#group_mixed3b_pattern)

[![](images/neuron/mixed3b_157.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_157.html)157[![](images/neuron/mixed3b_431.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_431.html)431[![](images/neuron/mixed3b_444.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_444.html)444[![](images/neuron/mixed3b_311.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_311.html)311[![](images/neuron/mixed3b_470.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_470.html)470[![](images/neuron/mixed3b_33.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_33.html)33[![](images/neuron/mixed3b_115.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_115.html)115[![](images/neuron/mixed3b_316.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_316.html)316[![](images/neuron/mixed3b_372.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_372.html)372

Low confidence category.

### [**Bumps** 2%](#group_mixed3b_bumps)

[![](images/neuron/mixed3b_167.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_167.html)167[![](images/neuron/mixed3b_206.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_206.html)206[![](images/neuron/mixed3b_312.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_312.html)312[![](images/neuron/mixed3b_292.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_292.html)292[![](images/neuron/mixed3b_194.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_194.html)194[![](images/neuron/mixed3b_140.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_140.html)140[![](images/neuron/mixed3b_223.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_223.html)223[![](images/neuron/mixed3b_254.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_254.html)254

Low confidence category.

### [**Double Boundary** 1%](#group_mixed3b_double_boundary)

[![](images/neuron/mixed3b_318.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_318.html)318[![](images/neuron/mixed3b_332.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_332.html)332[![](images/neuron/mixed3b_286.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_286.html)286[![](images/neuron/mixed3b_258.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_258.html)258[![](images/neuron/mixed3b_229.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_229.html)229[![](images/neuron/mixed3b_138.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_138.html)138[![](images/neuron/mixed3b_314.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_314.html)314

Units that detect boundary transitions on two sides, with a ‘foreground’ texture in the middle.

### [**Bar / Line-Like** 1%](#group_mixed3b_bar_line_like)

[![](images/neuron/mixed3b_81.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_81.html)81[![](images/neuron/mixed3b_97.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_97.html)97[![](images/neuron/mixed3b_98.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_98.html)98[![](images/neuron/mixed3b_107.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_107.html)107[![](images/neuron/mixed3b_282.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_282.html)282[![](images/neuron/mixed3b_288.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_288.html)288[![](images/neuron/mixed3b_295.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_295.html)295

Low confidence category.

### [**Boundary Misc** 1%](#group_mixed3b_boundary_misc)

[![](images/neuron/mixed3b_149.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_149.html)149[![](images/neuron/mixed3b_130.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_130.html)130[![](images/neuron/mixed3b_168.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_168.html)168[![](images/neuron/mixed3b_243.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_243.html)243[![](images/neuron/mixed3b_246.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_246.html)246[![](images/neuron/mixed3b_160.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_160.html)160[![](images/neuron/mixed3b_162.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_162.html)162

Boundary-related units we didn’t know what else to do with.

### [**Star** 1%](#group_mixed3b_star)

[![](images/neuron/mixed3b_263.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_263.html)263[![](images/neuron/mixed3b_262.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_262.html)262[![](images/neuron/mixed3b_205.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_205.html)205[![](images/neuron/mixed3b_135.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_135.html)135[![](images/neuron/mixed3b_94.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_94.html)94[![](images/neuron/mixed3b_304.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_304.html)304

Low confidence category.

### [**Line Grad** 1%](#group_mixed3b_line_grad)

[![](images/neuron/mixed3b_21.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_21.html)21[![](images/neuron/mixed3b_63.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_63.html)63[![](images/neuron/mixed3b_102.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_102.html)102[![](images/neuron/mixed3b_423.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_423.html)423[![](images/neuron/mixed3b_175.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_175.html)175

Low confidence category.

### [**Scales** 1%](#group_mixed3b_scales)

[![](images/neuron/mixed3b_461.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_461.html)461[![](images/neuron/mixed3b_466.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_466.html)466[![](images/neuron/mixed3b_90.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_90.html)90[![](images/neuron/mixed3b_478.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_478.html)478[![](images/neuron/mixed3b_419.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_419.html)419

We don’t really understand these units.

### [**Curves misc.** 1%](#group_mixed3b_curves_misc.)

[![](images/neuron/mixed3b_348.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_348.html)348[![](images/neuron/mixed3b_407.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_407.html)407[![](images/neuron/mixed3b_365.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_365.html)365[![](images/neuron/mixed3b_367.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_367.html)367

Low confidence organizational category.

### [**Shiny** 0.4%](#group_mixed3b_shiny)

[![](images/neuron/mixed3b_173.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_173.html)173[![](images/neuron/mixed3b_200.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_200.html)200

Units that seem to detect shiny, specular surfaces.

### [**Pointy** 0.4%](#group_mixed3b_pointy)

[![](images/neuron/mixed3b_166.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_166.html)166[![](images/neuron/mixed3b_111.jpg)](https://storage.googleapis.com/distill-circuits/inceptionv1-weight-explorer/mixed3b_111.html)111

Low confidence category.

---

Conclusion
----------

 The goal of this essay was to give a high-level overview of our present understanding of early vision in InceptionV1.
 Every single feature discussed in this article is a potential topic of deep investigation.
 For example, are curve detectors really curve detectors? What types of curves do they fire for? How do they behave on various edge cases? How are they built?
 Over the coming articles, we’ll do deep dives rigorously investigating these questions for a few features, starting with curves.

 Our investigation into early vision has also left us with many broader open questions.
 To what extent do these feature families reflect fundamental clusters in features,
 versus a taxonomy that might be helpful to humans but is ultimately somewhat arbitrary?
 Is there a better taxonomy, or another way to understand the space of features?
 Why do features often seem to form in families?
 To what extent do the same features families form across models?
 Is there a “periodic table of low-level visual features”, in some sense?
 To what extent do later features admit a similar taxonomy?
 We think these could be interesting questions for future work.

![](images/multiple-pages.svg)

 This article is part of the [Circuits thread](/2020/circuits/), a collection of short articles and commentary by an open scientific collaboration delving into the inner workings of neural networks.

[Zoom In: An Introduction to Circuits](/2020/circuits/zoom-in/)
[Curve Detectors](/2020/circuits/curve-detectors/)

An Overview of Early Vision in InceptionV1

Chris Olah
Nick Cammarata
Ludwig Schubert
Gabriel Goh
Michael Petrov
Shan Carter

Summary: An overview of all the neurons in the first five layers of InceptionV1, organized into a taxonomy of 'neuron groups.'

Going deeper with convolutions

Network dissection: Quantifying interpretability of deep visual representations

Visualizing higher-layer features of a deep network

Feature Visualization

Deep inside convolutional networks: Visualising image classification models and saliency maps

Deep neural networks are easily fooled: High confidence predictions for unrecognizable images

Inceptionism: Going deeper into neural networks

Plug & play generative networks: Conditional iterative generation of images in latent space

The Building Blocks of Interpretability

TensorFlow-Slim: A lightweight library for defining, training and evaluating complex models in TensorFlow