Monkey V4 Data Driven + Attention Readout Model Card
Neural Encoding model for Macaque V4. The model is a combination of a data driven core and an attention readout layer.
Model Details
Model Description
This model is a combination of a data driven core and an attention readout layer. The data driven core is a convolutional neural network and the attention readout layer is a multihead attention layer with each head trained to predict the firing rates of a neuron in Macaque V4.
Model Sources
For research purposes, we recommend our nnvision Github repository (https://github.com/sinzlab/nnvision), which contains the code for the model defintions and training.
- Repository: https://github.com/sinzlab/nnvision
- Paper: https://www.biorxiv.org/content/10.1101/2023.05.18.541176v1
Intended Use
The model is intended for research purposes only.
Model Use
The model can be used to predict the firing rates of neurons in Macaque V4 given an image.
nnvision
The model can be used in Python with the nnvision package.
import torch
from nnvision.models.trained_models.v4_data_driven import v4_multihead_attention_ensemble_model
input_image = torch.rand(1, 100, 100)
firing_rate = v4_multihead_attention_ensemble_model(input_image, data_key="all_sessions")
energy-guided diffusion
The model can be used in Python with the energy-guided-diffusion package.
from egg.models import models
model = models['data_driven']['train']