#### Grad-CAM visualization of any VisionEncoderDecoder model # Step 1: Open /pytorch_grad_cam folder and make sure that in init.py all the CAM version is imported as the class name not the python file. For example from pytorch_grad_cam.grad_cam import GradCAM because when in the main python code (Grad_CAM_Visualization.py) we want to import every Class directly. # Step2: Open the main Grad-CAM code: Grad_CAM_Visualization.py and edit the following function according to your model. # "def reshape_transform(tensor, height=14, width=14): result = tensor[:, 1:, :].reshape(tensor.size(0), height, width, tensor.size(2)) result = result.transpose(2, 3).transpose(1, 2) # return result" here as the resized image tensor was [150,528] which should be equivalent to the reshaped transform of [1,14,14,768] ## The error message should be like this if any mismatch: RuntimeError: shape '[1, 16, 16, 768]' is invalid for input of size 150528 # Step 3: Choose your desired model from (DeIT_Base16_Pretrained with ImageNeT, Customized VisionTransformer, Dino_Base16_Pretrained with ImageNeT, My customized DeiT-CXR model, My customized EfficientNet model, and ##VisionEncoderDecoder Model) # Step 4: Open base_cam.py file and go to the "forward" function of Class BaseCAM. Write extra line "outputs = outputs.pooler_output" for ##VisionEncoderDecoder Model as we need to take the tensor of pooler_output of the model configuration. Follow the comment line as well. # Step 5: Then follow the comments in the Grad_CAM_Visualization.py: use model.encoder instead of model for ## VisionEncoderDecoder Model use different target_layers for different model target_layers = [model.encoder.encoder.layer[-1].layernorm_before] for ## VisionEncoderDecoder Model # Step 6: Change the image_path and output_path accordingly # Step 7: Run python Grad_CAM_Visualization.py --use-cuda --image-path "directory/image_path" --method "any grad-cam method defined in the code"