torch.nn.modules.module.ModuleAttributeError: 'ResidualAttentionBlock' object has no attribute 'attn_probs'

[TongLi97]

Execute line of code:
num_tokens = image_attn_blocks[0].attn_probs.shape[-1]

An error message appears:
torch.nn.modules.module.ModuleAttributeError: 'ResidualAttentionBlock' object has no attribute 'attn_probs'

What problem causes this?

[OHaiYo-lzy]

have the same problem here

[che011]

If you're using the original openai's clip, do use this git repo's clip folder. In their clip/model.py line 184 they defined a new attribute attn_probs that the original clip repo did not have. Hope this helps!

[asPagurus]

You may git clone current CLIP (place it instead of CLIP directory inside this module) and patch it with attached patch-file.
There are new file inside clip directory and some addition in module.py
But good result you may get only for old models - for example for ViT-B/32.
New models show many noise(( and I don't know why (for now)
If you want run notebook with clip examples you have to copy images from current CLIP folder to new
patch0.patch

[Marverlises]

If you're using the original openai's clip, do use this git repo's clip folder. In their clip/model.py line 184 they defined a new attribute attn_probs that the original clip repo did not have. Hope this helps!

From this-https://github.com/openai/CLIP/blob/main/clip/model.py ? I didn't find the replaced one, can you tell in detail? thanks

[che011]

Use this repo’s clip at this link: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP

The model file I was mentioning would be here: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP/clip/model.py

[Marverlises]

Use this repo’s clip at this link: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP

The model file I was mentioning would be here: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP/clip/model.py

ok thank u!

[zqs010908]

I used clip at this link: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP
but an error message appears:
AttributeError: 'NoneType' object has no attribute 'shape'

[HUOUO7]

I used clip at this link: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP but an error message appears: AttributeError: 'NoneType' object has no attribute 'shape'

I have the same problem, have you solved it?

[Marverlises]

I used clip at this link: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP but an error message appears: AttributeError: 'NoneType' object has no attribute 'shape'

I have the same problem, have you solved it?

Use this repo’s clip at this link: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP

The model file I was mentioning would be here: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP/clip/model.py

[HUOUO7]

I used clip at this link: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP but an error message appears: AttributeError: 'NoneType' object has no attribute 'shape'

I have the same problem, have you solved it?

Use this repo’s clip at this link: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP

The model file I was mentioning would be here: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP/clip/model.py

But after the modification, when I run the following code, an error is reported: TypeError: forward() got an unexpected keyword argument 'attention_probs_forward_hook', Is my pytorch version not new? Do you know why?

self.attn(x, x, x, need_weights=False, attn_mask=self.attn_mask, attention_probs_forward_hook=self.set_attn_probs,
                         attention_probs_backwards_hook=self.set_attn_grad)[0]

[Marverlises]

I used clip at this link: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP but an error message appears: AttributeError: 'NoneType' object has no attribute 'shape'

I have the same problem, have you solved it?

Use this repo’s clip at this link: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP
The model file I was mentioning would be here: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP/clip/model.py

But after the modification, when I run the following code, an error is reported: TypeError: forward() got an unexpected keyword argument 'attention_probs_forward_hook', Is my pytorch version not new? Do you know why?

self.attn(x, x, x, need_weights=False, attn_mask=self.attn_mask, attention_probs_forward_hook=self.set_attn_probs,
                         attention_probs_backwards_hook=self.set_attn_grad)[0]

I remember I also encountered the same issue when I was doing a project few months ago. You need to donwload this file:

auxilary.py and use the MultiheadAttention module from this file.

[HUOUO7]

I used clip at this link: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP but an error message appears: AttributeError: 'NoneType' object has no attribute 'shape'

I have the same problem, have you solved it?

Use this repo’s clip at this link: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP
The model file I was mentioning would be here: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP/clip/model.py

But after the modification, when I run the following code, an error is reported: TypeError: forward() got an unexpected keyword argument 'attention_probs_forward_hook', Is my pytorch version not new? Do you know why?

self.attn(x, x, x, need_weights=False, attn_mask=self.attn_mask, attention_probs_forward_hook=self.set_attn_probs,
                         attention_probs_backwards_hook=self.set_attn_grad)[0]

I remember I also encountered the same issue when I was doing a project few months ago. You need to donwload this file:

auxilary.py and use the MultiheadAttention module from this file.

Thank you very much for your reminder, you are very kind, but I have a new error:
RuntimeError: cannot register a hook on a tensor that doesn't require gradient.
I wonder if it is because I set model.eval() before running the model? Have you encountered this error?

[insaneyilin]

I used clip at this link: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP but an error message appears: AttributeError: 'NoneType' object has no attribute 'shape'

I have the same problem, have you solved it?

Use this repo’s clip at this link: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP
The model file I was mentioning would be here: https://github.com/hila-chefer/Transformer-MM-Explainability/blob/main/CLIP/clip/model.py

But after the modification, when I run the following code, an error is reported: TypeError: forward() got an unexpected keyword argument 'attention_probs_forward_hook', Is my pytorch version not new? Do you know why?

self.attn(x, x, x, need_weights=False, attn_mask=self.attn_mask, attention_probs_forward_hook=self.set_attn_probs,
                         attention_probs_backwards_hook=self.set_attn_grad)[0]

I remember I also encountered the same issue when I was doing a project few months ago. You need to donwload this file:
auxilary.py and use the MultiheadAttention module from this file.

Thank you very much for your reminder, you are very kind, but I have a new error: RuntimeError: cannot register a hook on a tensor that doesn't require gradient. I wonder if it is because I set model.eval() before running the model? Have you encountered this error?

I encountered the same issue, and here's how I resolved it.

In auxilary.py at line 248, I modified the code as follows:

    # Use hooks for the attention weights if necessary
    # Only apply the hook when `attn_output_weights.requires_grad` is True
    if attn_output_weights.requires_grad and attention_probs_forward_hook is not None and attention_probs_backwards_hook is not None:
        attention_probs_forward_hook(attn_output_weights)
        attn_output_weights.register_hook(attention_probs_backwards_hook)

This ensures that the hooks are only registered when gradients are required — which avoids the RuntimeError related to registering hooks on non-differentiable tensors.