Skip to content

Bias not taken into account in compute_Linear_madd and compute_Linear_flops? #45

@atndel

Description

@atndel

In compute_madds.py, for compute_Linear_madd, the number of additions computed is computed this way :

add = num_in_features - 1.

I guess it comes from the additions of the dot product between the input vector and the weights vector. But if the bias parameter of the torch.nn.Linear layer is True, an additional addition will be performed with that bias.So the correct formula should be
add = num_in_features, in case there is a bias.

There is the same problem with compute_Linear_flops in compute_flops,
inp.size()[1] * out.size()[1]
should be replaced by
(inp.size()[1]+1) * out.size()[1], if bias is True.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions