If X has symmetries, the derivative may is not just a product of identity matrices, but can be more complicated.
See See 2.8.2 Symmetric in the Cookbook: https://www2.imm.dtu.dk/pubdb/edoc/imm3274.pdf
See also how https://www.matrixcalculus.org/ handles this.
Right now we always take derivatives as if the "wrt" is not symmetric.
If
Xhas symmetries, the derivative may is not just a product of identity matrices, but can be more complicated.See See 2.8.2 Symmetric in the Cookbook: https://www2.imm.dtu.dk/pubdb/edoc/imm3274.pdf
See also how https://www.matrixcalculus.org/ handles this.
Right now we always take derivatives as if the "wrt" is not symmetric.