Question about backward() with create_graph=True warning

[Matthew-Vandergrift]

Hello,

I am using hessain().eigenvalues to compute the top eigenvalue of the training data Hessian of my PyTorch Model but I am getting the following python UserWarning.

UserWarning: Using backward() with create_graph=True will create a reference cycle between the parameter and its gradient which can cause a memory leak. We recommend using autograd.grad when creating the graph to avoid this. If you have to use this function, make sure to reset the .grad fields of your parameters to None after use to break the cycle and avoid the leak. (Triggered internally at ../torch/csrc/autograd/engine.cpp:1203.)
  return Variable._execution_engine.run_backward(  # Calls into the C++ engine to run the backward pass

I was wondering if anyone had encountered this issue and knew of potential reasons it might be coming up? Additionally I noticed this error appearing in a forked repo https://github.com/tabtoyou/3D-PyHessian/blob/master/3D_Visualization_TwoModels.ipynb if this is helpful at all. The code I am using which triggers this issue is,

hessian_comp = hessian(model, nn.BCELoss(), data=(X_train, y_train), cuda=True)
top_eigenvalues, top_eigenvector = hessian_comp.eigenvalues(top_n=1)
print(top_eigenvalues)

Where my model and training data are all already stored as cuda tensors.

Thanks for any possible help, I appreciate it a ton! I can share more code since this is not exactly a reproducible example if that would be helpful.