When training on my custom dataset, I noticed that at the beginning of training, loss_ce becomes extremely large (around 160), while all other loss components remain below 10. This causes the overall training to be dominated by the classification loss and results in poor final model performance.
I checked the implementation and found that the final loss value in sigmoid_focal_loss is scaled by a factor of 50. Could you please explain the reason for this scaling? Is this multiplier necessary, and how should I adjust it (or other settings) to achieve better training stability and performance on my dataset?
Any guidance would be greatly appreciated. Thanks in advance!
When training on my custom dataset, I noticed that at the beginning of training,
loss_cebecomes extremely large (around 160), while all other loss components remain below 10. This causes the overall training to be dominated by the classification loss and results in poor final model performance.I checked the implementation and found that the final loss value in
sigmoid_focal_lossis scaled by a factor of 50. Could you please explain the reason for this scaling? Is this multiplier necessary, and how should I adjust it (or other settings) to achieve better training stability and performance on my dataset?Any guidance would be greatly appreciated. Thanks in advance!