ValueError: Attempting to unscale FP16 gradients.

[MilesQLi]

The command is
~/repos/Open-Assistant/model/model_training$ python trainer_sft.py --configs galactica-125m webgpt_dataset_only

Python 3.10, libraries installed as instructed.

error_log.txt

[jeangeezhang]

I got same issue
Python == 3.10
peft == 0.3.0
torch == 1.13.1

[andreaskoepf]

Please try adding dtype: float32 to the model configuration.