Pytorch-transformer and Allennlp Compatibility

[heeh]

As the authors mentioned in README.md,
pytorch-transformers 1.20 is not compatible with the specified branch in environment.yml.
I tried:

• Downgrading pytorch-transformers to 1.10
• install master branch of allennlp with pytorch-transformers 1.20

However, I couldn't get this working.
Has anyone been able to run the basic model(Roberta) recently?

[kernelmachine]

Hi there, sorry about the delay. what sort of errors are you getting?

[wooogler]

Same here. environment.yml is not working right now....

[nihirv]

Late response, but in your environment.yml file, change the allennlp line to:
- allennlp==0.9.1.dev20200218

[mikeleatila]

Hi, I'm facing this issue too. Can you @ALL share your thoughts on this? Many thanks

[nihirv]

@mikeleatila Did my comment not fix it for you...? You obviously have to reinstall the package and/or environment

[mikeleatila]

@nihirv

Thank you for your reply. Actually I was following the very same advice w.r.t the conda environment and the scripts' executions, i.e. by running firstly

python -m scripts.download_model
--model allenai/dsp_roberta_base_dapt_cs_tapt_citation_intent_1688
--serialization_dir $(pwd)/pretrained_models/dsp_roberta_base_dapt_cs_tapt_citation_intent_1688

and then

python -m scripts.train
--config training_config/classifier.jsonnet
--serialization_dir model_logs/citation-intent-dapt-dapt
--hyperparameters ROBERTA_CLASSIFIER_SMALL
--dataset citation_intent
--model $(pwd)/pretrained_models/dsp_roberta_base_dapt_cs_tapt_citation_intent_1688
--device 0
--perf +f1
--evaluate_on_test

but I am always getting this error:

/home/mikeleatila/anaconda3/envs/domains/bin/python /home/mikeleatila/dont_stop_pretraining_master/scripts/train.py --config training_config/classifier.jsonnet --serialization_dir model_logs/citation-intent-dapt-dapt --hyperparameters ROBERTA_CLASSIFIER_SMALL --dataset citation_intent --model /home/mikeleatila/dont_stop_pretraining_master/pretrained_models/dsp_roberta_base_dapt_cs_tapt_citation_intent_1688 --device 0 --perf +f1 --evaluate_on_test
2022-11-23 23:59:51,838 - INFO - transformers.file_utils - PyTorch version 1.13.0 available.
2022-11-23 23:59:52,445 - INFO - pytorch_pretrained_bert.modeling - Better speed can be achieved with apex installed from https://www.github.com/nvidia/apex .
Traceback (most recent call last):
File "/home/mikeleatila/anaconda3/envs/domains/bin/allennlp", line 8, in
sys.exit(run())
File "/home/mikeleatila/anaconda3/envs/domains/lib/python3.7/site-packages/allennlp/run.py", line 18, in run
main(prog="allennlp")
File "/home/mikeleatila/anaconda3/envs/domains/lib/python3.7/site-packages/allennlp/commands/init.py", line 92, in main
import_module_and_submodules(package_name)
File "/home/mikeleatila/anaconda3/envs/domains/lib/python3.7/site-packages/allennlp/common/util.py", line 462, in import_module_and_submodules
import_module_and_submodules(subpackage)
File "/home/mikeleatila/anaconda3/envs/domains/lib/python3.7/site-packages/allennlp/common/util.py", line 462, in import_module_and_submodules
import_module_and_submodules(subpackage)
File "/home/mikeleatila/anaconda3/envs/domains/lib/python3.7/site-packages/allennlp/common/util.py", line 462, in import_module_and_submodules
import_module_and_submodules(subpackage)
File "/home/mikeleatila/anaconda3/envs/domains/lib/python3.7/site-packages/allennlp/common/util.py", line 451, in import_module_and_submodules
module = importlib.import_module(package_name)
File "/home/mikeleatila/anaconda3/envs/domains/lib/python3.7/importlib/init.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 1006, in _gcd_import
File "", line 983, in _find_and_load
File "", line 967, in _find_and_load_unlocked
File "", line 677, in _load_unlocked
File "", line 728, in exec_module
File "", line 219, in _call_with_frames_removed
File "/home/mikeleatila/dont_stop_pretraining_master/dont_stop_pretraining/modules/seq2vec_encoders/cls_pooler.py", line 13, in
class CLSPooler(Seq2VecEncoder):
File "/home/mikeleatila/anaconda3/envs/domains/lib/python3.7/site-packages/allennlp/common/registrable.py", line 123, in add_subclass_to_registry
raise ConfigurationError(message)
allennlp.common.checks.ConfigurationError: Cannot register cls_pooler as Seq2VecEncoder; name already in use for ClsPooler
Traceback (most recent call last):
File "/home/mikeleatila/dont_stop_pretraining_master/scripts/train.py", line 142, in
main()
File "/home/mikeleatila/dont_stop_pretraining_master/scripts/train.py", line 139, in main
subprocess.run(" ".join(allennlp_command), shell=True, check=True)
File "/home/mikeleatila/anaconda3/envs/domains/lib/python3.7/subprocess.py", line 512, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command 'allennlp train --include-package dont_stop_pretraining training_config/classifier.jsonnet -s model_logs/citation-intent-dapt-dapt' returned non-zero exit status 1.

Process finished with exit code 1

Many thanks in advance!

[nihirv]

Oh right - that's a different error.

In dont_stop_pretraining/modules/seq2vec_encoders/cls_pooler.py, you should see a class ClsPooler. Above that class there's a decorator which registers the name cls_pooler. Change that name to cls_pooler_x. Then, in the training_config/classifier.jsonnetfile, change the instances ofcls_poolertocls_pooler_x`. That should fix the error you're getting.

(I'm recalling this from memory, but I believe these are the right files and steps to follow)

[mikeleatila]

@nihirv Thanks a lot, that worked :)

[mikeleatila]

@nihirv Do you know how to fix this one btw? #41