Hi, thanks for sharing the code. i got into trouble when trying the runing the mmlu_hf task.

as i dived into the code, i found pkl and npy is needed:
|
def load_mmlu_multi_run(model_names, dataset_name): |
|
data_dir = os.path.join(RESULT_DIR, "mmlu") |
|
# find number of samples for each model |
|
data = [] |
|
choices = ["A", "B", "C", "D"] |
|
for model_n in model_names: |
|
arr_path = os.path.join(data_dir, model_n, dataset_name, f"latest_predictions.npy") |
|
outputs_path = os.path.join(data_dir, model_n, dataset_name, f"latest_outputs.pkl") |
however pkl and npy is not presented in the compressed file ur provided. There is only csv files.
|
## Basemodel Outputs |
|
|
|
The base model outputs of these datasets are can be downloaded from the link below and must be put inside `results/` |
|
|
|
``` |
|
https://drive.google.com/drive/folders/17_5JN5koFKsnyty9klNz1nBGYLKSE4tw?usp=sharing |
|
``` |
PS: the gsm8k folder has npy and pkl and was successful
Hi, thanks for sharing the code. i got into trouble when trying the runing the mmlu_hf task.
as i dived into the code, i found pkl and npy is needed:
llm-topla/helper.py
Lines 158 to 165 in a01e555
however pkl and npy is not presented in the compressed file ur provided. There is only csv files.
llm-topla/README.md
Lines 29 to 35 in a01e555
PS: the gsm8k folder has npy and pkl and was successful