Missing script to convert LIBERO HDF5 dataset to action.npy format in rel2abs.py pipeline

[Noietch]

Description

In the LIBERO data preprocessing documentation (preprocess.md), the workflow mentions:

1. Run regenerate_libero_dataset.py to regenerate the LIBERO dataset
2. Run rel2abs.py to convert relative actions to absolute actions

However, there is a missing intermediate step between these two scripts.

The Problem

regenerate_libero_dataset.py outputs HDF5 files with actions stored as:

  ep_data_grp.create_dataset("actions", data=actions)  # Line 238

But rel2abs.py expects to read action.npy from a data_jpg directory structure:

data_root = '/data/libero/data_jpg'
 action_path = os.path.join(data_root, task_suite_name, f"{task_name}_demo", f"demo_{i}", "action.npy")
 action = np.load(action_path)  # Line 108-109

Expected

There should be a script (or documentation) that converts the HDF5 output from regenerate_libero_dataset.py into the data_jpg directory structure containing:

• action.npy
• Images (jpg files)
• Other necessary files

Questions

1. Is there a missing script for this HDF5 → data_jpg conversion step?
2. Or should rel2abs.py be modified to read directly from the regenerated HDF5 files?
3. Could you please provide the h5 file converted to abs eef

Thanks for the great work on X-VLA!

[CathyF9600]

Hello, indeed there is a gap in the documentation/scripts regarding how to convert the provided raw dataset into that structure. While we are working on it, we are uploading this absolute-format Libero dataset on HuggingFace. Please stay tuned!

[gianlucageraci]

Hi, thanks again for the excellent work on X-VLA and for continuing to support the community by releasing additional datasets. It’s genuinely super helpful.

I have a question regarding this thread. I’ve been inspecting the dataset you shared, specifically the demo_0 files from:

• libero_object/pick_up_the_butter_and_place_it_in_the_basket_demo
• libero_90/KITCHEN_SCENE10_close_the_top_drawer_of_the_cabinet_and_put_the_black_bowl_on_top_of_it_demo

I’m still trying to fully understand how the original LIBERO dataset was preprocessed, particularly how the proprioceptive observations were constructed and later used as inputs to the policy.

So far, my understanding is that you converted the relative EEF actions from the original LIBERO dataset into an EEF 6D representation, which results in a 20-dimensional action prediction target.

I also noticed that during inference, the previously predicted 20-dim action is fed back into the policy as proprio state, with the gripper dimension masked out (as mentioned in [this issue](#21)). Based on this, I initially assumed that during training you similarly used the previous action from the dataset as the proprioceptive input.

However, when inspecting the dataset, this does not seem to be the case. In particular, the values stored in proprio and abs_action_6d diverge quite significantly. For example:

>>> f["proprio"][10]
array([ 0.03920502, -0.03929644, -0.1332037 , -0.04016251,  0.25590494,
        0.99869674, -0.03091343, -0.04041783, -0.00395263], dtype=float32)

>>> f["abs_action_6d"][8:12]
array([[-0.12789783, -0.06101989,  0.25661346,  0.07485608,  0.99700993,
         0.01917583,  0.99281925, -0.07271452, -0.09498733, -1.        ],
       [-0.1260635 , -0.07097673,  0.25620577,  0.08363482,  0.9962359 ,
         0.02278728,  0.992066  , -0.08108727, -0.09607212, -1.        ],
       [-0.12462157, -0.08139759,  0.255882  ,  0.09119736,  0.9954323 ,
         0.02823978,  0.99144614, -0.0881003 , -0.096296  , -1.        ],
       [-0.12292513, -0.09239007,  0.2556142 ,  0.09632501,  0.9947789 ,
         0.03371124,  0.9908552 , -0.09261999, -0.09811962, -1.        ]],
      dtype=float32)

My questions are therefore:
(1) How exactly was the proprio state constructed in this dataset? Did you actually use previous actions or did you transform the observation states already contained in LIBERO?
(2) Am I understanding correctly that the gripper state is missing in the proprio beacause it is supposed to be masked and the policy will pad it with 0 anyways?

Thanks a lot for the clarification!