Skip to content

Feature/meta quest3 support#65

Open
josue99999 wants to merge 6 commits intoNVlabs:mainfrom
josue99999:feature/meta-quest3-support
Open

Feature/meta quest3 support#65
josue99999 wants to merge 6 commits intoNVlabs:mainfrom
josue99999:feature/meta-quest3-support

Conversation

@josue99999
Copy link

Add Meta Quest 3 teleoperation support

Summary

This PR adds Meta Quest 3 as an alternative input device for the GEAR-SONIC teleoperation system. Users can run the manager with Quest controllers (USB or WiFi) in addition to the existing Pico-based flow. The default remains Pico; Quest is opt-in via --reader quest.

Changes

  • Quest reader (gear_sonic/utils/teleop/readers/quest_reader.py): New QuestReader that implements the same sample interface as PicoReader. Uses meta_quest_teleop to read controller poses, maps them to the existing VR 3-point pose format (L-Wrist, R-Wrist, Head), and corrects Quest coordinate axes to the robot frame (X=forward, Y=left, Z=up). Supports USB and WiFi (--quest-ip-address), calibration via right trigger, and optional ADB/synthetic modes for testing.
  • Manager integration (gear_sonic/scripts/pico_manager_thread_server.py): New CLI options --reader {pico|quest} (default: pico) and --quest-ip-address. When --reader quest, the manager uses QuestReader and reads controller state (A/B/X/Y, triggers, grips, joysticks) from the Quest sample for mode switching and locomotion. Calibration uses G1 FK default pose as reference, same as the validation script.
  • Validation and testing: validate_quest_raw for 3D visualization of Quest tracking (raw or G1-calibrated), and test_quest_meta_quest.py for a quick connection check. ADB test via python -m gear_sonic.utils.teleop.readers.quest_reader --test-adb.
  • Dependencies: Optional dependency meta_quest_teleop added under the existing gear_sonic[teleop] extra in pyproject.toml.
  • Docs: Setup guide for Quest 3 (requirements, ADB, calibration, full sim + deploy + Quest workflow) in TELEOPERATION_QUEST_SETUP.md and reader usage in gear_sonic/utils/teleop/readers/README.md.

Backward compatibility

  • Pico unchanged: Default reader is pico. All Pico code paths (XRoboToolkit, body tracking, SMPL 3-point pose) are unchanged. Existing users continue to use the same commands without modification.
  • Quest is additive: No breaking changes to existing APIs or CLI; Quest is selected only when --reader quest is passed.

How to use (Quest)

# USB (recommended for first use)
python gear_sonic/scripts/pico_manager_thread_server.py --manager --reader quest

# WiFi
python gear_sonic/scripts/pico_manager_thread_server.py --manager --reader quest --quest-ip-address 192.168.x.x

# With VR 3-point visualization
python gear_sonic/scripts/pico_manager_thread_server.py --manager --reader quest --vis_vr3pt

Connect the Quest, start the manager, then press right trigger to calibrate (adopt G1 default pose first). See TELEOPERATION_QUEST_SETUP.md for full setup (ADB, developer mode, install).

Tested

  • Meta Quest 3 hardware (controllers, USB and WiFi)
  • Ubuntu 22.04

This allows using the teleoperation system with Meta Quest 3 devices in addition to Pico, without affecting existing Pico users.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant