Skip to content

Fix simulator skill failure handling#414

Open
axelpey wants to merge 3 commits into
mainfrom
codex/sim-skill-error-handling
Open

Fix simulator skill failure handling#414
axelpey wants to merge 3 commits into
mainfrom
codex/sim-skill-error-handling

Conversation

@axelpey

@axelpey axelpey commented May 28, 2026

Copy link
Copy Markdown
Contributor

Summary

  • load the missing light runtime dependencies needed by shipped chess/Gemini skills in the sim OS image
  • validate skill inputs before executing so bad calls return clean user-facing messages instead of raw Python TypeErrors
  • fail manipulation skills fast in simulator mode when FK/IK topics are absent
  • expose missing physical artifacts (pick_socks, wave) as unavailable/in-training instead of silently hiding them
  • remove the unavailable open_door skill and stop the security guard directive from advertising or requesting it

Validation

  • python3 -m py_compile ros2_ws/src/brain/brain_client/brain_client/skill_loader.py ros2_ws/src/brain/brain_client/brain_client/skills_action_server.py workspace/innate_skills/scan_for_objects.py workspace/innate_agents/security_guard_agent.py
  • git diff --check
  • rg -n "open_door|open door" workspace/innate_agents workspace/innate_skills ros2_ws/src/brain -S returns no matches
  • direct import of SecurityGuardAgent().get_skills() now returns only innate-os/navigate_to_position and innate-os/send_email
  • rebuilt and launched the sim from this worktree before the open_door removal with ./innate sim up --once; action smoke matrix confirmed invalid inputs return clean validation messages, manipulation skills return the simulator FK/IK unavailable message, scan_for_objects returns missing Gemini key, wave returns missing runtime artifacts, and head_emotion succeeds

@greptile-apps

greptile-apps Bot commented May 28, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes simulator skill failure handling across five files: it adds proper validation so bad inputs return clean user-facing messages, marks replay skills with missing artifacts as in-training, fails manipulation skills fast when FK/IK topics are absent, removes the unimplemented open_door skill, and migrates scan_for_objects to the new google-genai SDK.

  • skills_action_server.py: adds _validate_skill_inputs (inspect-based call-shape check), an in-training skill branch in execute_callback, and _sim_manipulation_unavailable_reason to short-circuit manipulation calls when the simulator doesn't expose the required ROS topics.
  • skill_loader.py: replay skills whose replay_file is absent or missing on disk are now marked in-training rather than silently dropped.
  • scan_for_objects.py: migrated from google.generativeai to the google.genai client API, including Part.from_bytes for image data (decoded from base64) and types.GenerateContentConfig for response MIME type.

Confidence Score: 5/5

Safe to merge — all changes are defensive additions that improve error handling without altering the happy path.

The new validation paths only fire on bad inputs or missing runtime artifacts, leaving normal execution untouched. The open_door removal is clean and consistent across agent and skill layers. The google-genai SDK migration correctly decodes base64 before passing bytes and uses the new client API. The only out-of-diff artifact that needs a follow-up is the stale comment and redundant google-generativeai install in ci/Dockerfile.test.

ci/Dockerfile.test was not updated alongside the SDK migration in scan_for_objects.py — its comment and explicit google-generativeai install are now stale.

Important Files Changed

Filename Overview
ros2_ws/src/brain/brain_client/brain_client/skill_loader.py Adds early-exit guards for replay skills whose replay_file is absent or missing on disk, correctly marking them as in-training rather than silently failing.
ros2_ws/src/brain/brain_client/brain_client/skills_action_server.py Adds input validation (_validate_skill_inputs), in-training skill handling, and sim-mode manipulation guard (_sim_manipulation_unavailable_reason) to return clean user-facing errors instead of raw Python exceptions.
sim/Dockerfile Adds python-chess and google-genai to the sim image's Python dependencies to support the chess and Gemini-based skills.
workspace/innate_agents/security_guard_agent.py Removes the unavailable open_door skill from the skill list and cleans up all references to it from the patrol directive prompt.
workspace/innate_skills/scan_for_objects.py Migrates from the deprecated google.generativeai SDK to google.genai (google-genai package), updating client construction, model invocation, and image-part creation (Part.from_bytes with decoded bytes instead of a raw base64 dict).

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[execute_callback] --> B{Parse JSON inputs}
    B -- failure --> C[Return: Invalid inputs JSON]
    B -- success --> D{inputs is dict?}
    D -- no --> E[Return: Inputs must be JSON object]
    D -- yes --> F{Lookup skill type}
    F -- code_entry --> G[_execute_code_skill]
    F -- physical_entry --> H[_execute_physical_skill]
    F -- in_training_entry --> I[Return: Skill not executable yet]
    F -- not found --> J[Return: Skill not available]
    G --> K{_validate_skill_inputs}
    K -- error --> L[Return: validation error message]
    K -- ok --> M{_sim_manipulation_unavailable_reason}
    M -- topics missing --> N[Return: Arm manipulation unavailable]
    M -- ok --> O[Execute skill.execute inputs]
    style C fill:#ffcccc
    style E fill:#ffcccc
    style I fill:#fff3cc
    style J fill:#ffcccc
    style L fill:#ffcccc
    style N fill:#fff3cc
    style O fill:#ccffcc
Loading

Reviews (3): Last reviewed commit: "refactor: simplify skill input validatio..." | Re-trigger Greptile

Comment on lines +1253 to +1256
def _validate_skill_inputs(self, skill_id: str, skill, inputs) -> str | None:
"""Return a user-facing validation error, or None if inputs are usable."""
if not isinstance(inputs, dict):
return "Skill inputs must be a JSON object."

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The isinstance(inputs, dict) guard inside _validate_skill_inputs is unreachable in the current call flow. execute_callback already aborts and returns before reaching _execute_code_skill (the only caller of this method) whenever inputs is not a dict, so the check here will never fire. If the function is meant to be callable from other contexts in the future, a docstring note would be clearer; otherwise the guard is dead code.

Suggested change
def _validate_skill_inputs(self, skill_id: str, skill, inputs) -> str | None:
"""Return a user-facing validation error, or None if inputs are usable."""
if not isinstance(inputs, dict):
return "Skill inputs must be a JSON object."
def _validate_skill_inputs(self, skill_id: str, skill, inputs) -> str | None:
"""Return a user-facing validation error, or None if inputs are usable.
``inputs`` is guaranteed to be a dict by the time ``execute_callback``
dispatches here, but the guard is kept for safety in case this helper
is ever called from a different context.
"""
if not isinstance(inputs, dict):
return "Skill inputs must be a JSON object."

Comment on lines +1309 to +1310
topic_names = {name for name, _types in self.get_topic_names_and_types()}
missing_topics = [name for name in ("/fk_pose", "/ik_solution") if name not in topic_names]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 The get_topic_names_and_types() call traverses the full ROS 2 graph on every manipulation skill invocation in simulator mode. If many manipulation skills fire in quick succession this could add latency. Caching the result for a short TTL would be more efficient, though correctness is unaffected since the check already returns None in non-sim mode.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant