Skip to content

middle mouse button does not work in computer use #4390

Description

@rovle

Summary

Daytona computer use documents middle as a supported mouse button value, but in practice it does not behave as a middle click.

Instead, passing button="middle" appears to silently degrade to a left click in the current X11-backed implementation.

This is primarily a documentation / contract mismatch issue: the public API, SDK docs, and request schema say that middle is valid, but the actual runtime behavior does not honor that contract.

Reproduction

sandbox.computer_use.mouse.click(400, 200, "middle")

Expected behavior

The request should emit a real middle-click event.

Actual behavior

The request does not produce a middle click. In the current implementation it appears to fall back to a left click instead, with no validation error or warning.

Why this is a bug

The public contract explicitly advertises middle as a supported value.

Examples in the codebase currently describe the button as:

  • left, right, middle in apps/daemon/pkg/toolbox/computeruse/interface.go
  • ('left', 'right', 'middle') in SDK docs and docstrings

That means users are being told to rely on a value that does not work as documented.

Suspected cause

The computer-use plugin forwards the raw button string into the mouse backend without normalizing or validating it.

The backend uses go-vgo/robotgo, which expects "center" for the middle mouse button rather than "middle". Because "middle" is not a recognized value, it misses the mapping and falls back to the default left button behavior.

Impact

  • Middle-click workflows cannot be automated reliably
  • Users receive no error, so failures are misleading and hard to debug
  • The behavior violates documented API / SDK expectations

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions