Linux Computer Use for Codex

linux-computer-use is a practical substitute for native Computer Use when Codex needs to inspect and operate a Linux desktop application. It implements a controlled screenshot-act-verify workflow with deterministic shell tooling.

It supports:

• local Linux X11 and XWayland desktop sessions;
• Ubuntu and other Linux guests controlled from a macOS Parallels host;
• desktop screenshots;
• visible-window discovery and geometry;
• window focus;
• keyboard input, mouse movement, clicking, and scrolling;
• screenshot-backed smoke testing and UI QA;
• explicit safety boundaries for sensitive or consequential actions.

This project does not emulate or replace OpenAI's security model, visual reasoning, or native Computer Use plugin. It provides the mechanical desktop-control layer that Codex can call through the shell and pairs it with a skill that requires visual verification.

Project layout

LinuxComputerUse/
├── README.md
├── install.sh
└── linux-computer-use/
    ├── SKILL.md
    ├── agents/
    │   └── openai.yaml
    ├── references/
    │   ├── backends.md
    │   ├── safety.md
    │   └── troubleshooting.md
    └── scripts/
        └── linux-computer-use

The linux-computer-use child directory is the installable Codex skill. The repository root contains human-facing documentation and installation support.

Requirements

For local Linux:

• Bash;
• an active graphical session;
• xdotool and wmctrl;
• one screenshot utility: gnome-screenshot, grim, spectacle, scrot, or ImageMagick import.

For a Linux VM controlled from macOS:

• Parallels Desktop and prlctl on the host;
• Parallels Tools in the guest;
• a running graphical Linux session;
• xdotool and wmctrl installed in the guest.

Ubuntu dependency example:

sudo apt update
sudo apt install xdotool wmctrl gnome-screenshot

The Parallels backend captures screenshots with prlctl capture, so it does not require a guest screenshot utility.

Installation

Install into the active Codex home:

./install.sh

With no arguments, install.sh uses ${CODEX_HOME:-$HOME/.codex}/skills. Pass one or more skills directories to install elsewhere:

./install.sh "/path/to/first/skills" "/path/to/second/skills"

Use --source "/path/to/skill" to install another local skill directory. Existing installations are moved to a sibling skill-backups directory before replacement; set SKILL_BACKUP_DIR to choose another backup location. The installer contains no machine-specific source, destination, or backup paths.

Restart Codex after installation so a new session discovers the skill. Invoke it explicitly with $linux-computer-use, or ask Codex to inspect, drive, or visually test a Linux desktop application.

Quick start: Parallels Linux VM

Set the driver path:

SKILLS_DIR="${CODEX_HOME:-$HOME/.codex}/skills"
LCU="$SKILLS_DIR/linux-computer-use/scripts/linux-computer-use"
VM_NAME="<vm-name-or-id>"
GUI_USER="<linux-gui-user>"
CAPTURE_DIR="${TMPDIR:-/tmp}"

Check the VM and graphical session:

"$LCU" \
  --backend parallels \
  --vm "$VM_NAME" \
  --user "$GUI_USER" \
  doctor

Capture and inspect the display:

"$LCU" \
  --backend parallels \
  --vm "$VM_NAME" \
  screenshot "$CAPTURE_DIR/linux-before.png"

If the VM display is idle and the capture is black, wake it and capture again:

"$LCU" --backend parallels --vm "$VM_NAME" --user "$GUI_USER" wake

If the desktop is awake but the intended app is behind another window, cycle-window sends one physical Alt+Tab through Parallels. Inspect a screenshot immediately afterward.

List windows:

"$LCU" \
  --backend parallels \
  --vm "$VM_NAME" \
  --user "$GUI_USER" \
  windows

Interact with the selected application one action at a time:

"$LCU" --backend parallels --vm "$VM_NAME" --user "$GUI_USER" focus "<window-title-or-class>"
"$LCU" --backend parallels --vm "$VM_NAME" --user "$GUI_USER" type "example text"
"$LCU" --backend parallels --vm "$VM_NAME" --user "$GUI_USER" key Return
"$LCU" --backend parallels --vm "$VM_NAME" screenshot "$CAPTURE_DIR/linux-after.png"

Codex should inspect both images and verify the expected text. A successful process exit is not visual proof.

Quick start: local Linux

SKILLS_DIR="${CODEX_HOME:-$HOME/.codex}/skills"
LCU="$SKILLS_DIR/linux-computer-use/scripts/linux-computer-use"
"$LCU" --backend local doctor
"$LCU" --backend local screenshot "${TMPDIR:-/tmp}/linux-desktop.png"
"$LCU" --backend local windows
"$LCU" --backend local geometry "<window-title-or-class>"

Run Codex from a terminal belonging to the graphical session so it inherits DISPLAY, XAUTHORITY, WAYLAND_DISPLAY, and D-Bus environment values.

Command reference

Global options must precede the command:

Option	Purpose
--backend auto|local|parallels	Select the execution backend
--vm NAME	Select a Parallels VM
--user USER	Select the guest GUI user
--display DISPLAY	Override the X11 display
--xauthority PATH	Override the Xauthority file
--delay MS	Control typing delay

Command	Purpose
doctor	Show target, session environment, and tools
wake	Wake an idle display without typing text
cycle-window	Send one Alt+Tab and require visual verification
screenshot /absolute/path.png	Capture the current desktop
windows	List visible X11/XWayland windows
active-window	Show the active window
focus TARGET	Focus a title match or explicit ID
geometry TARGET	Print window coordinates and size
move X Y	Move the pointer
click X Y [BUTTON]	Move and click
type TEXT	Type ordinary non-secret text
key KEY...	Send xdotool key names
scroll up|down [COUNT]	Send wheel events

Run linux-computer-use --help for the authoritative CLI help.

Safety model

The skill requires Codex to:

1. verify the target before input;
2. capture the display before and after each meaningful action;
3. avoid secrets in command arguments;
4. pause before irreversible or consequential actions unless already authorized;
5. stop if focus, layout, or visible state is unexpected;
6. report what changed and where the evidence was stored.

The driver intentionally does not install packages, start privileged input daemons, handle credentials, bypass confirmations, or perform OCR-driven clicking.

Wayland limitations

Input and window discovery use X11 tools. They work with X11 and XWayland applications, but do not reliably control native Wayland-only windows.

System-wide Wayland input injection commonly requires privileged uinput access. This project deliberately avoids enabling it automatically. Prefer XWayland, application test APIs, accessibility interfaces, browser automation, or a future native Linux Computer Use implementation.

Validation

Validate the skill metadata:

python3 \
  "<skill-creator-directory>/scripts/quick_validate.py" \
  ./linux-computer-use

Resolve <skill-creator-directory> from the active Codex installation before running validation.

Check the scripts:

bash -n ./install.sh
bash -n ./linux-computer-use/scripts/linux-computer-use
./linux-computer-use/scripts/linux-computer-use --help

For an end-to-end smoke test, run doctor, capture a screenshot, list windows, focus a harmless application, perform one reversible input action, and capture a second screenshot.

Troubleshooting

The skill contains targeted guidance in:

• references/backends.md for setup and backend selection;
• references/safety.md for authorization and sensitive workflows;
• references/troubleshooting.md for display, focus, screenshot, scaling, and Parallels failures.

Start with doctor. Most failures come from selecting the wrong GUI user, missing X11 environment values, targeting a native Wayland window, or using stale coordinates.