Skip to content

[BUG] SauteAdapter device mismatch on GPU with custom envs #388

Description

@96sanjay

Required prerequisites

What version of OmniSafe are you using?

0.6.0

System information

Python 3.10, Linux, installed from source (git clone). PyTorch 2.1 with CUDA. Tested on both RTX 3050 and A100.

Problem description

Running PPOSaute on GPU with a custom environment that returns CPU tensors crashes on the very first step.

_safety_obs is initialized on cuda in reset(), but the tensors from env.step() (obs, reward, cost, terminated, truncated) stay on cpu. When _safety_step tries
self._safety_obs -= cost.unsqueeze(-1) / self._safety_budget, it blows up because of the device mismatch.

Same issue in _safety_reward, _augment_obs, and the done computation in step().

Doesn't happen with mujoco/safety-gym envs because their wrapper chain handles device coercion, but any custom registered env that returns cpu tensors hits this immediately.

Reproducible example code

Python snippets:
import omnisafe

any custom env that returns cpu tensors

agent = omnisafe.Agent('PPOSaute', 'YourCustomEnv-v0', custom_cfgs={'train_cfgs': {'device': 'cuda'}})
agent.learn()

crashes at first env.step()

Traceback

runtimeError: Expected all tensors to be on the same device, but found at least two devices, cuda:0 and cpu!

Expected behavior

SauteAdapter should coerce env outputs to the training device before doing arithmetic with internal state tensors. Other adapters like OnPolicyAdapter work fine because they don't do
in-place ops mixing internal state with env outputs.

Additional context

The fix is straightforward — coerce env outputs to self._device at the top of step() and reset(). I have a PR ready: #387

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions