The lightweight agentic operating system for general-purpose robots
MARS, a small agentic robot for your home.
Start with skills, agents, additional inputs, the simulator, or the ROS reference.
Tip
Don't have a robot? You can still experiment with the simulator!
 Web app |
 Mobile app |
 Simulator |
Innate OS is developed for MARS; if you want to port it to your robot, we are happy to feature it.
With the Innate web app, you can control the robot in real time. Use the virtual joystick controls to drive the base, move the arm, and trigger skills manually.
It is available at https://<robot-address> which can either be its IP or hostname.
The Innate mobile app is available on both iOS and Android. It allows you to control the robot in real time, just like the web app, but with a more convenient interface.
|
Innate Controller app |
Download the Innate Controller app Connect to MARS, drive the robot, run agents and skills, record training data, and manage maps from your phone. Android APK (1.3.0) Direct APK download. iOS TestFlight Join the iOS beta. Controller app docs |
Skills are the core unit of action on Innate robots.
A skill can be digital, like calling a tool, a service or another agent; or physical, like navigating, waving, grasping, recording a demonstration, or executing a learned manipulation policy.
Two standalone skill examples, shown sequentially: moving a chess piece, then opening a door.
- Execute manually — Run skills from the
innateCLI. - Operate from apps — Trigger skills through the web app or Innate mobile apps.
- Run autonomously — Let agents select and interrupt skills as the world changes.
On the robot, skills can be inspected and called through the CLI:
innate skill type innate-os/arm_zero_position
innate skill run innate-os/arm_zero_position @duration=3Custom skills use the same @name=value input syntax.
Some physical skills can be learned from demonstrations.
- Record episodes from the phone app or web app.
- Train a policy with one of the models available on Innate Cloud or locally.
- Deploy the trained model as a skill.
Start here: Training overview. To ship a trained model back to the robot, see Deploy a trained skill.
You will find skills in two different directories:
- Built-in skills — Located in
workspace/innate_skills/. - Your custom skills — Stored in
workspace/custom_skills/. Gitignored and yours to play with.
|
Replay skill — replay a recorded motion file. Saved as workspace/custom_skills/greet/metadata.json:
{
"name": "greet",
"type": "replay",
"guidelines": "Greet the user with a friendly arm wave.",
"inputs": {},
"wheeled": false,
"downloads": {
"episode_0.h5": "https://your-cdn.com/greet/episode_0.h5"
},
"execution": {
"model_type": "replay",
"replay_file": "episode_0.h5",
"replay_frequency": 50.0,
"start_pose": [1.57693225, -0.6, 1.4772235, -0.73784476, 0.0, 0.0],
"end_pose": [1.57693225, -0.6, 1.4772235, -0.73784476, 0.0, 0.0]
}
} |
Code skill — call the mobility interface to move forward. Saved as workspace/custom_skills/move_forward.py:
from brain_client.skills.types import Interface, InterfaceType, Skill, SkillResult
import time
class MoveForward(Skill):
"""Move the robot forward by a given distance."""
mobility = Interface(InterfaceType.MOBILITY)
def __init__(self, logger):
super().__init__(logger)
self._cancelled = False
@property
def name(self):
return "move_forward"
def guidelines(self):
return "Move the robot forward by a given distance in meters."
def execute(self, distance_m: float = 0.5):
self._cancelled = False
if self.mobility is None:
return "Mobility interface not available", SkillResult.FAILURE
speed = 0.2 # m/s
duration = distance_m / speed
self.mobility.send_cmd_vel(linear_x=speed, duration=duration)
stop_at = time.time() + duration
while True:
remaining = stop_at - time.time()
if not remaining > 0:
break
if self._cancelled:
self.mobility.send_cmd_vel(linear_x=0.0)
return "Move cancelled", SkillResult.CANCELLED
time.sleep(min(0.05, remaining))
return f"Moved forward {distance_m} m", SkillResult.SUCCESS
def cancel(self):
self._cancelled = True
return "Move cancelled" |
Agents allow Innate robots to run autonomously following your instructions.
They make the robot think in a high-frequency loop using a multimodal model, for example a VLM that is constantly observing the world.
An agent consists of:
- A set of skills the robot is allowed to use
- A system prompt that defines the robot's behavior
- An agent loop that connects the model to observations, memory, tools, and robot actions
Pick up and put away skills chained in an agent to clean a room.
Multimodal agents have different constraints than purely digital agents: they need to observe continuously, run at a high frequency to react, and to be able to interrupt a running skill when the world has changed.
You can find agents in two different directories:
workspace/innate_agents/— Built-in agents shipped with Innate OS.workspace/custom_agents/— Your local agents. Gitignored and yours to play with.
Here is an example of a simple agent to navigate:
A minimal agent file, saved as workspace/custom_agents/navigate_agent.py:
from brain_client.agents.types import Agent
class NavigateAgent(Agent):
"""An agent that can navigate to requested positions."""
@property
def id(self):
return "navigate_agent"
@property
def display_name(self):
return "Navigate"
def get_skills(self):
return ["innate-os/navigate_to_position"]
def get_inputs(self):
return ["micro"]
def get_prompt(self):
return "You are a helpful robot. When asked, navigate to the requested location using the navigate_to_position skill."Use the simulator to test custom agents before running them on a physical robot.
Innate OS includes a high-level simulator running a replica of MARS. Use it to play with skills, agents, and input devices before you have a robot on your desk.
./innate-sim setup
./innate-sim upThis starts the Docker-based Innate OS runtime, the simulator, and the built frontend at http://localhost:8000. The terminal opens a live dashboard with startup logs, simulator logs, brain logs, and runtime health.
./innate-sim up --vis # open the native simulator viewer
./innate-sim status # show current runtime state
./innate-sim logs simulator # inspect simulator logs
./innate-sim down # stop the runtimeSee sim/launcher/README.md for the full local simulator workflow.
Innate OS provides an SDK for streaming new data into running agents. Innate robots are designed to be naturally expandable: add a new sensor, expose it as an input device, and let agents request it by name.
Input devices live in workspace/inputs/ and are pure Python. They should not import ROS directly.
Thermometer input example
# workspace/inputs/thermometer_input.py
import threading
import time
from brain_client.input_types import InputDevice
def read_thermometer_celsius() -> float:
# Replace this with your hardware, websocket, serial, or API read.
return 21.5
class ThermometerInput(InputDevice):
def __init__(self, logger=None):
super().__init__(logger)
self._stop_event = threading.Event()
self._thread = None
@property
def name(self) -> str:
return "thermometer"
def on_open(self):
self._stop_event.clear()
self._thread = threading.Thread(target=self._loop, daemon=True)
self._thread.start()
def on_close(self):
self._stop_event.set()
if self._thread:
self._thread.join(timeout=1.0)
def _loop(self):
while not self._stop_event.is_set():
self.send_data(
{"celsius": read_thermometer_celsius(), "timestamp": time.time()},
data_type="custom",
)
time.sleep(1.0)An agent or directive can then request the input by name:
def get_inputs(self):
return ["thermometer"]See docs/INPUT_DEVICES.md for the full input-device lifecycle.
Innate OS is currently based on ROS 2, the reference framework for robotics operating systems. Most builders should start with skills, agents, inputs, and the simulator. Changing the core OS is not recommended for normal usage, but it is possible.
| ros2_ws/ Robot runtime workspace. |
System Overview Architecture reference. |
Startup Robot node wiring. |
Updates Services and CLI commands. |
config/ DDS, systemd, udev, audio, Bluetooth, sounds, and shell setup. |
Main ROS 2 runtime packages
- mars_control - top-level robot app node, rosbridge websocket server for the mobile/web app, and low-latency UDP receiver for leader-arm teleop.
- mars_bringup - hardware bringup for motors, base, IMU, and LiDAR, plus
robot_state_publisherfor the TF tree. - mars_arm - arm and head servo driver, MoveIt
move_group, and KDL-based IK solver. - mars_cam - stereo main camera, arm camera, VPI stereo depth estimator, WebRTC streamer, and stereo calibration action server.
- mars_nav - Nav2-based navigation, SLAM mapping, and the mode manager that switches between
mapfree,mapping, andnavigation. - brain_client - bridge to the Innate cloud brain, websocket client, skills action server, and user input manager.
- manipulation - records and replays manipulation demonstrations and runs learned or scripted manipulation policies.
- innate_logger - uploads robot logs and telemetry to the Innate cloud.
- innate_training_node - collects training episodes and pushes them to the training cloud.
- innate_uninavid - UniNaVid vision-language navigation client.
- Innate documentation - canonical docs for setup, simulator, skills, agents, training, and robot operation.
We welcome contributions to Innate OS and will be happy to feature applications written on top of it here–and robots using it.
A huge thanks to all people in the community who helped by contributing, providing feedback, and building on Innate OS.
If you want to help, feel free to reach out on Discord.