Teaching a six-legged robot to walk, navigate and chase — learned in simulation.
Spider Bot AI trains a 24-DoF hexapod on NVIDIA's GPU physics simulator to cross procedurally generated rough terrain and carry out three behaviours — waypoint, patrol and chase — from one reinforcement-learning policy, then exports it to ONNX for sim-to-real deployment.
Learning to walk where rules can't be hand-coded
A many-legged robot has to decide which foot to place where, and when, every single moment — and the rougher the ground, the faster that problem explodes. Locomotion like this is hard to script by hand, so letting the robot learn to walk through reinforcement learning has become the standard approach.
Spider Bot AI drops many hexapods at once onto procedurally generated terrain inside NVIDIA Isaac Lab (PhysX) and trains a walking policy with PPO. The result doesn't just walk: the same network routes to a destination, patrols unexplored ground, and chases an evading target, switching between modes on a signal.
One recurrent policy that perceives, walks and plans
Recurrent multi-modal policy
A GRU-based PPO_RNN built on skrl. Four independent encoders — proprioception (MLP), a 64×64 heightmap, a 3-channel BEV, and a 33×33 nav grid (ResBlock CNNs) — fuse to 320-D, pass through a GRU (hidden 512) over 32-step sequences, then a shared trunk branches into a Gaussian policy head and a value head.
One policy, three modes
Waypoint, patrol and chase are handled by a single network rather than three. A one-hot mode input scales 18+ reward terms — task rewards (progress, reach, exploration, pursuit) and motion-quality rewards (torque, smoothness, foot air-time, fall penalties) — so the same weights produce different behaviour.
Geodesic progress reward
Progress is measured by wall-aware geodesic distance (BFS over the nav grid), not straight-line distance, so the robot routes around obstacles instead of pushing into walls. Asymmetric log shaping around a ~1.5 m/s target speed suppresses overspeeding and reversing.
Procedural terrain & exploration
Perlin-noise hills, random cube/sphere obstacles and terraced zones are generated procedurally, sampled for flat spawn points, and converted to USD meshes — with SHA-256 config hashing for deterministic, reproducible caching. In patrol, a decaying staleness map drives the robot toward long-unvisited ground.
Sim-to-real via ONNX
On export, the running-standardisation statistics are baked into the model and the GRU hidden state is exposed as explicit I/O, so raw sensor input drives single-step recurrent inference with no external normalisation — built for edge deployment.
A robot, its senses and its world — built from scratch
Manager-based RL environment
Built on Isaac Lab's ManagerBased architecture, with action, command, observation, reward, termination and event logic split into modules — so new reward terms, sensors or behaviour modes drop in without breaking the rest.
Custom 24-DoF hexapod
A self-built URDF: 6 legs × 4 joints, each modelled as an implicit PD actuator with its own range and torque limits. The network outputs target joint positions (offset from a default pose, scale 0.75) rather than raw torque, at 60 Hz control over 120 Hz physics.
Multi-modal sensing
Overlapping contact sensors on all 8 feet, a 64×64 raycast height scanner, a 32-channel 360° LiDAR and a frame transformer combine proprioception with a read on the surrounding terrain.
Reproducibility & containerisation
TensorBoard logging, timestamped run outputs, Omniverse debug markers and debug plots track training, all packaged in Docker Compose for both single-GPU and multi-GPU runs.
From parallel rollouts to a deployable policy
PPO_RNN training loop
Many environments roll out in parallel on PhysX, GAE estimates advantages, and a clipped PPO objective updates the recurrent policy. KL-adaptive learning rate and running standardisation keep it stable, torch.distributed (DDP/NCCL) scales it across GPUs, and the trained policy exports to ONNX for sim-to-real.
- Rollout · 32 steps
- Standardize obs & reward
- GAE λ=0.95
- PPO update · clip 0.2
- KL-adaptive LR
- ONNX export → deploy
What it's built on
- Simulation
- Isaac Lab 3.0Isaac Sim 6.0PhysX (GPU)Omniverse Kit
- Reinforcement learning
- skrl (custom PPO_RNN)PyTorchGRU policyGAEKL-adaptive LRRunningStandardScaler
- Perception & environment
- Multi-modal obsContact · raycast · LiDARBFS geodesic navProcedural terrain
- Robot
- 24-DoF hexapod URDFImplicit PD actuatorsJoint-position control
- Deploy & infra
- ONNX (sim-to-real)torch.distributed DDPDocker ComposeTensorBoardPython 3.12
Redesigned and rebuilt end to end — robot model, simulation, reward design, recurrent policy and the training-to-deployment pipeline.