AERO-CONTROLLERS
A Live Interactive Documentation By Team Aero-Controllers
System Architecture

Technical Stack &
System Architecture

Closed-loop reinforcement learning stabilization system built on Unreal Engine 5 and Cosys-AirSim.

01 — High-Level System Architecture

Layered Simulation Stack

This architecture forms a real-time cyber-physical simulation and learning loop.

Windows 11 OS
Host Environment
Unreal Engine
5.5.4
Cosys-AirSim Plugin
Physics Extension
RPC Bridge
msgpack API
Python PPO Engine
Policy Optimization Backend
Core RL
stable-baselines3 2.7.1gym 0.26.2gymnasium 1.2.3Shimmy 2.0.0
Deep Learning
torch 2.7.1+cu118torchvision 0.22.1+cu118torchaudio 2.7.1+cu118CUDA 11.8
Simulation Interface
cosysairsim 3.3.0rpc-msgpack 0.6msgpack 1.1.2
Data & Analysis
numpy 2.2.6pandas 2.3.3matplotlib 3.10.8opencv-python 4.12.0.88
System Monitoring
pynvml 13.0.1nvidia-ml-py 13.590.48psutil 7.2.2
CUDA-enabled PyTorch backend with Stable-Baselines3 PPO implementation.
Logging + Evaluation
02 — Closed-Loop Control Flow

Policy Optimization Cycle

This is the heart of the system.

Sensor Data
IMU, Vision, Odometry
State Vector
Network Inputs
PPO Policy
Forward Pass
Motor Commands
Thrust PWM
AirSim Physics
Rigid Body Sim
Updated State
Next Iteration
Real-Time
Cyber-Physical
Learning Loop
600,000 Iterative Updates
03 — RPC Communication Pipeline

Bidirectional Data Flow

Python Agent
PyTorch Policy
Receives sensor data, outputs actions.
RPC
AirSim API
C++ Wrapper
Bridges Python RPC to Unreal physics layer.
NATIVE
UE5 Physics
Rigid Body Env
Computes collisions, aerodynamics, constraints.
  • Python sends motor command
  • AirSim converts to thrust/torque
  • UE5 updates physics frame
  • Sensor state returned via API
  • PPO computes next optimal action
04 — Quadrotor Physics Pipeline

Force-to-State Integration Flow

Full nonlinear rigid body simulation with aerodynamic disturbance injection.

Control Inputs
Rotor Speeds
PWM Signals
Quadratic thrust curves mapping RPM to Force.
Thrust Forces
Torque Generation
Rigid Body Dynamics
(6-DOF)
Updated Pose
Quaternion Integration
Orientation representation avoiding gimbal lock.
Sensor Output
IMU & Vision
Environment
Wind Model
Disturbance
Stochastic Dryden wind turbulence and steady gusts.
Aerodynamic Force
Physics Engine
Unreal Engine Layer
05 — Wind & Domain Randomization

Disturbance-Robust Learning

Domain randomization ensures stability across extreme operational envelopes.

Calm
Stochastic Smooth
Strong Gust
Mixed
Training Distribution
Policy Optimization
06 — Training Evolution

Monotonic Stabilization Improvement

Tracking performance gains across discrete training phases.

0 Steps
50k Steps
200k Steps
600k Steps
M₀ > M₅₀ₖ > M₂₀₀ₖ > M₆₀₀ₖ
07 — Final Evaluation Pipeline

System Rollout Assessment

Worst-case wind regime minimized through robust policy evaluation.

Trained Policy
Wind Regime i
Physics Rollout
Stability Score S_i
Comparison Across Regimes
08 — Toolchain & Version Table

Software Stack

Component
Version
Python
3.10
Unreal Engine
5.5.4
Cosys-AirSim
April 2025
Visual Studio
2022 v17.13.6
MSVC
14.43
Windows SDK
10.0.22621.0
09 — Hardware Specifications

Compute Infrastructure

NVIDIA GPU
CUDA Acceleration
Host CPU
Workstation Node
System RAM
Allocated Memory
Selected Node Telemetry

NVIDIA GPU Compute

Massively parallel hardware acceleration for matrix operations and policy optimization.

Compute API
CUDA 11.8
Backend Map
cu118
VRAM Req.
12 GB minimum
Workload
PPO Backprop, Env Vectorization
10 — System Characteristics

Engineering Summary

  • 6-DOF rigid body physics
  • Quaternion orientation dynamics
  • Stochastic aerodynamic disturbance
  • Policy gradient optimization
  • Domain-randomized robustness
11 — Reproducibility Workflow

Execution Sequence

1
Launch Engine
2
Load plugin
3
Start Environment
4
Run Python Agent
5
Optimize Policy
6
Evaluate Robustness
STEP 01
Launch Engine
Initialize Unreal Engine 5.5.4 in headless or editor mode.
$UnrealEditor.exe "AeroProject.uproject" -game
Made by Team Aero-Controllers