DreamMotion

LLM agent that generates Unitree G1 humanoid robot motions from natural language. Speak or type what you want the robot to do, and DreamMotion plans the motion, generates it with a diffusion model, and deploys it to simulation or real hardware.

Note

Built in 48 hours at HACK26, a hackathon organized by ETH Robotics Club.

graph LR
    subgraph Input
        MIC[Microphone] --> WEB[Web UI]
        TEXT[Text] --> CLI[Agent CLI]
    end

    subgraph Agent
        WEB --> AGENT[Strands Agent<br/>GPT-4.1]
        CLI --> AGENT
        AGENT --> REFINER[Prompt Refiner<br/>GPT-4.1]
        REFINER --> KIMODO[Kimodo<br/>Motion Diffusion]
    end

    KIMODO --> PT[.pt files<br/>output/]

    subgraph Deploy
        PT --> WATCHER[File Watcher]
        WATCHER --> MIMIC[Motion Tracking<br/>BeyondMimic]
        LOCO[Locomotion<br/>AMO] <--> MIMIC
    end

    LOCO --> ROBOT[MuJoCo Sim<br/>or Real G1]
    MIMIC --> ROBOT

Loading

Quick start

git clone --recurse-submodules https://github.com/maedmatt/DreamMotion.git
cd DreamMotion
uv sync
cp .env.example .env   # fill in your keys

Set OPENAI_API_KEY and KIMODO_URL in .env (see Kimodo server setup).

Dependencies by component

DreamMotion is modular — you only need the dependencies for the parts you use.

Agent + Web UI (motion generation from text/voice):

• Python 3.11+, uv
• OPENAI_API_KEY — powers the agent (GPT-4.1), prompt refinement, speech-to-text, and TTS
• A running Kimodo server — NVIDIA GPU with 17GB+ VRAM, Docker

Sim2Sim deployment (play motions in MuJoCo):

• RoboJuDo cloned as a sibling directory: git clone https://github.com/HansZ8/RoboJuDo ../RoboJuDo
• Install with: uv sync --extra deploy

Sim2Real deployment (physical Unitree G1):

• Everything from Sim2Sim, plus:
• unitree_sdk2_python: git clone https://github.com/unitreerobotics/unitree_sdk2_python ../unitree_sdk2_python
• CycloneDDS installed on the system
• Set UNITREE_NETWORK_INTERFACE in .env to your Ethernet adapter connected to the G1 (find with ip link or ifconfig)
• Install with: uv sync --extra deploy --extra tts

Warning

Always preview motions in MuJoCo before running on the real robot. Generated trajectories can produce weird joint angles that may damage hardware. Run uv run deploy first and check the motion looks right before using --config g1_agent_locomimic_real.

Entry points

Command	Description	Docs
uv run web	Web UI with voice input, 3D preview, and TTS	docs/web-ui.md
uv run agent	Text CLI for motion generation	-
uv run deploy	Robot control loop — watches output/ and plays motions	docs/sim2sim.md, docs/sim2real.md

Docs

• Kimodo server — Docker-based motion generation backend
• Web UI — voice/text interface with 3D preview
• Sim2Sim deployment — playing motions in MuJoCo
• Sim2Real deployment — deploying to a physical Unitree G1
• Development — linting, type checking, contributing

Credits

• Kimodo (NVIDIA) — motion diffusion model
• RoboJuDo — RL deploy pipeline with BeyondMimic tracking
• Strands Agents — LLM agent framework
• g1_description — G1 URDF/meshes for 3D viewer

Contributors

shafeef901 · maedmatt · Thisanwerss · SloDamn · saadl27 · gguidone