Hebbian Attractor Networks for Robot Locomotion

Alexander Dittrich, Fuda van Diggelen, and Dario Floreano
Laboratory of Intelligent Systems, EPFL

Hebbian Attractor Networks (HANs) are plastic neural networks that continuously self-modify during deployment using local Hebbian learning rules. By combining dual-timescale plasticity and temporal activation averaging, HANs give rise to distinct attractor dynamics in weight space — fixed-point attractors for stable gaits and limit-cycle attractors for co-dynamic locomotion.

Analysis of weight dynamics under three Hebbian learning condition

Gymnasium Benchmark

Gymnasium Benchmark tasks with Hebbian Attractor Networks.

Attractor Dynamics of Unitree Go1 Locomotion

Left: Limit-cycle attractor (condition B) — weights oscillate in sync with the gait. Right: Fixed-point attractor (condition E) — weights converge to a stable configuration.

250904185953-go1walkflatterrain-mlp-hebbian-openai-es-29_checkpoint499_lca.mp4

250916172552-go1walkflatterrain-mlp-hebbian-openai-es-4_checkpoint799_lca.mp4

Key Results

• Max normalization transforms unbounded Hebbian updates into structured attractor dynamics
• Slower Hebbian updates + activation averaging induce fixed-point weight attractors
• Fixed-point HANs are robust to perturbations — the gait survives even when Hebbian updates are interrupted
• HANs outperform static MLPs and match GRUs on standard locomotion benchmarks
• Results generalize to quadrupedal locomotion on a simulated Unitree Go1 robot
• HANs in different conditions retain adaptive behavior in Ant-task with morphologial damage as demonstrated by Najarro, Leung.

Method

HANs use a feedforward network with online Hebbian weight updates following the generalized ABCD rule:

$$\Delta w_{ij}^{(k)}(t) = \eta_{ij}^{(k)} \cdot h_{\theta_{ij}^{(k)}}\left(\bar{x}_j^{(k-1)}(t),, \bar{x}_i^{(k)}(t)\right)$$

where activations $\bar{x}$ are computed as a moving average of length $M$. After each update, layerwise max normalization bounds the weights and enables attractor dynamics. Hebbian coefficients and learning rates are optimized via evolutionary strategies (ES).

Condition	Max Norm	Window $M$	$f_\text{NN}/f_\text{hebb}$	Emerging Attractor
(A) HNN	-	1	1	Unbounded
(B) HAN	Yes	1	1	Limit cycle
(C) HAN	Yes	1	4	Limit cycle / Fixed point
(D) HAN	Yes	10	1	Limit cycle / Fixed point
(E) HAN	Yes	10	4	Limit cycle / Fixed point

Implementation

• Framework: JAX — all training and rollouts are fully vectorized
• Optimization: Evolutionary strategies — OpenAI-ES, CMA-ES, $(\mu, \lambda)$-ES — via evosax
• Environments: Gymnasium (Swimmer, HalfCheetah, Hopper, Walker2d, Ant) and MuJoCo Playground (Unitree Go1, Cheetah, etc.) via MuJoCo XLA (MJX)
• Configuration: Hydra for composable YAML configs
• Logging: Weights & Biases

Installation

git clone git@github.com:alexanderdittrich/hebbian-attractor-networks.git
cd hebbian-attractor-networks
pip install .

For GPU support, install JAX with CUDA separately before pip install . — see JAX installation.

Usage

Training is configured via Hydra. The main parameters controlling HAN conditions are:

Parameter	Config key	Options
Learning rule	policy.learning_rule_cls	eta-abcd, abcd, eta-plain
Max normalization	policy.learning_rule_cfg.regularizer	abs-scale (on), null (off)
Activation averaging	policy.activation_buffer_cls	direct (M=1), moving-average
Window size	policy.activation_buffer_cfg.window_size	integer (e.g., 10, 20)
Hebbian update interval	update_interval	float in seconds (null = same as sim)

Gymnasium benchmarks (HalfCheetah, Hopper, Walker2d, Swimmer)

# Condition (B): HAN with max norm, no averaging, synchronized updates
python scripts/train_gymnasium_kandel.py \
  gymnasium=HalfCheetah-v5 \
  policy=mlp-hebbian \
  strategy=openai-es

# Condition (E): HAN with max norm, moving average (M=10), decoupled updates
python scripts/train_gymnasium_kandel.py \
  gymnasium=HalfCheetah-v5 \
  policy=mlp-hebbian \
  policy.activation_buffer_cls=moving-average \
  policy.activation_buffer_cfg.window_size=10 \
  update_interval=0.2

MuJoCo Playground (Cheetah, Unitree Go1)

# Cheetah locomotion
python scripts/train_playground_kandel.py \
  playground=CheetahRun \
  policy=mlp-hebbian \
  strategy=openai-es

# Unitree Go1 quadruped
python scripts/train_unitree_kandel.py \
  policy=mlp-hebbian \
  strategy=openai-es

Morphological adaptation (mutilated Ant)

For the Mutilated-Ant-task, we use the MuJoCo Playground API. For PPO-training, we use Brax instead of Stable-Baselines3.

python scripts/train_ant_mutilated_kandel.py \
  policy=mlp-hebbian \
  strategy=openai-es

python scripts/train_ant_mutilated_ppo.py

Static baselines (MLP, GRU without plasticity)

python scripts/train_gymnasium_kandel.py \
  gymnasium=HalfCheetah-v5 \
  policy=mlp \
  strategy=openai-es

Repository Structure

config/
  gymnasium/       # Gymnasium environment configs (Ant, HalfCheetah, ...)
  playground/      # MuJoCo Playground configs (CheetahRun, Go1Walk, ...)
  policy/          # Network architectures (mlp, mlp-hebbian, gru, ...)
  strategy/        # Evolution strategies (openai-es, cma-es, ...)
scripts/           # Training entry points
src/kandel/        # Core library
  model/           # Networks, learning rules, activation buffers
  strategy/        # ES implementations
  sim_api/         # Gymnasium interface
notebooks/         # Analysis and visualization notebooks

Citation

@INPROCEEDINGS{dittrich2026hebbian,
  title={Hebbian Attractor Networks for Robot Locomotion},
  author={Dittrich, Alexander and van Diggelen, Fuda and Floreano, Dario},
  booktitle={2026 IEEE International Conference on Robotics and Automation (ICRA)},
  year={2026},
}

Acknowledgements

This work was supported by the Swiss National Science Foundation (SNSF) and the Japan Society for the Promotion of Science (JSPS) under project number IZLJZ2_214053.

Disclaimer

This code is provided "as is" without warranty of any kind. The neural network models prioritize readability and simplicity over the modularity of libraries like Flax or Equinox.