Open-Source FPO++ for G1 Motion Tracking

[Ayanami-Yu]

Hi, thank you for such great work and open-sourcing the cleaned-up code!

I'd like to use FPO++ for G1 motion tracking task since the motion tracking results are present in your paper. But I didn't find any motion tracking pipeline in current code, so I wonder if you plan to also open-source the motion tracking part?

I've tried to modify the locomotion pipeline for motion tracking given the implementation details present in your paper. To be more specific, I:

1. Used the MDP parts of BeyondMimic.
2. Changed both actor_hidden_dims and critic_hidden_dims to [1024, 512, 256].
3. Changed sampling_steps to 50.
4. Changed learning_rate to 3e-4.
5. Changed clip_param to 0.01.
6. Changed num_learning_epochs to 5.

The full runner config is:

@configclass
class G1FlatFpoRunnerCfg(FpoRslRlOnPolicyRunnerCfg):
    max_iterations = 30000
    save_interval = 200
    experiment_name = "g1_flat_flow"
    num_steps_per_env = 24
    empirical_normalization = True
    flow_eval_modes = ["zero"]
    policy = FpoRslRlPpoActorCriticCfg(
        init_noise_std=1.0,
        actor_hidden_dims=[1024, 512, 256],
        critic_hidden_dims=[1024, 512, 256],
        activation="elu",
        sampling_steps=50,
    )
    algorithm = FpoRslRlPpoAlgorithmCfg(
        num_learning_epochs=5,
        n_samples_per_action=32,
        learning_rate=3e-4,
        clip_param=0.01,
    )

I used 4096 envs to train it but the reward curve looks like this:

So am I missing something or would you plan to open-source the motion tracking part? Thanks a lot!!

[hongsukchoi]

Hi, we used a previous version of BeyondMimic’s reward design and weights for the paper and the sim-to-real demo. We are currently verifying the same experiments with the latest BeyondMimic reward design and weights, and expect to release this within the week.

[roybake]

Hi, sorry to bother you! You mentioned you would release the motion tracking part within a week in your previous reply, and I was wondering if there are any updates on the release schedule? I’m really looking forward to using the official motion tracking pipeline for G1 and would appreciate it if you could let us know the current progress. Thanks a lot!