clamp value

[Luo624]

Hello, I have read your manuscript and think it is an extremely excellent work. At the same time, I have a doubt. As stated in the manuscript,

3. CFM loss implementation details: The FPO and FPO++ ratios (Equation 3) both exponentiate a difference between squared CFM losses. An early concern we had was that exponentiating a value computed from squares could cause numerical instabilities for outlier epsilon values. We ran experiments with details like Huber instead of squared CFM losses, as well as by applying both naive and gradient-preserving clamp operations to CFM loss differences. We found that a simple approach of (i) clamping CFM losses before taking differences and (ii) then clamping the difference before exponentiation was sufficient for stabilizing training.

But the specific clamp value has not been provided?

[brentyi]

Hi @Luo624, thanks for pointing this out! For locomotion we set [-10.0, 10.0] for the CFM loss difference (more important), and 20.0 for the CFM loss itself (less important). My experience is that the exact value is not too important, this is primarily there to filter out outliers.

I think these things will also be clear with the code release (cleanup done, sending in for approvals today/tomorrow), but I also made a note to update the Overleaf + arXiv.