Whole-body humanoid motion tracking, learned from exact simulator gradients.
Training time to reach it
Policy updates to reach it
to reach return
Full-clip return: once the robot falls, the rest of the clip scores zero. All 40 LAFAN1 clips, deterministic policy. ∇Mimic: residual actions (an offset to the reference pose; same reward) and a moving average of the policy weights, mean ± sd of 3 seeds, one RTX PRO 6000. SuperMimic: curve read off its paper's figure, one H100; its update counts are our estimates from the paper's figures. PPO: BeyondMimic's configuration, 6 h; updates are PPO iterations (20 gradient steps each); "on MuJoCo Warp" is the same run, estimated from the measured iteration-time ratio.2
Each clip is a full LAFAN1 take, 220 seconds long on average.
6/40
3 minutes
2.5/40
5 minutes
0/40
8 minutesevery seed
0/40
after 25 minutesevery checkpoint
Mean over seeds at the checkpoints nearest 3 and 5 minutes; at the checkpoints nearest 8 minutes no seed falls; from 25 minutes (update 3,750) on, none falls at any checkpoint.
Accelerating the scaling of humanoid foundation models.
Exact gradients carry rich information, but are hard to tame.
Methods that learn directly from simulator gradients cannot handle the real gradient signal of a full humanoid: the gradients explode.
LAFAN1 return against training time. × gradient explosion: SHAC's guard (gradient norm above 10⁶) stops training. "Patched": the same SHAC with exploding updates clipped and applied instead of stopping. ∇Mimic: mean ± sd of 3 seeds.2
∇Mimic backpropagates the return through 64 simulated steps (1.28 s of motion) at a time and bootstraps the rest with a learned critic.
Contact gradients are heavy-tailed: a few environments dominate the batch. ∇Mimic clips each sample's action gradient relative to the batch median.
Gradients become target actions of a fixed size; the policy regresses onto them, so the update no longer depends on the gradient's scale.
Three seeds, one curve.
How can it be so fast?
Introducing NablaEngine
We reimplement the dynamics of MuJoCo and MuJoCo Warp on the GPU with one warp per world and fused kernels, and compute the gradient of a whole control step in a single megakernel. Specialized for the Unitree G1, the first robot it supports.1
One control step (20 ms of simulated time) for 4096 G1 environments. Default model: the mjlab Unitree G1 (29 DoF, all 502 contact pairs); MJX 3.17 (Menagerie G1) and Isaac Lab use their own G1 models: Isaac Lab's task default (3 box colliders, no self-collision) saves collisions; "38 collision shapes" is its full-collision G1 with self-collision. MuJoCo C: the reference CPU implementation (float64), MuJoCo's own thread pool over the 4096 worlds on one AMD EPYC 9555 (16 threads); a typical CPU time, the same on every tab.1
Forward dynamics + backpropagation
The backward pass of a whole control step is one megakernel.
One control step and its gradient from the input state. No MuJoCo Warp release supports a backward pass; its adjoint exists only in an open pull request (#1535). Forward and backward parts come from a separate measurement, so they do not add up exactly to the headline time.4
The same PPO algorithm, with different simulators:
Time per PPO iteration (4096 environments × 24 steps, plus 20 gradient steps), same configuration, one RTX PRO 6000.
Full articulated rigid-body dynamics with MuJoCo's constraint-based contact model: no penalty springs, no simplified contact geometry. All 502 contact pairs, self-collision included, pyramidal friction cones, joint and actuator force limits, solved by Newton's method with implicit integration, as in MuJoCo.
Forward error against float64 MuJoCo, the reference C implementation, solved to convergence. Median over 16,384 held-out states. Lower is better.3
3.1×10⁻⁷
MuJoCo Warp against itself: the same inputs, run twice.
0
NablaEngine against itself: bitwise identical, every run.
MuJoCo Warp accumulates contacts, constraint rows and forces with GPU atomic operations, so the order of floating-point additions, and with it the float32 rounding, changes from run to run.
Gradient error against float64 central finite differences of the converged MuJoCo step, per input block. Median over held-out states. Lower is better.3
f: the simulator's dynamics over one control step (20 ms of simulated time); x = (q, q̇). c: a random output direction with i.i.d. standard-normal entries (fixed seed, unnormalised; three per state).
MuJoCo C finite differences: MuJoCo's own one-sided differences (step 10⁻⁶) through its 10-iteration solver, the way they are used in practice. Reference floor: the truth computed at two step sizes agrees to about 10⁻⁷, so no method can be measured below it. Log scale, shared with the forward chart. MJX: forward-mode Jacobian (MJX has no reverse mode). MuJoCo Warp's adjoint is available only in an open pull request (google-deepmind/mujoco_warp #1535; see also etaoxing/mujoco_warp #1). Genesis's native reverse mode differentiates only with respect to the initial velocity.3
One gradient, component by component.
The velocity gradient of one held-out state: the residual of each engine against the float64 finite-difference truth, per component, on a log scale. The state is chosen by rule as representative of every engine's median error. Hover a column for the joint name.
…but faster.
The same algorithm, ∇Mimic, trained on two simulators: NablaEngine and the MuJoCo Warp adjoint.
Mean ± sd of 3 seeds on each simulator. Same hyper-parameters, same GPU (RTX PRO 6000).
Inside NablaEngine
Constraint-based contact as in MuJoCo, implicit integration, joint and actuator limits.
All 502 contact pairs, self-collision included.
Heightfields and box obstacles, for differentiable policy learning across multiple terrains.
✓ supported ◐ partial ✗ no.
Partial: Brax offers MuJoCo-style constraints with an iterative surrogate solve, differentiated through its iterations; Genesis's differentiable mode is restricted to an approximate implicit integrator and a subset of contact features. Sources: the respective papers and source code.
from nablaengine import jax_ffi # one control step: advance 20 ms of simulated time (4 × 5 ms substeps) qpos1, qvel1, warm1 = jax_ffi.step(qpos, qvel, warm, ctrl, friction, com) loss = lambda ctrl: (jax_ffi.step(qpos, qvel, warm, ctrl, friction, com)[1] ** 2).sum() g = jax.jit(jax.grad(loss))(ctrl) # analytic adjoint
from nablaengine import torch_ffi # one control step: advance 20 ms of simulated time (4 × 5 ms substeps) ctrl.requires_grad_() qpos1, qvel1, warm1 = torch_ffi.step(qpos, qvel, warm, ctrl, friction, com) (qvel1 ** 2).sum().backward() # ctrl.grad: analytic adjoint
jax.jit, jax.grad, jax.vjp and lax.scan rollouts; torch autograd, CUDA graphs and torch.compile. The same results from both, bit for bit. Any batch size.
One setup script; the CUDA library compiles once on first use, in about 15 s, and is cached.
Tested on V100, RTX 3090, H100 and RTX PRO 6000. CUDA 12, compute capability 7.0+.
Grey: the simulated robot; blue: the reference motion, shifted sideways. PPO: BeyondMimic's configuration.
Here the reference starts on top of the robot, so any anchor drift shows at a glance. The policy never observes its position offset from the reference.
The reference is drawn on top of the robot. Anchor drift: position and orientation of the robot's torso against the reference torso. Body motion error: mean position error of 14 tracked bodies, with the reference aligned to the robot's torso. Both update with the videos.
NablaEngine, MuJoCo Warp, its gradient and the "MJX 3.17" row run the same G1 model with all 502 contact pairs. Isaac Lab and the Menagerie-G1 MJX row run their own G1 setups in their standard configurations. Those rows show the time a practitioner pays per control step. Speedups are always within one GPU.
The analytic gradient of the discrete simulation step, with the step's discrete decisions (contact set, active constraints, actuator clamps) held fixed. In exact mode it matches float64 finite differences.
SuperMimic's implementation is not public. We compare with the numbers stated in its paper and with its LAFAN1 curve, which we read off the paper's figure. Its paper defines the normalized return as we do. All our runs observe the relative anchor orientation of the reference. SuperMimic's paper ran on an H100; our main runs use one RTX PRO 6000.
The Unitree G1: free base and 29 actuated joints, 31 bodies, 502 contact pairs among capsules, spheres and the floor.
Mimic: Ultra-Efficient Humanoid Control with Differentiable Simulation
UC Davis
@article{lu2026nablamimic,
title = {$\nabla$Mimic: Ultra-Efficient Humanoid Control with Differentiable Simulation},
author = {Lu, Weyl and Chen, Yubei},
journal = {arXiv preprint arXiv:2610.41654},
year = {2026}
}
∇Mimic is pronounced “Nabla-Mimic”.