Straightening Graph Paths: How Meta Sped Up Neural Network Training by 63% ✛

‹›

Recommendation systems that decide which post you’ll see next run on deep neural networks made up of thousands of operations — intricate computation Graphs (blueprints of operations on data). These systems are living, ever-evolving Dynamical Systems: the slightest architecture change cascades through the entire graph. As model accuracy grows, the paths get tangled: redundant forks like “slice a tensor (a multidimensional array of numbers) into pieces and immediately glue them back together” create real traffic jams on GPUs. Every such twist launches a separate compute kernel, and over a day of training trillions of superfluous launches can pile up. Engineers try to clear these paths manually, but models change weekly, and keeping order in this entropic chaos without an automated copilot has become impossible.

Optimus is a module inside the PyTorch 2.x compiler that works as a greedy cartographer. It’s armed with a library of over thirty optimization patterns — typical junk blueprints like a split-then-cat chain. The algorithm starts from anchors, critical operations around which the tangle coils, and walks through neighboring nodes, checking each step against a reference pattern. Where it finds a match, it cuts out the bulky structure and inserts a compact equivalent. Horizontal cleanup merges dozens of identical layers into one; vertical cleanup removes pointless splits and concatenations. The whole procedure doesn’t change the final result but radically simplifies the internal graph. Graphs of any topology are amenable to such optimization, because the rules operate on the universal Torch-IR intermediate representation.

The effect on Meta’s production systems was tangible. Five recommendation Machine Learning models of different architectures showed throughput gains of up to 63%—more training examples per second without hardware changes. Vertical cleanups slashed hundreds of parasitic Triton kernels, while horizontal ones merged a scattering of identical operations into one; the more clones, the bigger the win. Peak memory use dropped by 2–6%, which is critical for models with activation quantization, where every megabyte counts. In an illustrative TorchBench test, the number of kernels in training plunged from 12 772 to 24, and in inference from 6 320 to 6, delivering 11× and 364× speedups respectively. This numerical simulation triumph was possible because Optimus operates at the compiler level, seeing the whole graph.

This framework turns a chaotic web of footpaths into a data autobahn, and what’s best — all its tools are open to everyone. The code is baked into PyTorch, and any developer can not only use the ready-made patterns but also add their own, teaching the system new straightening tricks. Going forward, the authors plan to push the optimization down to the even lower Inductor-IR level and to teach the algorithm to choose the best sequence of rules autonomously — without human input. This isn’t just faster training; it’s a step toward self-optimizing neural nets that adapt to hardware on their own, saving electricity and time. The tangled graph paths will no longer waste resources: they’ll be straightened by autopilot.