Authors: Jubin Chheda, Sundararajan Subramanian, and Viral Raghuwanshi

Imagine a swarm of agents attempting to climb to the top of the hill, most of them failing, some struggling, but some making it to the top. These agents take every success and failure as learnings and replicate them into a better swarm that is better at scaling even higher peaks. Imagine this happening in parallel hundreds, thousands, even millions of times. This is recursive self-improvement. In this post, we will see how we do this in improving efficiency tools and also fine-tuning the models.

For years, efficiency work on Meta’s web tier depended on engineers spotting code inefficiencies. Engineers identified and tested each fix behind an A/B testing framework and posted the result. Much of that knowledge stayed in silos.

With AI, the volume of diffs and output has increased substantially, while capacity remains constrained. Verification and learning from results have become bottlenecks. We need to be able to optimize the new code faster and also enable the learnings to flow into the development phase.

Wally, Meta’s internal compute-optimization platform, connects measure → optimize → verify → ship → learn. It turns production inefficiencies into optimization hypotheses, checks their potential and risk, and tests promising candidates in small A/B experiments. The results update Wally’s own knowledge base, thereby improving itself. The aim is recursive self-improvement (RSI): Each round should help the optimizer make better choices in the next. We need to measure both the capacity saved and whether its success rate improves. To do that, RSI connects two loops. One loop improves the code. The other loop uses test results to improve how Wally chooses to apply optimization patterns in the future.

How Wally works

Every attempt produces three things: a measured outcome, a ship-or-reject decision, and a learning for the next run.

We’ll follow one example all the way through: an expensive read sitting above a low resource early exit. Move the check first, and the read disappears on the early path.

Profile

Wally starts with profiling: It identifies which functions are computationally burdensome, based on profiling data consumption. Wally picks 100 functions at random to run evaluations.

Propose

Wally relies on a knowledge base built by gathering past efficiency optimizations across Meta. 

Starting from a resource use profile of the target, the proposer checks every hot subfunction against the knowledge-base catalog of optimization categories (see Table 1), applying each category’s own anti-patterns to reject false matches, then runs a separate analysis for algorithmic or structural wins the catalog misses. 

Each surviving candidate gets a savings estimate from the profiled resource use, is screened against prior attempts on the same code, and must clear a minimum-efficiency gate before the top-ranked few are implemented behind an experiment.

Verify offline

Wally applies a targeted set of evals and judges to do an independent review of the recommended optimizations, and assigns a risk label. This phase helps to reduce hallucinations and prevent regressions from reaching production.

Test in production

Optimizations with high confidence are published using a A/B test framework to create a safe testing environment. The intent of the A/B test is twofold: to measure compute savings and monitor regressions for the optimized path.

Decide

A/B tests are run for three days to get statistically significant data on compute savings. Optimizations that show positive results are auto-shipped.

Optimizations showing negative/neutral savings or metric regressions are rejected.

Learn (recursive self-improvement) 

Continuous self-improvements assist in improving results over time.

Wally has two self improving loops.

• Evaluate results: A weekly job analyzes results of the week, what shipped, which were wins, and which got reverted. The job proposes anti-patterns that need to be updated within the knowledge base to ensure bad optimizations are rejected upfront.
• Pattern-catalog updates: Data-driven analysis of patterns based on their success data, sunsets patterns that are not performing well, and analyzes data to find new patterns to exercise.

RSI starts with changes to Wally’s knowledge base. Successful experiments add examples to the optimizer for future use. Failed experiments add rules to prevent the same error. Independent verifier agents test proposed rules before feeding them as a learning for Wally to use. These proposed rules are sent through a code-review process on a periodic basis to ensure there is human oversight. 

How we ensure quality

Given Wally’s scale, proposed recommendations should be high quality. Even though A/B testing is low lift, there are real resource drains from shipping, unshipping, and code reviews.

To ensure only quality recommendations are proposed, we have built certain checks and balances in the system. Each check is a script, not advice. Any one can kill a candidate, and each exists because skipping it once could waste resources.

Has this already been tried?

Wally checks whether the same change was already shipped, lost, got rejected, or was reverted after an incident. Re-proposals were our biggest source of wasted verification. Reviews grounded in platform data caught them 71–94% of the time; reviews reading only source history, 0–9%.

Can we prove the new code behaves the same?

Our A/B system measures compute, not correctness. Changes that are safe by construction, such as removing dead code, go straight to experiment. Anything that assumes equivalence must first show identical results on identical inputs. 

Would our own verifier reject this?

The generator screens its output against eight defect patterns from past rejections, such as dropped logging and bad cache keys. On one day, roughly 30% of pending diffs were retired for exactly these defects: generated, verified, and wasted.

Survivors become small, reversible patches with an evidence packet (computational cost inputs, assumptions, affected callers, rollback path), so a reviewer never has to replay the agent’s session.

Let the model judge; let code decide

This is the most transferable lesson from building Wally. Models are good at judgment: spotting a hot path that looks wasteful, matching it to a pattern, explaining why a change should be safe. They are bad at being the same every time. So we draw a hard line between the two (Figure 2).

The model owns the open-ended questions. What could be optimized here? Which catalog pattern applies? What could this change break? Why is it safe?

Code owns everything that must not vary. The order of the steps, retry caps, the minimum-savings gate, the three-day experiment window, and the ship/reject thresholds all live in scripts. A check that fails to run counts as a fail. No prompt can talk its way past a gate.

The rule we use is: if a step’s outcome should be the same on every run, it doesn’t belong in a prompt. Each time the model got a deterministic step wrong, that step moved into a script.

Verification: simple checks first, followed by production

“Verify” hides four different questions, and each kind of evidence has blind spots (Table 2).

Our verifier runs thirteen specialized checks, from the efficiency thesis to blast radius, and takes the strictest verdict: ship, fix-and-ship, or block. It also screens every diff against failure shapes drawn from 54 past-production incidents. A check that fails to run is considered a fail. All the checks need to pass to land it in production..

Don’t just ship it, learn from it

A loop that ships wins and forgets the rest isn’t learning; it’s just producing. We keep the full record of every attempt—what we tried, what we assumed, what we measured, what we’d change.

Two jobs run on that record:

• A weekly pass turns wins into catalog updates. A new pattern needs two corroborating wins before it counts—one win is only an anecdote.
• A separate pass autopsies weak results against a fixed list of causes, so failures become tests, filters, or preconditions instead of just vibes.

Repeat failures become verification tests, filters, or preconditions contributing to anti-patterns. The anti-pattern catalog now holds about fifty entries.

Is the optimizer actually getting smarter?

Increased efficiency alone doesn’t prove it. The product (less compute), the process (better targets, faster verification), and the model can each improve on their own, but progress in one says nothing about the others.

So every catalog or process change is tested against the current version on held-out cases, run in shadow or canary, then approved. When the learner agent creates the learnings and applies it to Wally through a diff, these are called “learning modifications.” The optimizer success rate across multiple experiments is tied to each of these learning modifications. When the optimizer efficiency drops or regresses, this also helps us to directly pinpoint what went wrong. 

Improving the RSI with fine-tuned models:SFT

We created a model & harness that learns to optimize CPU & GPU hotspots by training on the outcomes of its own optimization attempts—a closed loop from fleet hotspots → proposals → verification → new training data → a better model → back to the fleet. We leverage on work from Wally—which has an optimizer, verifier, and learner and is recursively self-improving on its harness.

We have set up an RSI loop on the model (Figure 3).This helps us to fine-tune a smaller specialist model from the frontier teacher. We can RSI the effectiveness of this smaller model to land better optimization experiments. We also extend this to further applications where product metrics can be recursively improved with new experiments and the learnings being fed back .

What we learned that applies beyond compute capacity

Start your own loop

1. Pick one bounded target with measured computational cost: Name the metric and the smallest effect worth chasing—in our case, the success rate of experiments, i.e., the actual capacity-wins ratio.
2. Verify it against anti-patterns. Start the verifiers with learnings from past experiments or common anti-patterns to watch for. In addition, also determine the right exposure, a mechanism to watch the results, and quick rollback mechanisms. Test with a few bad candidates to prove your verifier rejects them.
3. Automate gradually. Turn each failure into an executable check, and grow the volume of the experiments at a pace that allows the verifier to catch up. A very eager automation of the loop could cause additional high severity issues.
4. Define stopping states up front and cap retries on some of the experiments. Also maintaining an earlier-attempt knowledge base helps in not retrying the same optimization over and over.

We started with compute capacity because it was easy to measure. The loop fits anywhere outcomes are measurable and verifiable.

The output is not just efficient code. It is a better, auditable, and self-improving way (RSI) of making code efficient.

Thanks to Robert Glickman, Scott Valentine, Alex Lei, Peter Podniesinski, Kareem Kamel, Zach Zhou, Lucy Lin, Tommy Tran, Taylor Schoettle, and Walter Mao, who have contributed in numerous ways to the optimization loop in Wally.