LLM-to-Flow Distillation: Teaching Natural Language Goals to Scientific Generators
#Method: LLM-to-Flow Distillation
LLM-to-Flow Distillation (LFD) iteratively aligns a pretrained scientific generator (the student) towards a natural-language goal g with Direct Preference Optimization (DPO) on preferences induced by an LLM teacher.
Choose an attainable subgoal
Because the pretrained model may rarely generate samples aligned with the natural-language goal g, the teacher also proposes intermediate subgoals that are from the currently generated samples.
At each round t, the teacher uses observables O(Xt) of the current sample batch Xt to propose an attainable subgoal gt.
-
Turn judgments into preferences
For each candidate, the teacher returns a positive, negative, or abstain judgment ℓt(xi) ∈ {+, −, ∅}. Excluding abstentions, the positive and negative sets induce the dataset:
Pt = {xi ∈ Xt : ℓt(xi) = +}, Nt = {xi ∈ Xt : ℓt(xi) = −}.Why preferences?
We use a preference learning signal instead of an LLM’s pointwise estimate of a hidden quantity because such estimates can be unreliable. However, even strongly miscalibrated scores can induce an ordinal learning signal that we can exploit. Across peptide, crystal, and small-molecule tasks, numerical score differences often depart from the true differences. Yet the same frontier LLMs can induce relative preferences above chance through different elicitation methods (scoring, ranking, positive/negative labels, and pairwise comparisons).
Hover or tap a task or method to read the prompt. Select a model in the legend to show or hide it.
Relative judgments provide a learning signal even when score magnitudes are unreliable. LFD uses positive/negative batch judgments: each positive is preferred to each negative, while abstentions are excluded. This yields many training pairs from a single pass over the batch, avoiding a separate LLM query for every pair.
-
Update the student and repeat
All positive–negative pairs in Dt = Pt × Nt train the student, using the current generator pθₜ as a frozen DPO reference. We minimize the DPO loss, which favors positive over negative candidates relative to this reference:
Δtθ = log[pθ(x⁺)/pθₜ(x⁺)] − log[pθ(x⁻)/pθₜ(x⁻)], Lt(θ, Dt) = −EDₜ log σ(βΔtθ).Here, σ is the logistic function and β > 0 controls regularization. Starting from θt, optimizer steps yield θt+1; we then draw fresh samples and repeat.
A design-space view. A goal g defines a target region Ωg in design space. When that region is rarely reached, LFD learns through attainable subgoals that tighten as the generator improves:
Each Ωgₜ contains designs satisfying gt. The final goal stays fixed.
Function details
i.e., such that the LLM judgement yields enough positive and negative examples for DPO.
Draw a fresh batch from the current student pθₜ.
Resampling after every update keeps the teacher’s feedback tied to what the generator can currently produce.
The observables O(Xt) describe the current sample batch to the teacher. They provide evidence about the part of the design space the student can reach, informing the next subgoal.
The choice of observables depends on the scientific domain.
Use the original goal, previous subgoal, and current sample observables to propose an attainable relaxation.
Accept the subgoal only when the teacher identifies enough positive and negative candidates for a useful learning signal. As the student improves, the teacher tightens the goal online.
For each candidate, request a positive, negative, or abstain judgment under the current subgoal, with a rationale and sources of uncertainty:
The positive set Pt and negative set Nt induce preference pairs:
In the experiments, two independent judgment passes must agree on a positive or negative label. Abstained and inconsistent candidates are excluded from preference training. The feedback is categorical; uncertainty is described in words.
For a preference pair (x⁺, x⁻) ∼ Dt, measure how its relative likelihood changes from the current generator:
Minimize the teacher-induced DPO objective:
σ is the logistic function; β > 0 controls regularization. Keep the reference pθₜ fixed during this round.
Start from θt and take optimizer steps on the denoising-loss surrogate, since terminal log-densities are intractable. Return the updated parameters, then resample for the next round.
Experiments on peptides and small molecules
#BibTeX
@misc{desanti2026llmtoflowdistillationteachingnatural,
title={LLM-to-Flow Distillation: Teaching Natural Language Goals to Scientific Generators},
author={Riccardo De Santi and Federico Di Gennaro and Angelo Gnazzo and Sophia Tang and Pranam Chatterjee and Fanny Yang and İsmail İlkan Ceylan and Michael M. Bronstein and Andreas Krause},
year={2026},
eprint={2610.17246},
archivePrefix={arXiv},
primaryClass={cs.LG},
url={https://arxiv.org/abs/2610.17246},
}