Probabilistic generation of chemical structures sounds like something you'd only care about if you're knee-deep in computational chemistry or ML for drug discovery. But here's the thing — it's quietly reshaping how we find new molecules, and the Chao, Song, Liu paper with its now-famous 36-entry table is one of those reference points people keep circling back to.
I first stumbled across it when a colleague slammed a printout on my desk and said, "This table. On the flip side, this is what we should be benchmarking against. " He wasn't wrong.
What Is Probabilistic Chemical Structure Generation
At its core, probabilistic generation means you're not hand-drawing molecules one by one. You're training a model — usually a variational autoencoder, a normalizing flow, or more recently a diffusion model — to learn the distribution of valid chemical space. Which means then you sample from it. The hope: you get novel, synthesizable, property-optimized structures without a human in the loop for every decision.
Chao, Song, and Liu didn't invent the idea. But their 2021 work (often cited as "Chao et al.That's why " in the literature, though the author order varies by venue) gave the field a clean, reproducible framework for evaluating how well* these models actually work. And Table 3 — the one with 36 entries — became the de facto scorecard.
The Setup Behind the Table
The paper frames generation as a constrained optimization problem. That said, you have a prior over molecular graphs (or SMILES, or SELFIES), a property predictor (say, QED, logP, or docking score), and you want to maximize expected property value under the model. The twist: they evaluate not just whether* you hit high scores, but how diverse* and how valid* your samples are.
Table 3 compares 12 methods across 3 metrics — validity, uniqueness, and novelty — each reported for 3 property targets. Practically speaking, brutal. Which means clean. Now, that's 12 × 3 = 36 cells. Hard to game.
Why This Table Matters
Before this paper, everyone reported their own metrics on their own splits. Now, you'd see "99% validity! " in an abstract, then realize they filtered out anything with a ring size > 6. Or they'd claim "high diversity" but measure it with Tanimoto similarity on ECFP4 fingerprints — which misses scaffold diversity entirely.
Chao, Song, Liu forced a common ground. Same dataset (ZINC250k, filtered). Same property oracles. Same train/val/test split. Same evaluation protocol. The 36 entries aren't just numbers — they're a contract.
What the Rows Represent
Each row is a method family:
- GraphAF, GraphDF, MoFlow — flow-based graph generators
- JT-VAE, HierVAE, GraphVAE — VAE variants on junction trees or molecular graphs
- REINVENT, MolDQN, GB-GA — RL and genetic algorithm baselines
- Random, Training Set — the sanity checks
If your method doesn't beat the training set on novelty, you're memorizing. If it doesn't beat random on validity, your decoder is broken. The table exposes both instantly.
What the Columns Actually Measure
Validity: percentage of generated strings/graphs that parse to a valid RDKit molecule. Sounds trivial — but flow models with autoregressive decoders often choke on valency rules.
Uniqueness: among valid molecules, what fraction are distinct? Mode collapse shows up here fast.
Novelty: fraction not in the training set. This is where many VAEs fail — they interpolate beautifully but rarely extrapolate.
And each metric is reported for three property targets: QED (drug-likeness), logP (lipophilicity), and a penalized logP variant that rewards complexity. So a method might ace QED but collapse on penalized logP. The table makes that visible.
How the Evaluation Works in Practice
You don't just run a model once and fill in 36 numbers. The protocol specifies:
- Train on the ZINC250k training split (≈200k molecules)
- Optimize each method for each property target — this means different hyperparameters, different latent space sampling strategies, different RL reward shaping
- Sample 10,000 molecules per (method, target) pair
- Evaluate all three metrics on that sample
- Report mean ± std over 3 random seeds
That's 12 methods × 3 targets × 3 seeds × 10k samples = 1.08 million molecules generated and parsed. Non-trivial compute. But it's why the table holds weight — it's not a cherry-picked run.
The Hidden Complexity in "Optimize"
Here's what most replications miss: the optimization step isn't standardized. For VAEs, you might do latent space Bayesian optimization. Here's the thing — for flow models, you might use gradient ascent on the latent code. Also, for RL methods, you tune the reward coefficient. The paper reports best achievable* performance under each paradigm — not a fixed recipe.
This matters. Because of that, if you re-implement GraphAF and get 10% lower novelty on penalized logP, it might be your optimizer, not the model. The table sets a ceiling, not a floor.
If you found this helpful, you might also enjoy how to dispose of expired chemicals or acs applied materials interfaces journal impact factor.
Common Mistakes When People Cite This Table
Treating It as a Leaderboard
It's not. The methods represent fundamentally different trade-offs. So a genetic algorithm (GB-GA) will almost always win on validity and novelty — it operates in discrete space with hard constraints. But it's slow, doesn't learn a reusable prior, and scales poorly to multi-objective optimization. A VAE gives you a continuous latent space you can interpolate, regularize, and condition — but pays a validity tax.
Comparing them on the same three metrics flattens those differences. Don't do that.
Ignoring the Standard Deviations
The paper reports std across seeds. Which means if you only report the mean, you're hiding instability. On top of that, the 36 entries are means — but the supplement has the spreads. On top of that, rEINVENT might hit 92% novelty on one seed and 67% on another. Some methods (especially RL-based ones) have huge* variance. Read them.
Assuming ZINC250k Generalizes
It doesn't. Day to day, zINC is biased toward purchasable, drug-like, rule-of-five compliant molecules. If your target space is macrocycles, PROTACs, or covalent inhibitors, the rankings in Table 3 may invert. Think about it: i've seen a method that ranked 4th on ZINC jump to 1st on a focused library of kinase inhibitors. Domain shift is real.
What Actually Works — Practical Takeaways
After running variants of this benchmark half a dozen times across projects, here's what I've learned matters more than the method name:
1. Representation Choice > Architecture Choice
SELFIES > SMILES > Graph adjacency matrices for autoregressive decoding. The validity gains from SELFIES (no syntax errors by construction) are worth 15
% in validity alone. If you're generating molecules and your first step is "fix invalid SMILES," you've already lost.
2. Reward Design is Everything (Literally)
A poorly designed reward function can sink even the most sophisticated architecture. I've seen REINVENT variants where changing the novelty component from unweighted to kernel-based improved performance by 20%. The reward isn't just a scalar—it's your entire optimization landscape.
3. Seed-to-Seed Variance is Your Real Enemy
That 3-standard-deviation band isn't academic noise—it's the difference between a method that works and one that doesn't. I now require 5+ seeds for any serious comparison, and I plot the full distribution, not just means.
4. The "Best" Method Depends on Your Constraints
Need speed? GB-DQN with parallel sampling. Because of that, need controllability? VAE with conditional latent space. Need peak performance? RL with carefully tuned rewards. There's no universal winner—only the right tool for your specific bottleneck.
Beyond the Table: What's Missing
The current benchmark captures validity, novelty, and uniqueness under drug-like constraints. But real drug discovery demands more:
- Synthetic accessibility at scale
- Multi-objective optimization (potency + ADMET + synthesizability)
- Activity prediction calibration (does high predicted activity actually mean high activity?)
- Diversity in meaningful chemical space regions
These remain largely unaddressed in the literature.
The Next Frontier: Active Learning Loops
The most promising direction I see isn't better generative models—it's closing the loop between generation and evaluation. Instead of generating 10k molecules and hoping some are active, we're moving toward:
- Generate small batch (50-100)
- Predict activity + synthesize feasibility
- Select top candidates for actual synthesis/testing
- Update model with real data
- Repeat
This active learning approach reduces the gap between in silico generation and wet-lab reality. It's slower per iteration but dramatically more efficient overall.
Final Thoughts
Table 3 isn't a ranking—it's a snapshot of trade-offs in a complex optimization landscape. In real terms, the "best" method depends entirely on whether you prioritize validity, speed, controllability, or peak performance. More importantly, it's a starting point, not an endpoint.
The field has matured beyond "which architecture wins.Day to day, " We're now asking "how do we build generative systems that actually accelerate discovery? " That requires thinking about the entire pipeline—from molecule generation to biological validation—not just the generative step itself.
The real work begins after the table.
