Paper review - RFDiffusion

2025.03.17

Hojae Choi <chjko0206@gmail.com>

Metrics of this paper: RFDiffusion

Authors group: Baker Lab & collaborators ( MIT, etc.)

• David Baker (receiving 1/3 the 24" Novel Prize in chemistry)

D. Baker found 'Rosetta Commons'

Backgounds - Why de novo protein design?

• Nature has explored only a tiny subset of the possible protein landscape
• Evolution does not necessarily select for protein attributes that are desirable from a pharmaceutical/biotechnological perspective
  • in vitro solubility, stability, ease of production, low immunogeinity etc.
• de novo protein design allows us to derive new proteins with new functions and desirable attributes

Protein design Workflow

Backbone generation is the limiting factor

Diffusion models as an attrative framework for protein design

• can generate endlessly diverse outputs
• can operate directly on amino acid coordinates
• can condition on a wide range of inputs, and can be guided with auxiliary potentials

Related works

1. DDPM: It was used as main architecture of RFDiffusion model, generating 3D coordinate of frames.
2. ProteinMPNN_{(Dauparas et al. 2022)}: Generating residues (amino acids) from fixed protein backbone. It was used in generating aminoacids for fixed backbone
3. RoseTTaFold_{(Baek et al. 2021)}: Protein Folding prediction model. It was used in denoising module in RFDiffusion. It was used to prcessed by in fine-tuning
4. Alphafold2_{(Jumper et al. 2021)}: Protein Folding prediction model. It was used in evaluation of designed proteins structures

Preview

Generating (designing) protein binder. Various applications of RFdiffusion

How can we learn on protein structures

Challenges of proteins vs Images

• Strong geometric constraints (4 backbone heavy atoms, 3 covalent bonds, continuous chain)
• Must also have a sequence that can encode it

_{(Jumper et al., 2020;AF2)}

Challenges of proteins vs Images

How to represent a protein backbone

• atom level : too complicated translation and rotation
  

Challenges of proteins vs Images

Diffusion models as an attrative framework for protein design

RoseTTA Fold (author's previous work) can be easily used without much changes.

Training summary

• Dataset: PDB (384 amino acids; i.e. no cropping), clustered by sequence similairty
• 200 time steps (t=0: true structures, t=200: 3D Gaussian coordinates, uniform frame rotations)
• Self-conditioning of predicted structures between timesteps
• Model was initialised from RoseTTAFold structure prediction weights
  
  Main inputs: Input sequence vs. Diffused coordinates

Allowing the model to 'self-condition' improves RFdiffusion

• In RFdiffusion, the model receives its previous prediction as a template input (‘self-conditioning’).

• At each timestep of a trajectory (e.g. 200 steps), RFdiffusion takes from the previous step and and then predicts an updated structure ( ).

• The next coordinate input to the model ( ) is generated by a noisy interpolation towards .

  

Allowing the model to 'self-condition' improves RFdiffusion

Generate B.B. find sequence with Protein MPNN recapitulate the designs with AlphaFold2

Training from pre-trained RoseTTAFold weights makes training computationally tractable

Ablation study

performance: RF diffusion >> MSE Loss self-conditioning > fine-tuning > pretraining

Strengths

Unconditional generation

Discussion

• How to evaluate the designed proteins
  • Directly evaluated the designed proteins with experiments
• Is the RFdiffusion novel method?
  • my thought: More impact on the evaluatio with experiments
• What are the other factors affect to designing performance

Conclusion

• There are various use cases of protein design model impactful
• Finetunning generative model with pre-trained structual prediction models (or simulations)

TIL

• Protein can be represented by frames attributed with coordinates and rotation

Recommendations

Presentation of main authors

Q&A

References

1. Watson, Joseph L., David Juergens, Nathaniel R. Bennett, Brian L. Trippe, Jason Yim, Helen E. Eisenach, Woody Ahern, et al. 2023. “De Novo Design of Protein Structure and Function with RFdiffusion.” Nature 620 (7976): 1089–1100. https://doi.org/10.1038/s41586-023-06415-8.
2. Joe Watson & David Juergens, presented at Tuesday February 14th, 4-5 pm EST, RFDiffusion: Accurate protein design using structure prediction and diffusion generative models, https://www.youtube.com/watch?v=wIHwHDt2NoI, accessed at 2025.03.17
3. Baek, Minkyung, Frank DiMaio, Ivan Anishchenko, Justas Dauparas, Sergey Ovchinnikov, Gyu Rie Lee, Jue Wang, et al. 2021. “Accurate Prediction of Protein Structures and Interactions Using a Three-Track Neural Network.” Science 373 (6557): 871–76. https://doi.org/10.1126/science.abj8754.
4. Dauparas, J., I. Anishchenko, N. Bennett, H. Bai, R. J. Ragotte, L. F. Milles, B. I. M. Wicky, et al. 2022. “Robust Deep Learning–Based Protein Sequence Design Using ProteinMPNN.” Science 378 (6615): 49–56. https://doi.org/10.1126/science.add2187.
5. Jumper, John, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, et al. 2021. “Highly Accurate Protein Structure Prediction with AlphaFold.” Nature 596 (7873): 583–89. https://doi.org/10.1038/s41586-021-03819-2.

Supplementary

motif scaffolding