Questions about 'initialize_from_state' for Fine-Tuning in NequIP-allegro

[hyunjinlee12]

Hi everyone,
I've been really enjoying working with NequIP recently!

I have a few questions and am looking forward to your response!

We’ve been working with a well-defined dataset for a specific purpose (e.g., bulk amorphous phases) and now aim to adapt it to a slightly different system (a surface amorphous structure). Based on this existing dataset, we’re planning to fine-tune our potential, and we believe that the initialize_from_state feature could be particularly useful in this process. I have a few questions about it and would appreciate your insights:

1. I noticed that NequIP recommend using initialize_from_state to start a new training run from an existing model state. Can this be considered a form of fine-tuning or transfer learning? How do you typically integrate this feature into your workflow, and what has been your experience with it?
2. Is there any theoretical background or reference material available on how initialize_from_state works? I couldn’t find much detail in the manual. Are there any resources you would recommend that discuss fine-tuning or transfer learning in the context of NequIP?
3. Performance Comparison: Fine-tuning a NequIP model pre-trained on a large dataset with a small additional dataset using initialize_from_state resulted in a noticeably faster training process. Is such a speedup generally expected? Additionally, does a potential trained with initialize_from_state typically achieve comparable accuracy to one trained entirely from scratch?

Thank you very much for your time!