Why is memo work in LLM unlearning?

[Yuda-Jin]

In my opinion, memo only provide a set of Inversion Fact as training data.

[Carol-gutianle]

Thank you for your attention. MEMO not only provides a set of inverted facts, but also provides memory signals for unlearn. Because too many inverted facts will cause the unlearned model to be more inclined to normal wrong answers, we need to find the smallest subset that meets the unlearn requirements, and MEMO provides an anchor point for this process.

[Carol-gutianle]

And why do memory signals work? I think models have different abilities to master new knowledge with different levels of memory. For example, llama2 is more likely to learn to inverted facts with a high level of memory.