How to debug merging performance failure

[Vincent-Li-9701]

Hi merging experts,

New to this methodology here. I tried to merge one classification finetuned weight with one relevance prediction weight. I noticed that the classification finetuned weight always have precedent over the relevance one. The merging destroyed the performance on relevance task. I tried out slerp, and breadcrumbs. This is really surprising to me. Just curious if it originates from the method or I did anything wrong. My configs attached below

models:
  - model: ../classification
merge_method: slerp
base_model: ../relevance
parameters:
  t:
    - value: 0.3 # fallback for rest of tensors
dtype: bfloat16

models:
  - model: ../classification
    parameters:
      density: 0.9 #[1, 0.7, 0.1] # density gradient
      gamma: 0.01
      weight: 1.0
  - model: ../relevance
    parameters:
      density: 0.9
      gamma: 0.01 #[0, 0.3, 0.7, 1] # weight gradient
      weight: 1.0
merge_method: breadcrumbs
base_model: ../Mistral-7B-Instruct-v0.2
dtype: float16

[Vincent-Li-9701]

After some extensive trial and errors.

1. It’s better to work with some existing mult-task checkpoints.
2. Tasks matter. Merging a checkpoint trained with hard task with one that’s trained with easy task is going to be destructive for the hard task.
3. Point 2 can be alleviate with adding a checkpoint that’s also mult-task tuned .