Why the depth always miss some lower depth or high depth part after training of the Depth_Head

[liyuan-qiu]

Hi, I am training the depth_head layer of depth-anything v2 with my own dataset, but it seems like that the original vits parameter performance is good, but after several training, the depth always seems like miss some part.
And I draw the line of the middle depth picture from the "ground truth", "vits" from depth-anything v2, "trained result" from my own training parameter as shown in the following picture.

I have tried different learning rate, different max/min_depth, but there always seems to be this kind of missing depth part. Can you give some hint about the missing depth?