How to do ORPO with DPO template?

[yuanf8]

May I ask how we can do ORPO with the DPO template? Do we just set the pref_loss to orpo? And which parameter corresponds to the $\lambda$ in $L_{ORPO} =E(x,y_w,y_l)[L_{SFT} +λ·L_{OR}]$?

[hiyouga]

just set pref_loss: orpo

[yuanf8]

Which parameter is for $\lambda$ though? Is it pref_beta or pref_ftx?

[hiyouga]

pref_beta