Add Glaive conversation format support

[brianfitzgerald]

Adds support for the Glaive function calling dataset. This dataset has 2 columns, system and chat; and an additional tool role in the conversation. This role contains the output from a tool call.

In this PR we add support for the Glaive dataset, which we convert to the ShareGPT format; tool calls are masked, and consecutive tool calls are merged to one message.

How has this been tested?

I SFT trained a TinyLlama lora with the config below.

base_model: TinyLlama/TinyLlama-1.1B-Chat-v1.0
model_type: LlamaForCausalLM
tokenizer_type: LlamaTokenizer
is_llama_derived_model: true

load_in_8bit: true
load_in_4bit: false
strict: false

datasets:
  - path: glaiveai/glaive-function-calling-v2
    type: sharegpt.load_glaive
    conversation: chatml_glaive

[winglian]

@brianfitzgerald thanks for this PR. the ability to train models for function calling is very much needed. Currently, this PR breaks some of the existing tests for sharegpt datasets. I think if we can get the changes fixed so as not to break existing functionality as well as add new data fixtures and tests for the new functionality, we can get this merged. lmk if you'd like some help with this.

[ehartford]

This is an exciting development.
I will use this to add glaive to all my models

[winglian]

thank for putting this all together! 🚀

[hasan9090]

Thanks for the updated functionality. I would like to similarly treat any other tool handling dataset like for example this version of glaive lilacai/glaive-function-calling-v2-sharegpt , which already is in sharegpt, or similarly any other custom dataset of that format. When trying to do so and orientating at the above mentioned configs, I cannot make it work to recognize the tool role as 3rd role in axolotl and don't really know if it is possible and how the config has to look like.
From the above , I think setting type also to sharegpt.load_glaive would not work as the dataset is already in sharegpt and this function seems to convert it first to that format? But I do think I need to set the conversation to "chatml_glaive" since this is handling the tool role? When using the following config it is not working for me:

datasets:
-path: lilacai/glaive-function-calling-v2-sharegpt
type: sharegpt
conversation: chatml_glaive
field_human: human
field_model: gpt

It would be nice if someone could help.
Thanks Hasan