Add new functionality to binarize preference datasets directly from distilabel

[plaguss]

Description

This PR adds a function to binarize a CustomDataset with a PreferenceTask:

from distilabel.utils import prepare_dataset
from datasets import load_dataset
from distilabel.tasks import JudgeLMTask
from distilabel.dataset import prepare_dataset

dataset = load_dataset("argilla/distilabel-intel-orca-dpo-pairs", split="train")
# Add the task that originally was used for the labelling
dataset.task = JudgeLMTask()
dataset_binarized_random = prepare_dataset(dataset, strategy="random", keep_ties=True)

Closes #263

[davidberenstein1957]

Looking good, added some comments :)

[davidberenstein1957]

again, some people might be like: "what is binarizing?"

[plaguss]

Could you both @davidberenstein1957 and @dvsrepo take a look at the new section of the docs for the dataset preparation/binarization? This commit e4602d4 contains the updates.

[dvsrepo]

@plaguss it looks good, I don't have cycles to do an in-depth review.

One suggestion: would it be possible to return the chosen and rejected in the OpenAI format, like this?

https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized?row=0

I think this would make it more interoperable

[plaguss]

@plaguss it looks good, I don't have cycles to do an in-depth review.

One suggestion: would it be possible to return the chosen and rejected in the OpenAI format, like this?

https://huggingface.co/datasets/HuggingFaceH4/ultrafeedback_binarized?row=0

I think this would make it more interoperable

It should be easy, will take a look and update the code
Done!

[davidberenstein1957]

@plaguss can you resolve the merge conflicts. After that you can merge it. @sdiazlor, this might be interesting to include or at least briefly mention in your tutorial too. #247

"this dataset is already binarized don't know about binarization or do you want to know how to binarize a dataset?" look here