Fine Tuning Language Models With Axon

The fine tuning, evaluation, Nx serving and embedding code from ElixirConf 2023

Tune

The tune module for fine tuning with Roberta

Example.tune()

Evaluate

The evaluate module for evaluation

Example.evaluate()

Embeddings

The language model embeddings can be generated from the token ids

Example.get_token_ids() |> Example.embeddings()

Nx Serving

The cancellations can be generated for a given dataset with Nx Serving

Example.cancellations()

To use the cancellations function above you must first fine tune the model to generate cancel.axon then you need to uncomment this in the mix.exs file and run iex -S mix run