Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

supporting fine tuning for OAI #441

Closed
Intex32 opened this issue Sep 21, 2023 · 4 comments
Closed

supporting fine tuning for OAI #441

Intex32 opened this issue Sep 21, 2023 · 4 comments

Comments

@Intex32
Copy link
Member

Intex32 commented Sep 21, 2023

contains multiple subtasks

  • server: adapt query endpoint to accept custom model name (depends on [DRAFT] Server conversations #413)
  • xef-core: implement an API for querying fine-tuned models
  • clone OAI endpoints for fine tuning (to later intercept and collect metrics)

https://platform.openai.com/docs/guides/fine-tuning
web api: https://platform.openai.com/docs/api-reference/fine-tuning/create
to estimate fine tuning costs: https://colab.research.google.com/drive/11Yl7cQ3vzYZzrzRaiQEH9Y9gAfn5-Pe6?usp=sharing

my experience with fine-tuning

  • general high level knowledge of ML applies (accuracy, learning rate)
  • overfitting possible, after a couple of epochs no more progress as training accuracy was 1.0
  • basic validation on file upload (complained about double line break instead of single)
  • actual validation performed during when training job is started (eg. too few lines)
  • training of ron-v2 model with 12 epochs and 50 lines took about half an hour
@Intex32
Copy link
Member Author

Intex32 commented Sep 26, 2023

As this issue partly depend on #413, querying fine tuned models from the xef-server is not support yet. Currently, all requests are streamed from OpenAI without going through the xef-core logic.

Resolving a model based on it's name and it's base model's name might look like this later:

fun spawnCustomModel(provider: Provider, baseModelName: String, fineTunedModelName: String): LLM {
    val baseModel = when(provider) {
        Provider.OPENAI -> com.xebia.functional.xef.conversation.llm.openai.OpenAI().supportedModels().find { it.modelType.name == baseModelName }
        else -> TODO()
    } ?: error("base model $baseModelName not found")
    return if(baseModel is FineTuneable)
        baseModel.fineTuned(fineTunedModelName)
    else error("model $baseModelName supports no fine tuning")
    // we cannot know at this point if the fine tuned model exists
}

@raulraja
Copy link
Contributor

This issue should not depend on #415 , we are not following that approach for now. We need to add the fine-tuning endpoint to the Xef server following what it's doing in the main now and forwarding directly to Open AI. I am happy to discuss this online in Slack if you need further clarification.

@Intex32 Intex32 linked a pull request Sep 28, 2023 that will close this issue
Intex32 added a commit that referenced this issue Oct 3, 2023
* query fine tuned models (from branch #441-fine-tuning-oai)

* spotless

* clean build, make tests and mocks compile

* changes according to pr comments

---------

Co-authored-by: José Carlos Montañez <[email protected]>
@Intex32
Copy link
Member Author

Intex32 commented Oct 4, 2023

Aallam just closed my issue (aallam/openai-kotlin#236) regarding implementing the new fine tuning API. Foreseeably, there is going to be a new release soon. We could implement the actual fine tuning now more easily. But I question if this is actually of any value at this point. You @raulraja have to decide what has priority now. For later, I can imagine capturing the metrics like accuracy etc from the training that OAI provides to us.

@Intex32
Copy link
Member Author

Intex32 commented Oct 4, 2023

@Intex32 Intex32 closed this as completed Nov 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants