Fine tuning guide?

[niccolox]

congratulations on doing this for world peace and the future of humanity

can you give some information, perhaps even a dedicated document on fine tuning Sky-T1-32B-Flash?

fine tuning is an important but often overlooked aspect to LLM use and I think would remove much of the WTF factor when implemented

[tyler-griggs]

Thank you for the kind comments!

Good point, we don't have a simple document that explains how to reproduce the Flash model, end-to-end.

As a start, the README page found here (https://github.com/NovaSky-AI/SkyThought/tree/main/skythought/tools) describes all of the tools and scripts used to produce both Sky-T1-32B-Preview and Flash, and all necessary code is readily available in the directory.

[niccolox]

working towards tuning

first step, basic mechanics of ollama releases

fwiw, I created my own ollama release and uploaded to the site, just a shim around prior work including the hf.co link above

https://ollama.com/niccolox/sky-t1-flash-cline

blog post https://www.niccolox.com/p/355-making-an-ollama-model-release-with-sky-t1-flash-for-cline
github https://github.com/Devekko-Dev/devekko.llms/tree/master/sky-t1-flash-cline

its really a shim, but I intend to swim upstream and next fine tune, than train a model

THANKS!

[niccolox]

btw, this https://github.com/NovaSky-AI/SkyThought/tree/main/skythought/tools gives a 404

[lynnliu030]

@niccolox Hi, thanks for sharing! We refactor our codebase in the latest PRs so the tools folder no longer exists. Did you find this link in the current main README?