switch to the granite provided by rhel ai rather than huggingface

[cooktheryan]

ilab model download --repository docker://registry.redhat.io/rhelai1/granite-7b-starter --release latest

Transition the above command into the pipeline rather than using huggingface

[tumido]

Sadly ilab model download makes things only complicated:

• It requires ilab config init or a precreated config file - we can fake it but still - it's a nuisance
• If an OCI reference is used (which has to be prefixed with docker:// 🤦), it basically does following:
  1. transform the --repository, --release and --model-dir args into skopeo copy {repository}@{release} oci:/{model-dir} command
  2. Path magic to transform OCI blob into a OCI artifact layer/file name via symlinks

IMO it would be much better if we could use https://github.com/containers/ramalama or oras or something else instead of relying on this head-scratch logic by ilab...

The most demanding part would be to re-create/undo the symlinking, since we can't rely on that in KFP.

[tumido]

Additionally, ilab model download also allows to download models from HuggingFace, however their solution requires HuggingFace token to be always present, compared to our solution which doesn't require that. This would result in regression or added logic/requirement to provide a HuggingFace token...

https://github.com/instructlab/instructlab/blob/fa6073daf084f24346af05c021feb8f1eaa4cc44/src/instructlab/model/download.py#L73-L78

[leseb]

@tumido, for a small clarification, the token is not needed if we download an image from the instructlab repository. By the way, I'm curious how other projects handle the token.

[tumido]

Yeah, correct. However so far we've been using ibm-granite/granite-7b-base as the default base model. This model can be downloaded without auth, but not via ilab model download (as a workaround, setting HF_TOKEN to invalid, stub value, works just fine here).

[tumido]

FTR, we took this back to stakeholders, since we're not sure at the moment how aligned is OCI Artifacts or even Hugging face as model sources relevant to Phase 3 as it should stay close to Phase 2 experience. Our offer is to revert to sourcing the model from object storage in the phase 3 as well.