Skip to content

Latest commit

 

History

History
50 lines (38 loc) · 1.48 KB

04-LOADING-MODEL-ARGS.md

File metadata and controls

50 lines (38 loc) · 1.48 KB
Raw

4. LOADING MODEL ARGS

loadModelArgs is called if includeTensors is true.

In this phase, the model configuration file "params.json" is parsed as ModelArgs object via JSON parser.

Then, loadModelArgs function takes its result and assigns it to result object's ModelArgs property: model.ModelArgs.

from src/model/loader.go

func LoadModelEx(modelDir string, includeTensors bool, includeVocab bool) (*Model, error) {
    if includeTensors {
        ...
        err = loadModelArgs(modelDir, model)
        if err != nil {
            return nil, err
        }
    }
    ...
}

Now, we have model.ModelArgs object with similar values:

Dim: 4096
N_Layers: 32
N_Heads: 32
N_KVHeads: 8 // if -1, it will be equal to N_Heads
VocabSize: 128256 // if -1, it will be size of tokenizer.model
MultipleOf: 1024 // make SwiGLU hidden layer size multiple of large power of 2
FFNDimMultiplier: 1.3
NormEpsilon: 0.00001
RopeTheta: 500000
UseScaledRope: true
MaxSequenceLength: 2048
N_Rep: 0 // will be calculated further
HeadDim: 0 // will be calculated further