DEVELOPMENT.md

Transformers

GenerationMixin.generate

https://github.com/huggingface/transformers/blob/main/src/transformers/generation/utils.py#L1704

GenerationConfig

https://github.com/huggingface/transformers/blob/main/src/transformers/generation/configuration_utils.py#L94

Llama.cpp

llama.__call__

https://github.com/abetlen/llama-cpp-python/blob/main/llama_cpp/llama.py#L1836

OpenAI

https://platform.openai.com/docs/api-reference/chat/create

vLLM

https://github.com/vllm-project/vllm/blob/main/vllm/entrypoints/llm.py#L312

Model

Meta-Llama-3_1-8B-Instruct

{
  "bos_token_id": 128000,
  "do_sample": true,
  "eos_token_id": [
    128001,
    128008,
    128009
  ],
  "temperature": 0.6,
  "top_p": 0.9,
  "transformers_version": "4.42.3"
}