Skip to content

Commit

Permalink
rewording doc and updated parameters
Browse files Browse the repository at this point in the history
Co-authored-by: Sébastien Han <[email protected]>
Co-authored-by: Russell Bryant <[email protected]>
Signed-off-by: Oindrilla Chatterjee <[email protected]>
  • Loading branch information
3 people committed Jun 24, 2024
1 parent ea64144 commit 79fe0fd
Showing 1 changed file with 22 additions and 10 deletions.
32 changes: 22 additions & 10 deletions docs/sdg/sdg-api-interface.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ We propose the following structure for the SDG library. There will be config fil

## CLI

The CLI client uses the instructlab SDG library and provides it a run configuration with input parameters.
The CLI client uses the instructlab SDG library and provides it a run configuration with input parameters. The following represents a sample of what Library usage could look like.

```python
# cli_driver.py
Expand All @@ -40,14 +40,27 @@ cli_sdg = SDG(run_config, client) # run config has all the variables like num_s
generated_samples = cli_sdg.generate()
```

The run configuration contains the parameters needed to run the SDG code library as well as points to the templates needed to run the SDG code as well as the prompt template and the default model system prompt template.
As an initial integration milestone, we will modify the `generate_data` function in `instructlab.sdg.generate_data` to make use of the updated SDG API. This is the function the `ilab` CLI already uses, so modifying this implementation will allow us to get the updated SDG API in place without disrupting the CLI integration.

CLI integration will require additional changes later to allow passing in customizations to the SDG pipeline, but we will treat that as a follow-up implementation milestone.

The run configuration includes the necessary parameters for executing the SDG code library. It specifies the templates required for running the SDG code, the prompt template, and the default model system prompt template.

* `num_samples` is the number of synthetic samples that you wish to generate per seed example.
* `num_procs` is the number of parallel processes that you want to run
* `max_retry` is the maximum number of non-greedy retries you want to make if the `num_samples` is not reached. The number of samples in the generated output will be the samples achieved until `max_retry` is reached.
* Pipeline steps contains the steps that you want to invoke in the SDG pipeline and the prompt configurations per step. The variable names of the blocks can be anything and the prompt configurations must be compatible with the teacher model.
* `max_new_tokens` is the maximum number of tokens we want to generate. In other words, the size of the output sequence, not including the tokens in the prompt.
* `model_name` is the teacher model we would want to use to generate the synthetic data.
* `model_template` and `stop_token` are the parameters for generation template.
* `model` is the name of the served up teacher model we would want to use to generate the synthetic data.
* `model_prompt`:

Check failure on line 55 in docs/sdg/sdg-api-interface.md

View workflow job for this annotation

GitHub Actions / markdown-lint

Trailing spaces

docs/sdg/sdg-api-interface.md:55:18 MD009/no-trailing-spaces Trailing spaces [Expected: 0 or 2; Actual: 1] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md009.md
* `client` points to an OpenAI client used to interface with the model. Example of a client:

```python
client = OpenAI(
api_key=openai_api_key,
base_url=openai_api_base,
)
```

```yaml
# run_config.yaml
Expand All @@ -61,9 +74,8 @@ pipeline_steps:
prompt_template: "configs/filter_q.yaml"
max_new_tokens: 10000
# model parameters for generation
model_name: mistralai/Mixtral-8x7B-Instruct-v0.1
# generation template
model_template: <s> [INST] {sys_prompt} {prompt} [/INST]
stop_token: </s>

```
model_name: mixtral-model
model_prompt: '<s> [INST] {prompt} [/INST]'
client: client
num_procs: 8
```

0 comments on commit 79fe0fd

Please sign in to comment.