Skip to content

Commit

Permalink
docs: typos conditioning.md
Browse files Browse the repository at this point in the history
  • Loading branch information
0xlws committed Oct 13, 2023
1 parent 92b9916 commit f4673bb
Showing 1 changed file with 7 additions and 7 deletions.
14 changes: 7 additions & 7 deletions docs/CONDITIONING.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ For now, we support 3 main types of conditioning within AudioCraft:
* Joint embedding conditioning methods for text and audio projected in a shared latent space.

The Language Model relies on 2 core components that handle processing information:
* The `ConditionProvider` class, that maps metadata to processed conditions leveraging
* The `ConditionProvider` class, that maps metadata to processed conditions, leveraging
all the defined conditioners for the given task.
* The `ConditionFuser` class, that takes preprocessed conditions and properly fuse the
conditioning embedding to the language model inputs following a given fusing strategy.
Expand All @@ -29,7 +29,7 @@ conditioning signals and feed them to the language model.

### Conditioners

The `BaseConditioner` torch module is the base implementation for all conditioners in audiocraft.
The `BaseConditioner` torch module is the base implementation for all conditioners in AudioCraft.

Each conditioner is expected to implement 2 methods:
* The `tokenize` method that is used as a preprocessing method that contains all processing
Expand All @@ -47,8 +47,8 @@ providing the processing logic for the given attribute.

Similarly to the conditioners, the condition provider works in two steps to avoid sychronization points:
* A `tokenize` method that takes a list of conditioning attributes for the batch,
and run all tokenize steps for the set of conditioners.
* A `forward` method that takes the output of the tokenize step and run all the forward steps
and runs all tokenize steps for the set of conditioners.
* A `forward` method that takes the output of the tokenize step and runs all the forward steps
for the set of conditioners.

The list of conditioning attributes is passed as a list of `ConditioningAttributes`
Expand Down Expand Up @@ -111,15 +111,15 @@ frozen or fine-tuned at train time to extract the text embeddings.
### Waveform conditioners

All waveform conditioners are expected to inherit from the `WaveformConditioner` class and
consists of conditioning method that takes a waveform as input. The waveform conditioner
consist of a conditioning method that takes a waveform as input. The waveform conditioner
must implement the logic to extract the embedding from the waveform and define the downsampling
factor from the waveform to the resulting embedding.

The `ChromaStemConditioner` conditioner is a waveform conditioner for the chroma features
conditioning used by MusicGen. It takes a given waveform, extract relevant stems for melody
conditioning used by MusicGen. It takes a given waveform, extracts relevant stems for melody
(namely all non drums and bass stems) using a
[pre-trained Demucs model](https://github.com/facebookresearch/demucs)
and then extract the chromagram bins from the remaining mix of stems.
and then extracts the chromagram bins from the remaining mix of stems.

### Joint embeddings conditioners

Expand Down

0 comments on commit f4673bb

Please sign in to comment.