Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streaming ww dev #168

Merged
merged 183 commits into from
Jan 20, 2025
+209,143 −5
Merged

Streaming ww dev #168

Changes from 1 commit
Commits
Show all changes
183 commits
Select commit Hold shift + click to select a range
b16b8d9
moved training_torch to experimental and added a README
jeremy-syn Oct 9, 2023
31dd4be
starting to move code into here
jeremy-syn Jan 22, 2024
0626c97
some updates to streaming wakeword
jeremy-syn Jan 27, 2024
03c9d8b
updated streaming wakeword model to be the actual candidate DS-TCN mo…
jeremy-syn Jan 27, 2024
725cb69
set default features to 40-D LFBEs
jeremy-syn Jan 27, 2024
8e533a3
changed num_classes to 3 in train.py
jeremy-syn Jan 29, 2024
9c15b56
demo notebook (in progress) added
jeremy-syn Jan 29, 2024
5475c7b
demo notebook runs through small training run
jeremy-syn Jan 29, 2024
c7138c4
added count_labels and is_batched()
jeremy-syn Jan 29, 2024
b089ce0
demo now adds some silence waveforms (which then have noise added) to…
jeremy-syn Jan 29, 2024
6c8f06d
updated get_dataset (mostly copied from demo.ipynb) and removed use_s…
jeremy-syn Jan 29, 2024
e7922f7
fixed default model architecture flag
jeremy-syn Jan 29, 2024
5a9a235
fixed some issues with building model
jeremy-syn Jan 29, 2024
7193979
added from_logits argument to model.compile
jeremy-syn Jan 31, 2024
5e74320
cleanup changes to get_dataset and demo notebook
jeremy-syn Feb 2, 2024
4e02106
catching up on edits
jeremy-syn Feb 2, 2024
5d9932c
keras_model does not need tf datasets module
jeremy-syn Feb 2, 2024
4edc86e
Merge branch 'streaming_ww_dev' of github.com:mlcommons/tiny into str…
jeremy-syn Feb 2, 2024
b72d850
cleaning up demo notebook
jeremy-syn Feb 5, 2024
fc7a4d8
cleaning up demo notebook
jeremy-syn Feb 5, 2024
e9de194
made path to speech commands dataset easier to config per location/us…
jeremy-syn Feb 5, 2024
73e3d63
beginning of code to test long waveform in python
jeremy-syn Feb 5, 2024
2d69501
some updates
jeremy-syn Feb 7, 2024
6916250
added option to read in model config file
jeremy-syn Feb 7, 2024
e91a3ed
set validation set to incorporate background noise. also fixed issue…
jeremy-syn Feb 7, 2024
eb4d120
moved code to add silent (or white noise) frames to dataset into its …
jeremy-syn Feb 7, 2024
29da207
fixed argument error
jeremy-syn Feb 7, 2024
c6f23f1
added post-training quantization
jeremy-syn Feb 10, 2024
f478259
several changes in order to use QAT and evaluate on long waveforms:
jeremy-syn Feb 11, 2024
b46d797
notebook updated to work with last commits on get_data, keras_model
jeremy-syn Feb 11, 2024
c6185a7
changed default LR schedule to reduce_on_plateau so it scales better …
jeremy-syn Feb 11, 2024
9dd2f1b
some more edits to get QAT working
jeremy-syn Feb 11, 2024
19470b0
changed labels to one-hot to work with precision/recall metrics. Also…
jeremy-syn Feb 13, 2024
772e6cc
added notebook to develop tflite model for feature extraction
jeremy-syn Feb 13, 2024
a6ac856
removed some prints from get_dataset. added an evaluation to train
jeremy-syn Feb 13, 2024
7302a8d
adjusted reduce lr on plateau settings
jeremy-syn Feb 13, 2024
20abafc
Merge branch 'streaming_ww_dev' of github.com:mlcommons/tiny into str…
jeremy-syn Feb 13, 2024
8ef587c
fixed plotting error
jeremy-syn Feb 13, 2024
deec7c9
working on different options to run the feature extractor on MCU
jeremy-syn Feb 14, 2024
468a344
small changes to notebook
jeremy-syn Feb 17, 2024
2964369
removed old commented-out code that loaded pre-built dataset
jeremy-syn Feb 17, 2024
ee12fe4
tflite_feature_extractor.ipynb very much a work in progress
jeremy-syn Feb 17, 2024
ac40098
added setup instructions and a to the streaming wakeword benchmark (…
AlexMontgomerie Mar 6, 2024
380f117
Merge branch 'master' of gh-syn:mlcommons/tiny
jeremy-syn Apr 15, 2024
9798093
cache datasets after spectrogram computation to avoid recomputing the…
jeremy-syn May 21, 2024
c584f23
fixed data_dir default to point to speech_commands_v0.02
jeremy-syn May 21, 2024
12d727f
fixed data_dir default to point to speech_commands_v0.02
jeremy-syn May 21, 2024
0e122cf
added BooleanOptionalAction to correctly parse boolean Flags
jeremy-syn May 21, 2024
7bfefc4
fixed parsing of bool args (use_qat, run_test_set) to work with pytho…
jeremy-syn May 21, 2024
49330a9
changed so parse_command raises exception on unrecognized flags
jeremy-syn May 26, 2024
6dc9314
changed so parse_command raises exception on unrecognized flags
jeremy-syn May 26, 2024
252cafc
added foreground scaling args foreground_volume_min, _max to train on…
jeremy-syn May 28, 2024
2d80a04
set is_training true for ds_val so it gets noise added
jeremy-syn Jun 3, 2024
7270a9a
Merge branch 'streaming_ww_dev' of github.com:mlcommons/tiny into str…
jeremy-syn Jun 3, 2024
5cf7657
edits to str ww model
jeremy-syn Jun 3, 2024
eca617c
edits to data set building
jeremy-syn Jun 3, 2024
0e7ac0f
saved training history along with plot
jeremy-syn Jun 3, 2024
8bedf74
Merge branch 'streaming_ww_dev' of gh-syn:mlcommons/tiny into streami…
jeremy-syn Jun 3, 2024
7ac2852
removed average pooling, increased initial feature stride
jeremy-syn Jun 3, 2024
01470d7
Merge branch 'streaming_ww_dev' of github.com:mlcommons/tiny into str…
jeremy-syn Jun 3, 2024
3b0fb65
Fixed bug where np.random is only evaluated at graph creation, so all…
jeremy-syn Jun 4, 2024
3f3dabf
fixed several places where np.random was used in a tf graph, resultin…
jeremy-syn Jun 5, 2024
4fd29c1
widened filters in 2nd,3rd layers to 128
jeremy-syn Jun 5, 2024
6f8a272
changed back from 32 LFBEs to 40
jeremy-syn Jun 5, 2024
ac85b48
Merge branch 'streaming_ww_dev' of github.com:mlcommons/tiny into str…
jeremy-syn Jun 5, 2024
d7c607c
minor cleanup -- whitespace, removing old commented out lines, etc.
jeremy-syn Jun 6, 2024
870ac09
fixed error - val set was using target words from training set
jeremy-syn Jun 6, 2024
8e9a0ee
minor cleanup -- whitespace, removing old commented out lines, etc.
jeremy-syn Jun 6, 2024
264eae7
Merge branch 'streaming_ww_dev' of github.com:mlcommons/tiny into str…
jeremy-syn Jun 6, 2024
6a3b15e
changed ordering in data prep, now shuffle before batching
jeremy-syn Jun 6, 2024
76413de
adding current version of trained and quantized streaming ww model
jeremy-syn Jun 6, 2024
dec0abb
minor edits/cleanup
jeremy-syn Jun 7, 2024
b11d123
changed Flags.num_train_samples to num_samples_training. same for tes…
jeremy-syn Jun 7, 2024
b1b1204
added 1st pass at get_data_config(), refactoring dataset build
jeremy-syn Jun 8, 2024
d313d65
refactored dataset building. train.py runs now, have not tested perfo…
jeremy-syn Jun 9, 2024
c4c654b
setup_example is work in progress, just capturing progress
jeremy-syn Jun 10, 2024
a6028cf
train.py runs but gives random-level validation accuracy. demo noteb…
jeremy-syn Jun 10, 2024
d6e23bb
flag parsing used 'train' instead of 'training' and therefore was not…
jeremy-syn Jun 14, 2024
de7a198
updated demo to match changes in data
jeremy-syn Jun 15, 2024
af713d8
minor updates
jeremy-syn Jun 22, 2024
b50d8fc
dumps options as json into plot_dir
jeremy-syn Jun 22, 2024
7168f50
fixed demo to work with new get_data code. moved take after shuffle …
jeremy-syn Jun 22, 2024
859b3b3
moved softmax to inside the model; adjusted loss function accordingly
jeremy-syn Jun 24, 2024
36cd5b8
moved softmax calculation into the model
jeremy-syn Jun 29, 2024
ea3eb2e
working on true pos/false pos computation
jeremy-syn Jun 29, 2024
c7d2f1c
resolved merge
jeremy-syn Jun 29, 2024
35d4394
fixed error, post-wakeword extension was being added twice
jeremy-syn Jun 30, 2024
9184f66
fixing notebook counting of true/false positives
jeremy-syn Jul 1, 2024
8e671d0
removed commented-out code; added zero2nan()
jeremy-syn Jul 1, 2024
a1b5673
added multiple background noise paths, can split long bg files into s…
jeremy-syn Jul 1, 2024
37ca39d
change QAT initial LR to Flags.lr, LR is too small after float pre-tr…
jeremy-syn Jul 1, 2024
7f14c47
Merge branch 'streaming_ww_dev' of github.com:mlcommons/tiny into str…
jeremy-syn Jul 1, 2024
811db67
fixed cmd line arg processing to accomodate multile bg noise paths
jeremy-syn Jul 1, 2024
53bb590
removed commented out code from demo notebook
jeremy-syn Jul 2, 2024
e1d75f9
convert only-target dataset to numpy array and back so cardinality() …
jeremy-syn Jul 7, 2024
3eb0329
refactored num_silent, num_repeats in to fraction_silent and fraction…
jeremy-syn Jul 9, 2024
ec0d024
fixed cmd line arg processing to accomodate multile bg noise paths
jeremy-syn Jul 9, 2024
5ecfeec
fixing code for smaller datasets
jeremy-syn Jul 16, 2024
6a2d715
catching up on demo edits
jeremy-syn Jul 16, 2024
a2a5f8b
added code to run quantized model on long waveform
jeremy-syn Jul 16, 2024
45d0b72
working on long wav file creation; added poisson process to place wak…
jeremy-syn Jul 16, 2024
609f901
updated long wave creation, need to move it to a separate file soon. …
jeremy-syn Jul 16, 2024
d059f4a
increased number of background files from 50 to 100
jeremy-syn Jul 16, 2024
9d11002
added code to illustrate false detects/rejects
jeremy-syn Jul 18, 2024
95dda3e
updating background noise creation to avoid train/val duplicates
jeremy-syn Jul 18, 2024
3473dbb
added exclude_background_files.txt
jeremy-syn Jul 18, 2024
5114246
put code to build the long test wav into its own (two) files
jeremy-syn Jul 19, 2024
ccec9f4
added eval_long_wav.py to test fpr, fnr on a long wav
jeremy-syn Jul 20, 2024
6910b1c
made build_long_wav work with the musan_path from streaming_config.json
jeremy-syn Jul 20, 2024
04b0608
made build_long_wav work with the musan_path from streaming_config.json
jeremy-syn Jul 20, 2024
c599598
fixed a typo
jeremy-syn Jul 20, 2024
0dced76
fixed issue with path construction in long wav spec
jeremy-syn Jul 20, 2024
223f315
added l2 reg to conv layers
jeremy-syn Jul 21, 2024
76de629
added L2 reg to conv layers
jeremy-syn Jul 21, 2024
2c77d4e
removed some old commented-out code
jeremy-syn Jul 21, 2024
1d1a112
eval_long_wav can now test either h5 models or tflite models
jeremy-syn Jul 21, 2024
c6332f2
Merge branch 'streaming_ww_dev' of gh-syn:mlcommons/tiny into streami…
jeremy-syn Jul 21, 2024
7724e98
added script to create indices into the val set for calibration
jeremy-syn Jul 21, 2024
05cf6d3
code to create calibration set is working
jeremy-syn Jul 22, 2024
7cbcd9f
fixed quantize.py to work with extracted npz calibration set
jeremy-syn Jul 22, 2024
df1d3e8
adjusted volume of foreground and background for testing
jeremy-syn Jul 22, 2024
d341993
added code to save spectrogram in build_long_wav.py
jeremy-syn Jul 22, 2024
e30cbd3
demo notebook should now work with current code
jeremy-syn Jul 23, 2024
0fba129
separated augmentation (built by get_augment_wavs_func()) and feature…
jeremy-syn Jul 25, 2024
40cbecf
made l2 reg parameter a commmand line flag
jeremy-syn Jul 25, 2024
6c6c330
fixed eval_long_wav to work with feature extractor changes
jeremy-syn Jul 25, 2024
f480533
added validation set measurments to eval_long_wav.py
jeremy-syn Jul 25, 2024
8cfc991
moved eval_long_wav to evaluate.py
jeremy-syn Jul 25, 2024
7801dd2
added threshold=0.95 to precision/recall metrics to match evaluate.py
jeremy-syn Jul 25, 2024
23e4434
added a list of 'bad' marvin wav files. modified build_long_wav_spec …
jeremy-syn Jul 26, 2024
2672489
edited comment on saved_model_path to reflect evaluate.py
jeremy-syn Jul 26, 2024
d0f30f8
added bad_marvin_files.txt
jeremy-syn Jul 26, 2024
6d8cb29
fixed error in number of unknown samples for reduced runs
jeremy-syn Jul 27, 2024
ba80cc1
renamed build_long_wav_spec.py -> build_long_wav_def.py to avoid ambi…
jeremy-syn Jul 27, 2024
d626e61
renamed features back to audio to allow easy skipping of feature extr…
jeremy-syn Jul 28, 2024
8eb1f6c
minor edits
jeremy-syn Jul 29, 2024
5960e11
removed debug print statement
jeremy-syn Jul 29, 2024
b0c424a
fixed code for tflite models
jeremy-syn Jul 29, 2024
0ba2b18
adjusted some default training params
jeremy-syn Jul 29, 2024
547569b
catching notebook up to other code
jeremy-syn Jul 29, 2024
6341ee8
clearing out some debug prints
jeremy-syn Jul 29, 2024
5a16bb9
added trained model
jeremy-syn Jul 29, 2024
fad10d4
Merge branch 'streaming_ww_dev' of github.com:mlcommons/tiny into str…
jeremy-syn Jul 29, 2024
0c0c89f
moved label_count out of model_settings into a flag
jeremy-syn Jul 29, 2024
a348974
Merge branch 'streaming_ww_dev' of gh-syn:mlcommons/tiny into streami…
jeremy-syn Jul 29, 2024
a707678
minor edits
jeremy-syn Jul 29, 2024
6836643
added random timeshift
jeremy-syn Aug 4, 2024
143dd8e
added a couple more bad marvins to exclude
jeremy-syn Aug 4, 2024
2461453
added random timeshifting to augmentation function
jeremy-syn Aug 4, 2024
9c89eb7
added flag to enforce a minimum SNR
jeremy-syn Aug 5, 2024
c91b792
centralized data paths in streaming_config.json (no command line argu…
jeremy-syn Aug 5, 2024
c02b502
removed some obsolete cmd line args and modified get_dataset to respe…
jeremy-syn Aug 5, 2024
0a88769
fixed evaulate.py to work with changes on speech_commands_path
jeremy-syn Aug 5, 2024
f0cd836
changed evaluate and quantize to use model_init_path, so by default t…
jeremy-syn Aug 5, 2024
61153a4
adjusted trainign params
jeremy-syn Aug 5, 2024
42d51a0
adjusted trainign params
jeremy-syn Aug 5, 2024
9e9e2ad
updated long wav info
jeremy-syn Aug 5, 2024
2258d55
updated README
jeremy-syn Aug 5, 2024
15a2d89
updated reference model
jeremy-syn Aug 5, 2024
b49eb7d
removed some info messages
jeremy-syn Aug 6, 2024
9c655d2
add line to create plot_dir if it does not exist
jeremy-syn Aug 9, 2024
b9cd142
reduced noise level in long wav
jeremy-syn Aug 9, 2024
54de3e9
refactored command line argument parsing
jeremy-syn Aug 11, 2024
c354ac6
refactored command line argument parsing
jeremy-syn Aug 12, 2024
7e69344
Merge branch 'streaming_ww_dev' of github.com:mlcommons/tiny into str…
jeremy-syn Aug 12, 2024
478f6ae
fixed some errors in README
jeremy-syn Aug 12, 2024
7d97168
fixed quantize to use saved_model_path instead of model_init_path
jeremy-syn Aug 12, 2024
bd96237
added calibration_samples.npz
jeremy-syn Aug 12, 2024
2bc1175
fixing argument processing for evaluate.py to work with either tflite…
jeremy-syn Aug 12, 2024
8e7b718
fixed typo in evaluate.py
jeremy-syn Aug 12, 2024
8c63415
fixed typo in evaluate
jeremy-syn Aug 12, 2024
7608316
fixing merge
jeremy-syn Aug 12, 2024
30308c9
updated tflite model
jeremy-syn Aug 12, 2024
1fea3a0
fixed issue with plot_dir
jeremy-syn Aug 12, 2024
0c73cf8
Merge branch 'streaming_ww_dev' of gh-syn:mlcommons/tiny into streami…
jeremy-syn Aug 12, 2024
fb4e6a2
ignoring trained models other than reference model
jeremy-syn Aug 12, 2024
68f53b8
updated readme
jeremy-syn Aug 12, 2024
02953c9
updated demo notebook
jeremy-syn Aug 12, 2024
c280e5d
added note about the demo notebook to readme
jeremy-syn Aug 12, 2024
900e126
merged gitignore from master branch
jeremy-syn Aug 12, 2024
f9ec337
Merge branch 'master' into streaming_ww_dev
Peter-Chang Sep 9, 2024
5cfabb6
merged work from runner_dev_jeremy
jeremy-syn Jan 19, 2025
50104d0
Merge branch 'master' into streaming_ww_dev
jeremy-syn Jan 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Loading
Loading status checks…
ignoring trained models other than reference model
@jeremy-syn
jeremy-syn committed Aug 12, 2024

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode
commit fb4e6a2bf4d07cb42cc4c1aaf671eebabd4304d9
1 change: 1 addition & 0 deletions benchmark/training/streaming_wakeword/.gitignore
View file
Original file line number Diff line number Diff line change
@@ -2,3 +2,4 @@
.ipynb_checkpoints/
streaming_config.json
plots/
trained_models/str_ww_model.h5