Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pull in DPO loss #48

Open
wants to merge 50 commits into
base: main
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
e330ae3
changes for dpo
gdharuman11 Mar 20, 2024
5df25e7
moved dpo_training script
gdharuman11 Mar 20, 2024
461524e
commented out assert in load checkpoint func
gdharuman11 Mar 21, 2024
2c66771
tuple structure removed
gdharuman11 Mar 21, 2024
f983241
loading correct env
gdharuman11 Mar 21, 2024
1aeebca
modified main func
gdharuman11 Mar 21, 2024
aa9d9b2
run script with modified exec
gdharuman11 Mar 21, 2024
ded92e3
added modified load model weights func
gdharuman11 Mar 21, 2024
96a1d26
component wise loss func
gdharuman11 Mar 21, 2024
fa587cf
added arguments and datapaths for pref unpref datasets
gdharuman11 Mar 22, 2024
3c5b89a
added arguments for pref unpref datapaths
gdharuman11 Mar 22, 2024
4ef1df7
pref unpref dataloaders and iterators
gdharuman11 Mar 22, 2024
d89121e
testing model forward and loss compute
gdharuman11 Mar 22, 2024
e550a43
additional args for pref unpref data files
gdharuman11 Mar 22, 2024
a40fb14
returning tensor parallel logps
gdharuman11 Mar 22, 2024
e1f0396
tensor parallel logps for pref and unpref batches
gdharuman11 Mar 22, 2024
c22d090
computing pref and unpref logps from reference model
gdharuman11 Mar 22, 2024
5c3db71
dpo loss func restructured
gdharuman11 Mar 23, 2024
8915f2a
tracing the forward backward func
gdharuman11 Mar 25, 2024
eec8697
train step func
gdharuman11 Mar 25, 2024
09aa61b
training iteration
gdharuman11 Mar 25, 2024
6402e62
restored return in cross entropy
gdharuman11 Mar 27, 2024
1a04e84
post processing cross entropy return
gdharuman11 Mar 27, 2024
f956f30
calls a different script
gdharuman11 Mar 27, 2024
66ae844
working backprop
gdharuman11 Mar 27, 2024
de88521
training log func
gdharuman11 Mar 27, 2024
b50e140
loggin training metrics
gdharuman11 Mar 27, 2024
24737c0
text-sequence data files
gdharuman11 Apr 2, 2024
cbbd2a6
generate func post model training
gdharuman11 Apr 5, 2024
e1d3e6f
flops profiler enabled
gdharuman11 Apr 5, 2024
a976fd7
flops profiler enabled
gdharuman11 Apr 5, 2024
b964d58
removed dtype assertion that throws error
gdharuman11 Apr 5, 2024
ad780cb
changed to generation after dpo
gdharuman11 Apr 5, 2024
6a3b03e
newer preprocess data script
gdharuman11 Apr 5, 2024
9cb0089
batch preprocess
gdharuman11 Apr 5, 2024
a2aba88
convert to megatron format
gdharuman11 Apr 5, 2024
9e13d4e
write megatron data file
gdharuman11 Apr 5, 2024
702bb26
Add @hzheng s example `run_hzheng.sh`
saforem2 Apr 11, 2024
8d048ac
Prefer `torch.optim` for `--optimizer`
saforem2 Apr 11, 2024
859ec88
Remove global batch from `run_hzheng.sh`
saforem2 Apr 11, 2024
02961dc
minor changes
Jul 19, 2024
984fcf0
scaling run launch script for dgx cluster
Jul 19, 2024
907d158
minor changes
Jul 22, 2024
f7f7da3
dsflops log to file
Aug 1, 2024
f91ba8c
all data
Aug 1, 2024
be896de
reference optimizer removed
Aug 1, 2024
67ff707
with reference optimizer
Aug 1, 2024
4804eb7
scaling 3.5B model
gdharuman11 Aug 1, 2024
14e2bf3
scaling 70B model
gdharuman11 Aug 1, 2024
71db627
scaling 7B model
gdharuman11 Aug 1, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
tracing the forward backward func
gdharuman11 committed Mar 25, 2024
commit 8915f2ab2e632904c299b48b0a038c50f99f8522
1 change: 1 addition & 0 deletions megatron/core/pipeline_parallel/schedules.py
Original file line number Diff line number Diff line change
@@ -102,6 +102,7 @@ def forward_step(data_iterator, model):
else:
forward_backward_func = forward_backward_pipelining_without_interleaving
else:
print(f'On forward_backward_no_pipelining branch ..')
forward_backward_func = forward_backward_no_pipelining
return forward_backward_func