Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge real #28

Open
wants to merge 720 commits into
base: real-eval-test
Choose a base branch
from
Open

Merge real #28

Changes from 1 commit
Commits
Show all changes
720 commits
Select commit Hold shift + click to select a range
3ea2121
Iteration on the relative actions, likely not fully working yet
ankile Jul 15, 2024
a395766
run update
anthonysimeonov Jul 17, 2024
8402f56
Increase exploration noise and make noise constant in RPPO
ankile Jul 17, 2024
93d6379
Add a couple more conditions to test
ankile Jul 17, 2024
1006ef7
More experiments with more exploration noise
ankile Jul 24, 2024
3158ede
Update reference to right furniture bench
ankile Jul 24, 2024
d2be47a
Pull in rppo bug of not properly clearing actions
anthonysimeonov Jul 9, 2024
a0b52c2
fix conflicts
anthonysimeonov Jul 25, 2024
5d9193c
Small fix for better SPS measurement when restarting RPPO runs
ankile Jul 25, 2024
d8ce847
point to correct folder in furniture bench for saving rewards during …
anthonysimeonov Jul 25, 2024
f442a00
Merge branch 'main' into real-to-sim
ankile Jul 25, 2024
2f4aff8
Merge branch 'real-to-sim' of github.com:ankile/robust-rearrangement …
ankile Jul 25, 2024
75995d5
Merge pull request #18 from ankile/real-to-sim
ankile Jul 25, 2024
481b184
Update README.md
ankile Jul 26, 2024
118a2c6
new slurm run files
anthonysimeonov Jul 26, 2024
69dea38
small fixes for mlp ppo training
anthonysimeonov Jul 26, 2024
7213aa0
Some cleanup of rppo code
ankile Jul 26, 2024
0c77a96
Resolve merge conflict
ankile Jul 26, 2024
7ba5290
Implement first iteration of relative actions for vision
ankile Jul 27, 2024
195f6f6
Update README.md
ankile Jul 27, 2024
4576eee
Fix bug in relative actions
ankile Jul 28, 2024
902ec42
Add a GPU id option to RPPO
ankile Jul 28, 2024
9bc204c
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Jul 28, 2024
7abf8f2
Add more relative action experiments
ankile Jul 29, 2024
a9d39b6
Merge branch 'main' into relative-actions
ankile Jul 29, 2024
9cb5bcf
Add Dinov2 and CLIP ViT Vision encoders
ankile Jul 29, 2024
23a273d
Change some BC training parameters and add clipping and tracking of g…
ankile Jul 29, 2024
27a6373
Merge in main
ankile Jul 29, 2024
e81e713
runs for mug rack
anthonysimeonov Jul 29, 2024
d777758
Add vision encoder files for the new ViTs
ankile Jul 29, 2024
28fe1c3
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Jul 29, 2024
ecca026
Merge branch 'main' into relative-actions
ankile Jul 29, 2024
a9c6de5
Add launch scripts for vit encoders for relative actions
ankile Jul 30, 2024
47713fe
Revert changes to implement relative actions in state-dataset
ankile Jul 30, 2024
c783d00
Implement better checkpoint book-keeping and loading
ankile Jul 30, 2024
2cbe938
Normalize the the velocity and gripper part of the robot state in rel…
ankile Jul 30, 2024
4efd239
Add optional masking of proprioceptive pose information
ankile Jul 30, 2024
51ff066
Add a base actor config yaml
ankile Jul 30, 2024
c7fe0d5
Merge in main
ankile Jul 30, 2024
971ef53
Merge pull request #19 from ankile/relative-actions
anthonysimeonov Jul 30, 2024
b7241b4
reshape tensor to account for possible obs history in relative action…
anthonysimeonov Jul 30, 2024
1821ace
Clean up the way we do checkpointing
ankile Jul 30, 2024
a540120
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Jul 30, 2024
5f03ee4
Small fix of missing variable in bc resume runs
ankile Jul 30, 2024
bfe8666
multi gpu with ddp verified on ant local, tbd on cluster (issues with…
anthonysimeonov Jul 30, 2024
e5ed581
Update ViT experiments
ankile Jul 31, 2024
7f75a2e
Change the forward call logic to use the DDP internals
ankile Jul 31, 2024
79d9281
Fix multiple wandb runs for each GPU
ankile Jul 31, 2024
8a8f2d0
Moved the sampler.set_epoch call inside the epoch loop for proper ran…
ankile Jul 31, 2024
8ed6291
starting to get reading from disk (+ little ddp bug fix with transfor…
anthonysimeonov Jul 31, 2024
9cbfe50
add option to set chunksize in zarr store (chunksize 1 is better for …
anthonysimeonov Jul 31, 2024
1eb09dd
Tune the learning rate of the encoder and some niceities for ddp trai…
ankile Jul 31, 2024
33fb64d
Merge branch 'multi-gpu-dev' of github.com:ankile/robust-rearrangemen…
ankile Jul 31, 2024
5584b14
Add relative action real-world co-training runs
ankile Jul 31, 2024
b496475
Update the supercloud relative action scripts
ankile Jul 31, 2024
e1e6df0
Updates to experiment runs
ankile Aug 1, 2024
377bf74
Merge in main and resolve merge conflict
ankile Aug 1, 2024
4637e31
Small fix with module attr mixin
ankile Aug 1, 2024
d172775
Clean and slightly improve the ddp bc script
ankile Aug 1, 2024
e9d922d
Merge pull request #20 from ankile/multi-gpu-dev
ankile Aug 1, 2024
c1034b9
basic implementation of bc re-fitting during residual ppo
anthonysimeonov Aug 1, 2024
6e239d2
Update relative action experiments
ankile Aug 2, 2024
c6e24cb
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Aug 2, 2024
ee8786f
Small change to the done returned by the rppo env wrapper
ankile Aug 2, 2024
0b9f979
Run ablation experiments on demo number and residual action scale
ankile Aug 5, 2024
40dbdad
Add an integrated residual actor with MLP as base model and accompany…
ankile Aug 5, 2024
d0e2681
Fix tiny error in run script
ankile Aug 5, 2024
f19ae11
Add mug-rack baseline bc experiment launch script
ankile Aug 5, 2024
0355ddf
Collect only eval rollouts for bc-in-the-loop and iron out some kinks
ankile Aug 5, 2024
494a31f
Debug the bc-in-the-loop high loss issue
ankile Aug 6, 2024
36805d6
Fix tiny error in run script
ankile Aug 5, 2024
0f4816f
Add mlp rppo oneleg medium script
ankile Aug 6, 2024
a18772e
Merge branch 'mlp-residual-ppo' of github.com:ankile/robust-rearrange…
ankile Aug 6, 2024
bf4da78
The debugging continues
ankile Aug 6, 2024
7e1a54f
Update DARPPO code to allow for more control of number gradient steps…
ankile Aug 7, 2024
bbd9a86
Fix some small bugs with the DARPPO updated implementation
ankile Aug 7, 2024
94f2ae7
peg hole bc scripts
anthonysimeonov Aug 7, 2024
8609c49
add factory to list of tasks
anthonysimeonov Aug 7, 2024
71fdc04
update runs
anthonysimeonov Aug 7, 2024
c5a4ba5
First rough implementation of DAgger for states implemented
ankile Aug 7, 2024
d537767
Add a threshold for when to start doing base BC updates
ankile Aug 7, 2024
e36eaa6
Merge branch 'main' into state-based-dagger
ankile Aug 7, 2024
0cf3cf4
Fix bug with normalizer on wrong device
ankile Aug 7, 2024
05d7800
State-based DAgger seems to be working nicely
ankile Aug 7, 2024
93a7463
Add better way to autmatically requeue premtible job in bc
ankile Aug 7, 2024
97c4622
Temporarily take out some logging logic that crashed
ankile Aug 7, 2024
980093b
Add back in the opportunity to resume dagger jobs
ankile Aug 7, 2024
b0a3fa6
Make sure to step the lr scheduler
ankile Aug 7, 2024
516f57b
factory rppo run script
anthonysimeonov Aug 8, 2024
7a7548f
updated fph bc runs
anthonysimeonov Aug 8, 2024
bf6d219
larger exp noise for fph
anthonysimeonov Aug 8, 2024
8ba819d
Merge branch 'main' into state-based-dagger
ankile Aug 8, 2024
8830e1c
Dagger experiment runs
ankile Aug 8, 2024
0945181
Make it so when continuing a run, the original config will be used
ankile Aug 8, 2024
b82c0d4
Merge in main
ankile Aug 8, 2024
2e73ab3
Add cosine scheddule with minimum value and run dagger with beta at e…
ankile Aug 8, 2024
2bf0a4b
Merge in main
ankile Aug 8, 2024
dc147db
Updates to dagger training to debug collapse
ankile Aug 8, 2024
018ea6c
Merge pull request #24 from ankile/mlp-residual-ppo
anthonysimeonov Aug 8, 2024
9eeaf50
Fix resuming bug
ankile Aug 8, 2024
6f59c79
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Aug 8, 2024
3a2b84d
Merge branch 'main' into state-based-dagger
ankile Aug 9, 2024
db1e8af
Further probing of why dagger is not performing
ankile Aug 9, 2024
973360e
Make a change to where the residual action correction is placed for d…
ankile Aug 9, 2024
33c383d
Make updates to bc to make training a bit faster and checkpointing wo…
ankile Aug 9, 2024
9cb2a8c
Small fixes in wt loading, dataset loading, and ep length calc in rppo
ankile Aug 11, 2024
372563c
Fix some small issues with bc run resuming
ankile Aug 11, 2024
41223ad
Make the rollout procedure use less memory and update demo scaling ex…
ankile Aug 11, 2024
c635867
Fix bug in rollout reward calculation when not storing rollouts
ankile Aug 12, 2024
bafcbbc
Fix bug in episode stats calculation when 0 successes in RPPO
ankile Aug 12, 2024
7b65ae4
update hyperparams for MLP PPO exp and add fph mlp BC run
anthonysimeonov Aug 12, 2024
576de26
add factory_peg_hole to list of tasks in eval
anthonysimeonov Aug 12, 2024
3aa645e
Add the option to specify no weight in the weight loading function
ankile Aug 12, 2024
37bfc2a
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Aug 12, 2024
d3726a1
Merge in main
ankile Aug 12, 2024
adcd79f
Remove unnecessary path prefix in state scaling configs
ankile Aug 12, 2024
98f89c9
Add real-world demo scaling training runs
ankile Aug 12, 2024
df6a7f8
Add load into memory argument to dataset in normal bc file
ankile Aug 12, 2024
a3b04ca
Add missing export for env variable in launch script
ankile Aug 12, 2024
d1dc0d5
Add real-world demo scaling experiment to cannon
ankile Aug 12, 2024
038f36d
Add the final supercloud files for real-world scaling
ankile Aug 12, 2024
0907670
trying out beta decay and new dagger run hparams
anthonysimeonov Aug 12, 2024
317e470
linear decay to start
anthonysimeonov Aug 12, 2024
390bf87
linear decay to start
anthonysimeonov Aug 12, 2024
d5ee7aa
fix
anthonysimeonov Aug 12, 2024
2d04db1
fix
anthonysimeonov Aug 12, 2024
00dd5f7
fix
anthonysimeonov Aug 12, 2024
4f95054
dagger runs
anthonysimeonov Aug 12, 2024
976ce03
Add demo scaling runs for cannon
ankile Aug 12, 2024
0bcade9
Some more experiments
ankile Aug 12, 2024
a03247a
Move the rest of the real-world scaling experiments to cannon
ankile Aug 13, 2024
ea0036b
Update the wandb mode in config when resuming runs
ankile Aug 13, 2024
99cd5cb
Fix gradient step tracking in dagger
ankile Aug 13, 2024
a96c34a
dagger runs + make minimum beta value a config param
anthonysimeonov Aug 13, 2024
9cbd8a3
Update the sim scaling experiment files
ankile Aug 14, 2024
987676c
hacky solution for image based dagger, WIP to clean up
Aug 15, 2024
dad2377
Add separate wamup steps arguments for actor and critic in rppo
ankile Aug 15, 2024
fe4bf07
fix memory pre-allocation in replay buffer
anthonysimeonov Aug 15, 2024
9330b69
Merge in main
ankile Aug 16, 2024
a737d00
merge conflict
anthonysimeonov Aug 21, 2024
1ee2daf
fix up dagger (buffer attributes, adjust config to make evaluation ea…
anthonysimeonov Aug 21, 2024
55ca993
Clean the dagger file somewhat
ankile Aug 23, 2024
955de3a
Merge in from remote
ankile Aug 23, 2024
bacabf6
Start adding more vision models and fix naming issue in rppo
ankile Aug 23, 2024
642083a
Add cotraining experiment files for confusion and upweighting
ankile Aug 23, 2024
618a22b
Add frozen encoders for co-training for curiosity
ankile Aug 23, 2024
b17f883
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Aug 23, 2024
8149dce
Merge branch 'main' into state-based-dagger
ankile Aug 26, 2024
4ce705c
Merge pull request #27 from ankile/state-based-dagger
ankile Aug 26, 2024
c954b14
intermediate fixes to real world eval (ensure all actor modules in ev…
anthonysimeonov Aug 26, 2024
d3059eb
Analyze features of sim and real images
ankile Aug 27, 2024
a25e6ab
some local run scripts
anthonysimeonov Aug 28, 2024
82067b7
add little sim2real readme
anthonysimeonov Aug 28, 2024
df3c510
wip readme's for real and sim2real
anthonysimeonov Aug 28, 2024
97bcab7
add isaac lab version + more notes on sim2real
anthonysimeonov Aug 28, 2024
916a4d2
Update README.md
ankile Sep 1, 2024
3d7e691
Make the repo less furniture-specific
ankile Sep 7, 2024
057cf03
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Sep 7, 2024
9581801
Fix a bug in the ddp code with imports in the right order
ankile Sep 7, 2024
c63db7a
Add the centroid formulation of the confusion loss
ankile Sep 8, 2024
3903aa1
Add the centroid formulation of the confusion loss
ankile Sep 8, 2024
5d67673
Analyze confusion with centroid formulation
ankile Sep 8, 2024
7691cce
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Sep 8, 2024
112de59
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Sep 8, 2024
f0073ed
Add vision-based diffusion model using attention pooling for encoder …
ankile Sep 13, 2024
a6586d9
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Sep 13, 2024
7516ab3
Add Supercloud attention pool launchscript
ankile Sep 13, 2024
a335dd0
Run more thorough DAgger evals
ankile Sep 14, 2024
3254695
Merge in changes from main
ankile Sep 14, 2024
a39ccb4
Merge pull request #21 from ankile/vas
ankile Sep 14, 2024
20c7cd4
Make updates to VAS for new tasks
ankile Sep 15, 2024
c579589
Small changes for real-world deployment
anthonysimeonov Sep 15, 2024
6d14c13
Merge branch 'main' of github.com:ankile/robust-rearrangement into main
anthonysimeonov Sep 15, 2024
716c91c
Fix to handle missing time dimension in transformer diffusion model
ankile Sep 15, 2024
c1aaa97
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Sep 15, 2024
ebe3ece
Make image-dagger improvements
ankile Sep 19, 2024
9909406
Clean up obsolete files
ankile Sep 20, 2024
8c3c146
Update README.md
ankile Sep 20, 2024
af3d1f6
Purge the terminology of furniture for task and remove obsolete files
ankile Sep 20, 2024
3b25c06
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Sep 20, 2024
fb13d23
Update README.md
ankile Sep 20, 2024
c4d6032
Fix an issue with task/furniture naming
ankile Sep 20, 2024
892e29d
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Sep 20, 2024
d0f8112
Fix up in requirements file
ankile Sep 20, 2024
6d26a71
Update README.md
ankile Sep 24, 2024
6e8fd9b
Produce and run scaling experiments and uncut videos for more tasks
ankile Sep 26, 2024
09394f5
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Sep 26, 2024
f031b7a
Small changes to reorganize data for different purposes
ankile Sep 27, 2024
228fbb5
Update README.md
ankile Sep 27, 2024
67dedd1
Update README.md
ankile Sep 27, 2024
34ffc54
Add checkpoints to S3 bucket and option to eval raw checkpoints
ankile Sep 27, 2024
a8139da
Change camera serial
anthonysimeonov Sep 28, 2024
40048e4
Update plots for paper
ankile Sep 30, 2024
39a9da9
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Sep 30, 2024
82d611f
Increase LR somewhat for bigger batch
ankile Oct 2, 2024
7a84175
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Oct 2, 2024
9055df4
Change partition and gpu for cannon scripts
ankile Oct 2, 2024
f969b76
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Oct 2, 2024
a4cea9b
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Oct 2, 2024
9bfcb54
Increase time limit
ankile Oct 2, 2024
837957f
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Oct 2, 2024
48cc303
Fix superfluous argument in bc rollout code
ankile Oct 2, 2024
d8ce9d9
Add round_table scaling experiment with bigger UNet
ankile Oct 4, 2024
e9cd6c5
Reduce RAM usage for scaling experiments
ankile Oct 4, 2024
a4ee8dc
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Oct 4, 2024
0cf078c
Fix spurious debug true
Oct 4, 2024
0b237ad
Continue fph scaling runs
Oct 5, 2024
f5c6c82
Update repo to allow for easier running and loading of pre-trained ch…
ankile Oct 7, 2024
6094e13
Add lamp demo scaling experiments
ankile Oct 7, 2024
82ea150
Implement zarrs storing all low dim data as one chunk
ankile Oct 8, 2024
4aeb667
Add experiment config for unchunked mlp for peg-hole
ankile Oct 8, 2024
d9dd7c9
Further peg-hole scaling analyses
ankile Oct 8, 2024
30a1cd0
Add round_table dagger
ankile Oct 8, 2024
aac06b0
Update eval_utils.py
ankile Oct 17, 2024
26c1e80
Update bc.py
ankile Oct 17, 2024
1f993e4
Keep pushing the scaling analyses
ankile Oct 17, 2024
033da3a
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Oct 17, 2024
070cd99
Start converting data formats
ankile Oct 21, 2024
6e371f7
Plot new scaling plots
ankile Oct 21, 2024
a8a7982
Pushing the fph scaling analyses
ankile Oct 21, 2024
fde139b
Implement data conversion script
ankile Oct 21, 2024
d3aebc6
BC without rollouts running
ankile Oct 21, 2024
14b0b99
Make changes to uproot pytorch3d and reliance on furniture bench
ankile Oct 21, 2024
f1c1b82
MJ single env running and seemingyl working
ankile Oct 22, 2024
1936cbb
Fix the gripper action to be the raw unthresholded one
ankile Oct 22, 2024
7e6814e
Simplify the IK layer
ankile Oct 22, 2024
5fcf552
Remove stochasticity for debugging
ankile Oct 22, 2024
ddf76be
Fix gripper width calculation
ankile Oct 22, 2024
01162d1
Debugging vec env
ankile Oct 22, 2024
440c2fc
Fix issue with VecEnv, parallel envs running now
ankile Oct 22, 2024
1440feb
Residual RL code running for new task
ankile Oct 23, 2024
af5c9bc
Fix visualization issue in bimanual env
Oct 23, 2024
f31e013
Load the init poses in the env init
ankile Oct 23, 2024
158bfbf
Add randomness seeing in bc code for better resuming
ankile Oct 23, 2024
f889a92
Merge branch 'main' into bimanual-task
ankile Oct 23, 2024
62c8262
Run bimanual rppo with more envs
ankile Oct 24, 2024
beb935e
Merge in a fix in eval model
ankile Oct 24, 2024
85aadf6
Add rate limiter for real time viewing
ankile Oct 24, 2024
95a387d
Add logs to gitignore
ankile Oct 24, 2024
b0cde00
Merge branch 'main' into bimanual-task
ankile Oct 24, 2024
007ed3a
Clean up data storage and processing for dexhub tasks
ankile Oct 25, 2024
f1132f3
Improve mujoco environment, runs very nicely now
ankile Oct 25, 2024
726e851
Add scaling analysis work
ankile Oct 28, 2024
c8be2d2
Launch fph dagger run from pre-trained weights
ankile Oct 28, 2024
3175d85
Merge branch 'main' into bimanual-task
ankile Oct 28, 2024
eb90e97
Adapt the mlp ppo and vas bvaselines to work with the new Mujoco env
ankile Oct 30, 2024
ad812ba
Numerous changes to integrate with baselines and delay before success
ankile Nov 13, 2024
272f2e3
Merge pull request #34 from ankile/bimanual-task
ankile Nov 13, 2024
6e35161
Update plots
ankile Nov 26, 2024
41c99a0
Merge branch 'main' of github.com:ankile/robust-rearrangement
ankile Nov 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add logs to gitignore
@ankile
ankile committed Oct 24, 2024

Verified

This commit was signed with the committer’s verified signature.
atlowChemi Chemi Atlow
GPG key ID: E2DE4855C6AF0350
Verified
Learn about vigilant mode
commit 95a387d5b3179c5a391d04fb1da12fe7b61710ef
4 changes: 3 additions & 1 deletion .gitignore
View file
Original file line number Diff line number Diff line change
@@ -115,4 +115,6 @@ tmp

*.out

*.csv
*.csv

logs/