Integrate icenet-gan

[ampersandmcd]

Hey team!

As most of you know, I've developed the icenet-gan library in PyTorch over the course of my MRes, replacing the U-Net backbone of the IceNet pipeline with a GAN in the hopes of generating more spatially-sharp sea ice forecasts.

I'd love to see the icenet-gan library integrated into the main IceNet project such that we're able to swap out the U-Net backbone for a GAN backbone during training and inference, and am keen to help make that happen however I can! We'll need to port it from PyTorch to TensorFlow or wait for the PyTorch implementation of IceNet to get everything moving, but that's no biggie.

I'll be intermittently online throughout September as my PhD doesn't officially start until 2 October, but will be consistently online and in the office from there on out to help operationalise this integration.

Happy to chat on Slack/Zoom/in-person to make this a reality!

Bryn edit:

• Get a demo notebook running using PyTorch UNet & cGAN from above code.
• Get it running on BAS HPC, and create a reproducible environment (e.g. conda) that can be shared.
• Demonstrator notebook of Torch UNet + GAN with the IceNet library/pipeline.
  • https://github.com/icenet-ai/icenet-notebooks/tree/main/pytorch
• Integration of UNet with the icenet-pipeline for end to end runs.
• Integration of GAN with the icenet-pipeline for end to end runs.

[JimCircadian]

Nice one @ampersandmcd, thanks for raising this, it's very much appreciated. @rchan26 has opened a PR for the pytorch example in the IceNet ecosystem, so we're well on the way. We'll leave this here unassigned until someone starts it in earnest, whomever that will be! Great to have this here. 😉

[scotthosking]

It would also be useful to add some documentation on how to add other models to the pipeline, this can come later but wanted to highlight it while I remembered

[JimCircadian]

Good show @scotthosking, this is currently covered trivially by notebooks, but we need to expand on it for sure. This issue would be a good catalyst for making a better template for different architectures.

[bnubald]

Ongoing work integrating your code @ampersandmcd (UNet and GAN) using Icenet's dataloader as backend for generating samples, currently in a fork of icenet-notebooks.

1. UNet
2. GAN

[JimCircadian]

@bnubald it'll be good to look at wrapping the outstanding items for this work up now you're moving onto #57

[bnubald]

Todo:

• Re-organise notebook ordering
• Submit PR to merge into main repo.

[bnubald]

New PyTorch dataset class included in IceNet library v0.2.8_dev.

This splintered into an issue in icenet-notebooks repo: icenet-ai/icenet-notebooks#13.

Ongoing work integrating your code @ampersandmcd (UNet and GAN) using Icenet's dataloader as backend for generating samples, currently in a fork of icenet-notebooks.

1. UNet
2. GAN

Initial notebooks from above have been tidied up, and can be found in the main branch. I've also included a conda env file in the same path since getting pytorch and tensorflow to co-exist with cuda can be temperamental. These notebooks show the usage of the IceNet library for much of the heavy lifting, except for the model side.

For more details on changes from the original code, and details on the notebooks, please see the relevant PR: icenet-ai/icenet-notebooks#18

[bnubald]

Integration with the pipeline via refactored icenet-gan codebase can be found below (currently a private repo), it currently shows end-to-end pipeline run with UNet, GAN to be added.

https://github.com/icenet-ai/icenet-gan/

ToDo:

• Pipeline integration demo script (currently drafted, to be pushed)
• GAN integration (currently UNet).