Vision Titans

[ShunkaiZhou]

Thank you very much for the excellent code!
May I ask if you will build Vision Titans?
Best wishes!!!

[lucidrains]

@ShunkaiZhou hey Shunkai, thanks for your interest

could you elaborate a bit more? do you mean in the context of tokenized video or something different?

[ShunkaiZhou]

@lucidrains Hi, thank you for your quick reply.
For example, online video analytics?
Best wishes.

[kanghuazhao]

VLM-Titans？

[RefractAI]

@lucidrains

I would like to share that, by replacing the logits with a continuous pixel prediction, this repo trains a class-conditional MNIST digit image generator in under 2 minutes on a 4090. Hyperparameters unchanged from default.

[lucidrains]

@RefractAI nice! it could just be sliding window attention though

local attention with even the slightest overlap is incredibly strong

[ShunkaiZhou]

@RefractAI
Hi, your changes are very interesting. Could you share the changed code please? Thanks a lot.
Best wishes!

[lucidrains]

@RefractAI did you adapt it from the transfusion repo? i can port one over here if needed

[piwawa]

Can you add support for image input? Just like ViT or Swin-Transformer.

@RefractAI did you adapt it from the transfusion repo? i can port one over here if needed