Repositories list

• InternVL

  Public
  [CVPR 2024 Oral] InternVL Family: A Pioneering Open-Source Alternative to GPT-4o. 接近GPT-4o表现的开源多模态对话模型

  image-classificationgptmulti-modal+ 9semantic-segmentationvideo-classificationimage-text-retrievalllmvision-language-modelgpt-4vvit-6b+ 2

  Python

  •

  MIT License

  •496•6.5k•145•3•Updated Dec 19, 2024Dec 19, 2024

• PVC

  Public
  PVC: Progressive Visual Token Compression for Unified Image and Video Processing in Large Vision-Language Models

  Python

  •

  MIT License

  •0•11•1•0•Updated Dec 18, 2024Dec 18, 2024

• STM-Evaluation

  Public
  Python

  •

  MIT License

  •6•70•1•0•Updated Dec 15, 2024Dec 15, 2024

• V2PE

  Public
  [ArXiv] V2PE: Improving Multimodal Long-Context Capability of Vision-Language Models with Variable Visual Position Encoding

  Python

  •

  MIT License

  •0•8•0•0•Updated Dec 13, 2024Dec 13, 2024

• PIIP

  Public
  [NeurIPS 2024 Spotlight ⭐️] Parameter-Inverted Image Pyramid Networks (PIIP)

  computer-visionimage-classificationobject-detection+ 3semantic-segmentationinstance-segmentationvision-transformer

  Python

  •

  MIT License

  •2•63•0•0•Updated Dec 12, 2024Dec 12, 2024

• InternVideo

  Public
  [ECCV2024] Video Foundation Models & Data for Multimodal Understanding

  benchmarkaction-recognitionvideo-understanding+ 17video-dataself-supervisedmultimodalvideo-datasetopen-set-recognitionvideo-retrievalvideo-question-answering+ 10

  Python

  •

  Apache License 2.0

  •91•1.5k•92•4•Updated Dec 11, 2024Dec 11, 2024

• VLMEvalKit_InternVL2_5

  Public
  Open-source evaluation toolkit of large vision-language models (LVLMs), support 160+ VLMs, 50+ benchmarks

  Python

  •

  Apache License 2.0

  •215•0•0•0•Updated Dec 9, 2024Dec 9, 2024

• Hulk

  Public
  An official implementation of "Hulk: A Universal Knowledge Translator for Human-Centric Tasks"

  Python

  •

  MIT License

  •4•110•13•0•Updated Dec 4, 2024Dec 4, 2024

• Ask-Anything

  Public
  [CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

  chatvideogradio+ 10big-modelvideo-understandingcaptioning-videosvideo-question-answeringfoundation-modelslarge-modellarge-language-models+ 3

  Python

  •

  MIT License

  •253•3.1k•63•5•Updated Nov 26, 2024Nov 26, 2024

• MM-NIAH

  Public
  [NeurIPS 2024] Needle In A Multimodal Haystack (MM-NIAH): A comprehensive benchmark designed to systematically evaluate the capability of existing MLLMs to comprehend long multimodal documents.

  benchmarklong-contextvision-language-model+ 1multimodal-large-language-models

  Python

  •6•105•1•0•Updated Nov 25, 2024Nov 25, 2024

• OmniCorpus

  Public
  OmniCorpus: A Unified Multimodal Corpus of 10 Billion-Level Images Interleaved with Text

  Python

  •7•288•0•0•Updated Nov 17, 2024Nov 17, 2024

• GUI-Odyssey

  Public
  GUI Odyssey is a comprehensive dataset for training and evaluating cross-app navigation agents. GUI Odyssey consists of 7,735 episodes from 6 mobile devices, spanning 6 types of cross-app tasks, 201 apps, and 1.4K app combos.

  Python

  •4•76•1•0•Updated Nov 12, 2024Nov 12, 2024

• Vision-RWKV

  Public
  Vision-RWKV: Efficient and Scalable Visual Perception with RWKV-Like Architectures

  Python

  •

  Apache License 2.0

  •15•387•17•0•Updated Oct 31, 2024Oct 31, 2024

• .github

  Public
  1•0•0•0•Updated Oct 30, 2024Oct 30, 2024

• OV-OAD

  Public
  This repo takes the initial step towards leveraging text learning for online action detection without explicit human supervision.

  1•1•0•0•Updated Oct 28, 2024Oct 28, 2024

• InternVL-MMDetSeg

  Public
  Train InternViT-6B in MMSegmentation and MMDetection with DeepSpeed

  object-detectionsemantic-segmentationvision-foundation

  Jupyter Notebook

  •5•69•1•0•Updated Oct 25, 2024Oct 25, 2024

• PhyGenBench

  Public
  The code and data of Paper: Towards World Simulator: Crafting Physical Commonsense-Based Benchmark for Video Generation

  Python

  •1•78•3•0•Updated Oct 25, 2024Oct 25, 2024

• VisionLLM

  Public
  VisionLLM Series

  object-detectionlarge-language-modelsgeneralist-model

  Python

  •

  Apache License 2.0

  •29•957•14•0•Updated Oct 18, 2024Oct 18, 2024

• VideoMAEv2

  Public
  [CVPR 2023] VideoMAE V2: Scaling Video Masked Autoencoders with Dual Masking

  video-understandingaction-detectionself-supervised-learning+ 4temporal-action-detectionfoundation-modelcvpr2023action-recognition

  Python

  •

  MIT License

  •64•546•13•0•Updated Oct 8, 2024Oct 8, 2024

• EfficientQAT

  Public
  EfficientQAT: Efficient Quantization-Aware Training for Large Language Models

  Python

  •18•232•5•0•Updated Oct 8, 2024Oct 8, 2024

• OmniQuant

  Public
  [ICLR2024 spotlight] OmniQuant is a simple and powerful quantization technique for LLMs.

  quantizationlarge-language-modelsllm

  Python

  •

  MIT License

  •56•742•23•1•Updated Oct 8, 2024Oct 8, 2024

• MMIU

  Public
  MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Models

  Python

  •2•57•3•0•Updated Sep 14, 2024Sep 14, 2024

• ChartAst

  Public
  [ACL 2024] ChartAssistant is a chart-based vision-language model for universal chart comprehension and reasoning.

  Python

  •

  Other

  •8•108•6•0•Updated Sep 7, 2024Sep 7, 2024

• EgoExoLearn

  Public
  [CVPR 2024] Data and benchmark code for the EgoExoLearn dataset

  Python

  •

  MIT License

  •0•48•2•0•Updated Sep 3, 2024Sep 3, 2024

• InternGPT

  Public
  InternGPT (iGPT) is an open source demo platform where you can easily showcase your AI models. Now it supports DragGAN, ChatGPT, ImageBind, multimodal chat like GPT-4, SAM, interactive image editing, etc. Try it at igpt.opengvlab.com (支持DragGAN、ChatGPT、ImageBind、SAM的在线Demo系统)

  samclickvqa+ 17image-captioningllamagptgradiohuskymultimodalvideo-generation+ 10

  Python

  •

  Apache License 2.0

  •232•3.2k•18•1•Updated Aug 20, 2024Aug 20, 2024

• all-seeing

  Public
  [ICLR 2024 & ECCV 2024] The All-Seeing Projects: Towards Panoptic Visual Recognition&Understanding and General Relation Comprehension of the Open World"

  datasetall-seeingregion-text

  Python

  •17•466•12•0•Updated Aug 9, 2024Aug 9, 2024

• Diffree

  Public
  Diffree: Text-Guided Shape Free Object Inpainting with Diffusion Model

  Python

  •

  Apache License 2.0

  •13•233•3•0•Updated Aug 6, 2024Aug 6, 2024

• InternImage

  Public
  [CVPR 2023 Highlight] InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions

  backbonesemantic-segmentationdeformable-convolution+ 2foundation-modelobject-detection

  Python

  •

  MIT License

  •237•2.6k•184•8•Updated Aug 1, 2024Aug 1, 2024

• MMT-Bench

  Public
  ICML'2024 | MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI

  Python

  •3•95•6•0•Updated Jul 18, 2024Jul 18, 2024

• ControlLLM

  Public
  ControlLLM: Augment Language Models with Tools by Searching on Graphs

  Python

  •8•188•6•0•Updated Jul 15, 2024Jul 15, 2024