Merge pull request #279 from davidberenstein1957/add-structure-extrac…

…tion-vlm Add structure extraction vlm
huggingface · Jan 31, 2025 · 8273c06 · 8273c06
2 parents 0eec604 + 0f3d86b
commit 8273c06
Show file tree

Hide file tree

Showing 3 changed files with 405 additions and 1 deletion.
diff --git a/notebooks/en/_toctree.yml b/notebooks/en/_toctree.yml
@@ -108,6 +108,8 @@
           title: Smol Multimodal RAG, Building with ColSmolVLM and SmolVLM on Colab's Free-Tier GPU
         - local: fine_tuning_vlm_dpo_smolvlm_instruct
           title: Fine-tuning SmolVLM using direct preference optimization (DPO) with TRL on a consumer GPU
+        - local: structured_generation_vision_language_models
+          title: Structured Generation from Images or Documents Using Vision Language Models
 
     - title: Search Recipes
       isExpanded: false

diff --git a/notebooks/en/index.md b/notebooks/en/index.md
@@ -7,11 +7,11 @@ applications and solving various machine learning tasks using open-source tools
 
 Check out the recently added notebooks:
 
+- [Structured Generation from Images or Documents Using Vision Language Models](structured_generation_vision_language_models)
 - [Vector Search on Hugging Face with the Hub as Backend](vector_search_with_hub_as_backend)
 - [Multi-Agent Order Management System with MongoDB](mongodb_smolagents_multi_micro_agents)
 - [Scaling Test-Time Compute for Longer Thinking in LLMs](search_and_learn)
 - [Signature-Aware Model Serving from MLflow with Ray Serve](mlflow_ray_serve)
-- [Fine-tuning SmolVLM using direct preference optimization (DPO) with TRL on a consumer GPU](fine_tuning_vlm_dpo_smolvlm_instruct)
 
 You can also check out the notebooks in the cookbook's [GitHub repo](https://github.com/huggingface/cookbook).