Skip to content

Run Generative AI models with simple C++/Python API and using OpenVINO Runtime

License

Notifications You must be signed in to change notification settings

openvinotoolkit/openvino.genai

Folders and files

NameName
Last commit message
Last commit date

Latest commit

a3d4153 · May 8, 2024
May 8, 2024
May 8, 2024
May 8, 2024
May 6, 2024
May 8, 2024
Mar 22, 2024
Mar 14, 2024
Feb 15, 2024
Oct 23, 2023
Feb 5, 2024
Nov 28, 2023
May 8, 2024

Repository files navigation

GenAI Pipeline Repository

The GenAI repository contains pipelines that implement image and text generation tasks. The implementation uses OpenVINO capabilities to optimize the pipelines. Each sample covers a family of models and suggests certain modifications to adapt the code to specific needs. It includes the following pipelines:

  1. Benchmarking script for large language models
  2. Text generation C++ samples that support most popular models like LLaMA 2
  3. Stable Diffuison (with LoRA) C++ image generation pipeline
  4. Latent Consistency Model (with LoRA) C++ image generation pipeline

License

The GenAI repository is licensed under Apache License Version 2.0. By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.

Requirements

Requirements may vary for different samples. See respective readme files for more details, and make sure to install the OpenVINO version listed there. Refer to documentation to see how to install OpenVINO.

The supported devices are CPU and GPU including Intel discrete GPU.

See also: https://docs.openvino.ai/2023.3/gen_ai_guide.html.