Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test GitHub actions #465

Closed
wants to merge 123 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
123 commits
Select commit Hold shift + click to select a range
ba91fde
initial generate
pavel-esir Mar 26, 2024
9d85a0e
LLM pipeline
pavel-esir Mar 28, 2024
b21c6c1
Added calculating for several batches
pavel-esir Apr 2, 2024
e52e90d
Greedy search works
pavel-esir Apr 3, 2024
745a804
rename to GenerationConfig
pavel-esir Apr 4, 2024
8895ed0
Add fluent interface
pavel-esir Apr 5, 2024
b24977d
Update text_generation/causal_lm/cpp/generate_pipeline/generate_pipel…
pavel-esir Apr 5, 2024
c933ca0
cosmetic changes in main
pavel-esir Apr 5, 2024
c43e901
greedy search with batches and left padding works
pavel-esir Apr 10, 2024
5a914f6
combine LLModel with LLMPipeline
pavel-esir Apr 10, 2024
c1e0c9d
wip: enable calling tokenize/detokenize for LLMPipeline
pavel-esir Apr 10, 2024
8d66353
add callback to generate
pavel-esir Apr 11, 2024
fa12da7
cleanup generate_sample.cpp
pavel-esir Apr 11, 2024
5ceb9d5
add speculative decoding
pavel-esir Apr 16, 2024
a5083c7
separate Tokenizer
pavel-esir Apr 17, 2024
7692160
wip
pavel-esir Apr 23, 2024
d3f6339
add start/stop conversation
pavel-esir Apr 24, 2024
3776433
use text in streamer instead of raw tokens
pavel-esir Apr 23, 2024
964a5e8
add apply_chat_template
pavel-esir Apr 23, 2024
e57aa4c
fix difference between accumulating conversation as text and keeping …
pavel-esir Apr 26, 2024
d0c1341
cleanup
pavel-esir Apr 26, 2024
8dcea1f
add Jinja2cpp submodule
pavel-esir Apr 26, 2024
754a462
add ov namespace
pavel-esir May 2, 2024
9b19c6f
return scores for batched outputs
pavel-esir May 2, 2024
9bf6caa
add AnyMap
pavel-esir May 3, 2024
39fd73c
Merge remote-tracking branch 'upstream/master' into generate_pipeline
pavel-esir May 3, 2024
63d8f6d
cleanup
pavel-esir May 3, 2024
a833760
before moving to pimpl
pavel-esir May 6, 2024
1681654
move to separate include & src
pavel-esir May 6, 2024
9fe73c6
pimpl implementation
pavel-esir May 6, 2024
053708f
temporary disable jinja2cpp
pavel-esir May 6, 2024
bd6849a
add python api draft, hide implementations from user & refactor imple…
pavel-esir May 7, 2024
62c471e
extract decoding methods to separate files
pavel-esir May 7, 2024
f1d54f4
extended python api, added python api test
pavel-esir May 7, 2024
3c82e11
remove call method
pavel-esir May 8, 2024
5543cee
init
Wovchena May 6, 2024
abb8835
add_subdirectory
Wovchena May 7, 2024
0998abc
add files
Wovchena May 8, 2024
15492c4
add __init__.py
Wovchena May 8, 2024
005d3fb
removed set_streamer
pavel-esir May 8, 2024
cc44bc8
use std::optional
pavel-esir May 8, 2024
d8cab05
started to add Readme docs
pavel-esir May 8, 2024
2535394
reoder Readme
pavel-esir May 8, 2024
95c1bfb
rm generate_pipeline/python
Wovchena May 9, 2024
4510f71
update Readme; cleanup LLMPipeline and add docstring
pavel-esir May 9, 2024
507bc49
refactor folder structure
pavel-esir May 9, 2024
af747d4
cleanup generation_config and ov::Tokenizer
pavel-esir May 9, 2024
c6620d9
move includes to a separate openvino/genai folder
pavel-esir May 10, 2024
59c3e0b
Merge branch 'generate_pipeline' into package
Wovchena May 10, 2024
be84345
align names
Wovchena May 10, 2024
bced64a
Dont modify text_generation/causal_lm/cpp/CMakeLists.txt
Wovchena May 10, 2024
f4e82b6
rm -r text_generation/causal_lm/cpp/generate_pipeline/python-bindings/
Wovchena May 10, 2024
5b2b0ca
fix build
Wovchena May 10, 2024
0dd8f59
add tokenizers only once
Wovchena May 10, 2024
23638ff
change cmake.source-dir
Wovchena May 10, 2024
d8c5349
restore openvino/genai inits
Wovchena May 10, 2024
24faefe
Integrate JinjaCpp
ilya-lavrenov May 10, 2024
598dda3
install genai lib
Wovchena May 10, 2024
f274b93
Merge pull request #2 from ilya-lavrenov/jinja-integration-pavel
pavel-esir May 10, 2024
02d0eae
import openvino for win and lin
Wovchena May 10, 2024
e6695f3
Merge branch 'generate_pipeline' into package
Wovchena May 10, 2024
a27c5a7
put the line back
Wovchena May 10, 2024
0849c41
Added cmake build type before project clause
ilya-lavrenov May 10, 2024
34cddff
one line properties
Wovchena May 10, 2024
023cf1e
Merge pull request #3 from ilya-lavrenov/cmake-build-type
pavel-esir May 10, 2024
6a5d750
Export API symbols
ilya-lavrenov May 10, 2024
27f385e
Merge pull request #4 from ilya-lavrenov/generate_pipeline
pavel-esir May 10, 2024
a9332f0
Merge branch 'generate_pipeline' into package
Wovchena May 10, 2024
9ef488c
rename
Wovchena May 10, 2024
4fad7d5
add .github/workflows/genai_lib.yml
Wovchena May 10, 2024
51e03a2
on: pull_request
Wovchena May 10, 2024
e23a7bb
spelling
Wovchena May 10, 2024
fc5b753
install openvino
Wovchena May 10, 2024
09f8806
add syntacis sugar for geenrate, optimize value passing by reference
pavel-esir May 10, 2024
af22a8a
remove speculative decoding
pavel-esir May 11, 2024
e7db7e8
update
Wovchena May 13, 2024
f279363
add rpath
Wovchena May 13, 2024
83d77c8
add rpath to libopenvino.so
Wovchena May 13, 2024
167f924
py_generate_pipeline
Wovchena May 13, 2024
a111a3f
reorder tokenizer.cpp, add comments to BaseStreamer
pavel-esir May 11, 2024
813d80a
install centos7
Wovchena May 13, 2024
6227b65
install nightly
Wovchena May 13, 2024
74fc107
Merge branch 'generate_pipeline' into package
Wovchena May 13, 2024
9b83a7e
propagate _GLIBCXX_USE_CXX11_ABI
Wovchena May 13, 2024
2d15752
Populate python with the libraries to allow skipping wheel installation
Wovchena May 13, 2024
8025554
run setupvars
Wovchena May 13, 2024
2b14286
update .gitignore, install numpy
Wovchena May 13, 2024
1c11bc7
quotes
Wovchena May 13, 2024
e7fce82
fix PYTHONPATH
Wovchena May 13, 2024
64608d1
fix PYTHONPATH
Wovchena May 13, 2024
43b87c7
quotes
Wovchena May 13, 2024
fef9674
reorder vars
Wovchena May 14, 2024
b21286c
openvino.genai-
Wovchena May 14, 2024
d393f89
Merge pull request #1 from Wovchena/package
pavel-esir May 14, 2024
2b8954d
Merge branch 'master' into generate_pipeline
pavel-esir May 14, 2024
11e872b
Update CMakeLists.txt
pavel-esir May 14, 2024
442dcbf
move group beam searcher to src
pavel-esir May 13, 2024
53d534e
Update .gitignore (#5)
Wovchena May 15, 2024
dcb4b86
Merge remote-tracking branch 'origin/generate_pipeline' into generate…
pavel-esir May 15, 2024
72c045e
fixed difference between old greddy sample and generate
pavel-esir May 15, 2024
11fbaa2
tokenizer minor fixes
pavel-esir May 15, 2024
264e99f
apply comments
pavel-esir May 15, 2024
11032b4
remove accidentally added test_cpp_samples.py
pavel-esir May 15, 2024
7d0c80b
fix build
pavel-esir May 15, 2024
2e3cd73
fix causal_lm comparison error
pavel-esir May 15, 2024
e7fa974
fix different outputs
pavel-esir May 15, 2024
78d0b88
Archive (#7)
Wovchena May 20, 2024
5eb59ea
add tests
pavel-esir May 16, 2024
ce4eb00
Apply suggestions from code review
pavel-esir May 22, 2024
aa90e9d
names correction
pavel-esir May 22, 2024
54cbb52
update URL_HASH
Wovchena May 22, 2024
82a9449
remove submodules from .gitmodules
Wovchena May 22, 2024
5a0079b
install openvino_tokenizers for genai_python_lib
pavel-esir May 22, 2024
73e4312
Update Jinja2Cpp fork commit
Wovchena May 22, 2024
75b7c37
remove group_beam_searcher.hpp; copy fast_tokenizer
pavel-esir May 22, 2024
70f1177
Fix archive (#8)
Wovchena May 23, 2024
da729ba
Apply suggestions from code review
pavel-esir May 24, 2024
28c313b
add groups to GenerationConfig docstring
pavel-esir May 24, 2024
c395a8d
refactor namespace ov::* -> ov::genai::*
pavel-esir May 24, 2024
bbc8c25
removed ov_tokenizers_path when ov::gena::Tokenizer is passed to LLMP…
pavel-esir May 24, 2024
9e37273
Add sampling decoding (#6)
as-suvorov May 27, 2024
81ec069
Fix library loading by updating dependencies (#10)
Wovchena May 28, 2024
88c44fe
Add extension near to genai library, tokenizers from fork (#11)
Wovchena May 29, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
names correction
pavel-esir committed May 22, 2024
commit aa90e9d229cc2357acee7e1d202c1a7d5871a63b
10 changes: 6 additions & 4 deletions .github/workflows/genai_python_lib.yml
Original file line number Diff line number Diff line change
@@ -16,11 +16,13 @@ jobs:
- run: source ./ov/setupvars.sh && cmake -DCMAKE_BUILD_TYPE=Release -S ./ -B ./build/
- run: source ./ov/setupvars.sh && cmake --build ./build/ --config Release -j
- run: python -m pip install --pre openvino --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly # Can't load CentOS libraries from the archive
- run: PYTHONPATH=./src/python/ python -c "from openvino_genai.py_generate_pipeline import LLMPipeline"
- run: PYTHONPATH=./src/python/ python -c "from openvino_genai import LLMPipeline"
- run: source ./ov/setupvars.sh && python -m pip install --pre . --extra-index-url https://storage.openvinotoolkit.org/simple/wheels/nightly
- run: source ./ov/setupvars.sh && python -m pip install ./thirdparty/openvino_tokenizers/[transformers]
- run: python -c "from openvino_genai import LLMPipeline"
- name: Install optimum-cli and run for each model
- name: GenAI Python API tests
run: |
source ./ov/setupvars.sh
cd ./tests/
python -m pip install -r requirements.txt
models=$(python3 generate_models.py)
@@ -47,6 +49,6 @@ jobs:
- run: call w_openvino_toolkit_windows_2024.2.0.dev20240515_x86_64\setupvars.bat && cmake -DCMAKE_BUILD_TYPE=Release -S ./ -B ./build/
- run: call w_openvino_toolkit_windows_2024.2.0.dev20240515_x86_64\setupvars.bat && cmake --build ./build/ --config Release -j
- run: python -m pip install "numpy<1.27"
- run: set "PYTHONPATH=./src/python;" && call w_openvino_toolkit_windows_2024.2.0.dev20240515_x86_64\setupvars.bat && python -c "from openvino_genai.py_generate_pipeline import LLMPipeline" # cmd evaluates variables in a different way. Setting PYTHONPATH before setupvars.bat instead of doing that after solves that.
- run: set "PYTHONPATH=./src/python;" && call w_openvino_toolkit_windows_2024.2.0.dev20240515_x86_64\setupvars.bat && python -c "from openvino_genai import LLMPipeline" # cmd evaluates variables in a different way. Setting PYTHONPATH before setupvars.bat instead of doing that after solves that.
- run: call w_openvino_toolkit_windows_2024.2.0.dev20240515_x86_64\setupvars.bat && python -m pip install .
- run: python -c "from openvino_genai.py_generate_pipeline import LLMPipeline"
- run: python -c "from openvino_genai import LLMPipeline"
4 changes: 2 additions & 2 deletions src/cpp/include/openvino/genai/llm_pipeline.hpp
Original file line number Diff line number Diff line change
@@ -73,7 +73,7 @@ class OPENVINO_GENAI_EXPORTS LLMPipeline {
*/
LLMPipeline(std::string& path, std::string device="CPU",
const ov::AnyMap& plugin_config={},
const std::string& ov_tokenizer_path="");
const std::string& ov_tokenizers_path="");

/**
* @brief Constructs a LLMPipeline when ov::Tokenizer is initialized manually using file from the different dirs.
@@ -88,7 +88,7 @@ class OPENVINO_GENAI_EXPORTS LLMPipeline {
const ov::Tokenizer& tokenizer,
const std::string device="CPU",
const ov::AnyMap& plugin_config = {},
const std::string& ov_tokenizer_path=""
const std::string& ov_tokenizers_path=""
);

~LLMPipeline();
2 changes: 1 addition & 1 deletion src/cpp/include/openvino/genai/tokenizer.hpp
Original file line number Diff line number Diff line change
@@ -21,7 +21,7 @@ class OPENVINO_GENAI_EXPORTS Tokenizer {
* @param tokenizer_path openvino_tokenizer.xml and openvino_detokenizer.xml should be located in the tokenizer_path
* @param device device. Currently only 'CPU' is supported
*/
Tokenizer(const std::string& tokenizers_path, const std::string& device="CPU", const std::string& ov_tokenizer_path="");
Tokenizer(const std::string& tokenizers_path, const std::string& device="CPU", const std::string& ov_tokenizers_path="");

/**
* @brief encode a single prompt
14 changes: 7 additions & 7 deletions src/cpp/src/llm_pipeline.cpp
Original file line number Diff line number Diff line change
@@ -45,11 +45,11 @@ class LLMPipeline::LLMPipelineImpl {
const ov::Tokenizer& tokenizer,
const std::string device,
const ov::AnyMap& plugin_config,
const std::string& ov_tokenizer_path=""
const std::string& ov_tokenizers_path=""
);

LLMPipelineImpl(std::string& path, std::string device, const ov::AnyMap& config);
LLMPipelineImpl(std::string& path, std::string device, const ov::AnyMap& config, const std::string& ov_tokenizer_path="");
LLMPipelineImpl(std::string& path, std::string device, const ov::AnyMap& config, const std::string& ov_tokenizers_path="");

GenerationConfig generation_config() const;

@@ -71,7 +71,7 @@ ov::LLMPipeline::LLMPipeline(
const ov::Tokenizer& tokenizer,
const std::string device,
const ov::AnyMap& plugin_config,
const std::string& ov_tokenizer_path
const std::string& ov_tokenizers_path
) {
m_pimpl = make_unique<LLMPipelineImpl>(model_path, tokenizer, device, plugin_config);
}
@@ -81,7 +81,7 @@ ov::LLMPipeline::LLMPipelineImpl::LLMPipelineImpl(
const ov::Tokenizer& tokenizer,
std::string device,
const ov::AnyMap& plugin_config,
const std::string& ov_tokenizer_path
const std::string& ov_tokenizers_path
): m_tokenizer(tokenizer), m_device(device), m_plugin_config(plugin_config) {
ov::Core core;

@@ -95,12 +95,12 @@ ov::LLMPipeline::LLMPipelineImpl::LLMPipelineImpl(
}
}

ov::LLMPipeline::LLMPipeline(std::string& path, std::string device, const ov::AnyMap& config, const std::string& ov_tokenizer_path) {
m_pimpl = make_unique<LLMPipelineImpl>(path, device, config, ov_tokenizer_path);
ov::LLMPipeline::LLMPipeline(std::string& path, std::string device, const ov::AnyMap& config, const std::string& ov_tokenizers_path) {
m_pimpl = make_unique<LLMPipelineImpl>(path, device, config, ov_tokenizers_path);
}

ov::LLMPipeline::LLMPipelineImpl::LLMPipelineImpl(std::string& path, std::string device,
const ov::AnyMap& config, const std::string& ov_tokenizer_path) {
const ov::AnyMap& config, const std::string& ov_tokenizers_path) {
std::string config_path = path + "/" + "config.json";
std::string tokenizer_config_path = path + "/" +"tokenizer_config.json";
std::string generation_config_path = path + "/" +"generation_config.json";
10 changes: 5 additions & 5 deletions src/cpp/src/tokenizer.cpp
Original file line number Diff line number Diff line change
@@ -53,17 +53,17 @@ class Tokenizer::TokenizerImpl {
int64_t m_eos_token_id = 2;

TokenizerImpl() = default;
TokenizerImpl(std::string tokenizers_path, const std::string device, const std::string& ov_tokenizer_path) {
TokenizerImpl(std::string tokenizers_path, const std::string device, const std::string& ov_tokenizers_path) {
ov::Core core;

if (ov::generate_utils::is_xml(tokenizers_path))
OPENVINO_THROW("tokenizers_path should be a path to a dir not a xml file");

if (ov_tokenizer_path.empty()) {
if (ov_tokenizers_path.empty()) {
// OPENVINO_TOKENIZERS_PATH is defined in CMakeLists.txt
core.add_extension(OPENVINO_TOKENIZERS_PATH);
} else {
core.add_extension(ov_tokenizer_path + "/libopenvino_tokenizers.so");
core.add_extension(ov_tokenizers_path + "/libopenvino_tokenizers.so");
}
std::shared_ptr<ov::Model> tokenizer_model, detokenizer_model;
try {
@@ -144,8 +144,8 @@ class Tokenizer::TokenizerImpl {
}
};

Tokenizer::Tokenizer(const std::string& tokenizers_path, const std::string& device, const std::string& ov_tokenizer_path) {
m_pimpl = std::make_shared<TokenizerImpl>(tokenizers_path, device, ov_tokenizer_path);
Tokenizer::Tokenizer(const std::string& tokenizers_path, const std::string& device, const std::string& ov_tokenizers_path) {
m_pimpl = std::make_shared<TokenizerImpl>(tokenizers_path, device, ov_tokenizers_path);
}

std::pair<ov::Tensor, ov::Tensor> Tokenizer::encode(const std::string prompt) {
10 changes: 5 additions & 5 deletions src/python/py_generate_pipeline.cpp
Original file line number Diff line number Diff line change
@@ -62,9 +62,9 @@ std::string call_with_config(ov::LLMPipeline& pipe, const std::string& text, con
return pipe(text, config);
}

std::string genai_module_path() {
std::string ov_tokenizers_module_path() {
py::module_ m = py::module_::import("openvino_tokenizers");
py::list path_list = m.attr("__path__");
py::list path_list = m.attr("__path__");
return std::string(py::str(path_list[0])) + "/lib";
}

@@ -74,9 +74,9 @@ PYBIND11_MODULE(py_generate_pipeline, m) {
py::class_<LLMPipeline>(m, "LLMPipeline")
.def(py::init<const std::string, const ov::Tokenizer&, const std::string, const ov::AnyMap&, const std::string&>(),
py::arg("model_path"), py::arg("tokenizer"), py::arg("device") = "CPU",
py::arg("plugin_config") = ov::AnyMap{}, py::arg("ov_tokenizer_path") = genai_module_path())
py::arg("plugin_config") = ov::AnyMap{}, py::arg("ov_tokenizers_path") = ov_tokenizers_module_path())
.def(py::init<std::string&, std::string, const ov::AnyMap&, const std::string>(),
py::arg("path"), py::arg("device") = "CPU", py::arg("plugin_config") = ov::AnyMap{}, py::arg("ov_tokenizer_path") = genai_module_path())
py::arg("path"), py::arg("device") = "CPU", py::arg("plugin_config") = ov::AnyMap{}, py::arg("ov_tokenizers_path") = ov_tokenizers_module_path())
.def("__call__", py::overload_cast<ov::LLMPipeline&, const std::string&, const py::kwargs&>(&call_with_kwargs))
.def("__call__", py::overload_cast<ov::LLMPipeline&, const std::string&, const ov::GenerationConfig&>(&call_with_config))
.def("generate", py::overload_cast<ov::LLMPipeline&, const std::string&, const py::kwargs&>(&call_with_kwargs))
@@ -105,7 +105,7 @@ PYBIND11_MODULE(py_generate_pipeline, m) {
.def(py::init<std::string&, const std::string&, const std::string&>(),
py::arg("tokenizers_path"),
py::arg("device") = "CPU",
py::arg("ov_tokenizer_path") = py::str(genai_module_path()))
py::arg("ov_tokenizers_path") = py::str(ov_tokenizers_module_path()))

// todo: implement encode/decode when for numpy inputs and outputs
.def("encode", py::overload_cast<const std::string>(&ov::Tokenizer::encode), "Encode a single prompt")