Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom samplingconfig addition #2633

Open
wants to merge 48 commits into
base: main
Choose a base branch
from

Conversation

buddhapuneeth
Copy link

Based on v0.15 tag.

Related to issue: #2609

I made the changes as mentioned in this commit, but still not able to compile the TRT-LLM.

Error:

[100%] Linking CXX shared module bindings.cpython-310-x86_64-linux-gnu.so
lto-wrapper: warning: using serial compilation of 39 LTRANS jobs
/usr/bin/ld: /tmp/ccFZkJmj.ltrans12.ltrans.o: in function `tensorrt_llm::pybind::executor::InitBindings(pybind11::module_&)':
<artificial>:(.text+0x2158): undefined reference to `tensorrt_llm::executor::SamplingConfig::setCustom(std::optional<float> const&)'
/usr/bin/ld: <artificial>:(.text+0x215f): undefined reference to `tensorrt_llm::executor::SamplingConfig::getCustom() const'
/usr/bin/ld: <artificial>:(.text+0x4fad): undefined reference to `tensorrt_llm::executor::SamplingConfig::SamplingConfig(int, std::optional<int> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int> const&, std::optional<float> const&, std::optional<unsigned long> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int> const&, std::optional<int> const&, std::optional<int> const&)'
/usr/bin/ld: /tmp/ccFZkJmj.ltrans17.ltrans.o: in function `pybind11::cpp_function::initialize<pybind11::detail::initimpl::factory<tensorrt_llm::pybind::executor::InitBindings(pybind11::module_&)::{lambda(int, std::optional<int> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int> const&, std::optional<float> const&, std::optional<unsigned long>, std::optional<unsigned long> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int>, std::optional<int> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int> const&, std::optional<int> const&, std::optional<int> const&)#6}, pybind11::detail::void_type (*)(), std::unique_ptr<tensorrt_llm::executor::SamplingConfig, std::default_delete<tensorrt_llm::executor::SamplingConfig> > (int, std::optional<int> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int> const&, std::optional<float> const&, std::optional<unsigned long>, std::optional<unsigned long> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int>, std::optional<int> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int> const&, std::optional<int> const&, std::optional<int> const&), pybind11::detail::void_type ()>::execute<pybind11::class_<tensorrt_llm::executor::SamplingConfig>, pybind11::arg_v, pybind11::kw_only, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v>(pybind11::class_<tensorrt_llm::executor::SamplingConfig>&, pybind11::arg_v const&, pybind11::kw_only const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&) &&::{lambda(pybind11::detail::value_and_holder&, int, std::optional<int> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int> const&, std::optional<float> const&, std::optional<unsigned long>, std::optional<unsigned long> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int>, std::optional<int> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int> const&, std::optional<int> const&, std::optional<int> const&)#1}, void, pybind11::detail::value_and_holder&, int, std::optional<int> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int> const&, std::optional<float> const&, std::optional<unsigned long>, std::optional<unsigned long> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int>, std::optional<int> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int> const&, std::optional<int> const&, std::optional<int> const&, pybind11::name, pybind11::is_method, pybind11::sibling, pybind11::detail::is_new_style_constructor, pybind11::arg_v, pybind11::kw_only, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v, pybind11::arg_v>(pybind11::class_<tensorrt_llm::executor::SamplingConfig>&&, void (*)(pybind11::detail::value_and_holder&, int, std::optional<int> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int> const&, std::optional<float> const&, std::optional<unsigned long>, std::optional<unsigned long> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int>, std::optional<int> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int> const&, std::optional<int> const&, std::optional<int> const&), pybind11::name const&, pybind11::is_method const&, pybind11::sibling const&, pybind11::detail::is_new_style_constructor const&, pybind11::arg_v const&, pybind11::kw_only const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&, pybind11::arg_v const&)::{lambda(pybind11::detail::function_call&)#3}::_FUN(pybind11::detail::function_call&) [clone .lto_priv.0]':
<artificial>:(.text+0x3c2e): undefined reference to `tensorrt_llm::executor::SamplingConfig::SamplingConfig(int, std::optional<int> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int> const&, std::optional<float> const&, std::optional<unsigned long> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int> const&, std::optional<int> const&, std::optional<int> const&)'
/usr/bin/ld: <artificial>:(.text+0x3d3c): undefined reference to `tensorrt_llm::executor::SamplingConfig::SamplingConfig(int, std::optional<int> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int> const&, std::optional<float> const&, std::optional<unsigned long> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<float> const&, std::optional<int> const&, std::optional<int> const&, std::optional<int> const&)'
collect2: error: ld returned 1 exit status
gmake[3]: *** [tensorrt_llm/pybind/CMakeFiles/bindings.dir/build.make:282: tensorrt_llm/pybind/bindings.cpython-310-x86_64-linux-gnu.so] Error 1
gmake[2]: *** [CMakeFiles/Makefile2:1681: tensorrt_llm/pybind/CMakeFiles/bindings.dir/all] Error 2
gmake[1]: *** [CMakeFiles/Makefile2:1688: tensorrt_llm/pybind/CMakeFiles/bindings.dir/rule] Error 2
gmake: *** [Makefile:439: bindings] Error 2
Traceback (most recent call last):
  File "/code/tensorrt_llm/./scripts/build_wheel.py", line 434, in <module>
    main(**vars(args))
  File "/code/tensorrt_llm/./scripts/build_wheel.py", line 208, in main
    build_run(
  File "/usr/lib/python3.10/subprocess.py", line 526, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command 'cmake --build . --config Release --parallel 48 --target tensorrt_llm nvinfer_plugin_tensorrt_llm th_common bindings   executorWorker  ' returned non-zero exit status 2.

juney-nvidia and others added 30 commits October 15, 2023 21:26
* Fix deadlines in README.md

* Update README.md
Co-authored-by: Guoming Zhang <[email protected]>
* Update TensorRT-LLM

---------

Co-authored-by: Shixiaowei02 <[email protected]>
* Update TensorRT-LLM

---------

Co-authored-by: Shixiaowei02 <[email protected]>
* Update TensorRT-LLM

* update

* Bump version to 0.7.0
Shixiaowei02 and others added 18 commits December 22, 2023 08:00
* Update TensorRT-LLM

---------

Co-authored-by: Shixiaowei02 <[email protected]>
* Update TensorRT-LLM

---------

Co-authored-by: Shixiaowei02 <[email protected]>
* Update TensorRT-LLM

---------

Co-authored-by: Bhuvanesh Sridharan <[email protected]>
Co-authored-by: Morgan Funtowicz <[email protected]>
Co-authored-by: Eddie-Wang1120 <[email protected]>
Co-authored-by: meghagarwal <[email protected]>
* TensorRT-LLM Release 0.10.0

---------

Co-authored-by: Loki <[email protected]>
Co-authored-by: meghagarwal <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants