tensorrt-llm 0.12.0.dev2024073000, triton 2.46.0 #52

yorickvP · 2024-08-08T14:04:04Z

Breaking changes:

enable_trt_overlap now useless
max_queue_size is now required, 0 is a good default
max_seq_len required until next week, but will default to the model max input size after

Other changes:

removed bls

Remove the tensorrt_llm python script, since it confuses `maybe_download_tarball_with_pget`

hopefully fixes `ValueError: Invalid pattern: '**' can only be an entire path component`

Should fix #51

adds TRTLLM_BUILDER_VARIANT=h100 to work around "It looks like a copy of this model version already exists on Replicate"

yorickvP and others added 30 commits July 19, 2024 13:54

tensorrt-llm: 0.9.0 -> 0.10.0, triton: 2.42.0 -> 2.44.0

d12ae7b

update cognix

c3eb6f3

update triton_templates to trtllm-0.10

cec6d28

don't tokenize in postprocessing (see #27)

0626e44

instantiate triton_model_repo with the default config

4cd5533

Adjust decoding_mode after testing, omit missing optional params

25cb314

Remove the tensorrt_llm python script, since it confuses `maybe_download_tarball_with_pget`

ignore empty SSE's, remove decoding_mode, just omit topk instead

a19b28b

bump pget==0.8.2, cog==0.10.0-alpha16

d86927d

tensorrt-llm: decrease closure size by cleaning up kernels

098f7e7

bump cognix to exclude train: null

cd3ce0e

tensorrt-llm: 0.10.0 -> 0.11.0

e5ac906

update triton_templates

57e97ee

update triton_model_repo

7129024

Add a $TRTLLM_PYTHON environment with pydantic 2

8a2170f

use pytriton 0.5.8

494d476

add nvidia-modelopt setuptools dependency

5c3eded

datasets: 2.14.4 -> 2.20.0

dfd19b4

hopefully fixes `ValueError: Invalid pattern: '**' can only be an entire path component`

update cognix

d32b60c

tensorrt-llm: 0.11.0 -> 0.12.0.dev2024072300

a2fc518

update triton_model_repo, triton_templates

8cc2174

don't add 2 copies of tensorrt-llm in builder

92000d5

tensorrt-lllm: 0.12.0.dev2024072300 -> 0.12.0.dev2024072301

dc78a43

github action: limit to 12 cores

5fb8fb7

remove rogue .gitkeep files

6888d8f

tensorrt-cu12: 10.2.0 -> 10.2.0.post1

49ccca6

Should fix #51

Push builder to triton-builder-h100

73567de

fix: work around replicate unique model check

d2adf71

adds TRTLLM_BUILDER_VARIANT=h100 to work around "It looks like a copy of this model version already exists on Replicate"

build tensorrt-llm from source

2805da2

trtllm: fix patch location to work in nix develop

d10ce53

Add support for building torch & trtllm(python) from source

95e44d8

yorickvP added 5 commits August 7, 2024 15:10

tensorrt-llm: 0.12.0.dev2024072301 -> 0.12.0.dev2024073000

75c6f70

update triton_model_repo

b25f096

Update triton_model_repo python and configs, remove bls

56557e8

update triton_model_repo and default config

4152b5a

tensorrt-llm: always autoPatchelf $out

67438c8

yorickvP requested a review from joehoover August 8, 2024 14:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

tensorrt-llm 0.12.0.dev2024073000, triton 2.46.0 #52

tensorrt-llm 0.12.0.dev2024073000, triton 2.46.0 #52

yorickvP commented Aug 8, 2024

tensorrt-llm 0.12.0.dev2024073000, triton 2.46.0 #52

Are you sure you want to change the base?

tensorrt-llm 0.12.0.dev2024073000, triton 2.46.0 #52

Conversation

yorickvP commented Aug 8, 2024