Skip to content

Commit

Permalink
feat: venus adapter
Browse files Browse the repository at this point in the history
  • Loading branch information
jimpang committed Sep 12, 2024
1 parent ad44916 commit 5dac321
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion vllm/entrypoints/api_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ async def stream_results() -> AsyncGenerator[bytes, None]:
text_outputs = [
output.text for output in request_output.outputs
]
output_tokens = [output.token_ids for output in request_output.outputs]
output_tokens = [list(output.token_ids) for output in request_output.outputs]
logprobs = [[{k: asdict(v) for k, v in logprobs.items()} for logprobs in
output.logprobs] if output.logprobs is not None else None for output in request_output.outputs]
ret = {"text": text_outputs, "output_token_ids": output_tokens, "logprobs": logprobs}
Expand Down

0 comments on commit 5dac321

Please sign in to comment.