Add metrics endpoint

[AllentDan]

Open http://xxxx:23333/metrics/ to view the metrics.

[zhulinJulia24]

感觉gpu和cpu是不是不用放里面，可以自己起一个nvidia的端口就能获得metrics，如 docker run -d --gpus all --rm -p 9400:9400 nvcr.io/nvidia/k8s/dcgm-exporter:3.1.7-3.1.4-ubuntu20.04

[zhulinJulia24]

能不能添加tokens相关性能指标？

[nathan-az]

Can we add first token time as well, so the difference between scheduling time and first token time can be used to estimate prefill time?

Here is vLLM's source as a reference

[uzuku]

latency类的指标可以考虑用Histogram/Summary类型的metric，方便计算分布以及不同时段的平均值，也可以简化计算逻辑

[uzuku]

对于有单位的指标，可以在metrics_name中声明单位，如duration_queue_seconds，写PromQL的时候会比较直观

[uzuku]

Request数量类指标建议使用Counter类型，单调递增

[uzuku]

gpu的index可以用label来指定，用info的话指标变成str了，不好在PromQL中做计算

[AllentDan]

@uzuku
I removed postprocess metrics since it is weird for me to align lmdeploy api_server with triton inference server. There are no clear pre-infer-post stages when streaming mode is applied.

[Huarong]

Thanks for the great work. The metrics is import in production observation. Can we expect this feature to be merged in the next release? @AllentDan

[ao-zz]

If our metrics can be compatible with vllm, it will greatly facilitate the comparison of deployment performance between lmdeploy and vllm.
For instance, vllm supports histogram_e2e_time_request , which is not currently considered in this PR.

如果我们的 metrics 可以和 vllm 的兼容，将为 lmdeploy 和 vllm 之间的部署性能比较，带来很大的方便。
例如，vllm 支持 histogram_e2e_time_request 这个指标，我们是否也会考虑支持呢

[lvhan028]

If our metrics can be compatible with vllm, it will greatly facilitate the comparison of deployment performance between lmdeploy and vllm. For instance, vllm supports histogram_e2e_time_request , which is not currently considered in this PR.

如果我们的 metrics 可以和 vllm 的兼容，将为 lmdeploy 和 vllm 之间的部署性能比较，带来很大的方便。 例如，vllm 支持 histogram_e2e_time_request 这个指标，我们是否也会考虑支持呢

好。我们尽量对齐