[WIP] porting Medusa

[zhyncs]

Motivation

As titled, porting FasterDecoding/Medusa

Modification

finished

1、Medusa weights conversion
2、Medusa weights loading
3、Porting FasterDecoding/Medusa Heads code with LMDeploy components and utilities
4、TP support: Distribute the weights equally based on hidden_size

We've used https://github.com/zhyncs/medusa-whl-centos7/releases/tag/2024.02.27, https://huggingface.co/FasterDecoding/medusa-vicuna-13b-v1.3, https://huggingface.co/lmsys/vicuna-13b-v1.3 to verify the correctness of porting code (fp16 and bf16).

under debugging

1、Porting generate_candidates and evaluate_posterior
2、Integrating with LlamaBatch

todo

1、add docs
2、add tests
3、benchmark

Checklist

1. Pre-commit or other linting tools are used to fix the potential lint issues.
2. The modification is covered by complete unit tests. If not, please add more unit tests to ensure the correctness.
3. If the modification has a dependency on downstream projects of a newer version, this PR should be tested with all supported versions of downstream projects.
4. The documentation has been modified accordingly, like docstring or example tutorials.

[zhyncs]

After creating a new commit, the updates I made are not showing up in this PR. I believe the changes were submitted correctly. It appears to be a bug on GitHub, so I will close this PR and open a new one.

[zhyncs]

refer to #1231 just close this

[zhyncs]

https://www.githubstatus.com/

It is indeed a problem with GitHub.