Skip to content

Latest commit

 

History

History
38 lines (32 loc) · 924 Bytes

README.md

File metadata and controls

38 lines (32 loc) · 924 Bytes
Raw

DeepSpeed Huggingface Text Generation Script

Setup

Python dependencies:

pip install -r requirements.txt

Usage

The test-run-generation.py example can be run using test-gpt.sh, which serves as an example of how to run the script.

deepspeed --num_nodes 1 --num_gpus 1 test-run-generation.py \
    --model_type=gpt2 \
    --model_name_or_path=gpt2-xl \
    --sample_input single_query.txt \
    --fp16 \
    --ds-inference

Example Output

Command:

deepspeed --num_nodes 1 --num_gpus 1 test-run-generation.py \
    --model_type=gpt2 \
    --model_name_or_path=gpt2-xl \
    --sample_input single_query.txt \
    --fp16 \
    --ds-inference

Output:

=== GENERATED SEQUENCE 1 ===
What is DeepSpeed?

DeepSpeed is a multi-dimensional data compression framework designed to achieve high compression ratio on human readable