Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bad results, how to solve? #537

Open
qmzpg opened this issue Nov 15, 2024 · 2 comments
Open

bad results, how to solve? #537

qmzpg opened this issue Nov 15, 2024 · 2 comments

Comments

@qmzpg
Copy link

qmzpg commented Nov 15, 2024

EulerAncestralDiscrete_0_gs7.5_s100.mp4

The command is:
CUDA_VISIBLE_DEVICES=0,1,2,3 torchrun --nnodes=1 --nproc_per_node 1 --master_port 29514
-m opensora.sample.sample
--model_type "inpaint"
--model_path "/home//llm/models/Open-Sora-Plan-v1.3.0/any93x640x640_i2v"
--version v1_3
--num_frames 33
--height 352
--width 640
--max_hxw 236544
--crop_for_hw
--cache_dir "../cache_dir"
--text_encoder_name_1 "/home/
/llm/models/mt5-xxl"
--text_prompt examples/cond_prompt.txt
--conditional_pixel_values_path examples/cond_pix_path.txt
--ae WFVAEModel_D8_4x8x8
--ae_path "/home/*******/llm/models/Open-Sora-Plan-v1.3.0/vae"
--save_img_path "./save_path"
--fps 6
--guidance_scale 7.5
--num_sampling_steps 100
--max_sequence_length 512
--sample_method EulerAncestralDiscrete
--seed 1234
--num_samples_per_prompt 1
--rescale_betas_zero_snr
--prediction_type "v_prediction"
--noise_strength 0.0 \

@yunyangge
Copy link
Collaborator

Due to the data used during training, selecting num_frames=93 will yield relatively better results. Additionally, because of the current model's limited capability, inference in multi-person scenarios has a higher failure rate. This can be mitigated by adjusting the seed and prompt multiple times to achieve the best results. We are working on training a new version of the Open-Sora Plan model to provide better generation quality. Thank you for your understanding.

@WangRongsheng
Copy link

how to set --text_prompt examples/cond_prompt.txt?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants