Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request] Add traditional methods for comparison #4

Open
4 tasks done
calico-1226 opened this issue Aug 15, 2024 · 1 comment
Open
4 tasks done

[Feature Request] Add traditional methods for comparison #4

calico-1226 opened this issue Aug 15, 2024 · 1 comment
Assignees
Labels
enhancement New feature or request

Comments

@calico-1226
Copy link
Member

calico-1226 commented Aug 15, 2024

Required prerequisites

Motivation

While some traditional methods may not precisely capture human value preferences, they are usually faster and cheaper to compute. Therefore, we will integrate some traditional metrics into the SafeSora library, expecting they will complement feedback-based methods and enable comparative analysis.

Checklist

  • I have provided all relevant and necessary information above.
  • I have chosen a suitable title for this issue.
@calico-1226
Copy link
Member Author

In #5, three methods include:

  • PSNR vs. Informativeness: PSNR measures the quality of a reconstructed or compressed image/video by comparing it to the original, and assessing their similarity. We evaluate dynamic changes in the video by analyzing PSNR between the first and subsequent frames.
  • HPSv2 vs. Aesthetic: HPSv2 predicts human preferences for image beauty. We use it to assess the aesthetics of each video frame, averaging the results for an overall aesthetic measure.
  • CLIP vs. Instruction Following: CLIP assesses instruction adherence by evaluating the similarity between the prompt and video frames and averaging the results of frames for an overall instruction-following measure.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants