SimpleVQA

A Deep Learning based No-reference Quality Assessment Model for UGC Videos

Description

This is a repository for the model proposed in the paper "A Deep Learning based No-reference Quality Assessment Model for UGC Videos". Arxiv Version ACM MM 2022 Version

Usage

Install Requirements

pytorch
opencv
scipy
pandas
torchvision
torchvideo

Download databases

LSVQ KoNViD-1k Youtube-UGC

Train models

1. Extract video frames

python -u extract_frame_LSVQ.py >> logs/extract_frame_LSVQ.log

2. Extract motion features

 CUDA_VISIBLE_DEVICES=0 python -u extract_SlowFast_features_LSVQ.py \
 --database LSVQ \
 --model_name SlowFast \
 --resize 224 \
 --feature_save_folder LSVQ_SlowFast_feature/ \
 >> logs/extracted_LSVQ_SlowFast_features.log

3. Train the model

 CUDA_VISIBLE_DEVICES=0 python -u train_baseline.py \
 --database LSVQ \
 --model_name UGC_BVQA_model \
 --conv_base_lr 0.00001 \
 --epochs 10 \
 --train_batch_size 8 \
 --print_samples 1000 \
 --num_workers 6 \
 --ckpt_path ckpts \
 --decay_ratio 0.9 \
 --decay_interval 2 \
 --exp_version 0 \
 --loss_type L1RankLoss \
 --resize 520 \
 --crop_size 448 \
 >> logs/train_UGC_BVQA_model_L1RankLoss_resize_520_crop_size_448_exp_version_0.log

Test the model

You can download the trained model via Google Drive.

Test on the public VQA database

CUDA_VISIBLE_DEVICES=0 python -u test_on_pretrained_model.py \
--database KoNViD-1k \
--train_database LSVQ \
--model_name UGC_BVQA_model \
--feature_type SlowFast \
--trained_model ckpts/UGC_BVQA_model.pth \
--num_workers 6 \
>> logs/test_on_KoNViD-1k_train_on_LSVQ.log

Test on a single video

CUDA_VISIBLE_DEVICES=0 python -u test_demo.py \
--method_name single-scale \
--dist videos/2999049224_original_centercrop_960x540_8s.mp4 \
--output result.txt \
--is_gpu \
>> logs/test_demo.log

Citation

If you find this code is useful for your research, please cite:

@inproceedings{sun2022a,
title = {A Deep Learning Based No-Reference Quality Assessment Model for UGC Videos},
author = {Sun, Wei and Min, Xiongkuo and Lu, Wei and Zhai, Guangtao},
booktitle={Proceedings of the 30th ACM International Conference on Multimedia},
year = {2022},
pages = {856–865},
}