Video Corruptions When Using NVENC HEVC Encoding

[smyhlin]

Media player showing two identical frames side by side with noticeable visual distortions, such as glitching and pixelation. These effects could be the result of encoding or compression issues during video processing. On multiple video and only witn NVENC. x265, x264, Loseless works OK.

Here’s a Markdown-formatted version of console output:

# TensorRT Engine Generation and Upscaling Process

### Generating TensorRT Engine
**Command:**
```bash
python\VSPipe.exe -c y4m --arg "slot=1" --arg "video_path=F:\Media\Serials\Avatar\Avatar.The.Last.Airbender.720p.BDRip.H264.DD2.0[UKR].DTS2.0[ENG].SRT[ENG]-Hurtom\Avatar.The.Last.Airbender.2005.S01E01.720p.BDRip.H264-Hurtom.mkv" --start 0 --end 1 "C:\Users\myhli\Desktop\VideoJaNai-win-Portable\current\backend\animejanai\core\animejanai_encode.vpy" -p .

AnimeJanai Upscaling Process

Configuration:

{
  "min_px": 0.0,
  "max_px": "inf",
  "min_resolution": "0x0",
  "max_resolution": "0x0",
  "min_fps": 0.0,
  "max_fps": "inf",
  "models": [
    {
      "resize_factor_before_upscale": 100.0,
      "resize_height_before_upscale": 0.0,
      "name": "2x_AnimeJaNai_HD_V3_Compact"
    }
  ],
  "rife": false,
  "rife_factor_numerator": 2,
  "rife_factor_denominator": 1,
  "rife_model": 422,
  "rife_ensemble": false,
  "rife_scene_detect_threshold": 0.15,
  "final_resize_height": 0.0,
  "final_resize_factor": 100.0,
  "tensorrt_engine_settings": ""
}

Upscaling Log:

• Upscale Mode: Scaling 2x from 960x720 using engine 2x_AnimeJaNai_HD_V3_Compact
• TensorRT Settings:

  --bf16 --minShapes=input:1x3x8x8 --optShapes=input:1x3x1080x1920 --maxShapes=input:1x3x1080x1920 --inputIOFormats=fp32:chw --outputIOFormats=fp32:chw --tacticSources=+CUDNN,-CUBLAS,-CUBLAS_LT --skipInference

Warnings:

• Using an engine plan file across different models of devices is not recommended and may cause performance issues or errors.

Performance:

• Script evaluation: Done in 5.64 seconds
• Frames processed: 1/2
• Output speed: 2 frames in 0.11 seconds (17.65 fps)

---

FFmpeg Encoding Process

Command:

python\VSPipe.exe -c y4m --arg "slot=1" --arg "video_path=F:\Media\Serials\Avatar\Avatar.The.Last.Airbender.720p.BDRip.H264.DD2.0[UKR].DTS2.0[ENG].SRT[ENG]-Hurtom\Avatar.The.Last.Airbender.2005.S01E01.720p.BDRip.H264-Hurtom.mkv" "C:\Users\myhli\Desktop\VideoJaNai-win-Portable\current\backend\animejanai\core\animejanai_encode.vpy" - | "C:\Users\myhli\AppData\Roaming\VideoJaNai\ffmpeg\ffmpeg.exe" -y -i pipe: -i "F:\Media\Serials\Avatar\Avatar.The.Last.Airbender.720p.BDRip.H264.DD2.0[UKR].DTS2.0[ENG].SRT[ENG]-Hurtom\Avatar.The.Last.Airbender.2005.S01E01.720p.BDRip.H264-Hurtom.mkv" -map 0:v -c:v hevc_nvenc -preset p7 -profile:v main10 -b:v 50M -max_interleave_delta 0 -map 1:t? -map 1:a? -map 1:s? -c:t copy -c:a copy -c:s copy "F:\janai_result\Avatar.The.Last.Airbender.2005.S01E01.720p.BDRip.H264-Hurtom-corruption.mkv"

FFmpeg Version and Details

• Version: 2025-01-08-git-251de1791e-essentials_build-www.gyan.dev
• Build: gcc 14.2.0 (Rev1, Built by MSYS2 project)

---

Stream Mapping and Metadata

Input #0:

• Type: yuv4mpegpipe
• Video: 1920x1440, yuv420p, 23.98 fps

Input #1:

• Type: matroska,webm
• Duration: 00:23:40.51
• Streams:
  1. Audio (ukr): ac3, 192 kb/s
  2. Video (eng): h264, 960x720, 23.98 fps
  3. Audio (eng): dts, 1536 kb/s
  4. Subtitle: Full (eng)
  5. Subtitle: SDH (eng)

Output #0:

• Format: matroska
• Video: hevc (Main 10), 1920x1440, 50 Mbps
• Audio (ukr): ac3, 192 kb/s
• Audio (eng): dts, 1536 kb/s
• Subtitle: Full (eng) and SDH (eng)

---

Final Output

• File: F:\janai_result\Avatar.The.Last.Airbender.2005.S01E01.720p.BDRip.H264-Hurtom-corruption.mkv

---

Markdown:

### System Configuration

**Operating System:**
- Name: Microsoft Windows 11 Pro
- Version: 10.0.22621, Build 22621

**System Details:**
- Manufacturer: Gigabyte Technology Co., Ltd.
- Model: B550 AORUS ELITE V2
- Type: x64-based PC
- BIOS Version/Date: American Megatrends International, LLC. F18d, 2024-09-02
- CPU: AMD Ryzen 7 5800X3D, 8-Core Processor, 3.4 GHz, 8 Cores, 16 Logical Processors
- Installed RAM: 32.0 GB

**Storage and Boot:**
- Windows Directory: `C:\Windows`
- System Directory: `C:\Windows\system32`
- Boot Device: `\Device\HarddiskVolume1`

**Virtualization and Security:**
- Secure Boot State: Disabled
- Virtualization-Based Security: Enabled

---

### NVIDIA System Configuration

**GPU:**
- Model: NVIDIA GeForce RTX 3080
- Driver Version: 566.36 (DCH)
- DirectX Version: 12.0
- CUDA Cores: 8704
- Core Clock: 1740 MHz
- Memory Data Rate: 19.00 Gbps
- Memory Interface: 320-bit
- Memory Bandwidth: 760.08 GB/s
- Dedicated Video Memory: 10,240 MB (GDDR6X)
- Shared System Memory: 16,336 MB
- Total Available Graphics Memory: 26,576 MB

**Bus:**
- PCI Express x16 Gen4

**BIOS Version:**
- 94.02.71.40.6C

---