Comparative Study of AI-Based Video Optimization Techniques: Super-Resolution, Denoising, and Frame Interpolation

Authors

  • Untoro Apsiswanto✉ Department of Information Systems, Universitas Dharmawacana Metro, Metro, Indonesia Author
  • Ridwan Yusuf Department of Information Systems, Universitas Dharmawacana Metro, Metro, Indonesia Author
Diajukan 7 Okt 2024 Diterbitkan 31 Des 2024
DOI: https://doi.org/10.59025/ab894b41

Keywords:

AI video optimization, super-resolution, denoising, frame interpolation, deep learning

Abstract

Video quality degradation due to low resolution, noise, and limited frame rate poses a significant challenge in modern digital content ecosystems, ranging from streaming services to real-time surveillance systems. This study presents a systematic comparative analysis of three categories of artificial intelligence-based video optimization techniques super-resolution, denoising, and frame interpolation benchmarked against conventional methods through simulation on five representative video test clips. Six state-of-the-art deep learning models were evaluated using four quantitative metrics: PSNR, SSIM, LPIPS, and VMAF. Results demonstrate that AI-based methods consistently outperform conventional approaches, with VRT achieving optimal performance in super-resolution (PSNR 34.2 dB) and RIFE attaining the best frame interpolation score (SSIM 0.941).

References

  1. A. Vaswani et al., "Attention is all you need," in Proc. NeurIPS, vol. 30, 2017, pp. 5998-6008.
  2. K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, "Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising," IEEE Trans. Image Process., vol. 26, no. 7, pp. 3142-3155, Jul. 2017.
  3. J. Chen et al., "VideoSwin: A pure transformer for video understanding," in Proc. IEEE/CVF CVPR, 2022, pp. 3202-3211.
  4. C. Dong, C. C. Loy, K. He, and X. Tang, "Image super-resolution using deep convolutional networks," IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, no. 2, pp. 295-307, Feb. 2016.
  5. H. Shi et al., "RIFE: Real-time intermediate flow estimation for video frame interpolation," in Proc. ECCV, 2022, pp. 624-642.
  6. Z. Liu, H. Yao, H. Li, and F. Wu, "Video compression through image interpolation," in Proc. IEEE/CVF CVPR, 2018, pp. 696-705.
  7. J. Liang et al., "VRT: A video restoration transformer," IEEE Trans. Image Process., vol. 32, pp. 2171-2184, 2023.
  8. W. Xue et al., "TDAN: Temporally-deformable alignment network for video super-resolution," in Proc. IEEE/CVF CVPR, 2019, pp. 3360-3369.
  9. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, "Image quality assessment: From error visibility to structural similarity," IEEE Trans. Image Process., vol. 13, no. 4, pp. 600-612, Apr. 2004.
  10. R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, "The unreasonable effectiveness of deep features as a perceptual metric," in Proc. IEEE/CVF CVPR, 2018, pp. 586-595.
  11. Netflix Technology Blog, "VMAF: The journey continues," Netflix Tech Blog, 2018. [Online]. Available: https://netflixtechblog.com/vmaf-the-journey-continues-44b51ee9ed12
  12. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proc. IEEE/CVF CVPR, 2016, pp. 770-778.
  13. J. Liang et al., "SwinIR: Image restoration using Swin Transformer," in Proc. IEEE/CVF ICCV Workshops, 2021, pp. 1833-1844.
  14. D. B. Huynh-Thu and M. Ghanbari, "Scope of validity of PSNR in image/video quality assessment," Electron. Lett., vol. 44, no. 13, pp. 800-801, Jun. 2008.
  15. W. Shi et al., "Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network," in Proc. IEEE/CVF CVPR, 2016, pp. 1874-1883.
  16. X. Tao, H. Gao, R. Liao, J. Wang, and J. Jia, "Detail-revealing deep video super-resolution," in Proc. IEEE/CVF ICCV, 2017, pp. 4472-4480.
  17. A. Lugmayr et al., "Repaint: Inpainting using denoising diffusion probabilistic models," in Proc. IEEE/CVF CVPR, 2022, pp. 11461-11471.
  18. X. Gu et al., "FLOP: Flow estimation and prediction for video enhancement," in Proc. ICML, 2023, pp. 11892-11908.
  19. H. Zhou, C. Li, and X. Yang, "A survey of deep learning methods for video enhancement and restoration," J. Vis. Commun. Image Represent., vol. 89, p. 103628, 2022.
  20. ITU-T Recommendation H.265, "High efficiency video coding," International Telecommunication Union, Geneva, 2013.
Cover JTDS Vol 2 No 2 (2024)

Downloads

How to Cite

Comparative Study of AI-Based Video Optimization Techniques: Super-Resolution, Denoising, and Frame Interpolation. (2024). Journal of Technology and Data Science, 2(2), 73-84. https://doi.org/10.59025/ab894b41