Unmasking Deepfakes by Fusing Rich Features from Two-Stream CNN Model

Abstract

In contrast to the traditional object detection methods, image manipulation detection focuses on tampering artifacts instead of image content, indicating that more depth features must be learned to detect image manipulation. Deepfakes are one of these techniques that have appeared in recent times and need to learn a lot of the richer features to be detected. Deepfakes are a harmful application that affects all segments of society. It is meant to change the person's face and replace it with another person using deep learning techniques. In this paper, we contribute to finding a solution to detect the fakes. A new two-stream CNN model-based deep learning is developed, where two streams are combined, exploiting the fusion layer. Following the fusion layer, the data is classified using the classification layer. The first stream is a semantic stream to extract specific features from the RGB image input to identify manipulation artifacts such as blurring variation, the boundary of the face mask, and lighting difference. The second stream is a texture stream that exploits the texture features extracted from a Gabor bank filters layer. The proposed strategy significantly outperformed the previous methods that were in use. The measured performance metrics have an accuracy of more than 99.5%.