Deepfake Detection

Abstract

The rapid development of deepfake technology has led to the creation of very realistic manipulated videos, thereby causing serious threats to media authenticity, cybersecurity, and public trust. Existing detection methods cannot generalize to diversified data sets and are strongly reliant on spatial differences or asynchrony of audio and video streams. This work proposes an automated deepfake detection system relying on Convolutional Neural Networks (CNNs) and Long Short Term Memory (LSTM) for the analysis of lip synchrony and authentication of audiovisual content. Our system relies on prominent visual and auditory characteristics to identify synchronization differences in lip movements. MediaPipe FaceMesh detects the mouth area from video frames, which is then processed by a VGG16-based feature extractor. At the same time, Mel-Frequency Cepstral Coefficients (MFCCs) are applied to extract speech patterns from audio. A multimodal deep learning system classifies videos as authentic or synthetic depending on these characteristics. Experimental outcomes indicate enhanced accuracy in speech-lip inconsistency detection, with 98% accuracy and better than conventional methods. The system is extremely robust against adversarial deepfake attacks by incorporating spatial and temporal features. Scaling up the dataset to include other Indian languages and real-time detection of political videos from various states is future work, further strengthening the battle against disinformation.