The Deepfakes Analysis Unit (DAU) analysed a video that features former Indian Prime Minister Dr. Manmohan Singh apparently promoting a financial scheme. After analysing the video we noticed that most of the content in its audio track was identical to a manipulated financial scam video featuring billionaire businessman Gautam Adani. After expert analysis we were able to conclude that this video too was fabricated using synthetic audio.
The almost five-minute video was discovered by us through social media monitoring. It was posted from the same Facebook account which had posted the video featuring Mr. Adani, recently debunked by us. In that video there is a moment when the synthetic voice accompanying the visuals of Adani refers to itself as “Manmohan Singh”.
Chronologically, the video featuring Dr.Singh was posted first followed by the one with Mr. Adani. We do not know if the uploader is also the creator of both the manipulated videos. However, on comparing the audio tracks word-by-word, we noticed that the same script was largely used for both the videos but for the references to the investment amount, returns, and the sequence of a few lines explaining details of the dubious scheme.
Elevenlabs, a company specialising in voice A.I. research and deployment, had confirmed to us that the audio track for the Adani video was synthetic and that the user who had misused their platform by creating that audio had been blocked from their platform. For the Singh video as well they told us that the audio track was A.I.-generated and the user who broke their “terms of use” by generating the audio had been previously identified as a bad actor through their in-house automated moderation system, and had been blocked from their platform.
There are several visual oddities that we noticed in the video featuring Singh. His mouth region seemed to be moving independent of the rest of his face, making his lips appear unnaturally animated. His lip movements matched the spoken words for the most part, but in some instances his lips seemingly quivered even as no words were heard in the audio track.
The voice in the video sounds somewhat like Singh’s when compared with his recorded interviews and speeches. His signature mellow tone is decently captured in the audio, but the voice sounds hastened, lacking the characteristic pauses of his delivery. The synthetic voice carries a slight accent, which is unlike his and sounds western.
A reverse image search using screenshots from the video led us to this video published on Sept.1, 2019 from the official Youtube channel of the Indian National Congress (INC) party. This video carries a “Bharat Jodo Nyay Yatra” logo on the bottom right corner, which is missing from the manipulated video.
We also found a version of the same video published on the same date from the official YouTube channel of The Wire, an online news platform. This video carries the logo of The Wire on the top right corner, which seems to have been replaced in the manipulated video with an animation of the Indian flag — the tricolour appears to flutter throughout.
The content of the video carried by the INC as well The Wire on their respective Youtube channels is identical but for their logos. It seems that the version seen on the Youtube channel of The Wire has been lifted to create the manipulated video given the placement of the animated flag, which hides their logo.
The clothing, backdrop, and body language of Singh are identical in both versions of the authentic video as well as the manipulated video. He speaks in English in the authentic video content as well as the manipulated video but there is no mention of any A.I.-powered financial scheme in the original versions nor do they have captions.
The overall delivery in the manipulated video sounds scripted. However, given Singh’s natural inscrutable oratory, it is difficult to pick on the nuances of pitch and expression in the original versions for comparison with the manipulated.
At the DAU, we refrain from sharing manipulated content since we do not want to give oxygen to bad actors behind harmful and misleading content. However, to highlight the copycat nature of the financial scam videos we are sharing below a comparison of the scripts of the videos featuring Singh and Adani.
On the basis of our observations and expert analysis, we assessed that original footage featuring Singh was used with an A.I.-generated audio track to fabricate yet another financial scam video. The similarities in the script of this scam video with the one featuring Adani could be an attempt by bad actors to expedite the process of generating such fake videos using synthetic voice.
(Written by Debraj Sarkar and Debopriya Bhattacharya, edited by Pamposh Raina.)
Kindly Note: The manipulated audio/video files that we receive on our tipline are not embedded in our assessment reports because we do not intend to contribute to their virality.
You can read below the fact-checks related to this piece published by our partners:
Viral Video Of Manmohan Singh Promoting Investment Scheme Is A Deepfake
This Video of Manmohan Singh Promoting an Investment Platform Is a Deepfake!
Video Of Manmohan Singh Promoting Investment Opportunity Is An AI-Edited Fake
