The Deepfakes Analysis Unit (DAU) analysed a video featuring Yogi Adityanath, Chief Minister of Uttar Pradesh, apparently promoting a gaming platform. After putting the video through A.I. detection tools and getting experts to weigh in, we were able to conclude that the video was fabricated with synthetic audio.
The 26-second video in Hindi, embedded in a Facebook post, was sent to the DAU by a fact-checking partner for analysis. Packaged as a news story, it carried a logo on the bottom left of the frame resembling that of an English news channel, and seemingly narrated how Mr. Adityanath has made financial gains using a supposed pilot gaming platform.
The video opened with a female voice, however, the source of that voice was not very obvious as a clip of Adityanath in a public setting was predominantly visible. That voice seemed to be from another clip, featuring a female anchor, placed as an inset on the bottom left of the video frame, right above the logo. Both clips played out concurrently.
The video also featured another clip of Adityanath in which he was purportedly speaking about his investment and returns from the gaming platform and encouraging people to invest. Even as that clip played on in one half of the frame, the other half had a mobile phone screen with the interface of the supposed platform highlighting multiplier returns. Superimposed text in Hindi across the bottom of the video, similar to a news ticker seen on television channels, complimented the visuals and audio.
The other tiny clips in the video showed bundles of Indian currency running through cash sorting machines, and a mobile device being used to credit money to an account through an app, which bore a logo resembling that of the State Bank of India.
The overall video quality was poor, despite that it was apparent that the anchor’s lip movements did not match the words heard in the female voice. The lip-sync for Adityanath was not perfect either, however, in some bits it appeared that there was some consonance between his lip movements and the male voice being attributed to him. Though that voice sounded similar to his own voice, the tone seemed to have a synthetic quality to it when compared to his usual style of delivery.
To trace the origin of the various clips interspersed throughout the video we carried out a reverse image search using screenshots from the video as well as a keyword search, particularly for one of the videos featuring Adityanath, which appeared to be from the television show “Aap Ki Adalat”.
The anchor’s visuals seem to have been lifted from this video published on May 28, 2024 from the official YouTube channel of WION, a television news channel. One of the clips featuring Adityanath was taken from this video of him addressing a rally, published from his official YouTube channel on May 30, 2024. The other clip of his was taken from this video published from the official YouTube channel of India TV Aap Ki Adalat on May 11, 2024.
The backdrop, clothing, and body language of the subjects seen in the doctored video matched with those in the original videos. However, the anchor spoke in English in the original video and did not mention anything about a supposed gaming platform or Adityanath. The two clips of Adityanath only focus on politics.
The other visuals in the doctored video or the text presented in the news ticker format could not be found in any of the original videos.
We wanted to ascertain if A.I. had been used to manipulate the video, for which we ran it through A.I. detection tools.
Hive AI’s deepfake video detection tool suggested that there were no A.I. elements in the visuals. Their audio tool, however, indicated that there was a high probability that the audio track had been tampered with A.I. particularly toward the end, the portions of the audio being attributed to Adityanath.
We also ran the video through TrueMedia’s deepfake detector which suggested substantial evidence of manipulation in the video. In a further breakdown of the overall analysis, it gave a 100 percent confidence score to the subcategories of “face manipulation” and “A.I.-generated audio detection”, both indicating that A.I. was used to tamper with the visuals and audio track featured in the video under review.
We wanted to get a further analysis on the audio track featured in the video for which we reached out to ElevenLabs, a company specialising in voice A.I. research and deployment. They confirmed to the DAU that the audio is synthetic, implying that it was generated using A.I. They added that the user who broke their “terms of use” while generating the audio has been blocked from their platform.
To get an expert view on the possible tampering using A.I. especially around the mouth area of both Adityanath and the anchor, we reached out to our partners at RIT’s DeFake Project. Saniat Sohrawardi from the project noted that Adityanath’s mouth animation looked disjointed, it was animated or generated with the assumption that his face was looking at the camera. He was referring to certain points in the doctored video when Adityanath’s face is turned to his left side but his mouth moves as if he is looking straight.
Mr. Sohrawardi added that this could be a result of the algorithm failing to extract proper facial landmarks, which are essentially the structural details of the face.
For the anchor’s clip, he further said that despite the bad quality of the video, it was easy to tell that when the audio was playing and loud vowels were mentioned, her face did not open up as wide, for example, when she said “Yogi”.
On the basis of our observations and analyses from experts we can conclude that Adityanath did not utter the words attributed to him, and that a fake news package was created and overlain with synthetic audio to push a false narrative.
(Written by Debopriya Bhattacharya and edited by Pamposh Raina.)
Kindly Note: The manipulated audio/video files that we receive on our tipline are not embedded in our assessment reports because we do not intend to contribute to their virality.
.png)
