The Deepfakes Analysis Unit (DAU) analysed a video that features Gautam Adani, billionaire businessman, founder and chairperson of the Adani Group, apparently promoting a financial scheme. After putting the video through A.I. detection tools and getting our expert partners to weigh in, we were able to conclude that the video was fabricated with synthetic audio.
The four-minute-and-11-second video in English, embedded in a Facebook post, was sent to the DAU for verification by a fact-checking partner. It opens with visuals of Mr. Adani in a studio-like setting with artwork visible in the backdrop. The male voice accompanying his visuals makes a strong case for signing up for the supposed financial scheme for a once-in-a-lifetime chance of making enormous wealth.
The voice mentions that the scheme is driven by a software that uses A.I. for financial analysis and that unlike other schemes this one is not a financial scam. There is a noticeable sense of urgency in the voice, imploring the audiences to trust the program and take the leap by clicking on a certain link, which is not visible anywhere in the video though.
The male voice introduces itself as Gautam Adani initially but later refers to itself with the name of a former prime minister of India. The words heard in the audio track do not match the lip movements. The lips of the subject seem unnaturally shaky and continue to quiver even as the mouth is shut. The shape of the subject’s chin appears to be changing. An odd black spot can be seen in the video after it plays for about nine seconds.
The speech heard in the audio track sounds scripted without much change in tone or pitch. On comparing the audio track with Adani’s natural voice and accent, as heard in his publicly available interviews and speeches, there is similarity in the voice quality but the accent is different. However the long pauses, characteristic of his style of delivery, are missing from the video.
A reverse image search using screenshots of Adani from the video led us to this video published on Feb. 2, 2023, from the official YouTube channel of Adani Group.
The clothing, backdrop, and body language of Adani are identical in the original video as well as the one we analysed. He speaks in English in both the videos but the content of the audio is completely different. There is no mention of any A.I.-powered financial scheme in the original video nor does it have any captions.
To discern if A.I. was used to manipulate the visual and audio elements in the video under review, we put it through A.I. detection tools.
The voice detection tool of Hiya, a company that specialises in artificial intelligence solutions for voice safety returned results which indicated that there is a very high probability that an A.I.-generated audio track was used in the video.
Hive AI’s deepfake video detection tool indicated that the video was manipulated using A.I. and pointed out markers on the face of Adani throughout the video. Their audio tool also indicated A.I. tampering throughout the audio except for the last 10-second segment.
The deepfake detector of our partner TrueMedia suggested substantial evidence of manipulation in the video. The “AI-generated insights” offered by the tool provided additional contextual analysis by stating that the audio transcript resembles a classic scam pitch filled with exaggerated promises of wealth and urgency tactics.
The tool gave a 100 percent confidence score to “A.I.-generated audio detector” and 99 percent confidence score to “audio authenticity detector”, both subcategories indicate that the audio was generated using A.I. The tool also gave a 93 percent confidence score to the “audio foundation model detector” subcategory, which detects A.l.-generated audio using a foundation audio model that has been trained on a vast amount of data such as speech, music, and environmental sounds.
The tool gave a 96 percent confidence score to “video facial analysis”, a subcategory which analyses the video frames for unusual patterns and discrepancies in facial features.
For a further analysis on the audio, we also put it through the A.I. speech classifier of ElevenLabs, a company specialising in voice A.I. research and deployment. It returned results indicating that it was highly likely that the audio track featured in the video was generated using their software.
We reached out to ElevenLabs for a comment on the analysis. They told us that they were able to confirm that the audio is synthetic, implying that it was generated using A.I. They added that the user who broke their “terms of use” by generating the audio was subsequently blocked from their platform.
To get another expert to weigh in on the audio featured in the video, we escalated it to IdentifAI, a San-Francisco based deepfake security startup. They used their audio detection software to analyse the audio.
First they took two real voice samples of Adani to generate an audio profile of his which served as a representation of his real voice. Then they retrieved the audio from the video escalated to them, and used a heat-map analysis to compare the generated audio profile and the retrieved audio.
The image on the left displays a comparison between his real voice and the audio profile of his generated by our partner. The image on the right represents the comparison of the voice sample from the manipulated video with the generated audio profile. The patterns on both images are not identical, indicating that A.I. was used to generate the speech that is being attributed to Adani.
Based on the heat-map analysis and iterative testing, the team at IdentifAI were able to establish that the audio featured in the video is not Adani’s voice, and that it is an attempted deepfake.
To get an expert analysis on the video, we escalated it to the Global Deepfake Detection System (GODDS), a detection system set up by Northwestern University’s Security & AI Lab (NSAIL). They used a combination of 22 deepfake detection algorithms and analyses from two human analysts trained to detect deepfakes, to review the video escalated by the DAU.
Of the 22 predictive models used to analyse the video, 12 models gave a higher probability of the video being fake, while the remaining 10 models indicated a lower probability of the video being fake. They also tested the media with deepfake detectors specifically trained for audio and noted that 13 out of the 20 predictive models returned results with a high probability that the audio track is likely to be fake.
The team noted in their report that the subject’s jaw and mouth seem to move independently from the rest of the face, seeming unnatural and not in synchrony with the subject’s overall facial appearance. They also pointed to the appearance of a black dot over the subject’s mouth, which possibly indicates digital manipulation on the subject’s face.
The team also highlighted oddities and inconsistencies in the subject's speech and mouth movements, which corroborated our own observations of the video. In the overall verdict, the team concluded that the video is likely to be fake and created using artificial intelligence.
To get another expert to weigh in on the manipulations in the video, we reached out to our partners at RIT’s DeFake project. Akib Shahriyar and Kelly Wu from the project stated that the video contains several artefacts which suggest the use of a lip-sync deepfake generation model.
The team pointed to a moment where Adani’s mouth region abruptly went blank. According to them, this suggested a failure of the lip-sync model to generate realistic mouth movements, which resulted in a black, empty mask in place of his mouth.
They added that his head remains static toward the end of the video, while his mouth movements become unnaturally prolonged without a corresponding change in tone. They also pointed to the appearance of a white blob that resembles teeth but looks highly unnatural.
They highlighted another anomaly in the video, which is that at various points in the video while his lips move the teeth are not visible at all. At a few instances even as Adani’s mouth appears to be wide open his teeth are not visible; the team pointed to this as the model’s limitation in producing realistic teeth in sync with mouth movements.
They remarked that beyond the visible artefacts, there are numerous instances throughout the video where the audio and lip movements do not sync properly. They also noted that the area around Adani’s mouth, especially his chin, exhibits unnatural quivering, and his lips change shape inconsistently at various points.
On the basis of our observations and expert analyses, we assessed that original footage featuring Adani was used with an A.I.-generated track to fabricate the video. It is yet another attempt to associate a prominent business person with a dubious financial scheme to scam the public.
(Written by Debraj Sarkar and Debopriya Bhattacharya, edited by Pamposh Raina.)
Kindly Note: The manipulated audio/video files that we receive on our tipline are not embedded in our assessment reports because we do not intend to contribute to their virality.
You can read below the fact-checks related to this piece published by our partners:
Scammers At It Again! Viral Video Of Gautam Adani Promoting Investment Platform Is Deepfake
Video Of Gautam Adani Promoting Investment Opportunity Is An AI Edited Fake
