The Deepfakes Analysis Unit (DAU) analysed a video featuring Sundar Pichai, the chief executive of Google and Alphabet, apparently endorsing an investment platform supposedly launched by Google. After putting the video through A.I. detection tools and getting our expert partners to weigh in, we were able to conclude that his voice clone was used to fabricate the video.
The two-minute-and-14-second video in English, embedded in a Facebook post, was sent to the DAU by a fact-checking partner for analysis. The video features a looped clip of Mr. Pichai in which he is talking to the camera in an open area with trees in the backdrop. The male voice recorded over his visuals makes a case for profitable returns, especially for Indian citizens, from an algorithm-based investment platform, being linked to the company he leads.
The lip-sync of Pichai is imperfect. His lips quiver and his chin seems to distort when he opens and closes his mouth while talking; this peculiarity is more visible when the camera is zoomed in on his face. His teeth look blurry and his dentition seems inconsistent. The voice heard in the video somewhat resembles his voice quality, however, it is nasal and lacks inflection, giving it a synthetic quality and making it different from Pichai’s delivery.
After running a reverse image search using screenshots of Pichai from the video, we were able to locate this video published on June 8, 2020, from the official YouTube channel of YouTube Originals, a platform that hosts original films and series. Pichai’s clothing, backdrop, and body language in the original video and the one we reviewed were identical but the audio is totally different.
The logo of YouTube Originals visible at the bottom right in the original video, and the text appearing in parts of the original video are missing from the doctored video. A series of separate clips from the original video seem to have been lifted to create a video of about one minute-and-eight-seconds, which plays in a loop for a little over two minutes in the doctored video.
We carried out a keyword search for “Google Invest”, to see if such an investment platform exists as this supposed platform is mentioned in the manipulated video. We couldn’t find any investment platform by that name.
To discern if A.I. had been used to manipulate the video under review, we put it through A.I. detection tools.
The voice detection tool of Loccus.ai, a company that specialises in artificial intelligence solutions for voice safety returned results which indicated that there is a high probability that an A.I.-generated audio track has been used in the video.
Hive AI’s deepfake video detection tool indicated that the video was manipulated using A.I. and pointed out markers on the face of Pichai in numerous frames throughout the video. Their audio tool also indicated strong A.I. tampering in the audio track of the video.
We also put the video through the deepfake detector of our partner TrueMedia, their analysis suggested substantial evidence of manipulation in the video. In a breakdown of the overall analysis, their tool gave a 100 percent confidence score to the subcategory of “AI-generated audio detection”, 92 percent confidence score to “audio forensics”, and the two separate confidence scores for “audio analysis” stood at 99 percent and 85 percent - all indicating the presence of synthetic audio.
The subcategories highlighting manipulation or generation in the faces, did not give very conclusive results. The tool gave a 72 percent confidence score to the subcategory of “generative convolutional vision transformer” and 51 percent confidence score to “face manipulation” — the former analyses the video frames for unusual patterns in facial features and the latter points to the probability of manipulation in the faces using A.I.
For a further analysis on the audio, we also put it through the A.I. speech classifier of ElevenLabs, a company specialising in voice A.I. research and deployment. It returned results indicating a 98 percent probability of the audio having been generated using their software.
Subsequently, we reached out to ElevenLabs to get a confirmation on the results from their classifier. They told us that they were able to confirm that the audio is synthetic, implying that it was generated using A.I. They added that the user who broke their “terms of use” while generating the audio track had been previously identified as a bad actor through their in-house automated moderation system, and blocked from their platform.
To get another expert to weigh in on the video, we escalated it to the Global Online Deepfake Detection System (GODDS), a detection service set up by Northwestern University’s Security & AI Lab (NSAIL). They used a combination of 22 deepfake detection algorithms and analyses from two human analysts trained to detect deepfakes, to review the video escalated by the DAU.
Of the 22 predictive models used to analyse the video, 13 models gave a higher probability of the video being fake, while the remaining 9 models indicated a lower probability of the video being fake.
In the report, the team noted that the human analysts identified several indicators which suggested that the video may have been created using A.I. technology. The team marked inconsistencies in the subject's face and voice which corroborated our own observations on the video. The team concluded that the video is likely to be fake.
We also sought the expertise of our partners at RIT’s DeFake Project to make sense of the oddities we noticed around the face of the subject featured in this video. Saniat Sohrawardi from the project stated that the video appears to be a lip-sync deepfake.
Mr. Sohrawardi mentioned that the traces on this video appear less apparent due to the overall low quality of the video. He noted that the video has many visual artefacts, and that the audio and video are out-of-sync in some frames.
Sohrawardi noted that the subject’s mouth opening seems to be jumping up unnaturally because of the bad mouth region localisation during the generation process on those specific frames. The artefacts visible in some of the other frames are also because of the same generation process, he added, in those the lower part of the mouth appears to split up.
Based on our observations and expert analyses, we assessed that original footage featuring Pichai has been used with A.I. voice to fabricate the video. It seems like yet another attempt to scam people by peddling false information about an investment platform that does not seem to exist.
(Written by Debopriya Bhattacharya and Debraj Sarkar, edited by Pamposh Raina.)
Kindly Note: The manipulated audio/video files that we receive on our tipline are not embedded in our assessment reports because we do not intend to contribute to their virality.
You can read below the fact-check related to this piece published by our partner:
Viral Video Of Sundar Pichai Endorsing ‘Google Invest’ Is A Deepfake
