The Deepfakes Analysis Unit (DAU) analysed a video that apparently shows Dr. Devi Prasad Shetty, a leading Indian cardiac surgeon, promoting a cure for joint pain. After putting the video through A.I. detection tools and getting our expert partners to weigh in, we were able to conclude that synthetic audio was used with unrelated clips to fabricate the video.
The video in Hindi, embedded in a Facebook post, was escalated to the DAU by a fact-checking partner for analysis. The original length of the video was 10 minutes of which only three-minutes-and-29-seconds had video content, the rest of the video duration carried a static floral image.
The video seems to have been recorded in an office setting since Dr. Shetty is seen in scrubs and a surgical cap. The medium close-up in the video makes only a part of his chair and the backdrop visible. A male voice recorded over his video track claims that a “homemade remedy of baking soda and turmeric powder can cure bone-related ailments within five days”; and the bone health of an individual can be restored to what it was like in their youth.
The same male voice also mentions that bone disorders are caused by the accumulation of harmful crystals in the joints and not because of ageing or genetics; this supposed discovery is credited to some research by an organisation referred to as “Akhil Bharatiya Chikitsa Parishad”.
The lip movements of Shetty mostly appear to be in sync with the accompanying audio track. However, his lower lip seems to quiver in some frames and the dentition looks warped. The teeth change shape when the subject is apparently talking, in some frames only a white patch is visible in place of the teeth.
The voice and accent in the audio track sound somewhat like Shetty’s when compared with his recorded interviews available online. But the pitch is different and the overall delivery is monotonous, lacking natural pauses.
The Hindi heard in the audio track is not conversational. There is heavy usage of medical jargon which is difficult to understand. The script of the audio flashes across the bottom of the video frame throughout. There is a glitch in the audio when the narration seems to mention some statistic regarding the number of people cured by the purported remedy. The audio script reads “165 Indians” but the voice gives out a different number, which is difficult to discern.
Shetty has been falsely linked to health misinformation in the past as well. He went to court last year to seek protection against the misuse of his registered trademark and personality rights. The DAU debunked one such video in December which had been produced by combining an A.I.-generated audio track with original footage featuring him. The video claimed that a supposed breakthrough cure for diabetes has been discovered which is superior to conventional allopathic treatment.
In the video that we are addressing through this report, short clips of Shetty are interspersed with unrelated clips, including those of medical professionals in laboratory settings, an elderly couple hiking, close-ups of people clutching their body in pain; and graphics of cellular molecules and bone anatomy. Most of those people appear to be westerners but for one elderly woman who looks Indian; text graphics show her apparent name and age.
The female voice recorded over the woman’s visuals mentions the benefits of the purported remedy and how it has transformed her life. However, the sentences in that audio sound grammatically incorrect as they use the male verb form for her several times.
The mouth area of the woman is blurred, her lip movements are puppet-like and are not in sync with the audio. Her teeth are barely visible with some frames showing a brown patch for the teeth.
A static floral image appears for a flash of a second at various instances in the video; it’s the same image that surfaces at the end of the longer version of the video. We have noticed similar random images inserted at frequent intervals in many A.I. manipulated videos that we have debunked previously.
Toward the end of the video, the male voice accompanying Shetty's video track urges viewers to click on a video link to get details about the recipe of the purported cure. An urgency is created in the tone by suggesting that the video will be soon taken down by pharmaceutical companies. The last few seconds of the video feature a clip of Shetty's with the audio track claiming that the supposed cure will supersede commonly used pain management drugs Diclofenac and Tramadol in 2025.
The video or the link for it was not clearly visible anywhere in the video frame. However, we found a link buried below the video on Facebook and clicked on it for the purpose of this report. We were led to a dubious website listing out a diet plan that had nothing to do with joint pain. The site lacked much information or content and was poorly designed. It displayed a form where one could fill in their contact information; we refrained from doing so. (We would like to caution our readers against clicking on suspicious links.)
We undertook a reverse image search using screenshots from the video and established that unrelated clips were stitched together to produce this manipulated video.
Shetty’s segments were traced to this video published on May 17, 2023 from the official YouTube channel of Narayana Health, a private hospital network in India of which Shetty is the founder and chairman. The woman’s segment was traced to this video published on Dec. 26, 2022 from a YouTube channel called “Let’s Nurture Our Minds”, which is run by her and posts content on alternative medicine.
The clothing as well as the backdrop of Shetty and the woman in their respective clips in the doctored video and the videos we traced are identical. However, the frames for both the speakers are more zoomed-in, in the doctored video. The logos visible in the original videos are missing from the manipulated video.
Shetty speaks in English in the original video while the woman speaks a mix of Hindi and English in the original version. None of them talk about joint pain in their videos or use diagrammatic representation to explain bone health. The video clips of westerners that feature in the manipulated video could not be found in the original videos; only some of those could be traced to websites hosting stock footage.
We also discovered that the identity of the woman is not the same as is mentioned in the doctored video. The voice recorded with her visuals bears similarity to her voice, the accent also sounds natural but the overall delivery sounds scripted and robotic.
To discern the extent of A.I. manipulation in the video under review, we put it through A.I. detection tools.
The voice tool of Hiya, a company that specialises in artificial intelligence solutions for voice safety, indicated that there is a 61 percent probability of the audio track in the video being A.I.-generated.
Hive AI’s deepfake video detection tool indicated that the video was manipulated using A.I. It pointed out markers of manipulation in a small segment featuring the woman. Their audio detection tool indicated that most of the audio track was manipulated using A.I. but for a 30-second segment.
For a further analysis on the audio track from the video we put it through the A.I. speech classifier of ElevenLabs, a company specialising in voice A.I. research and deployment. The classifier returned results as “very unlikely”, indicating that the audio track in the video was not generated using their software.
We also ran the video through Deepfake-o-meter, an open platform developed by Media Forensics Lab (MDFL) at UB for deepfake image, video, and audio detection. The tool gives an option of various classifiers through which a media file, in this case video, can be run to receive analysis.
We chose six audio detectors, out of which five gave strong indicators of A.I. in the audio. AASIST (2021) and RawNet2 (2021) focus on detecting audio impersonations, voice clones, replay attacks, and other types of audio spoofs. Linear Frequency Cepstral Coefficient (LFCC)-Light Convolutional Neural Network (LCNN) model classifies genuine versus synthetic speech to detect audio deepfakes.
RawNet3 (2023) allows for nuanced detection of synthetic audio and RawNet2-Vocoder (2023) is useful in identifying synthesised speech. Whisper (2023) analyses synthetic speech characteristics, primarily focusing on using whisper features in audio samples to improve detection ability.
For expert analysis, we shared the video with our detection partner ConTrailsAI, a Bangalore-based startup with its own A.I. tools for detection of audio and video spoofs.
The team ran the video through audio as well as video detection models, the results that returned indicated high confidence for A.I. manipulation in the video and the audio track.
In their report they added that there were clear signs of the use of the lip-sync method in the video. They also noted that the voice that can be heard in the video is very monotonous, indicating A.I.-generation.
To get another expert to weigh in on the manipulations in the video, we reached out to our partners at RIT’s DeFake Project. Saniat Sohrawardi from the project stated that the video is a lip-sync deepfake and marked out various anomalies in the video. He identified an instance of “lip flap” or “out-of-sync lips” in the video when the lady appears to be speaking. He noted that the lip mismatch with the words is fairly obvious to the naked eye.
Mr. Sohrawardi said that there is a high probability that A.I.-generated video snippets have been used in the video. He made that observation based on the speed of the movements in those visuals and the fact that his team has seen similarly generated snippets being used in videos before. He was making a specific reference to short random clips, such as those of westerners, that can be seen at various points in the video.
Dental warping, one of the many visual artefacts indicative of a lip-sync deepfake, was also highlighted by him in Shetty’s frames. He added that this is a good indicator of the video being a lip-sync video as teeth obviously don’t change shape based on how our mouth opens. He said that there were quite a few instances of dental warping in the video.
Yet another anomaly pointed out by him was the presence of “flash frames”, that appear randomly in the video. He said that referring to the static floral image that keeps reappearing in the video as we mentioned in our analysis above.
He added that his team would want to investigate and assess the intention behind adding those frames, which he believes could be to confuse deepfake detection algorithms that rely on temporal information. He said that in this case, the flash frames would be referred to as “adversarial frames”.
On the basis of our findings and expert analyses we can conclude that original footage was used with synthetic audio to fabricate the video. It is yet another attempt to tout a dubious cure by linking it to a prominent doctor in a bid to peddle health misinformation.
(Written by Debraj Sarkar and Debopriya Bhattacharya, edited by Pamposh Raina.)
Kindly Note: The manipulated video/audio files that we receive on our tipline are not embedded in our assessment reports because we do not intend to contribute to their virality.
You can read below the fact-checks related to this piece published by our partners:
Fact Check: जोड़ो के दर्द का इलाज बताते कॉर्डियक सर्जन डॉ. देवी शेट्टी का वायरल वीडियो डीपफेक है
%20(1).jpg)
