Mother and lawyer Emma Mason vividly remembers the phone call she received that changed everything for her family.
Warning: This article discusses suicide and self-harm.
It was a typical day for Ms Mason at her work when she received the call from her daughter, Matilda “Tilly” Rosewarne, at lunchtime in November 2020.
Hearing 14-year-old Tilly sobbing on the phone, Ms Mason said she knew “instantly something was very wrong”.
Her daughter explained that a fake nude image — which depicted Tilly’s likeness — was circulating at school.
Tilly was eight years old when she first started being bullied.
Ms Mason said the bullying Tilly experienced continued into high school, occurring around their hometown of Bathurst in NSW.
The bullying then morphed onto social media. The impact was swift.
The fake nude image that was circulating among Tilly’s peers was an example of deepfake image-based abuse, which is directly affecting young people — predominantly girls.
“Despite the nude image not being of Tilly, she knew that people believed that it was her. In a small town this was catastrophic,” Ms Mason said.
“We came to understand how far the image had been spread amongst students in Bathurst,” she noted in her submission to the Joint Select Committee on Social Media and Australian Society.
For months following, the cyberbullying worsened.
Just after 3am on February 16, 2022, Tilly took her own life. She was just 15 years old.
Ms Mason said: “Sadly, Tilly died from a thousand cuts that occurred over the course of her short life. This was a death from bullying — exacerbated by social media.“
How fake explicit images are impacting kids
Deepfake image-based abuse is where sexual images and/or videos falsely depict a person’s likeness without their consent.
Explicit deepfake abuse often involves people’s faces being superimposed onto pornography. The concern among parents and educators is the prevalence of this happening to children and teenagers.
Teens are being impacted by deepfake explicit image abuse. (Adobe Stock: De Visu)
Year-on-year, the eSafety commissioner has seen a doubling of child sexual abuse material, including reports of synthetic material generated by AI.
This figure includes reports of deepfakes being used by teenagers to target their female classmates.
Rebecca Johnson is an AI researcher and PhD candidate within the Faculty of Science at the University of Sydney.
“People have been making fakes for all sorts of nefarious reasons, particularly targeting women and girls, for a lot of years,” she said.
What’s different now is the accessibility to these technologies that make creating this abuse material far easier.
“This is not a problem that can just be solved by technologists or big tech companies,” she said.
“We know from AI lessons in the past, that we need a wide and diverse range of people in addressing these kinds of problems.
“We’re heading into an era where it becomes evermore difficult to tell truth from fiction.”
In an effort to balance the potentially positive and negative effects of social media on youth, the New South Wales and South Australian governments hosted a Social Media Summit earlier this week.
Ms Mason was in attendance during day one of the summit, and spoke as part of the event’s second day line-up alongside a fellow lived-experience parental advocate, Wayne Holdsworth, whose 17-year-old son died by suicide after he became a victim of sextortion.
Both spoke powerfully about their families’ losses, describing it as “soul and heart destroying”.
Reflecting on day one of the summit, which had minimal lived experience perspectives, Ms Mason described it as “a bit tick-of-the-boxing”.
“This shouldn’t be a political hot potato.
“We all have the research and knowledge and know the impact of social media, let’s sort something. When Tilly died two years ago there was, ‘Let’s do a summit.’ Nothing actually happened with it. I say that as an outsider, I’m not in the halls of power,” she said.
“That’s how it feels.”
Ms Mason is among many Australian parents who agree with the federal government’s push to limit children’s access to social media.
The concern among parents and national leaders
For eSafety Commissioner Julie Inman Grant, she said the evolving threat landscape that youth are facing online shouldn’t be ignored.
“The hyper-realism of high sensory immersive environments such as the metaverse means an assault there can happen in real time and lodge in a child’s hippocampus, with all the traumatic physiological impacts that occur in the real world,” she said.
eSafety Commissioner Julie Inman Grant says she is focused on stamping out image-based abuse. (Four Corners: Keana Naughton)
The commissioner added that in the past financial year, eSafety has received more than 7,000 reports of image-based abuse.
“We are already seeing ‘nudifying apps’ and powerful deepfakes being weaponised against young girls.”
NSW Premier Chris Minns and SA Premier Peter Malinauskas spoke of their plans to back an age limit for children accessing social media, though neither disclosed what the age-cut off would be.
Youth mental health organisation ReachOut has warned that removing social media access to teenagers could be potentially harmful.
Its research found that many young people routinely relied on these platforms for mental health support.
Days ago, the NSW government released the results of a survey they conducted among 19,185 parents, which showed overwhelming support for social media age restrictions.
For Ms Mason, she hopes no other child experiences what her daughter Tilly did.
“I think it’s hard to find a parent whose had a lived experience that says ‘No, we should keep this harm going.’
“It is about education, but that’s not the only lever you can pull.”
