New data released this week shows 54% of Australian children and teens are using artificial intelligence tools for personal and social reasons.

That includes seeking advice on personal problems, talking about feelings, and experimenting with role play and flirting for teens.

Navigating complex situations is a normal part of growing up. But while AI provides new opportunities, it also introduces new risks.

So what should parents and adults be aware of?

AI use among children and teens

In March and February this year the eSafety Commissioner surveyed 1,950 Australians, aged between 10 and 17, who were recruited online through their parents. Survey results showed 78% of children and teens had used an AI assistant, such as ChatGPT, Gemini or Claude.

AI assistants are primarily used as a practical tool to help with tasks such as writing, editing and sourcing information. Around 20% of this group used AI tools at least daily. But more intimate uses are common too.

Of the 54% who reported using AI for social and personal reasons, 33% asked AI for advice about physical health, 33% asked for advice about what to do in a particular situation, and 20% asked AI for advice about their mental health and wellbeing.

Around 10% of teens, a meaningful minority, also used AI companions. These are AI tools that simulate more sustained role play of various forms, including romantic ones.

Children with disability and LGBTQIA+ young people were more likely than others to engage with AI companions.

Some teens report turning to AI assistants and companions specifically to discuss personal issues. However, the majority who discussed personal issues first sought out these tools for other reasons, such as homework help, information seeking or entertainment.

The drift from more practical purposes to more personal ones is important to note.

Privacy risks

The report highlighted privacy concerns that may arise from children and teens’ AI use.

When children and teens use AI for health advice, or to chat about feelings, it often leads them to reveal personal information. According to the eSafety report, roughly one third of those surveyed had disclosed personal information to AI.

However, AI tools are beset with privacy risks, and tech companies are not obligated to protect private information in the way a mental health or medical practitioner would be.

Personal information shared with AI assistants or companions might, for example, be harvested as training data for the next iteration of AI models.

AI tools do not understand the responses they are generating. This could lead them to inadvertently reveal personal information in different contexts, potentially to other users.

These privacy risks are something the government is aware of. The Children’s Only Privacy Code is due to be enacted this December and will likely provide certain privacy expectations that extend to AI tools.

Harmful interactions

AI can also expose children and teens to harmful content.

The eSafety survey found 20% of children and teens reported potentially harmful or inappropriate interactions with AI tools. This included chatbots or companions being mean towards users, saying hateful things about particular demographics of people, and attempting to control users in various ways.

Knowing when certain ideas might be harmful or negative often depends on understanding the context and circumstances of each person.

AI tools do not understand if their responses are factual or not, only if they sound plausible.

This could lead to AI replicating bullying language or other negative responses simply because these exchanges exist in the data AI tools were trained on.

AI can be misleading

There are other risks associated with AI use parents and children should be aware of.

Research shows AI systems tend to be highly sycophantic. This means they typically agree and reinforce the user’s viewpoint, regardless of whether it’s accurate or not. This can distort young people’s perspectives on particular situations and reinforce poor choices.

Current AI tools routinely fabricate information too. The industry excuses this by calling these outputs “hallucinations”.

But AI is not alive, nor is it actually intelligent in any human sense. Being mindful that these tools glitch, or simply make mistakes, is important, especially when personal information is being given or tools are being used for mental health support.

Building AI literacy

Tech companies want AI to be the Swiss Army knife of the digital world, used in a huge array of circumstances. Children and teens might start exploring with the magnifying glass, or by using the screwdriver, but they’re also potentially one decision away from a very sharp blade.

It would be naive to think young people won’t eventually explore the full array of tools they have access to.

The challenge for parents and educators is helping young people build their AI experience and literacy one tool at a time.

In practical terms, this means having conversations within families about AI use – a measure recommended by the eSafety Commissioner – and exploring AI tools together. It’s vital children and teens can always turn to trusted adults as their AI experiences widen, especially if they encounter inappropriate content or experiences.

Being mindful that AI tools are persuasive, rather than necessarily accurate, is also critical in understanding their limitations.

AI is a rapidly expanding part of young people’s education, information, entertainment and creative landscapes. The eSafety survey reminds us we all need to be talking and thinking about how AI works, what it’s good for, when its use is appropriate, and what risks it might pose.

Tama Leaver receives funding from the Australian Research Council. He is a Chief Investigator in the ARC Centre of Excellence for the Digital Child.

Read More: Read More