“The illusion of friendship: Why generative AI demands unprecedented ethical vigilance”

Professor Zahid Islam FRSN
Professor in Computer Science and
Associate Dean (Research)
Faculty of Business, Justice, and Behavioural Sciences
Charles Sturt University

Date: Wednesday, 25 March 2026, 12.30–1.30 pm AEDT
Venue:  Live streaming
Registration: Please register through Humanitix
Entry: No charge
All are welcome

This meeting is a joint presentation of Charles Sturt University and the Western NSW Branch of the Royal Society of NSW

Please click here to register

Summary: Generative AI (GenAI) systems such as ChatGPT are increasingly used for drafting, summarisation, tutoring, and decision support, offering substantial gains in productivity and reduced cognitive load. However, the same natural-language fluency that makes these systems useful can also blur the boundary between tool and companion. This boundary confusion may also encourage some users to experience GenAI as empathic, benevolent, and relationally persistent.

Emerging reports and early findings suggest that some users may form emotionally significant attachments to conversational agents, in some cases with harmful consequences, including delayed help-seeking, dependency, and impaired judgment in high-stakes contexts.

In this lecture, Professor Zahis Islam develops a philosophical and ethical argument for why the resulting “illusion of friendship” is both understandable and ethically risky. Drawing on classical accounts of friendship (including Epicurean, Confucian, and Aristotelian perspectives), the presentation explains why users may understandably interpret sustained supportive interaction as friend-like and companionship-like.

The lecture then advances a counterargument that, despite relational appearances, GenAI lacks moral agency, i.e., consciousness, intention, and accountability, and therefore does not qualify as a moral agent or a true friend. To demystify the illusion, the talk presents for an everyday audience a mechanism-level explanation of how transformer-based GenAI generates responses via tokenisation, embeddings, self-attention, and probabilistic next-token prediction, often producing emotionally resonant language without inner states or commitments.

The lecture concludes by outlining a safeguard framework for safe and responsible GenAI use, education as a foundational defence, human-in-the-loop accountability, and design-level interventions to reduce possible anthropomorphic cues generated by the GenAI systems. The central contribution is to demystify the illusion of friendship and explain the computational background so that we can shift the emotional attachment with GenAI towards necessary human responsibility and thereby understand how institutions, designers, and users can preserve GenAI’s benefits while mitigating over-reliance and emotional misattribution.

Md Zahidul Islam (commonly known as Zahid Islam) is a Professor of Computer Science at Charles Sturt University. He is the Centre Director of the AI and Cyber Futures Centre and serves as Associate Dean (Research) in the Faculty of Business, Justice and Behavioural Sciences.

Professor Islam’s research spans data mining, privacy and cybersecurity, with a strong focus on applying advanced data analytics to real-world problems across industry and society. His work contributes to the responsible development and deployment of emerging digital technologies, particularly in high-impact and high-risk contexts.

Share link:

Share link: