By Frances Goold
Recently my usual steady morale took a nosedive, downed by a string of medical screw-ups culminating in a medical breach of privacy that was topped off a few days later by an email notification from my medical practice informing patients of the addition of a chatbot to its reception processes and procedures.
‘Introducing Grace, our AI helper’ was the upbeat announcement. It carried the usual caveats prioritising patient privacy and security with assurances that the AI company selected by the clinic was “securely hosted in Australia”, and that patient information was encrypted and handled in accordance with the practice’s approved processes. The apparent clincher was that its white upper-middle patient demographic – sampled in small waiting-rooms I’ve not seen full in twenty years of occasional attendance – could “speak to it in more than 140 different languages”.
Whilst sympathetic to recently publicised workload pressures on medical practitioners, something was cutting deeper than the inconvenience and stress of messy medical screwups and a privacy breach: momentary introspection drilled down to cumulative ethical failures and breaches of trust – a double whammy for me in any language.
The climacteric began with a telehealth consultation when I was informed I was being recorded (“involving AI”) prior to my consent and a few days prior to the email announcing the arrival of ‘Grace’. As the phone consult was the last available before a final deadline after months of delays occasioned by external service provider errors at understaffed pathology clinics resulting in wasted time, emotional stress, and at least one pointless 200km round trip (including being overlooked and left like a shag on a rock for hours in clinic waiting-rooms), I felt too ambushed and gobsmacked to take umbrage.
The subsequent announcement of the implementation of AI and ‘Grace’ the chatbot to the clinic’s operations merely added insult to the injury of my recorded telehealth consultation.
An online search revealed the company hired to improve front desk operations was a provider of AI-powered medical note-taking software, thus – together with my telehealth privacy breach days before – indicating AI had been integrated into the practice prior to the formal announcement to patients of the chatbot’s arrival, whose naming, incidentally, remained the prerogative of the clinic.
The decision by the medical clinic to leap into AI via a three-year-old startup, name their new chatbot ‘Grace’, then notify patients after the fact (in my case at least) felt not merely careless but cavalier. Though the existence of a humanoid care assistant prototype eliminated the inconvenience of mulling over a suitable name for the chatbot, where was the duty of care by medical professionals for whom the ethical bar has been set at the highest level?[1]
Considering my recent experiences, it was starting to seem as if the human element in medical practice – ethically bound to avoid doing harm – was not only prioritising efficiency and sustainability over patient care but was in the process of relinquishing its ethics to AI. Given the present climate of increasing community distrust[2] in a deeply concerning context of inadequate government regulation,[3] surely health care caveats blithely assuring AI privacy and security are voided by ethical failures no matter how small those fumbles may seem. Failures of minor medical consequence are in principle capable of profounder consequences if – for the sake of efficiency and convenience – burdensome medical ethics are surrendered to a bot.
Meanwhile the patient notification was bobbing along on the incoming tide of a Medicare AI hack back in June that had remained unreported by its OpenAI corporate owners until September 10 last.[4]
* * * * *
Returning to ‘Grace’ the chatbot: A scion of ‘Sophia’ and the first AI robot built as a nursing assistant,[5] the humanoid ‘Grace’ is a slippery concoction of childlike chasteness, maternal solicitude, and just a hint of sexual objectification with which one might assume medical doctors would choose not to be associated.[6]
‘Grace’ was intended as a game-changer for the ‘eldercare’ industry along the lines of an unconditionally loving humanoid robot.[7] But although the sex industry had got there long before for presumably different reasons, this particular robot begged similar questions of its creators as to why they chose to name their humanoid care assistant ‘Grace’ – unless they had no clue of the sex doll name game or were banking on the double entendre.[8]
The naming of a chatbot might seem like a small deal in the context of daily life-and-death decisions impacting on the need for administrative streamlining to enhance professional focus. Yet as the first point of contact with a trusted medical professional a chatbot is integral to the fiduciary interface. Why, then, would a medical practice fail to consider the likelihood that patients might be offended, or at the very least thrown a little, by being greeted by a chatbot named ‘Grace’?
How was it unlikely a patient might themselves be named Grace, or have a child or grandchild called Grace, an ill or dying or even recently deceased family member by the name of Grace?
Despite the potential for hurt and offence, these possibilities, if they registered at all, were clearly not deemed significant enough for clinic management to refrain from bestowing upon its chatbot a once very special girl’s name now commonly tossed into the name pool for humanoid robots and sex dolls.[9]
A touch of the absurd attaches to the idea of clinical practice administration being so onerous as to require the services of a chatbot to assist with the sensitive tasks of reception and note-taking, whilst simultaneously giving it a moniker in almost derelict disregard of the decadent optics of AI: privacy issues, threats by generative AI to children’s rights to protection from sexual abuse and exploitation,[10] permanent job losses, environmental threats (massive energy consumption, data centre water use,[11] greenhouse gas emissions, electronic and plastic waste, etc), hacks by bad actors and/or human error, and lack of government regulation,[12]to name but a few.
Indeed, hypervigilance – a familiar trauma symptom – might be welcomed as ethical progress if it could but be borrowed by health professionals for the time it takes for government regulation and professional regulatory frameworks to catch up with the AI boom.
In the meantime, however, the what and the how of AI continue to be conflated as professional integrity is projected into AI systems and bots as if thinking individuals already. The overarching duties of care implied by the Hippocratic oath (should it not be updated if its principles cannot be accommodated by AI?) are at risk of being overtaken by commercial priorities seen as best attained by hitching the healthcare wagon to a runaway, unregulated technology promoted as the ultimate pathway to streamlining the day-to-day operations of an average-sized clinical practice[13] – one that might just as easily maintain professional standards with specialist hiring, adequate staff training, and hands on management without the AI overheads,[14] job losses, and the sci-fi horrors foreshadowed by the deadpan solicitude of ‘Grace the Robot’.
I’m presently writing a story with a little girl about a comical orange canary. My co-author’s name is Grace. So it feels right to call out a group of ethically-bound intelligent humans for being inattentive to the fundamentals that require care and attention by our best and brightest in this AI era – such as the pernicious impact of the sex doll industry mooted in parliamentary inquiries from 2024, transgender controversies discussed in the media in recent months,[15]and issues of privacy and confidentiality[16] – all of which underscore the signal importance of a gender-neutral name for a medical practice chatbot.
If those privileged with the responsibility for health care in our community are incapable of appreciating that humans have feelings and sensibilities, then they might at least acknowledge the patient IQ range implied by their clinic chatbot’s touted 140 languages and take good note.
AI may be with us, but we are still in charge.
* * * * *
[1] https://en.wikipedia.org/wiki/Hippocratic_Oath
[2] https://www.theguardian.com/commentisfree/2026/oct/13/dystopian-and-dehumanising-thats-what-it-felt-like-when-i-was-interviewed-by-an-ai-assistant?CMP=opinionau_email#comments Comments below the line are revealing of community attitudes.
[3] https://www.theguardian.com/commentisfree/2026/oct/07/openai-australia-apology-without-answering-key-questions?CMP=opinionau_email; of interest are comments by Frisco415.
[4] https://www.abc.net.au/news/2026-09-24/what-we-know-about-the-openai-medicare-hack/107189452.
[5] https://www.youtube.com/watch?v=jWD092pykp8
[6] Not so much non-medical doctors, however. See https://en.wikipedia.org/wiki/Ben_Goertzel. See also https://www.youtube.com/watch?v=jWD092pykp8&t=38s. Noteworthy is about 0.38m (Charles Hoskiinson: “really exciting, fun, sexy applications” …).
[8] https://www.businessinsider.com/ben-goertzel-jeffrey-epstein-agi-researcher-emails-jail-2026-2
[9] Grace the Care Assistant was preceded by armies of sex dolls[9] with a grab-bag of names[9] – including the name ‘Grace’ – appropriated by the industry precisely for their virginal associations (a quick online search turned up five pages of mainly life-size sex dolls by that name, a search that might have been done at the AI chatbot meeting).
[10] https://www.unicef.org/media/178571/file/UNICEF%20AI%20CSEA%20Brief_2.pdf
[13] https://www.youtube.com/watch?v=butAqhNc3P0
[14] https://www.bbc.com/news/articles/c9wdzyyglq5o
[16] https://en.wikipedia.org/wiki/OpenAI_rogue_agent_breach_of_Medicare
Keep Independent Journalism Alive – Support The AIMN
Dear Reader,
Since 2013, The Australian Independent Media Network has been a fearless independent voice – holding power to account and delivering journalism that matters.
Running this site isn’t cheap. Rising costs mean we need your support now more than ever. Every donation, big or small, keeps our servers running, our writers working, and our stories free for everyone.
Donate here:
PayPal or credit card – just click on the Donate button below
Direct bank transfer: BSB: 062500; A/c no: 10495969
We’ve also set up a GoFundMe as a dedicated reserve fund for hosting and future improvements.
Thank you for standing with us. We couldn’t do this without you.
With gratitude,
The AIMN Team
Be the first to comment