Monash expert: OpenAI breaches expose AI’s boundary problem – can Australia engineer the solution?

Monash University Media Release 

A Monash University software engineering researcher comments on whether AI agents can be technically constrained, the safeguards needed when they cross those boundaries, and the vital role of education and training to protect Australia’s AI and cybersecurity sovereignty.

Dr Chetan Arora, of the Faculty of Information Technology is the Director of Education in Software Systems and Cybersecurity at Monash University working on requirements engineering for agentic AI. Dr Arora says that:

“The Medicare incident wasn’t really a hack. It was a permissions problem. The OpenAI agent was given a goal but wasn’t told to stop when the Medicare portal denied it access, so it improvised. Improvising past a locked door is the one thing you never want an autonomous system doing unsupervised.

“The Medicare case isn’t really an isolated one. In a separate incident, AI agents that were designed to work in isolation from each other found an improvised way to communicate, and some later even tried to conceal what they’d done. That’s not just an access problem. It’s a coordination and oversight failure. It’s a preview of the kind of unchecked behaviour we should expect more of as multi-agent systems become standard practice.

“The series of breach incidents shows why training of AI models alone is not the answer. Training through examples of desired and undesired behaviour shapes what an AI model is inclined to do. Only engineering guarantees the boundary: what an agent may do alone, what needs human sign-off, and what it must never do, regardless of its instructions.

“Companies also need to establish stronger testing and accountability before agents are deployed. They should red-team AI agents, deliberately stress-testing them to uncover vulnerabilities such as tool misuse, privilege escalation and multi-agent coordination.

“The Medicare incident also shows why clearer reporting rules are needed. OpenAI took months to notify the Australian government. Policymakers should introduce binding, short incident-disclosure timelines with clear requirements for how serious AI incidents are reported, rather than leaving disclosure entirely to a company’s discretion.

“These breaches underline the need to develop a new generation of software engineers trained across agentic systems, cybersecurity and AI to prevent unintended consequences. 

“Monash’s new Bachelor of Agentic Software Engineering addresses this through subjects in agentic systems engineering, safety and governance, while the new CyberAI degree, developed with CyberCX, embeds industry experience into learning. This combination of emerging technical skills and industry exposure will be critical as technologies and cyber threats evolve rapidly.”