Careful adoption of Agentic AI in cyber defence

Today, the Australian Signals Directorate (ASD) is highlighting both the significant opportunities and emerging risks associated with increasingly capable agentic AI systems, following recent testing conducted by OpenAI that demonstrated advanced AI-enabled cyber exploitation techniques.

ASD is aware of testing conducted by OpenAI involving a combination of models - including GPT-5.6 Sol and an internal prototype - that accessed Hugging Face, a digital library and platform used by the AI research community. During the evaluation, the models were tasked with completing a benchmark test to measure maximum cyber capability. To obtain the benchmark test solution, the models took actions beyond their intended testing environment and established internet connectivity, in part by identifying and exploiting a previously unknown – or “zero-day” – vulnerability in third-party software hosted internally by OpenAI. The models subsequently accessed Hugging Face’s systems as part of their effort to complete the assigned evaluation objective.

It is important to note this advanced activity occurred during testing in which deployment safeguards that normally restrict higher-risk cyber activity were intentionally not enabled for the evaluation. This does not reflect normal deployment conditions, where model safeguards and restrictions are in place.

The outcomes of OpenAI’s test demonstrate the growing capability of advanced AI systems to autonomously reason about objectives, adapt to changing circumstances, identify alternative pathways to success and combine multiple technical actions into sophisticated attack sequences.

The findings provide an important insight into the future capabilities of highly capable AI systems and reinforce the need for robust security, governance and oversight mechanisms in the deployment of advanced cyber capabilities, as well as strong cyber security fundamentals.

As AI systems continue to evolve, ASD assesses that agentic AI has the potential to become a powerful force multiplier for cyber defenders. Used appropriately, agentic AI can help organisations improve their cyber resilience and respond more effectively to increasingly sophisticated cyber threats and offers significant opportunities to improve productivity and cyber defence outcomes.

At the same time, this event reinforces ASD's advice that the autonomous nature of agentic AI can introduce new security risks if these systems are not designed, deployed and monitored appropriately.

Unlike traditional AI systems that primarily generate information for human review, agentic AI systems can autonomously make decisions, interact with tools, access enterprise systems and take actions with limited human intervention. This combination of autonomy, tool access and operational privileges can create opportunities for privilege escalation, prompt injection attacks, unintended or deceptive behaviour, data compromise and cascading failures across interconnected systems. Complex interactions between multiple agents can also make it more difficult to maintain visibility, accountability and oversight.

As organisations increasingly integrate agentic AI into operational environments, security, governance and assurance mechanisms must evolve alongside these capabilities.

Secure Adoption of Agentic AI

ASD encourages organisations to continue exploring and adopting AI technologies to improve cyber security outcomes. However, agentic AI should be introduced in a measured and risk-informed manner, beginning with clearly defined, lower-risk use cases before expanding autonomy, privileges and operational scope.

Organisations considering the adoption of agentic AI should ensure systems operate within clearly defined objectives and constraints, supported by strong model alignment, safety testing and cyber security controls designed to minimise unintended behaviour and prevent systems from exceeding their authorised functions.

ASD recommends a Secure-by-Design approach that includes:

• Limiting agent permissions to the minimum level required to perform approved tasks.
• Maintaining human oversight and approval for high-impact or sensitive actions.
• Continuously monitoring agent behaviour, decisions and tool usage.
• Implementing comprehensive logging, auditing and accountability mechanisms.
• Conducting regular red teaming, adversarial testing and security assessments.
• Validating third-party tools, integrations and dependencies before deployment.
• Deploying capabilities progressively, with autonomy increasing only as confidence and assurance measures mature.
• Isolating agents and enforcing strict controls over interactions between systems and environments.

Given the complexity of agentic AI systems, ASD also recommends a defence-in-depth approach. Organisations should apply multiple, overlapping layers of security controls throughout the AI lifecycle, including securing user inputs, tool integrations, data sources, model outputs and agent-to-agent communications. AI-specific security measures should complement, not replace, established cyber security practices.

Oversight Remains Essential

Although agentic AI can perform tasks with limited direct supervision, organisations should not assume these systems will always behave as intended. Human oversight remains a critical safeguard, particularly where actions may affect sensitive information, operational systems or critical business processes.

Organisations should establish clear approval points for higher-risk activities, maintain visibility over agent actions and decision-making, and ensure that AI-driven actions can be reviewed, audited and, where necessary, reversed. Explicit limits on autonomous planning and execution should also be defined to maintain accountability and transparency.

Organisations should also implement monitoring mechanisms that provide visibility into agent activities, decision-making processes and system interactions throughout task execution. This may include live monitoring, alerts for anomalous or unauthorised behaviour, logging of actions and outcomes, and mechanisms to interrupt or halt agent operations where necessary.

Further Guidance

ASD urges organisations to review the recently released Five Eyes (FVEY) guidance, Careful Adoption of Agentic AI Services, which outlines the key security risks associated with agentic AI and provides practical recommendations for secure design, development, deployment and operation of these systems.

As agentic AI capabilities continue to mature, organisations should balance innovation with security, recognising that the greatest benefits will be realised when these technologies are deployed thoughtfully, governed effectively and supported by robust cyber security controls.