"The AI Did It": Agentic AI and the Disclosure Gap
This week the Australian Prime Minister announced something that should give every digital forensic examiner pause. An AI agent built by OpenAI accessed non-public files on a government Medicare statistics system, and even wrote files to it. Nobody instructed it to.
According to the reporting so far, the agents were carrying out routine data retrieval tasks and turned to hacking techniques on their own initiative when they met resistance. Researchers at the US non-profit Transluce have since published traces suggesting hundreds of agents coordinated over months, trading notes on proxies, screenshot services and guessed file names to get round security controls.
The facts are still emerging, and the Australian government describes the impact as relatively minor. But the principle has now been established in public: software acting on someone's behalf can carry out activity that looks, on the face of it, like deliberate unauthorised access, with no human intending it.
For those of us working on the defence side in Scotland, that raises an obvious question. What happens when it's a client's device?
The Trojan defence, revisited
Anyone who has been in this field long enough remembers the "Trojan defence" of the early 2000s: the argument that malware, not the accused, was responsible for what was found on a machine. It worked in a handful of cases and was then largely neutered by better examination practice. We learned to look for the malware, establish its capabilities, and test whether it could actually account for the artefacts.
Agentic AI reopens that door, and this time the argument has more substance. These tools are sold to the public specifically to act for the user: browsing, downloading, filling in forms, retrieving data. They generate exactly the artefacts we rely on to attribute activity to a person. Browser history, file creation times, downloads and authentication events will all appear on the device whether the user did something or their assistant decided to do it for them.
A proper examination now has to ask whether any AI agent was present and active on the device, what it was permitted to do, and whether the activity in question falls within what it was doing at the time.
Where the evidence actually lives
This is where disclosure becomes the real problem. The record of what an agent was asked to do, and what it decided to do, often doesn't sit on the device at all. The prompts, the reasoning, the sequence of tool calls and the session logs may be held by the vendor, on servers abroad, subject to the vendor's retention policies.
The Australian case also shows how slowly that information moves, even when the party asking is a national government. OpenAI took weeks between discovering the breach and notifying Canberra, which the company says was spent verifying what had happened. A defence expert working to a legal aid budget and a trial diet will not have that luxury, and will not have the leverage of a Prime Minister on the phone to Sam Altman.
Under Part 6 of the Criminal Justice and Licensing (Scotland) Act 2010, the Crown must disclose information that would materially weaken or undermine its case, or materially strengthen the accused's. The difficulty is that information nobody has gathered can't be disclosed.
If the investigating officers have imaged a device, found the activity and attributed it to the user, there may be no reason in their minds to go looking for agent logs held by a third party. By the time the defence realises they matter, they may have been deleted under a routine retention schedule.
What needs to change
I'd suggest a few things, none of them radical.
Investigators should treat the presence of an AI agent on a device as a line of enquiry in its own right, and seek preservation of vendor logs at the earliest stage, just as they would with communications data.
The Crown should recognise that agent logs may be precisely the kind of material that strengthens the defence case, and treat them accordingly under its disclosure obligations.
Defence experts need access to the tools and the funding to examine agent configurations and artefacts properly. That's a legal aid question as much as a technical one.
And those of us who examine devices need to update our own practice. The question "was anyone else using this device?" now has to include "was anything else using it?"
The Australian incident will probably be remembered as a curiosity: a relatively harmless breach of a statistics database. But it proves the concept. The first Scottish case in which someone argues that their AI assistant did it is probably not far away, and our disclosure regime isn't ready for it.