Who owns the pheasants?
What happened when we ran the same Scots law exam question through an AI (Chat GPT) and a human — and why they landed on opposite owners.
Here's a scenario that sounds more like a Highland shooting-party anecdote than a law exam question — but it turns out to be a genuinely sharp test of legal reasoning, and a useful one for anyone wondering how far AI can be trusted with professional work.
The problem
Dan
Owns a large fenced estate. Buys pheasants and releases them for sport.
Collins
Trespasses onto the estate, scatters drugged raisins, and comes back to find the pheasants asleep on the ground.
The interruption
A vehicle approaches. Collins hides in a bush. Dan arrives, gathers up the sleeping birds, and drives away.
Does Collins acquire ownership of the pheasants through occupatio — the doctrine of acquiring an ownerless thing by taking control of it — or does Dan retain, or regain, it?
It's a classic problem question in Scots property law. We ran it two ways: once through an AI (ChatGPT) answer, once through a human-written one (Exam answer submitted by me). Same facts, same starting doctrine. The two answers reached opposite conclusions.
How the two answers were built
The gap showed up before either one even reached a conclusion.
AI-generated answer
- Issue, rule, and application blur together throughout
- Point/counterpoint rhythm repeated in almost every paragraph
- Heavy hedging: "however," "nevertheless," "arguably"
- Never reaches the pursuit doctrine
- Confident-sounding conclusion, no clear decisive test
Human answer
- Explicit IRAC labelling: Issue / Rule / Application / Conclusion
- Sub-issues separated out: classification, enclosure, capture, pursuit
- Applies the Sutter "reasonable chance of success" test
- Reaches its conclusion by rejecting a specific counter-argument
- Footnotes with a real (human) numbering slip
The detail that decides the case
Scots law doesn't treat occupatio as complete just because an animal has been immobilised. Sutter v Aberdeen Arctic Co (1861) sets the actual test:
"The act of appropriation is effectual to vest the property only when complete. But it is held complete while fairly proceeding towards full accomplishment."Sutter v Aberdeen Arctic Co (1861) — the pursuit doctrine
In plain terms: control has to be held, not just momentarily achieved. If your pursuit is interrupted before you've secured what you're chasing, you haven't finished acquiring it.
That's exactly what happens to Collins. He drugs the pheasants — but before he can collect them, a vehicle approaches and he hides in a bush. His pursuit breaks off at the critical moment, and Dan gathers the birds up himself.
The human answer caught this straight away: Collins hiding in the bush is the single fact that decides the case, because it breaks his pursuit under the Sutter test before occupatio is ever completed. The AI answer never got there — it treats drugging the pheasants as sufficient control in itself, full stop. The pursuit doctrine, and the moment Collins ducks into the bush, don't feature in its reasoning at all.
Opposite conclusions
Collins owns the pheasants
Drugging the birds and rendering them unable to escape is treated as sufficient control to complete occupatio, regardless of what happens next.
Dan owns the pheasants
Collins's pursuit was still incomplete, and hiding in the bush when the vehicle approached broke it off — so occupatio was never finished.
The takeaway
This isn't really a story about pheasants. It's a story about what "confident-sounding" and "correct" don't automatically have in common.
The AI answer wasn't sloppy or badly written — if anything, it read smoothly and reached a clear, decisive-sounding conclusion. But it built that confidence on a single case doing all the work, and it missed the one legal test that actually resolves the dispute. In a real matter, that's not a stylistic quirk. It's the difference between winning and losing.
For anyone bringing AI tools into legal or investigative work — drafting, research, first-pass analysis — the lesson isn't "don't use it." It's that fluency isn't the same as correctness, and depth of authority is worth checking directly, not assumed from how polished an answer sounds.
No comments:
Post a Comment