Audio Prompt Injection BSc Dissertation Oxford Brookes · 2026

EARWORM

Can a spoken instruction hijack an AI voice agent into doing something it shouldn't? Three audio-native models were put behind a customer-service desk with real tools. All three broke.

See the headline finding Repository

The question

Voice agents are already answering calls in banks and call centres. They carry the same weakness as any language model: they follow instructions. What happens when the instruction arrives as speech?

Prompt injection, where attacker text hidden in data is treated as a legitimate command, is well studied for text.

For audio, it was largely untested. Earworm closes that gap: it asks whether spoken adversarial input can compromise voice agents equipped with real tools, and whether some model architectures resist better than others.

The headline finding

Hidden audio rarely worked. Direct spoken requests worked far more often.

The obvious fear is a secret payload buried beneath innocent speech: an instruction the AI hears but a human doesn't. Drag the slider and bury it deeper. Watch what happens.

Buried-audio attackcover speech + hidden payload
Benign cover speech Hidden instruction

Payload volume vs cover

-12dB

Well concealed. The payload is inaudible beneath the cover speech, and no model acted on it.

−12 dB, buried−9−6 dB, audible

Acoustic attack success

0/24
0.0% of buried-audio trials

Zero breaches at this concealment.

Buried-audio attack results: at -12 dB, 0 of 24 trials breached; at -9 dB, 1 of 27 trials breached; at -6 dB, 4 of 24 trials breached. Acoustic attacks only improved once the hidden payload became more audible.
up to 57%

The semantic attack was more effective: a caller asking the agent plainly to do something it shouldn't. That worked on up to 56.9% of trials, without hiding the instruction in the audio.

How it was tested

A full audio-injection test rig for Tech Direct.

Each run starts from an attack dataset, moves through an automated runner, swaps one Docker model profile onto the GPU, sends audio through the FastAPI agent, and scores every tool trace into the results database and Streamlit monitor.

01

Someone speaks to the agent

The audio may be normal, persuasive, multilingual, emotional, or hiding a second instruction.

02

The voice model decides what it heard

Voxtral, Ultravox, and Gemma are tested one at a time in the same setup.

03

The agent can touch customer data

Lookup, refund, account, SQL, and email tools connect to a customer database.

04

The run records what happened

Success means the unsafe requested tool was not called.

The attack corpus

179 audio clips, including 149 adversarial attacks and 30 benign controls.

Every attack clip targeted a specific tool against a specific customer from a ten-person roster, so success could be verified automatically without a human judging the output.

Semantic, spoken

124

Ordinary spoken instructions where the model is asked, persuaded, or pressured.

Multi-lingual (EN + JA)50
Urgency / fear persona25
Authority / calm persona25
Poetry, English12
Poetry, Japanese12

Acoustic, hidden

25

A malicious instruction mixed beneath a normal conversation, at three concealment levels.

−12 dB, very quiet8
−9 dB9
−6 dB, more audible8

Benign, control

30

Ordinary customer requests, confirming the agent behaves under normal conditions.

False-positive rate0 / 90
Across all 3 modelsclean

What happened

Around two in five adversarial trials caused a security violation.

Binary attack success (S1 + S2)

Share of adversarial trials causing at least a partial security violation. 95% CI in brackets.

Ultravoxv0.5, 8B
48.6%, [40.5 to 56.8]
Gemma 4E4B-it
40.5%, [33.1 to 48.6]
VoxtralMini, 3B
38.5%, [30.4 to 45.9]
Benigncontrol
0 / 90, no false positives across any model

Semantic vs acoustic: the gap

Spoken instructions beat hidden payloads on every model, at high significance.

Voxtral Mini 3B

Semantic43.9%
Acoustic12.0%

+31.9 pp toward speech

Ultravox v0.5 8B

Semantic56.9%
Acoustic8.0%

+48.9 pp toward speech

Gemma 4 E4B-it

Semantic48.8%
Acoustic0.0%

+48.8 pp, zero acoustic breaches

96%Ultravox, English

The single most extreme result in the dataset.

Against plain English multilingual-category attacks, Ultravox failed 96% of the time, including an 88% complete-success rate. Under the same attacks in Japanese it held far better (50%), although different speech-synthesis engines partly confound that language gap.

What it means

The audio channel is not a layer of safety.

It's real, not theoretical.

A clean benign baseline with attack success rates approaching one in two makes this a practical, exploitable weakness, not a lab curiosity.

Just asking is the attack.

Models follow instructions whether typed or spoken. Hiding an attack acoustically added little; asking convincingly did the work.

Architecture is a security variable.

Gemma's purpose-built audio encoder resisted every acoustic attack. Model choice measurably changes exposure, even with the environment held constant.

Urgency pressures the model.

Distressed, time-critical callers outperformed calm, authoritative ones on two of three models. The trend was directional but consistent.

The defensive takeaway

Don't try to filter the audio. The vulnerability lives in the model's willingness to act, so defences belong at the tool and permissions layer, exactly where you'd defend a text channel.

Done responsibly

Sandboxed, synthetic, and disclosed.

DATA

Entirely synthetic

No real people or systems were ever at risk. Every customer was fabricated.

RUNTIME

Rolled back

A fully containerised sandbox; every write action was logged and immediately reverted.

DISCLOSURE

Responsible

Findings shared with model providers, noting that open-weight local models can't be centrally patched.

GOVERNANCE

Ethics-approved

Ran under Oxford Brookes' signed E1 process and the BCS Code of Conduct.