02 / BEHAVIORAL & ADVERSARIAL EVALUATION
Test the behavior.
Then test the test.
An unexpected output is a starting point. The harder work is finding out what caused it—and whether the evaluation supports the conclusion.
PRODUCTION WORK / BOOSTLINGO
Does the interpreter stay in its role?
I developed a direct-address guard and spoken role-redirection tests. A colleague later integrated the guard into the production codebase behind a feature flag. The evaluation included ordinary-speech controls and detector-on/off comparisons, so a guard could be checked for both missed redirections and interference with legitimate speech.
The important distinction was between correctly interpreting someone’s words and treating those words as instructions to the interpreter. This work tested a particular conversational role; the evaluation does not establish general prompt-injection resistance.
PERSONAL EXPERIMENT / INSPECT / AI-ASSISTED
Can a document invent its own authority?
I built a controlled résumé-screening experiment: the same fictional applicant and job requirements, with identical-looking PDFs. Conditions included a clean résumé, hidden direct instructions, a claimed reward, and a forged hiring-manager exception.
The decision was scored against the job requirements. Repeating an injected flag or mentioning an instruction did not count as an attack success; advancing the unqualified applicant did.
This was a small exploratory study of one fixed applicant/job pair and a specific PDF extraction pipeline. It does not estimate performance across real hiring decisions or test visual document understanding.
WORK SAMPLE / PETRI / AI-ASSISTED
When the judge and the transcript disagree
In a Petri behavioral-auditing work sample, I designed scenario variants, reviewed transcripts, and iterated on the simulation. The work surfaced judge inconsistencies and simulator drift—changes in the simulated environment that complicated comparisons between conditions.
The analysis separated awareness, attempted actions, completed actions, and disclosure. I used the action and tool-result evidence to distinguish observed behavior from the judge’s interpretation, and retained limits on what the small experiment could show.
How I worked
These independent experiments used AI assistance for implementation and analysis. My work included framing questions, reviewing evidence, and making iteration decisions. The conclusions remain scoped to the experiments performed.
Code & report
- Inspect résumé-injection experiment: code, PDF fixtures, scoring rules, and offline tests.
- Petri audit report: design, observations, and limitations from 18 samples. A report-only work sample; the supplied hiring-test harness and raw logs are not redistributed.
For a public experiment with code, model comparisons, and recorded limits, see harmful-request classification.