Make evaluation
part of the work.

The problem

A real-time AI interpreter can produce fluent output while changing meaning, using the wrong language, or stepping outside its role. Individual feature tests were not enough to give teams a shared basis for release decisions.

What I did

I built multilingual evaluation datasets, scoring pipelines, provenance records, and shared evaluation suites. I worked with language experts to connect automated scores to the behavior we actually cared about.

The infrastructure was only part of the job. Through tech talks, recurring knowledge shares, and hands-on support, I helped other engineers write and use evaluations. I drove the move toward eval-driven development across the engineering organization.

I introduced CI gates progressively. When evaluation runs were too noisy, making them mandatory would have blocked work without providing reliable evidence. Stable suites became required checks for AI interpreter changes.

What changed

Evaluation became a shared release practice across the organization. Interpreter suites became required CI checks, and other teams adopted the framework for their own systems. Language assessments and expert review helped determine what was ready for general availability.

I led this work alongside an architect and contributing engineers; language experts helped define and review quality. My contribution combined building the machinery with making it a practice people could sustain.

Getting a team to trust the tests starts with making the tests worth trusting.

A concrete measurement lesson

Translation-quality scores missed outputs in the wrong language. Investigating those failures led me to add independent language-ID checks. A favorable aggregate score was not enough to establish that the interpreter had done its job.

Why I’m bringing this into AI safety

Safety evaluations need both sound measurement and an engineering process that acts on the result. This work gave me experience in both: investigating misleading scores and building the infrastructure and habits that make findings consequential.

This is experience in production reliability and behavioral evaluation. The tests establish evidence about specific behaviors under tested conditions; they do not establish general model safety.

Related public work

• Medical AI interpreting accuracy study — designed and authored at Boostlingo.
• AMTA 2026: MT Evaluation for Spoken AI Interpreting — lead presenter; co-developed the tutorial.
• Behavioral and adversarial evaluation work.