Frame the consequential question.
Clarify the decision, the estimand, and the evidence needed. Surface assumptions and choose a design that can support the intended claim.
AI evaluation · Causal inference · Measurement
Better evidence.
Better decisions.
I’m Masha, a Senior Scientist at Amazon. I build AI evaluation systems, design experiments and quasi-experimental studies, and develop the measurement and structural models that tell teams what a result means.
I stay hands-on in the research and lead the scientists and engineers around it: choosing defensible methods, setting validation standards, and turning evidence into decisions people can act on.
Amazon / Senior ScientistHarvard / Ph.D.MIT · J-PAL / Measurement Fellow · Randomized evaluationsSeattle / Washington
01 / Research approach
Different methods.
A consistent standard.
The question determines the method. Across AI evaluation, causal studies, measurement, and structural models, I ask what supports the conclusion, and what would change it.
AI evaluation & validation
Where does the model fail—and does the result generalize?
I build human-grounded evaluations, analyze where LLM judges and human raters disagree, and break errors down by category. I use regression checks to compare models and prompts, and I validate the reference data itself before trusting it.
A high aggregate agreement rate can hide a weak category. Agreement with human labels also does not establish that the labels capture the right thing.
Causal inference & quasi-experiments
What variation identifies the effect?
I design experiments and quasi-experimental studies around the causal question, the assignment process, and the available comparison. I assess identification, run diagnostics, quantify uncertainty, and show how results move under different assumptions.
A comparison group has to be credible for the question being asked. Similar pre-trends are a diagnostic; they do not establish the identifying assumption.
Measurement & psychometrics
Are we measuring the intended construct?
My psychometrics training shapes how I design and validate measures. Before interpreting a change in a score, I check comparability, composition, and uncertainty.
When the mix of respondents changes, a score can move even if the underlying experience does not.
Structural economic models
What assumptions support the counterfactual?
I use structural models to make assumptions about behavior explicit and to examine counterfactuals. I assess identification, fit, and sensitivity to establish what a model can and can’t tell us.
A model can fit observed behavior while supporting very different counterfactual predictions under alternative assumptions.
02 / Scientific leadership
Research depth.
Shared scientific direction.
I work from the individual research problem to the standards a team can use.
I lead scientists and engineers, set the methodological and validation standards they work to, and work across disciplines with senior decision-makers. I stay close to the research while helping others frame questions and assess evidence.
Team leadership
Led eight scientists
and engineers
Scientific standards
Methods and validation
across disciplines
Decision areas
AI evaluation, measurement
and intervention impact
Build standards people can reuse.
Make diagnostics, validation criteria, uncertainty reporting, and scientific review explicit enough to carry across projects and teams.
Connect findings to judgment.
Communicate what is known, what remains uncertain, and which tradeoffs belong to the decision-maker. Help collaborators challenge the conclusion productively.