Home | Jane Castleman

I'm a Computer Science PhD student at Stanford University.

I'm passionate about designing methods to measure the social impacts of AI, drawing upon theory from machine learning, statistics, measurement science, and consumer protection law.

Most recently, I've been interested in evaluation science [1], particularly for longitudinal AI evaluation and under real-world conditions. I also design evaluations of AI systems in high-stakes decision-making settings [2, 3], and conduct third-party evaluations of deployed algorithmic systems [4].

Previously, I graduated from Princeton University, advised by Aleksandra Korolova, working in the Center for Information Technology Policy. I was also fortunate to collaborate with Jonathan Mayer and Lydia T. Liu.

I'm always happy to discuss research ideas or talk to students thinking about applying to graduate school.

News

[09/26] Starting as a PhD student at Stanford University!
[5/26] Graduated from Princeton University with a Master's in Computer Science!
[2/26] Traveled to Paris to present our paper "Measuring Validity in LLM-based Resume Screening" at IASEAI '26!
[12/25] Excited to receive a DDSS Grant from Princeton to support my research studying rideshare and delivery platforms alongside Professors Aleksandra Korolova and Jonathan Mayer!
[09/25] Honored to be a 2026 Siebel Scholar! Thank you to my advisor, classmates, and professors at Princeton for your support.
[09/25] Honored to be attending the 12th Heidelberg Laureate Forum!
[08/25] Excited to be selected as a Graduate Fellow of Social Data Science for Fall 2025!
[08/25] Looking forward to another semester as Head Teaching Assistant for COS350: Ethics of Computing with Aleksandra Korolova and Arvind Narayanan!
[06/25] I'm attending FAccT '25 in Athens! I'll be presenting my paper Adultification Bias in LLMs & Text-to-Image Models.
[03/25] I presented "Rethinking Math Benchmarks for LLMs using IRT" at the iRAISE Workshop @ AAAI in Philadelphia.
[10/24] I presented "Why Am I Still Seeing This?" at AIES 2024 and Tech Policy: The Next Ten Years.

Publications

Measuring Validity in LLM-based Resume Screening

Jane Castleman, Zeyu Shen, Blossom Metevier, Max Springer, and Aleksandra Korolova

International Association for Safe & Ethical AI (IASEAI), 2026. Paper
Adultification Bias in LLMs and T2I Models

Jane Castleman and Aleksandra Korolova

ACM Conference on Fairness, Accountability, and Transparency (FAccT '25), 2025. Paper
Missing the Mark: Rethinking Math Benchmarks for LLMs using IRT

Jane Castleman*, Nimra Nadeem*, Tanvi Namjoshi*, and Lydia T. Liu

AAAI 2025 Workshop on AI for Education (iRAISE), Spotlight, 2025. Paper * denotes equal contribution.
Why Am I Still Seeing This: Measuring the Effectiveness of Ad Controls and Explanations in AI-Mediated Ad Targeting Systems

Jane Castleman and Aleksandra Korolova

7th AAAI Conference on AI, Ethics, and Society (AIES '24), 2024. Paper

Misc

Outside of graduate school, I enjoy spending time outside and playing sports. I was a member of the inaugural Varsity Women's Rugby team at Princeton and was selected to the National Intercollegiate Rugby All-Academic team in 2023.