Imran Khan Calls for Systematic Measurement of AI's Psychosocial Effects

Imran Khan, head of psychosocial evaluation of AI at the Center for Humane Technology, warns that while developers rigorously benchmark model capabilities, they largely neglect measuring how deployed systems affect cognition, relationships and identity;

Imran Khan, who leads psychosocial evaluation of AI at the nonprofit Center for Humane Technology, argues that AI builders focus intensely on technical benchmarks while failing to measure how deployed systems change people’s minds and lives. In an essay on the organization’s Substack, he says the central question — what AI is doing to humans — is being overlooked even as models become more capable, and that early measurement of societal‑level effects is urgent because those effects can become entrenched.

Khan points to several high‑profile signals of psychosocial harm rather than isolated anomalies: reports of teenagers dying by suicide, accounts described as people succumbing to “AI psychosis,” and large amounts of time and money spent engaging with highly sycophantic chatbots. He notes one visible response: public pressure led OpenAI to alter a ChatGPT model after concerns about sycophancy became public, showing scrutiny can prompt labs to change behavior when problems are documented.

On the technical side, Khan contrasts the heavy investment in capability testing with the near‑absence of human‑outcome metrics. Researchers and companies pour resources into performance benchmarks and competitive platforms — examples he cites include SWE‑bench, the phrase “humanity’s last exam,” and arenas like LLM Arena — generating tidy charts that track model capabilities while downstream psychosocial effects remain largely unmeasured.

Khan frames the gap in measurement alongside lessons from social media: by the time robust evidence accumulated about platform harms, many negative effects were already entrenched. Quantifying psychosocial harms, he argues, is not merely academic: measurable human‑outcome data would give the public and developers “ammunition” to press for changes and would help guide model adjustments and deployment practices. He calls on researchers and labs to develop and adopt psychosocial metrics that track downstream impacts so deployments can be evaluated, compared, and corrected when harmful effects emerge.