HumAI: Can We Trust AI as a Scientific Instrument for the Humanities?

About the seminar

Image

Rapid advances in artificial intelligence are changing not only how we process language, but also what we can measure about human communication at scale. This creates exciting opportunities for research in the humanities and social sciences, but also a methodological problem: when can we trust AI-generated measurements enough to build scientific claims on them?

In this talk, I will approach this question from the perspective of computational linguistics and language technology. I will first discuss language and dialect benchmarks and what they tell us about both the remarkable capabilities and persistent blind spots of current language models. I will then move beyond text to speech, where recent models make it possible to analyse large collections of human communication across languages and populations while preserving phenomena that written transcripts leave out.

Drawing on recent research on South Slavic languages, multilingual parliamentary speech, political discourse, and the ever-changing World Wide Web, I will show how AI-based measurements can be turned into substantive findings about language and society. These examples also point to a broader methodological proposal for AI-supported humanities research: start from genuine human-generated data, validate computational operationalizations against human judgements, use scalable models as scientific instruments, and replicate findings across populations and analytical choices.

Finally, I will consider a different role for large language models beyond their use as scientific instruments: as increasingly capable, agentic co-researchers supporting literature synthesis, research design, data analysis, interpretation, and critical revision throughout the research process. The central question is therefore not whether AI will enter humanities research, but how we can integrate it into the research process while keeping our methods valid, our findings reproducible, and AI-assisted contributions verifiable.

Nikola Ljubešić  is a senior researcher at the Jožef Stefan Institute in Ljubljana, where he works at the intersection of computational linguistics, language technology, and empirical research on language and society. His research focuses particularly on South Slavic and other less-resourced languages and spans large-scale web corpora, multilingual and dialectal language technologies, speech processing, and computational analysis of political discourse. He leads the CLARIN ERIC CLASSLA Knowledge Centre for South Slavic languages and is closely involved in the CLARIN.SI research infrastructure. His recent work explores how modern AI methods can be used not only to process language, but as scalable scientific instruments for studying linguistic and social phenomena across languages, modalities, and domains. He is particularly interested in robust evaluation, computational methods for empirical language research, and how rapid developments in artificial intelligence are reshaping the scientific process.