---

Generative LLMs are increasingly used to annotate text in the social sciences, but an LLM’s label is a measurement, not ground truth. In this module, we treat the LLM as a measurement tool and partially reproduce Törnberg’s (2023) study of populism in political speeches. We cover:

1. Turning a theoretical concept into a prompt and a structured output schema;
2. A Python (LangChain) tutorial on using GPT-OSS 20B, through the free Groq API, to score speeches from the Global Populism Database;
3. Checking the model’s scores against human coding and examining where they disagree;
4. What this means for validity, reliability, reproducibility, and research ethics.

Prerequisites	Proficiency in Python is not required, but a basic understanding of writing functions and loops will be helpful. Primary knowledge of Text as Data will also be beneficial.
Materials	https://github.com/jyang32/LLMworkshop-fall2026/tree/main