Florian Eichin, Philipp Mondorf, Andrei Mircea, Yupei Du, Barbara Plank, Michael A. Hedderich. 2026. Don't Forget! Decomposing the Training Dynamics of Memorization in Language Models. Preprint.
We decompose LLM loss trajectories to characterize memorization training dynamics through gradient alignment and validate our results through early memorization prediction causal post-hoc ablation.
Florian Eichin, Yupei Du, Philipp Mondorf, Maria Matveev, Barbara Plank, and Michael A. Hedderich. 2026. ExPLAIND: Unifying Model, Data, and Training Attribution to Study Model Behavior. ICML 2026.
We introduce ExPLAIND — an interpretability framework for jointly attributing model components, data, and training dynamics and apply it to investigate Grokking.
Felicia Körner*, Maria Matveev*, Florian Eichin*, Gitta Kutyniok, Barbara Plank, Michael A. Hedderich. 2026. Copy First, Translate Later: Interpreting Translation Dynamics in Multilingual Pretraining. EMNLP 2026 Main.
We study the early pretraining dynamics of multilingual models.
Ryan Soh-Eun Shim, Kwanghee Choi, Kalvin Chang, Ming-Hao Hsu, Florian Eichin, Zhizheng Wu, Alane Suhr, Michael A. Hedderich, David Harwath, David R. Mortensen, Barbara Plank. 2026. Linear Script Representations in Speech Foundation Models Enable Zero-Shot Transliteration. ACL 2026 Findings.
We show that script is largely represented as a single linear direction in activation space of Whisper models and that steering activations towards that direction enables transcriptions and even generalizes across different languages.
Michael A. Hedderich, Anyi Wang, Raoyuan Zhao, Florian Eichin, and Barbara Plank. 2025. What's the Difference? Supporting Users in Identifying the Effects of Prompt and Model Changes Through Token Patterns. ACL 2025 Main.
We propose a method for evaluating the influence of prompts on the LLM outputs through token patterns.
Florian Eichin*, Yang Janet Liu*, Barbara Plank, Michael A. Hedderich. 2025. Probing LLMs for Multilingual Discourse Generalization Through a Unified Label Set. ACL 2025 Main.
We probe an extensive set of SOTA large language models for multilingual discourse relation information unifying framework-specific label sets.
Florian Eichin, Carolin M. Schuster, Georg Groh, and Michael A. Hedderich. 2024. Semantic Component Analysis: Introducing Multi-Topic Distributions to Clustering-Based Topic Modeling. EMNLP 2025 Findings.
We introduce multi-topic distributions to the clustering-based topic modeling pipeline and reduce the number of samples assigned to noise to zero.