Pietro Bernardelle

Persona Conditioning as an Assessor-Sensitivity Probe for LLM-Based IR Evaluation

Samaneh Mohtadi, Pietro Bernardelle, Joel Mackenzie, Gianluca Demartini

Large language models (LLMs) are increasingly used as relevance assessors in information retrieval (IR) evaluation, raising questions about how assessor framing affects judgment reliability and downstream system comparison. We study persona conditioning as a diagnostic mechanism for exposing LLM assessor sensitivity. Using …

LLMs Encode Relevance as a Layer-Wise Cross-Lingual Signal

Pietro Bernardelle, Samaneh Mohtadi, Stefano Civelli, Joel Mackenzie et autres

Large language models (LLMs) are increasingly used in information retrieval (IR) pipelines as relevance judges and re-rankers. Yet most analyses remain output-centric, evaluating generated labels or scores while offering limited insight into how relevance is represented inside the model. In this work, …

LLMs Encode Relevance as a Layer-Wise Cross-Lingual Signal

Pietro Bernardelle, Samaneh Mohtadi, Stefano Civelli, Joel Mackenzie et autres

Large language models (LLMs) are increasingly used in information retrieval (IR) pipelines as relevance judges and re-rankers. Yet most analyses remain output-centric, evaluating generated labels or scores while offering limited insight into how relevance is represented inside the model. In this work, …

au (code pays fourni par la source)

A Shared Geometry of Difficulty in Multilingual Language Models

Association for Computational Linguistics 2026, Pietro Bernardelle, Nicolò Brunello, Stefano Civelli et autres

Large language models (LLMs) encode problem difficulty as an internal signal that can be linearly decoded from their residuals. Given their multilingual capabilities, we investigate whether this meta-cognitive signal is language-agnostic and how it is organized across the model’s layers by training …

au (code pays fourni par la source)

Towards Detecting Persuasion on Social Media: From Model Development to Insights on Persuasion Strategies

Elyas Meguellati, Stefano Civelli, Pietro Bernardelle, Shazia Sadiq et autres

Political advertising plays a pivotal role in shaping public opinion and influencing electoral outcomes, often through subtle persuasive techniques embedded in broader propaganda strategies. Detecting these persuasive elements is crucial for enhancing voter awareness and ensuring transparency in democratic processes. This paper …

au, hk (code pays fourni par la source)

Political Advertising on Facebook During the 2022 Australian Federal Election: A Social Identity Perspective

Stefano Civelli, Pietro Bernardelle, Frank Mols, Gianluca Demartini

Political advertising on social media has become an important component of contemporary election campaigns, yet most research has focused on voluntary voting systems in which mobilizing supporters is a central goal. This study examines political advertising on Meta platforms (Facebook and Instagram) …

au (code pays fourni par la source)

SubData: Bridging Heterogeneous Datasets to Enable Theory-Driven Evaluation of Political and Demographic Perspectives in LLMs

Pietro Bernardelle, Leon Fröhling, Gianluca Demartini

As increasingly capable large language models (LLMs) emerge, researchers have begun exploring their potential for subjective tasks.While recent work demonstrates that LLMs can be aligned with diverse human perspectives, evaluating this alignment on downstream tasks (e.g., hate speech detection) remains challenging due …

au (code pays fourni par la source)

SubData: Bridging Heterogeneous Datasets to Enable Theory-Driven Evaluation of Political and Demographic Perspectives in LLMs

Pietro Bernardelle, Leon Froehling, Stefano Civelli, Gianluca Demartini

As increasingly capable large language models (LLMs) emerge, researchers have begun exploring their potential for subjective tasks.While recent work demonstrates that LLMs can be aligned with diverse human perspectives, evaluating this alignment on downstream tasks (e.g., hate speech detection) remains challenging due …

Ideology-Based LLMs for Content Moderation

Stefano Civelli, Pietro Bernardelle, Nardiena Pratama, Gianluca Demartini

Large language models (LLMs) are increasingly used in content moderation systems, where ensuring fairness and objectivity is essential. In this study, we examine how persona adoption influences the consistency and fairness of harmful content classification across different LLM architectures, model sizes, and …

au, gb (code pays fourni par la source)

Context Shapes LLMs Retrieval-Augmented Fact-Checking Effectiveness

Pietro Bernardelle, Stefano Civelli, Kevin Roitero, Gianluca Demartini

Large language models (LLMs) show strong reasoning abilities across diverse tasks, yet their performance on extended contexts remains inconsistent. While prior research has emphasized mid-context degradation in question answering, this study examines the impact of context in LLM-based fact verification. Using three …

Context Shapes LLMs Retrieval-Augmented Fact-Checking Effectiveness

Pietro Bernardelle, Stefano Civelli, Kevin Roitero, Gianluca Demartini

Large language models (LLMs) show strong reasoning abilities across diverse tasks, yet their performance on extended contexts remains inconsistent. While prior research has emphasized mid-context degradation in question answering, this study examines the impact of context in LLM-based fact verification. Using three …

A Shared Geometry of Difficulty in Multilingual Language Models

Stefano Civelli, Pietro Bernardelle, Nicolò Brunello, Gianluca Demartini

Predicting problem-difficulty in large language models (LLMs) refers to estimating how difficult a task is according to the model itself, typically by training linear probes on its internal representations. In this work, we study the multilingual geometry of problem-difficulty in LLMs by …