Lars Doorenbos

Post-Training VLMs for Video Mistake Detection

Federico Spurio, Olga Zatsarynna, Lars Doorenbos, Emad Bahrami et autres

Human mistakes are inevitable when following instructions, yet they can lead to severe consequences. As such, there has been an increased interest in developing methods for detecting mistakes in videos, with current methods mostly focusing on closed-set protocols. While successful in controlled …

Multi-Person Human Motion Forecasting in Complex Scenes

Serdar Ozsoy, Lars Doorenbos, Jüergen Gall

Accurately forecasting the movement of people in complex scenes requires reasoning over the past and present state of the entire environment. In this context, effectively incorporating object information and social interactions into a unified framework remains particularly challenging. To address this, we …

Modality-Aware Out-of-Distribution Detection for Multi-Modal Action Recognition

Lars Doorenbos, Duc Manh Vu, Serdar Ozsoy, Jüergen Gall

The incorporation of additional modalities into action recognition models increases their performance across a wide range of settings. However, how this additional information can contribute to making the models more robust remains underexplored, particularly for the case of multi-modal out-of-distribution (OOD) detection. …

The Unreasonable Effectiveness of VLMs for Zero-shot Procedural Mistake Detection

Serdar Ozsoy, Lars Doorenbos, Federico Spurio, Gianpiero Francesca et autres

Procedural mistake detection is important for quality control and user assistance across many disciplines. Recent work in this field has achieved significant gains by using the reasoning capabilities of Video-Language Models (VLMs) as components within multi-stage pipelines, which consist of separate modules …

The Unreasonable Effectiveness of VLMs for Zero-shot Procedural Mistake Detection

Serdar Ozsoy, Lars Doorenbos, Federico Spurio, Gianpiero Francesca et autres

Procedural mistake detection is important for quality control and user assistance across many disciplines. Recent work in this field has achieved significant gains by using the reasoning capabilities of Video-Language Models (VLMs) as components within multi-stage pipelines, which consist of separate modules …

TRAFA: Anticipating User Actions to Reduce Errors in Procedural Tasks with Predictive Feedback

Sassan Mokhtar, Lars Doorenbos, Fatemeh Jabbari, Marius Bock et autres

Interactive assistance systems typically provide feedback after an action has been completed, supporting error recovery but not preventing the error itself. We present TRAFA, a real-time predictive feedback system for procedural tasks that intervenes before errors are committed. TRAFA operationalizes predictive feedback …

TRAFA: Anticipating User Actions to Reduce Errors in Procedural Tasks with Predictive Feedback

Sassan Mokhtar, Lars Doorenbos, Fatemeh Jabbari, Marius Bock et autres

Interactive assistance systems typically provide feedback after an action has been completed, supporting error recovery but not preventing the error itself. We present TRAFA, a real-time predictive feedback system for procedural tasks that intervenes before errors are committed. TRAFA operationalizes predictive feedback …

Modality-Aware Out-of-Distribution Detection for Multi-modal Action Recognition

Lars Doorenbos, Duc Manh Vu, Serdar Ozsoy, Jüergen Gall

de (code pays fourni par la source)

Video Panels for Long Video Understanding

Lars Doorenbos, Federico Spurio, Jüergen Gall

Recent Video-Language Models (VLMs) achieve promising results on long-video understanding, but their performance still lags behind that achieved on tasks involving images or short videos. This has led to great interest in improving the long context modeling of VLMs by introducing novel …

Iterative Deployment Exposure for Unsupervised Out-of-Distribution Detection

Lars Doorenbos, Raphael Sznitman, Pablo Márquez-Neila

ch, de (code pays fourni par la source)

Inferring geometry and material properties from Mueller matrices with machine learning

Lars Doorenbos, Lucas Patty, Raphael Sznitman, Pablo Márquez-Neila

Mueller matrices (MMs) encode information on geometry and material properties, but recovering both simultaneously is an ill-posed problem. We explore whether MMs contain sufficient information to infer surface geometry and material properties with machine learning. We use a dataset of spheres of …

ch, de (code pays fourni par la source)

ULISSE: Determination of the star formation rate and stellar mass based on the one-shot galaxy imaging technique

O. Torbaniuk, Lars Doorenbos, Maurizio Paolillo, S. Cavuoti et autres

Context. Modern sky surveys produce vast amounts of observational data, which makes the application of classical methods for estimating galaxy properties challenging and time-consuming. This challenge can be significantly alleviated by employing automatic machine- and deep-learning techniques. Aims. We propose an implementation …