Validation of Simulation-Based Testing: Bypassing Domain Shift with\n Label-to-Image Synthesis

Le résumé fourni par la source

Many machine learning applications can benefit from simulated data for\nsystematic validation - in particular if real-life data is difficult to obtain\nor annotate. However, since simulations are prone to domain shift w.r.t.\nreal-life data, it is crucial to verify the transferability of the obtained\nresults. We propose a novel framework consisting of a generative label-to-image\nsynthesis model together with different transferability measures to inspect to\nwhat extent we can transfer testing results of semantic segmentation models\nfrom synthetic data to equivalent real-life data. With slight modifications,\nour approach is extendable to, e.g., general multi-class classification tasks.\nGrounded on the transferability analysis, our approach additionally allows for\nextensive testing by incorporating controlled simulations. We validate our\napproach empirically on a semantic segmentation task on driving scenes.\nTransferability is tested using correlation analysis of IoU and a learned\ndiscriminator. Although the latter can distinguish between real-life and\nsynthetic tests, in the former we observe surprisingly strong correlations of\n0.7 for both cars and pedestrians.\n

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.