Workshop on Computer Vision for Image Segmentation

Le résumé fourni par la source

Practical Workflow for Microscopy Image Segmentation: Data Preparation, Model Training, Inference, and Pretrained Models Author: Vladimir Pimonov, PhDAffiliation: team PNEC / ILM This Zenodo repository contains the full practical workflow for image segmentation with deep learning, developed on transmission electron microscopy (TEM) images of nanoparticles. The repository covers the complete sequence required to adapt a segmentation model to a microscopy dataset: extraction of masks from annotations, dataset preparation and balancing, patch generation, construction of PyTorch datasets and dataloaders, model training and validation, inference on patches and full images, and post-processing of prediction maps. This repository contains the files, including pretrained model checkpoints and a minimal dataset needed to demonstrate the training and inference workflow. The corresponding code is also available and maintained through the associated GitHub repository. The material can be used in two ways: as support for a practical workshop on segmentation with deep learning, or as a reusable technical base for microscopy image segmentation and related computer vision tasks. The code and models were developed for nanoparticle recognition, but the workflow itself is not restricted to this application.The same structure can be adapted to other segmentation tasks based on annotated images and mask prediction. Content The deposit is organized around four practical notebooks and a set of reusable helper scripts, together with pretrained model weights and example data. TP01 – Dataset preparation This notebook prepares the segmentation dataset from annotated microscopy images. It includes: search for annotation files, extraction of masks from annotation layers, matching source images with their masks, definition of patching rules depending on magnification, description of candidate patches in a dataframe, characterization of patches as empty, almost empty, touching borders, or containing relevant objects, balancing of the future training set, export of the final image and mask patches. The purpose of this notebook is to construct a controlled dataset before writing the cropped patches to disk. TP02 – Dataframe preparation and PyTorch data pipeline This notebook prepares the exported dataset for model training. It includes: definition of a custom PyTorch dataset, implementation of paired image and mask transformations, definition of augmentation rules, loading of the dataframe containing the train, validation, and test split, creation of datasets and dataloaders, visual inspection of augmentations and loaded samples. The purpose of this notebook is to connect the saved dataset to the PyTorch training pipeline. TP03 – Model training and validation This notebook contains the training workflow. It includes: model loading and configuration, preparation of train, validation, and test dataloaders, estimation of class imbalance, definition of losses, optimizers, and schedulers, training and validation loops, metric tracking, checkpoint saving and loading, first prediction checks on test patches and larger images. The purpose of this notebook is to train and evaluate a segmentation model on the prepared dataset. TP04 – Inference and post-processing This notebook contains the inference workflow. It includes: loading trained checkpoints, full-image tiled inference, patch inference, probability-map visualization, thresholded segmentation overlays, contour visualization, comparison of several confidence thresholds, confidence-based filtering of weak isolated detections. The purpose of this notebook is to apply a trained model to new images and inspect the predicted results. Supporting scripts The folder scripts contains the helper scripts used throughout the workflow. These scripts provide: training and evaluation utilities, inference functions for patches and full images, loss functions, dataset transformations, and model initialization code. They can be used both for direct inference with the pretrained checkpoints and for further fine-tuning or adaptation of the models to other segmentation tasks. Image data The folder Aggregates for training TP contains a subset of the TEM images used during model development and during the practical workshop. This subset is provided as a minimal example for demonstration and testing purposes.It does not correspond to the full dataset used to train the released model checkpoints. The pretrained models distributed in this repository were trained on the complete image set, while the folder included here contains approximately 30% of that data. Pretrained model weights This repository also contains 6 sets of pretrained models. The weights are distributed as separate .zip archives.Each archive contains: a set of checkpoints, the corresponding training history files, and a readme.txt file describing the differences between weight versions. Each model version includes two checkpoints: an early-stop checkpoint, typically obtained after approximately 61 to 101 epochs, and a fully trained checkpoint obtained after 201 epochs. The different versions of the same model correspond to different loss functions used during training: VX.0 – training with weighted binary cross-entropy loss, VX.1 – training with binary cross-entropy + Dice loss, VX.2 and VX.3 – training with focal loss, where: VX.2 uses standard hyperparameters, VX.3 uses optimized hyperparameters. The weights are organized into separate .zip files so that users can download only the model variants relevant to their intended use. Supporting files This repository contains two text files describing the software environment and installation procedure: requirements.txt – list of the libraries used in the notebooks, readme.txt – practical installation instructions for an Anaconda-based Python environment. More comprehensive explanations on the installation of the environment provided in GitHub repository. Available models The deposit contains several segmentation architectures from the U-Net family, implemented with ResNet backbones. Available models include: U-Net (weights available in U-Net (resnet34 5conv layers) v24.zip) Deep U-Net (weights for legacy model available in U-Net deep (resnet50 7conv layers) v25.zip) U-Net++ (weights available in U-Net++ (resnet34 5conv layers) v23.zip) Deep U-Net++ (weights for legacy model available in U-Net++ deep (resnet50 7conv layers) v26.zip) U-Net 3+ (weights available in U-Net3+ (resnet34 5conv layers) v30.zip) Deep U-Net 3+ (weights available in U-Net3+ deep (resnet50 7conv layers) v29.zip) Legacy versions are also included for compatibility and reference. Two implementation families are available: Classic models, corresponding to the standard forms described in the literature, Deep models, which extend the depth of the network and increase the receptive field. The deep variants require larger input patches because repeated downsampling reduces the spatial size of the feature maps more strongly. Intended use This material was developed for nanoparticle recognition in microscopy images, but the workflow is generic at the level of: mask extraction from annotated data, patch-based dataset preparation, supervised segmentation training, tiled inference on large images, threshold-based and confidence-based post-processing. It can therefore serve both as a worked example for microscopy segmentation and as a starting point for adapting the same approach to other image segmentation problems.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.