UNITE software

Résumé fourni par la source

This repository contains the code and resources used in the UNITE paper by the Rosenfeld Group. It includes instructions for setting up the analysis environment, accessing raw data and extracted features, using trained models, and running UNITE on new shallow whole-genome sequencing data. The codes are uploaded to this Zenodo record as a 3.9GB zip file (Please scroll to the bottom of the page to see the files session, access is restricted). Please contact bci-nrlab-admin@qmul.ac.uk for access. UNITE-paper Contents Using the pre-built Conda environment Raw data Features extracted from raw data Docker image Trained models Running UNITE on new data Key resources License 1. Using the pre-built Conda environment This repository provides a pre-built Conda environment, R4_1.tar.gz, for fully reproducible analysis on Linux systems, including HPC or local Linux machines. This file was generated using conda-pack. The environment can be used with micromamba, which is recommended, or Miniconda. If conda or mamba is already installed, no additional installation is required. Install micromamba curl -Ls https://micro.mamba.pm/api/micromamba/linux-64/latest | tar -xvj bin/micromamba chmod +x bin/micromamba export PATH="$PWD/bin:$PATH" micromamba shell init -s bash -p ~/micromamba source ~/.bashrc Download and unpack the environment Download R4_1.tar.gz, approximately 3.4 GB, from Zenodo: https://doi.org/10.5281/zenodo.18187489 Choose where to install the environment, for example: mkdir -p ~/conda_envs/R4_1 Unpack the environment: tar -xzf R4_1.tar.gz -C ~/conda_envs/R4_1 Fix paths This step is required. conda activate ~/conda_envs/R4_1 Alternatively, if using micromamba: micromamba activate ~/conda_envs/R4_1 Then run: conda-unpack Activate the environment conda activate ~/conda_envs/R4_1 Alternatively, if using micromamba: micromamba activate ~/conda_envs/R4_1 Verify the environment This step verifies that the environment works as expected. Open the R console and run: library(pacman) packages <- c( "parallel", "future", "furrr", "progressr", "parallelly", "tidyverse", "Biobase", "QDNAseq", "Homo.sapiens", "plyranges", "GenomicRanges", "GenomicAlignments", "matrixStats", "AnnotationHub", "BSgenome.Hsapiens.UCSC.hg19", "BSgenome.Hsapiens.UCSC.hg38", "Rsamtools", "rslurm", "abind", "tictoc", "patchwork", "EBImage", "gridExtra", "grid", "reshape2", "purrr", "fastDummies", "ggalluvial", "MultiAssayExperiment", "nord", "plotly", "htmlwidgets", "ggpubr", "argparse", "argparser", "bettermc" ) p_load(char = packages) Notes Linux only. Do not move the environment after running conda-unpack. Re-run conda-unpack if activation fails. An explicit list of packages installed using micromamba is provided in R4_1.explicit.txt, deposited in the same Zenodo record: https://doi.org/10.5281/zenodo.18187489. Instead of using the packed environment, the environment can also be built from scratch using the explicit file. 2. Raw data The shallow whole-genome sequencing data of the newly sequenced samples in the UNITE paper are deposited in Zenodo: https://doi.org/10.5281/zenodo.18155576 The metadata are recorded in Table S12 of the manuscript. The data files are in .tsv.gz format and are derived from 0.1x BAM files aligned to hg19. Paired reads were merged into fragments using cfDNAPro, developed by the lab: https://github.com/NRLAB-CRUK/cfDNAPro Reference: Wang et al., Genome Biology, 2021. The files are compatible with FinaleDB by Zheng et al., Bioinformatics, 2021: http://finaledb.research.cchmc.org/ The columns are as follows: Column Example Description seqname chr1 Chromosome start 1782520 Start position end 1782770 End position mapq 60 Fragment mapping quality, defined as the minimum mapping quality of the forward and reverse reads strand + Strand The data and code are for research purposes only. If necessary, the tsv.gz file can be converted to BAM format, for example when an in-house software or pipeline only accepts BAM as input. Please note that the output is a single-end BAM file, where each “read” represents a cfDNA fragment. First, convert the TSV file to a GRanges R object and save it as an .rds file: tsv_file="/path/to/tsv/file" Rscript --vanilla ./finaleDB/read_finaledb_frag_tsv_to_granges.r ${tsv_file} Second, convert the GRanges object to BAM format: rds_file="/path/to/rds/file" output_dir="/path/to/output_dir/" Rscript --vanilla ./ulyses_scale_up_scripts/galp_to_bam.R ${rds_file} ${output_dir} FALSE 3. Features extracted from raw data For best practices in reproducible science, all features used in the paper are stored in a MultiAssayExperiment object and deposited in Zenodo: https://doi.org/10.5281/zenodo.18163904 These include: Fragment length features SD features C/T ratio S/L ratio CNA features Multidimensional arrays Metadata for all samples The deposited files are: MAE_object_for_cross_validation_samples.rds MAE_object_for_heldout_test_samples.rds MAE_object_for_unseen_test_samples.rds MultiAssayExperiment provides a data structure for representing and analyzing multi-omics experiments. To extract features from the MultiAssayExperiment object, please refer to the official Bioconductor documentation: https://www.bioconductor.org/packages/release/bioc/vignettes/MultiAssayExperiment/inst/doc/MultiAssayExperiment.html 4. Docker image Singularity, version singularity-ce 3.11.5-1.el8, was used to create the CNN model training container using the following command: singularity pull docker://pytorch/pytorch The Docker image was obtained on Wednesday, 15 May 2024 at 22:13:28 UTC. The version number was 22.04. 5. Trained models The locked models can be downloaded from Zenodo: https://doi.org/10.5281/zenodo.19150139 These are the final locked models trained using all cross-validation samples and tested on the unseen samples. A .pkl file in Python is a file created using pickle, Python’s built-in object serialization module. Model Model file File size Description UNITE-CNN CNN.pkl 263 MB ResNet18 architecture using C4 array features as input. UNITE-XGB XGB.pkl 2 MB XGBoost model using X6 features as input. DELFI-XGB DELFI.pkl 2 MB XGBoost model using the same feature set as the DELFI model. ichorCNA-TF ichorCNA.pkl 19 KB Logistic regression model using tumour fraction inferred by ichorCNA. 6. Running UNITE on new data The code below shows how to run UNITE on new data on a Slurm cluster. 6.1 Calculate BAM statistics This step calculates basic BAM statistics, including the number of reads and sequencing coverage. For each BAM file, a corresponding *.bam.summary.csv file will be generated. for i in *.bam do sbatch --time=0-10 --mem=128G -J "count" -p "epyc" --wrap="source ~/.bashrc; conda activate R4_1; Rscript --vanilla ./ulyses_scale_up_scripts/summariseBam.R ${i} . FALSE" done 6.2 Downsample the BAM file to the target depth The downsampled BAM files will be stored in the downsampled_bam folder. The .bam.summary.csv files generated in the previous step are used as input for this step. The BAM files should be in the same directory as the .bam.summary.csv files. downsampled_dir="../downsampled_bam" target_depth=0.1 mkdir -p ${downsampled_dir} for i in *.bam.summary.csv do sbatch --time=0-20 --mem=32G -J "downsamp" -p "epyc" --wrap="source ~/.bashrc; conda activate R4_1; Rscript --vanilla ./ulyses_scale_up_scripts/downsample_gatk.R ${i} ${target_depth} ${downsampled_dir}" done 6.3 Run UNITE to extract features Set the working directory containing the downsampled BAM files: working_dir="./your_working_directory/" Run the script: bash ./ulyses_scale_up_scripts/run_ulyses.sh ${working_dir} 6.4 Generate UNITE plots In the previous step, *ulyses_image.rds files are generated for each BAM file. The code below generates corresponding plots for each generated RDS file: for i in *.ulyses_image.rds do sbatch --time=0-00:15:00 \ --mem=1G --partition=epyc \ -J "plot" \ --wrap="source ~/.bashrc; \ conda activate R4_1; \ Rscript ./explore_packaged/functions/ulyses_image_to_features.R \ --image_

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.