Response to ‘comment on rigorous benchmarking of T cell receptor repertoire profiling methods for cancer RNA sequencing’ by Davydov A.N.; Bolotin D.A.; Poslavsky S. V. and Chudakov D.M.

Le résumé fourni par la source

Huang et al. reply: Davydov et al. discuss potential pitfalls in the publication ‘Rigorous benchmarking of T cell receptor repertoire profiling methods for cancer RNA sequencing’. Below, we clarified and demonstrated that conclusions cannot be drawn on the basis of their response. Davydov et al. highlighted that our comparison of tools like CATT, TRUST4, and ImRep—which do not consider Phred nucleotide quality in their analyses—against MiXCR was not fully standardized. They suggested that to ensure a fair comparison, the quality filter in MiXCR should also be disabled. We acknowledge the valuable suggestion from the Davydov et al. group regarding the implementation of Phred nucleotide quality filters on the reads before executing MIXCR. However, our primary objective in this publication is to faithfully replicate the typical workflow employed by scientific researchers when utilizing computational tools. Consequently, it would be inequitable to omit the default command when applying various bioinformatics tools to the identical set of samples. Consequently, we opted to employ uniform default criteria across all tools, executing each tool with its default settings. Furthermore, Davydov et al. pinpointed a specific issue related to clonotype abundance calculation in version 1.0.2 of TRUST4. We certainly appreciate the concerns raised by the Davydov et al. group regarding a potential bug in version 1.0.2 of TRUST4 that might affect clonotype abundance calculations. It’s crucial to maintain the highest standards of data integrity, and we take such observations seriously. However, it’s important to note that the version of TRUST4 used in our study was the version available at the time we were preparing and conducting our analyses for publication. Therefore, our methodology was in accordance with best practices available at that time. Davydov et al. pointed out that our evaluation of immune repertoire extraction tools focused solely on quantitative metrics, such as the number of reads containing CDR3 sequences and the number of reported clonotypes. And in doing so, we overlooked the crucial issue of false positive clonotypes, which can originate from regions unrelated to immune receptor genes and have a significant impact on downstream analyses and interpretations. We appreciate Davydov et al. group’s point regarding the potential impact of reads originating from genome regions unrelated to immune receptor genes (Variable (V), Diversity (D), Joining (J), or Constant (C)). The group led by Davydov et al. pointed out that one of the top clonotypes identified by TRUST from the PRJNA812076 samples—specifically, the CDR3 with the amino acid sequence CANTGELFF—is a false positive. However, after careful consideration, we respectfully disagree with the suggestion that our evaluation methodology is incomplete without assessing these aspects. Our focus on quantitative measures, for example the fractions of repertoire captured by RNA-Seq based on clonotypes confirmed by T cell receptor sequencing (TCR-Seq), was a deliberate choice. It’s important to note, that this particular clonotype (CANTGELFF) does not appear in our gold standard TCR-Seq sample. As such, we did not classify it as a true positive in our analysis, thereby eliminating the concern of reporting this false positive clonotype in our results. However, we did acknowledge the limitations of using TCR-Seq as a gold standard in our original publication. We noted instances where TCR clonotypes were identified through RNA-Seq-based techniques but were not detected using TCR-Seq. These discrepancies could either be false positives arising from RNA-Seq methods or false negatives from TCR-Seq—meaning actual TCR clonotypes missed by the TCR-Seq approach. Unfortunately, our benchmarking strategy lacks the capability to differentiate between these two scenarios. While we understand the concern about the possibility of false positives affecting the downstream analysis, we believe our current approach already addresses these issues adequately, and adding more layers of evaluation might dilute the focus of our research. We hope this clarifies our methodology but still appreciate the specific input on the concerns on the false positives. Davydov et al. pointed out that we excluded singletons (clones supported by only one read) from our analysis, noting that the implications of this filtering approach on the overall quality of the results remain ambiguous, particularly in light of the significant presence of false positives. While Davydov et al. advocate for retaining singletons in the analysis, we have a different stance on the matter. Given that TCR-Seq cannot be considered an infallible gold standard, we find it essential to implement specific criteria to bring it closer to serving as an accurate ground truth. Consequently, we identify singletons as likely false positives and opt to exclude them. The procedure of removing singletons serves as a method to enhance the reliability of the results derived from the gold standard. We used default settings for all TCR profiling tools to better reflect how these tools are commonly used in real-world scenarios. While we acknowledged a potential bug in TRUST4 version 1.0.2, we want to emphasize that we diligently adhered to the best practices available during the study. Our deliberate emphasis was on quantitative measures, specifically the fractions of repertoire captured by RNA-Seq, with TCR-Seq serving as our gold standard. To enhance the reliability of our results, we intentionally excluded singletons, which differs from Davydov et al.’s approach. This exclusion contributes to making TCR-Seq a more accurate gold standard for benchmarking purposes.

Ce résumé expose les affirmations des auteurs. BNTIC ne l’interprète pas comme une validation indépendante des résultats.