Pipeline Olympics: continuable benchmarking of computational workflows for DNA methylation sequencing data against an experimental gold-standard

Abstract DNA methylation is a widely studied epigenetic mark and a powerful biomarker of cell type, age, environmental exposures, and disease. Whole-genome sequencing following selective conversion of unmethylated cytosines into thymines via bisulfite treatment or enzymatic methods remains the reference method for DNA methylation profiling genome-wide. While numerous software tools facilitate processing of DNA methylation sequencing reads, a comprehensive benchmarking study has been lacking thus far. In this study, we systematically compared complete computational workflows for processing DNA methylation sequencing data using a dedicated benchmarking dataset generated with five genome-wide profiling protocols. As an evaluation reference, we employed highly quantitative locus-specific measurements from our preceding benchmark of targeted DNA methylation assays. Based on this experimental gold-standard assessment and a number of comprehensive metrics, we ranked the evaluated workflows, identified workflows that consistently demonstrated superior performance, and revealed global workflow development trends. To facilitate the sustainability of our benchmark, we implemented an interactive workflow execution and data presentation platform, adaptable to user-defined criteria and seamlessly expandable to future workflows.