Evaluating the Performance of NGS Pipelines on Noisy WGS Data – Inside DNAnexus


The recent generation of gold-standard genome datasets – such as the Genome in a Bottle (GIAB) and Platinum Genomes – has given greater understanding about the quality of sequencing. Competitions held on PrecisionFDA have driven the innovation and refinement of new methods. SentieonEdico, and DeepVariant, each a winner of one or more of ConsistencyTruth, and Hidden Treasures Challenges, have proven the power of standards to drive innovation.These gold-standard datasets used to construct truth sets and challenges were produced with great care and attention to quality.

We wondered how directly these sets reflect common, real-world sequencing data. “Quality” of sequencing is complex – factors such as library complexitycoveragecontaminationindex hoppingPCR errors, and insert size distributionare all important. For our exploration, we use a quantifiable, unambiguous, and straightforward measure – the sequencer-reported base quality.

FASTQ base qualities estimate the probability that a base call (A, T, G, or C) is in error. These errors make reads more difficult to correctly map and produce noise that variant callers must contend with. Monitoring of base qualities is often a component in QC.

We score each pair of forward and reverse reads with the expected value for the number of errors present in the pair. The combination of these values for each read pair in a sequencing run creates a distribution with a mean and standard deviation for errors.

Source: Evaluating the Performance of NGS Pipelines on Noisy WGS Data – Inside DNAnexus