How a criterion aggregates an annotation's scores to gate the suite:
"average" — gate on overall quality: the mean score across all runs
must clear the criterion's threshold. A few weak runs are tolerated as
long as the mean holds.
"passRate" — gate on consistency: each run passes when the
criterion's passFn predicate returns true for its annotation, and the
suite passes when the fraction of runs that pass is at least
minPassRate (e.g. minPassRate: 0.9 ⇒ 90% must pass; 1 ⇒ all).
How a criterion aggregates an annotation's scores to gate the suite:
"average"— gate on overall quality: the mean score across all runs must clear the criterion'sthreshold. A few weak runs are tolerated as long as the mean holds."passRate"— gate on consistency: each run passes when the criterion'spassFnpredicate returnstruefor its annotation, and the suite passes when the fraction of runs that pass is at leastminPassRate(e.g.minPassRate: 0.9⇒ 90% must pass;1⇒ all).