gnomad.utils.constraint.counts
Functions to count observed and possible variants and sum them by group.
Return 1 if a variant meets frequency criteria, 0 otherwise. |
|
|
Return per-variant observed and possible count expressions. |
Return the weighted aggregate sum of |
|
Return an aggregation expression for variant and singleton counts. |
|
|
Aggregate observed and possible variant counts by substitution context group. |
|
Get an aggregation expression for the sum of specified fields and expressions. |
Functions to count observed and possible variants and sum them by group.
- gnomad.utils.constraint.counts.variant_observed_expr(freq_expr=None, ht=None, singleton=False, max_af=None, count_missing=False)[source]
Return 1 if a variant meets frequency criteria, 0 otherwise.
One of
htorfreq_exprmust be provided. Iffreq_expris not provided,ht.freqis used. Whenfreq_expris an ArrayExpression, the first element is used.The returned count is 1 when:
singletonis True andfreq_expr.AC == 1.max_afis not None andfreq_expr.AC > 0andfreq_expr.AF <= max_af.Neither
singletonnormax_afis set butfreq_expris available:freq_expr.AC > 0.Neither
singletonnormax_afis set and nofreq_expris available: unconditionally 1.
- Parameters:
ht (
Optional[Table]) – Input Table. Used to look upfreqwhenfreq_expris None.freq_expr (
Union[StructExpression,ArrayExpression,None]) – StructExpression (or ArrayExpression of structs) withACandAFfields. When an array,freq_expr[0]is used.singleton (
bool) – Count only singletons (AC == 1). When True,max_afis ignored. Default is False.max_af (
Optional[float]) – Maximum allele frequency threshold. Default is None (no cutoff).count_missing (
bool) – Whether to count a variant whose count expression is missing (e.g. frequency is None). True counts it as 1, False as 0. Default is False.
- Return type:
- Returns:
Int32Expression equal to 0 or 1.
- gnomad.utils.constraint.counts.variant_observed_and_possible_expr(freq_expr, max_af=None, use_possible_adj=True)[source]
Return per-variant observed and possible count expressions.
For each element of the frequency array,
observed_variantsis 1 when the variant meets the frequency criteria (AC > 0, and AF <=max_afif set) and 0 otherwise.possible_variantsuses the same criteria, except that a missing frequency counts as possible: a variant absent from the dataset could still have been observed. A variant with a defined frequency that fails the criteria (AC == 0, or AF >max_af) is therefore 0 in both, so it is excluded from the denominator as well as the numerator. Withmax_afset, this removes common variants from the possible counts.When
freq_expritself is missing (e.g. the variant is not in the dataset),observed_variantsis missing andpossible_variantsis 1, or missing whenuse_possible_adjis False.When
use_possible_adjis True (default), the possible count is a scalar derived from the first (adj) element of the frequency array. When False, it is an array with one entry per array element, matching the shape ofobserved_variants.- Parameters:
freq_expr (
ArrayExpression) – Array of frequency structs withACandAFfields.max_af (
Optional[float]) – Maximum allele frequency threshold. Variants above it are excluded from both counts. Default is None (no cutoff).use_possible_adj (
bool) – If True, compute possible count from only the adj (first) frequency element. If False, compute per-downsampling. Default is True.
- Return type:
- Returns:
Struct with
observed_variants(array of int) andpossible_variants(int ifuse_possible_adj, else array of int).
- gnomad.utils.constraint.counts.weighted_sum_agg_expr(expr, weight_expr)[source]
Return the weighted aggregate sum of
exprweighted byweight_expr.Both parameters may be scalar or array numeric expressions:
Both arrays: elements are multiplied pairwise and summed per-element across rows (
hl.agg.array_sum).Both scalars: standard
hl.agg.sum(expr * weight_expr).Mixed: the scalar is broadcast across the array elements and summed per-element across rows.
- Parameters:
expr (
Union[ArrayNumericExpression,NumericExpression]) – Numeric expression (scalar or array) to be weighted.weight_expr (
Union[ArrayNumericExpression,NumericExpression]) – Numeric expression (scalar or array) to weight by.
- Return type:
Union[ArrayExpression,NumericExpression]- Returns:
Weighted aggregate sum expression.
- gnomad.utils.constraint.counts.counts_agg_expr(freq_expr=None, ht=None, count_singletons=False, max_af=None, count_missing=False)[source]
Return an aggregation expression for variant and singleton counts.
Aggregates per-variant counts (via
variant_observed_expr) across rows. Each variant contributes 0 or 1 to the count based on its frequency metadata (AC, AF). The result is a struct withvariant_countand, whencount_singletonsis True,singleton_count.One of
freq_exprorhtmust be provided. Iffreq_expris not provided,ht.freqis used as the fallback.The shape of
freq_exprcontrols whether the output counts are scalars or arrays:StructExpression (single frequency entry with
AC/AF): returns scalarvariant_countandsingleton_count.ArrayExpression (array of frequency structs, e.g. one per downsampling): returns array-valued counts where each element corresponds to a position in the input array. Internally the array is mapped through
variant_observed_exprand summed element-wise withhl.agg.array_sum.
When
max_afis set, only variants withAF <= max_afandAC > 0are counted. Singleton counting (count_singletons=True) counts only variants withAC == 1, independent of themax_affilter.- Parameters:
freq_expr (
Union[StructExpression,ArrayExpression,None]) – Frequency expression — an ArrayExpression of structs or a single StructExpression withACandAFfields. If None, falls back toht.freq.ht (
Optional[Table]) – Input Table. Used to look upfreqwhenfreq_expris None.count_singletons (
bool) – If True, include asingleton_countfield in the result. Default is False.max_af (
Optional[float]) – Maximum allele frequency threshold. Variants withAF > max_afare excluded fromvariant_count. Does not affectsingleton_count. Default is None (no cutoff).count_missing (
bool) – Whether to count a variant whose frequency is missing. True counts it as 1, False as 0. Default is False.
- Return type:
- Returns:
Aggregation StructExpression with
variant_count(and optionallysingleton_count). Values are scalars whenfreq_expris a StructExpression, or arrays when it is an ArrayExpression.
- gnomad.utils.constraint.counts.count_observed_and_possible_by_group(ht, possible_expr, observed_expr, additional_grouping=('methylation_level',), partition_hint=100, weight_exprs=None, additional_agg_sum_exprs=None)[source]
Aggregate observed and possible variant counts by substitution context group.
Groups rows by
context,ref,alt, and any fields inadditional_grouping, then sums observed and possible counts within each group.- Parameters:
ht (
Table) – Input Table withcontext,ref,altfields and any fields named inadditional_grouping.possible_expr (
Int32Expression) – Per-variant possible count (scalar).observed_expr (
ArrayExpression) – Per-variant observed count (array, one element per array element of the frequency array).additional_grouping (
Union[List[str],Tuple[str,...]]) – Field names to append to the base(context, ref, alt)grouping. Default is("methylation_level",).partition_hint (
int) – Target number of partitions for thegroup_by. Default is 100.weight_exprs (
Union[List[str],Dict[str,Union[ArrayExpression,NumericExpression]],None]) – Weighted sums ofpossible_exprto include. Pass field names (looked up onht) or a dict mapping output names to weight expressions. Each producesweighted_sum_agg_expr(possible_expr, weight).additional_agg_sum_exprs (
Union[List[str],Dict[str,Union[ArrayExpression,NumericExpression]],None]) – Extra fields to sum alongside observed/possible. Pass field names (looked up onht) or a dict mapping output names to expressions. Arrays usehl.agg.array_sum; scalars usehl.agg.sum.
- Return type:
- Returns:
Grouped Table with
observed_variants,possible_variants, and any weighted/additional sum fields.
- gnomad.utils.constraint.counts.aggregate_constraint_metrics_expr(t, fields_to_sum=('mu_snp', 'mu', 'observed_variants', 'possible_variants', 'predicted_proportion_observed', 'coverage_correction', 'expected_variants'), additional_exprs_to_sum=None)[source]
Get an aggregation expression for the sum of specified fields and expressions.
An aggregate sum or array sum is created for each field in
fields_to_sum.- Parameters:
t (
Union[Table,StructExpression]) – Input Table or StructExpression.fields_to_sum (
Union[List[str],Tuple[str,...]]) – Fields intto get an aggregate sum expression for. Default isDEFAULT_FIELDS_TO_SUM.additional_exprs_to_sum (
Optional[Dict[str,Expression]]) – Dictionary of additional expressions to get an aggregate sum expression for. Field names are the keys and expressions are the values. Default is None.
- Return type:
- Returns:
StructExpression with the sum of each field and expression.