gnomad.utils.constraint.grouping

Functions to group variants for constraint.

Covers consequence groups, VEP grouping fields, and genetic ancestry downsamplings.

gnomad.utils.constraint.grouping.build_constraint_consequence_groups(...)

Build constraint consequence groups.

gnomad.utils.constraint.grouping.explode_downsamplings_oe(ht, ...)

Explode observed and expected downsampling counts for each genetic ancestry group and metric.

gnomad.utils.constraint.grouping.get_all_gen_anc_lengths(ht, ...)

Get the minimum length of observed variant counts array for each genetic ancestry group downsampling.

gnomad.utils.constraint.grouping.get_constraint_grouping_expr(...)

Collect annotations used for constraint groupings.

gnomad.utils.constraint.grouping.annotate_exploded_vep_for_constraint_groupings(ht)

Explode a VEP annotation and add constraint grouping fields.

Functions to group variants for constraint.

Covers consequence groups, VEP grouping fields, and genetic ancestry downsamplings.

gnomad.utils.constraint.grouping.build_constraint_consequence_groups(csq_expr, lof_modifier_expr, classic_lof_annotations=('stop_gained', 'splice_donor_variant', 'splice_acceptor_variant'), additional_groupings=None, additional_grouping_combinations=None)[source]

Build constraint consequence groups.

Builds constraint groups based on the consequence expression and LoF modifier expression. By default, the following groups are built:

  • csq_set: synonymous_variant, missense_variant

  • lof: classic, hc_lc, hc

The resulting meta and corresponding constraint group filters are:

  • {“csq_set”: “syn”}: synonymous_variant

  • {“csq_set”: “mis”}: missense_variant

  • {“lof”: “classic”}: classic LoF annotations (stop_gained, splice_donor_variant, splice_acceptor_variant)

  • {“lof”: “hc_lc”}: LOFTEE HC or LC modifier

  • {“lof”: “hc”}: LOFTEE HC modifier only

Additional groupings can be added via additional_groupings, and grouping combinations via additional_grouping_combinations.

Note

A combination of more than one grouping builds the union of the combined groups, not their intersection. For example, with an additional_groupings entry {"am": {"am_per_99": ...}}, the combination ["lof", "am"] produces groups such as {"lof": "hc", "am": "am_per_99"} that contain variants that are HC LoF or in the top 1% of AlphaMissense scores. This differs from frequency metadata (e.g. freq_meta), where multiple keys mean all of them apply.

Parameters:
  • csq_expr (StringExpression) – VEP most severe consequence expression (e.g., ht.most_severe_consequence).

  • lof_modifier_expr (StringExpression) – LOFTEE modifier expression (e.g., ht.modifier).

  • classic_lof_annotations (Tuple[str, ...]) – Classic LoF annotations. Default is ("stop_gained", "splice_donor_variant", "splice_acceptor_variant").

  • additional_groupings (Optional[Dict[str, Dict[str, BooleanExpression]]]) – Additional groupings to add to the constraint groups. Default is None.

  • additional_grouping_combinations (Optional[List[List[str]]]) – Additional grouping combinations to add to the constraint groups. Groupings combined in one entry are joined as a union (see the note above). Default is None.

Return type:

Tuple[List[BooleanExpression], List[Dict[str, str]]]

Returns:

Tuple of (constraint group filter expressions, meta dicts).

gnomad.utils.constraint.grouping.explode_downsamplings_oe(ht, downsampling_meta, metrics=['syn', 'lof', 'mis'])[source]

Explode observed and expected downsampling counts for each genetic ancestry group and metric.

The input ht must contain struct of downsampling information for genetic ancestry groups under each metric name. For example: ‘lof’: struct {gen_anc_exp: struct {global: array<float64>}.

Parameters:
  • ht (Table) – Input Table.

  • metrics (List[str]) – List of metrics to explode. Default is ‘[‘syn’, ‘lof’, ‘mis’]’.

  • downsampling_meta (Dict[str, List[str]]) – Dictionary containing downsampling metadata. Keys are the genetic ancestry group names and values are the list of downsamplings for that genetic ancestry group. Example: {‘global’: [‘5000’, ‘10000’], ‘afr’: [‘5000’, ‘10000’]}.

Return type:

Table

Returns:

Table with downsampling counts exploded so that observed and expected metric counts for each pair of genetic ancestry groups and downsamplings is a separate row.

gnomad.utils.constraint.grouping.get_all_gen_anc_lengths(ht, gen_ancs, obs_expr)[source]

Get the minimum length of observed variant counts array for each genetic ancestry group downsampling.

The observed variant counts for each genetic ancestry group in gen_ancs are specified by annotations on the obs_expr expression.

The function also performs a check that arrays of variant counts within genetic ancestry group downsamplings all have the same lengths.

Parameters:
  • ht (Table) – Input Table containing obs_expr.

  • gen_ancs (Tuple[str, ...]) – Genetic ancestry groups used to categorize observed variant counts in downsamplings.

  • obs_expr (StructExpression) – Expression for the genetic ancestry group observed variant counts. Should be a struct containing an array for each gen_anc in gen_ancs.

Return type:

List[Tuple[int, str]]

Returns:

List of (length, gen_anc) tuples, one per group in gen_ancs, where length is the minimum array length for that group.

gnomad.utils.constraint.grouping.get_constraint_grouping_expr(vep_annotation_expr, coverage_expr=None, include_transcript_group=True, include_canonical_group=True, include_mane_select_group=False)[source]

Collect annotations used for constraint groupings.

Function collects the following annotations:

  • annotation - most_severe_consequence from vep_annotation_expr

  • modifier - first non-missing of lof or polyphen_prediction from vep_annotation_expr, or the literal “None”

  • gene - gene_symbol from vep_annotation_expr

  • gene_id - gene_id from vep_annotation_expr

  • coverage - exome coverage if coverage_expr is specified

  • transcript - transcript_id from vep_annotation_expr (added when include_transcript_group is True)

  • canonical - from vep_annotation_expr (added when include_canonical_group is True)

  • mane_select - from vep_annotation_expr (added when include_mane_select_group is True)

Note

This function expects that the following fields are present in vep_annotation_expr:

  • lof

  • most_severe_consequence

  • gene_symbol

  • gene_id

  • polyphen_prediction (optional; missing used if absent)

  • transcript_id (if include_transcript_group is True)

  • canonical (if include_canonical_group is True)

  • mane_select (if include_mane_select_group is True)

Parameters:
  • vep_annotation_expr (StructExpression) – StructExpression of VEP annotation.

  • coverage_expr (Optional[Int32Expression]) – Int32Expression of exome coverage. Default is None.

  • include_transcript_group (bool) – Whether to include the transcript annotation in the groupings. Default is True.

  • include_canonical_group (bool) – Whether to include canonical annotation in the groupings. Default is True.

  • include_mane_select_group (bool) – Whether to include mane_select annotation in the groupings. Default is False.

Return type:

Dict[str, Union[StringExpression, Int32Expression, BooleanExpression]]

Returns:

Dict mapping annotation names to Hail expressions.

gnomad.utils.constraint.grouping.annotate_exploded_vep_for_constraint_groupings(ht, coverage_expr=None, vep_annotation='transcript_consequences', include_canonical_group=True, include_mane_select_group=False)[source]

Explode a VEP annotation and add constraint grouping fields.

Explodes the specified VEP annotation (vep_annotation) and adds the following annotations via get_constraint_grouping_expr:

  • annotation - most_severe_consequence from vep_annotation

  • modifier - first non-missing of lof or polyphen_prediction from vep_annotation, or the literal “None”

  • gene - gene_symbol from vep_annotation

  • gene_id - gene_id from vep_annotation

  • coverage - exome coverage if coverage_expr is specified

  • transcript - transcript_id from vep_annotation (added when vep_annotation is “transcript_consequences”)

  • canonical - from vep_annotation (added when include_canonical_group is True)

  • mane_select - from vep_annotation (added when include_mane_select_group is True)

Note

This function expects that a vep annotation is present in ht.

Note

Only the grouping fields are kept from the VEP annotation. The returned Table no longer has the vep annotation or the exploded vep_annotation struct, which keeps per-variant tables built from it small. All other fields of ht are kept.

Parameters:
  • ht (Table) – Input Table.

  • coverage_expr (Optional[Int32Expression]) – Expression that defines the coverage metric. Default is None.

  • vep_annotation (str) – Name of annotation in vep (one of "transcript_consequences" and "worst_csq_by_gene") that will be used for obtaining constraint annotations. Default is "transcript_consequences".

  • include_canonical_group (bool) – Whether to include canonical annotation in the groupings. Default is True. Ignored unless vep_annotation is "transcript_consequences".

  • include_mane_select_group (bool) – Whether to include mane_select annotation in the groupings. Default is False. Ignored unless vep_annotation is "transcript_consequences".

Return type:

Tuple[Table, Tuple[str, ...]]

Returns:

Tuple of (Table with one row per element of vep_annotation and the grouping fields added in place of the VEP annotation, names of the added grouping fields).