gnomad_qc.v5.resources.meta

Script containing metadata related resources.

Module Functions

`gnomad_qc.v5.resources.meta.get_project_meta`([...])	Get the VersionedTableResource for per-sample project-level metadata.
`gnomad_qc.v5.resources.meta.get_sample_id_collisions`([...])	Get the TableResource for sample IDs that collide between AoU and gnomAD v4.
`gnomad_qc.v5.resources.meta.aou_sample_artifact_path`(name)	Return the permanent path to a run-invariant AoU sample-level artifact.
`gnomad_qc.v5.resources.meta.load_aou_sample_artifact_json`(name)	Return the parsed JSON sample artifact, or None when it has not been written.
`gnomad_qc.v5.resources.meta.write_aou_vds_sample_jsons`([...])	Write the permanent sample artifact JSONs used to load the AoU VDS.
`gnomad_qc.v5.resources.meta.get_low_quality_samples`([...])	Get the ExpressionResource for AoU-flagged low-quality sample IDs.
`gnomad_qc.v5.resources.meta.get_failing_metrics_samples`([...])	Get the ExpressionResource for samples failing AoU genomic QC metrics.
`gnomad_qc.v5.resources.meta.get_samples_to_exclude_resource`([...])	Get the ExpressionResource for the combined set of samples to exclude.
`gnomad_qc.v5.resources.meta.get_consent_samples_to_drop`([...])	Get the TableResource for consent-withdrawn samples.
`gnomad_qc.v5.resources.meta.meta`([version, ...])	Get the v5 sample QC meta VersionedTableResource.

Script containing metadata related resources.

gnomad_qc.v5.resources.meta.get_project_meta(environment='batch')[source]

Get the VersionedTableResource for per-sample project-level metadata.

Parameters:: environment (str) – Environment to use. Default is “batch”. Must be one of “rwb” or “batch”.
Return type:: VersionedTableResource
Returns:: VersionedTableResource for project metadata.

gnomad_qc.v5.resources.meta.get_sample_id_collisions(environment='batch')[source]

Get the TableResource for sample IDs that collide between AoU and gnomAD v4.

Parameters:: environment (str) – Environment to use. Default is “batch”. Must be one of “rwb” or “batch”.
Return type:: TableResource
Returns:: TableResource of sample ID collisions.

gnomad_qc.v5.resources.meta.aou_sample_artifact_path(name, environment='batch', version='5.0')[source]

Return the permanent path to a run-invariant AoU sample-level artifact.

Artifacts are small precomputed files read at VDS load (or by pipeline workers) instead of rescanning the ~330-partition sample metadata tables: the VDS-loading JSONs written by write_aou_vds_sample_jsons (sample_id_collisions.json, high_quality_samples.json, release_samples.json).

Unlike _meta_root_path, this resolves to the writable bucket in the “batch” environment: artifacts are both written and read from batch.

Parameters:

name (str) – Artifact file name (with extension).
environment (str) – Environment to use. Default is “batch”. Must be one of “rwb” or “batch”.
version (str) – gnomAD version.

Return type:

str

Returns:

GCS path to the artifact.

gnomad_qc.v5.resources.meta.load_aou_sample_artifact_json(name, environment='batch')[source]

Return the parsed JSON sample artifact, or None when it has not been written.

Parameters:

name (str) – Artifact file name (with extension).
environment (str) – Environment to use. Default is “batch”. Must be one of “rwb” or “batch”.

Return type:

Optional[list]

Returns:

Parsed JSON value, or None if the file is absent.

gnomad_qc.v5.resources.meta.write_aou_vds_sample_jsons(environment='batch', overwrite=False)[source]

Write the permanent sample artifact JSONs used to load the AoU VDS.

Writes three artifacts so that VDS loads read small files instead of each rescanning the ~330-partition sample metadata tables:

sample_id_collisions.json: pre-prefix sample IDs that collide with gnomAD samples (from the sample-collisions Table).
high_quality_samples.json: post-prefix high-quality sample IDs, collected with the exact meta_ht.high_quality filter that get_aou_vds(high_quality_only=True) applies, so the two cannot drift.
release_samples.json: post-prefix release sample IDs, collected with the exact meta_ht.release filter that get_aou_vds(release_only=True) applies.

get_aou_vds prefers these files and falls back to the original scans for any that are absent.

Parameters:

environment (str) – Environment to use. Default is “batch”. Must be one of “rwb” or “batch”.
overwrite (bool) – Whether to replace existing artifacts; without it, a rerun fails instead of silently rewriting them. Default is False.

Return type:

None

gnomad_qc.v5.resources.meta.get_low_quality_samples(environment='batch')[source]

Get the ExpressionResource for AoU-flagged low-quality sample IDs.

SetExpression containing IDs of 3 samples with an unspecified data quality issue.

For more information, see Known Issue #1 in the AoU QC document: https://support.researchallofus.org/hc/en-us/articles/29390274413716-All-of-Us-Genomic-Quality-Report.

Parameters:: environment (str) – Environment to use. Default is “batch”. Must be one of “rwb” or “batch”.
Return type:: ExpressionResource
Returns:: ExpressionResource of low-quality sample IDs.

gnomad_qc.v5.resources.meta.get_failing_metrics_samples(environment='batch')[source]

Get the ExpressionResource for samples failing AoU genomic QC metrics.

SetExpression containing IDs of 4030 samples failing coverage hard filters and 1490 samples with non-XX/XY sex ploidies.

For more information about samples failing coverage hard filters, see docstring of get_aou_failing_genomic_metrics_samples.

Parameters:: environment (str) – Environment to use. Default is “batch”. Must be one of “rwb” or “batch”.
Return type:: ExpressionResource
Returns:: ExpressionResource of failing-metrics sample IDs.

gnomad_qc.v5.resources.meta.get_samples_to_exclude_resource(environment='batch')[source]

Get the ExpressionResource for the combined set of samples to exclude.

SetExpression containing IDs of 5514 samples to exclude from v5 analysis.

Contains samples that should not have been included in the AoU v8 release (3 samples with unspecified quality issues and 4030 samples failing coverage hard filters) and 1490 samples with non-XX/XY sex ploidies.

The total number of samples to exclude is 5514, not 5523 because 9 samples both fail coverage filters and have non-XX/XY sex ploidies.

Parameters:: environment (str) – Environment to use. Default is “batch”. Must be one of “rwb” or “batch”.
Return type:: ExpressionResource
Returns:: ExpressionResource of sample IDs to exclude.

gnomad_qc.v5.resources.meta.get_consent_samples_to_drop(environment='batch')[source]

Get the TableResource for consent-withdrawn samples.

Table containing IDs of 897 samples that are no longer consented to be in gnomAD.

Samples are from the following projects: - RP-1061: 776 samples. - RP-1411: 121 samples.

Parameters:: environment (str) – Environment to use. Default is “batch”. Must be one of “rwb” or “batch”.
Return type:: TableResource
Returns:: TableResource of consent-withdrawn sample IDs.

gnomad_qc.v5.resources.meta.meta(version='5.0', data_type='genomes', environment='batch')[source]

Get the v5 sample QC meta VersionedTableResource.

Note

Exome data is not currently supported in this function. The v4 sample QC meta uses a different structure, so this function does not pull or duplicate that data. If exome data are needed, please use the v4 resource directly.

Parameters:

version (str) – Sample QC version.
data_type (str) – Data type. Default is “genomes”. If “exomes” is supplied, a warning will be raised suggesting the use of v4 sample QC metadata.
environment (str) – Environment to use. Default is “batch”. Must be one of “rwb” or “batch”.

Return type:

VersionedTableResource

Returns:

Sample QC meta VersionedTableResource.