Building reference panels

A custom reference panel for the single-sample mode can be generated most easily using the GATKSVPipelineBatch workflow. This must be run on a standalone Cromwell server, as the workflow is unstable on Terra.

note

Reference panels can also be generated by running the pipeline through joint calling on Terra, but there is currently no solution for automatically updating inputs.

We recommend copying the outputs from a Cromwell run to a permanent location by adding the following option to the workflow configuration file:

  "final_workflow_outputs_dir" : "gs://my-outputs-bucket",
  "use_relative_output_paths": false,

Here is an example of how to generate workflow input jsons from GATKSVPipelineBatch workflow metadata:

1. Get metadata from Cromwshell.

   cromshell -t60 metadata 38c65ca4-2a07-4805-86b6-214696075fef > metadata.json

2. Run the script.

   python scripts/inputs/create_test_batch.py \
      --execution-bucket gs://my-exec-bucket \
      --final-workflow-outputs-dir gs://my-outputs-bucket \
      metadata.json \
      > inputs/values/my_ref_panel.json

3. Build test files for batched workflows (google cloud project id required).

   python scripts/inputs/build_inputs.py \
      inputs/values \
      inputs/templates/test \
      inputs/build/my_ref_panel/test \
      -a '{ "test_batch" : "ref_panel_1kg" }'

4. Build test files for the single-sample workflow

   python scripts/inputs/build_inputs.py \
       inputs/values \
       inputs/templates/test/GATKSVPipelineSingleSample \
       inputs/build/NA19240/test_my_ref_panel \
       -a '{ "single_sample" : "test_single_sample_NA19240", "ref_panel" : "my_ref_panel" }'

5. Build files for a Terra workspace.

   python scripts/inputs/build_inputs.py \
      inputs/values \
      inputs/templates/terra_workspaces/single_sample \
      inputs/build/NA12878/terra_my_ref_panel \
      -a '{ "single_sample" : "test_single_sample_NA12878", "ref_panel" : "my_ref_panel" }'

Note that the inputs to GATKSVPipelineBatch may be used as resources for the reference panel and therefore should also be in a permanent location.