CellProfiler Integration

CellProfiler is the workhorse of this pipeline. It’s an open-source tool designed specifically for high-throughput image analysis in biology. If you’re new to CellProfiler, think of it as a flexible image processing engine that you configure using CellProfiler pipeline files (.cppipe).

This page explains how the Nextflow pipeline uses CellProfiler and how to customize the integration for your own data.

Overview¶

The pipeline uses CellProfiler v4.2.8 for image analysis tasks:

• Illumination correction calculation and application

• Image preprocessing (such as masking debris or confluent areas or balancing channels for barcode calling)

• Cell segmentation

• Feature extraction

• Barcode calling (optional, can be done outside of the Nextflow pipeline)

CellProfiler Processes¶

Process Modules¶

All CellProfiler processes follow a similar pattern:

process CELLPROFILER_ILLUMCALC {
    container 'wave.seqera.io/cellprofiler/cellprofiler:4.2.8'

    input:
    tuple val(meta), path(images)
    path(cppipe)

    script:
    """
    # Generate load_data.csv
    generate_load_data_csv.py \\
        --metadata-json metadata.json \\
        --images-dir ./images \\
        --output load_data.csv

    # Run CellProfiler
    cellprofiler \\
        -c -r \\
        -p ${cppipe} \\
        -o . \\
        --data-file=load_data.csv
    """
}

Key Processes¶

Container Flavors¶

SEGCHECK and COMBINEDANALYSIS support swapping in an alternate CellProfiler image that bundles extra dependencies for alternative segmentation methods (e.g. Cellpose), controlled by two params:

• --cellprofiler_flavor {default,cellpose2,cellpose3} — selects an image from the cellprofiler_flavor_containers map defined in nextflow.config. Applies to both SEGCHECK and COMBINEDANALYSIS together for a given run.

• --cellprofiler_container_override <image> — bypasses cellprofiler_flavor entirely and uses the given image directly for both processes.

To ensure that the container has the appropriate version of CellProfiler-plugins, run with --update_cellprofiler_plugins true for the current version of the CellProfiler-plugins repository or point to a specific RunCellpose plugin commit with --runcellpose_plugin.

To add a new flavor (e.g. a different Cellpose version, or another plugin bundle):

1. Add a name: image entry to params.cellprofiler_flavor_containers in nextflow.config.

2. Add the name to the cellprofiler_flavor enum in nextflow_schema.json, then run pixi run build-schema.

Pipeline Files (.cppipe)¶

Structure¶

CellProfiler pipelines are text files (.cppipe) files defining:

• Input modules (LoadData, LoadImages)

• Processing modules (CorrectIlluminationCalculate, ApplyIllumination)

• Output modules (SaveImages, ExportToSpreadsheet)

Required Pipeline Files¶

Load Data CSV Generation¶

Purpose¶

CellProfiler requires load_data.csv files that specify:

• Image file paths

• Metadata (plate, well, site, frame, channel, cycle)

• Image grouping for processing

Generation Script¶

The generate_load_data_csv.py script creates these files:

generate_load_data_csv.py \
    --metadata-json metadata.json \
    --images-dir ./images \
    --output load_data.csv

Pipeline Types¶

Different CellProfiler stages require different CSV formats:

1. illumcalc - Illumination Calculation¶

2. illumapply - Illumination Correction¶

3. preprocess - Barcoding with Cycles¶

4. combined - Painting + Barcoding¶

Execution Details¶

Command-Line Invocation¶

CellProfiler is run in headless mode:

cellprofiler \
    -c \                    # Run without GUI
    -r \                    # Run pipeline
    -p pipeline.cppipe \    # Pipeline file
    -o output_dir/ \        # Output directory
    --data-file=load_data.csv  # Input CSV

Plugin Loading¶

For processes requiring plugins, Nextflow stages plugins into the process based on the plugin path provided. Default plugins are loaded from CellProfiler plugins via raw github links but local or other sources for the plugins can be specified if required.

# Stage plugins in nf-pooled-cellpainting.nf
file(params.callbarcodes_plugin)

# Stage plugins as input into process
path plugins, stageAs: "plugins/"

# use plugins in process with cellprofiler
cellprofiler -c -r \\
    -p combinedanalysis_patched.cppipe \\
    -o . \\
    --data-file=load_data.csv \\
    --image-directory ./images/ \\
    --plugins-directory=./plugins/

Default URLs are configured in nextflow.config:

callbarcodes_plugin = "https://raw.githubusercontent.com/CellProfiler/CellProfiler-plugins/ad3d9c031b97a31f6300372baa378f75ecb44426/active_plugins/callbarcodes.py"
compensatecolors_plugin = "https://raw.githubusercontent.com/CellProfiler/CellProfiler-plugins/da969556d1d5095e5737601f483c7d55da374f75/active_plugins/compensatecolors.py"
runcellpose_plugin = "https://raw.githubusercontent.com/CellProfiler/CellProfiler-plugins/0f1de5696e933f417f50a48eb40233dc490f98a0/active_plugins/runcellpose.py"

runcellpose_plugin is only staged into SEGCHECK/COMBINEDANALYSIS when cellprofiler_flavor is not default (i.e. a Cellpose-enabled container is in use) - the default flavor’s image doesn’t have Cellpose installed, so the plugin would fail to import there.

Output Organization¶

File Naming Conventions¶

CellProfiler outputs follow structured naming:

CSV Outputs¶

Feature measurements are saved as CSV:

• Image.csv: Per-image measurements

• Nuclei.csv: Per-nucleus measurements

• Cells.csv: Per-cell measurements

• Experiment.csv: Pipeline metadata

Best Practices¶

1. Validate pipelines: Test .cppipe files in CellProfiler GUI first

2. Test on subset data first: Test your dataset on 1 well first, since well is the smallest unit we can use for the pipeline. If the pipeline works on 1 well, it should work on a full plate.

3. Resource tuning: Profile memory usage and adjust allocation (to save time and cost)

4. Plugin versioning: Pin plugin versions for reproducibility

5. Output validation: Check output file counts match expectations