CellProfiler Integration

CellProfiler is the workhorse of this pipeline. It’s an open-source tool designed specifically for high-throughput image analysis in biology. If you’re new to CellProfiler, think of it as a flexible image processing engine that you configure using “pipeline” files (.cppipe).

This page explains how the pipeline uses CellProfiler and how to customize the integration for your own data.

Overview¶

The pipeline uses CellProfiler v4.2.8 for image analysis tasks:

• Illumination correction calculation and application

• Image preprocessing

• Cell segmentation

• Feature extraction

• Barcode calling

CellProfiler Processes¶

Process Modules¶

All CellProfiler processes follow a similar pattern:

process CELLPROFILER_ILLUMCALC {
    container 'wave.seqera.io/cellprofiler/cellprofiler:4.2.8'

    input:
    tuple val(meta), path(images)
    path(cppipe)

    script:
    """
    # Generate load_data.csv
    generate_load_data_csv.py \\
        --pipeline_type illumcalc \\
        --channels ${meta.channels.join(',')} \\
        --output load_data.csv

    # Run CellProfiler
    cellprofiler \\
        -c -r \\
        -p ${cppipe} \\
        -o . \\
        --data-file=load_data.csv
    """
}

Key Processes¶

Pipeline Files (.cppipe)¶

Structure¶

CellProfiler pipelines are text files (.cppipe) files defining:

• Input modules (LoadData, LoadImages)

• Processing modules (CorrectIlluminationCalculate, ApplyIllumination)

• Output modules (SaveImages, ExportToSpreadsheet)

Required Pipeline Files¶

Load Data CSV Generation¶

Purpose¶

CellProfiler requires load_data.csv files that specify:

• Image file paths

• Metadata (plate, well, site, frame, channel, cycle)

• Image grouping for processing

Generation Script¶

The generate_load_data_csv.py script creates these files:

generate_load_data_csv.py \
    --pipeline_type illumcalc \
    --channels DAPI,GFP,RFP,Cy5,Cy3 \
    --frames 0,1,2,3 \
    --output load_data.csv

Pipeline Types¶

Different CellProfiler stages require different CSV formats:

1. illumcalc - Illumination Calculation¶

2. illumapply - Illumination Correction¶

3. preprocess - Barcoding with Cycles¶

4. combined - Painting + Barcoding¶

Execution Details¶

Command-Line Invocation¶

CellProfiler is run in headless mode:

cellprofiler \
    -c \                    # Run without GUI
    -r \                    # Run pipeline
    -p pipeline.cppipe \    # Pipeline file
    -o output_dir/ \        # Output directory
    --data-file=load_data.csv  # Input CSV

Plugin Loading¶

For processes requiring plugins, Nextflow stages plugins into the process based on the plugin path provided. Default plugins are loaded from https://github.com/CellProfiler/CellProfiler-plugins via raw github links but local or other sources for the plugins can be specified if required.

# Stage plugins in nf-pooled-cellpainting.nf
file(params.callbarcodes_plugin)

# Stage plugins as input into process
path plugins, stageAs: "plugins/"

# use plugins in process with cellprofiler
cellprofiler -c -r \\
    -p combinedanalysis_patched.cppipe \\
    -o . \\
    --data-file=load_data.csv \\
    --image-directory ./images/ \\
    --plugins-directory=./plugins/

Default URLs are configured in nextflow.config:

callbarcodes_plugin = "https://raw.githubusercontent.com/CellProfiler/CellProfiler-plugins/refs/heads/master/active_plugins/callbarcodes.py"
compensatecolors_plugin = "https://raw.githubusercontent.com/CellProfiler/CellProfiler-plugins/refs/heads/master/active_plugins/compensatecolors.py"

Output Organization¶

File Naming Conventions¶

CellProfiler outputs follow structured naming:

CSV Outputs¶

Feature measurements are saved as CSV:

• Image.csv: Per-image measurements

• Nuclei.csv: Per-nucleus measurements

• Cells.csv: Per-cell measurements

• Experiment.csv: Pipeline metadata

Best Practices¶

1. Validate pipelines: Test .cppipe files in CellProfiler GUI first

2. Test on subset data first: Test your dataset on 1 well first, since well is the smallest unit we can use for the pipeline. If the pipeline works on 1 well, it should work on a full plate.

3. Resource tuning: Profile memory usage and adjust allocation (to save time and cost)

4. Plugin versioning: Pin plugin versions for reproducibility

5. Output validation: Check output file counts match expectations