SegmentationTool

Segmentation Tools: Cellpose, StarDist, and Classic

Omega includes a specialized CellAndNucleiSegmentationTool that supports three segmentation algorithms. You can ask Omega to segment cells or nuclei by simply describing what you want.

---

Available Algorithms

Cellpose

Cellpose is a deep learning-based generalist model for cell and nucleus segmentation. It works well on a wide variety of cell types and imaging modalities.

Models:

• cyto / cyto2 / cyto3 -- Cytoplasm (whole cell) segmentation
• nuclei -- Nucleus segmentation

Key parameters:

• model_type -- Which Cellpose model to use
• diameter -- Estimated cell diameter in pixels (auto-estimated if not provided)
• channel -- Channel specification for multi-channel images

Example prompts:

• "Segment the nuclei using Cellpose"
• "Use Cellpose cyto3 model to segment the cells with diameter 30"

Note: Cellpose uses CellposeModel from the cellpose package (v4.x API).

StarDist

StarDist detects cells using star-convex polygon shapes. It is particularly effective for round or convex nuclei.

Models:

• versatile_fluo -- Trained on a broad range of fluorescent images
• versatile_he -- Trained on H&E stained tissue (may generalize to other staining modalities)

Key parameters:

• model_type -- Which StarDist model to use
• scale -- Scaling factor for images with different resolution than the training data

Example prompts:

• "Segment nuclei using StarDist"
• "Use StarDist versatile_fluo model on this image"

Classic

The Classic segmentation method uses traditional image processing techniques (Otsu thresholding + optional watershed) and requires no additional dependencies.

Key parameters:

• min_distance -- Minimum pixel distance between peaks for watershed
• erosion_steps / closing_steps / opening_steps -- Morphological operations
• apply_watershed -- Whether to separate touching cells using the watershed algorithm

Example prompts:

• "Segment the image using the classic method"
• "Use classic segmentation with watershed"

---

Installation

Auto-Install

When you ask Omega to segment using Cellpose or StarDist for the first time, Omega will automatically install the required package using pip. No manual installation is required.

Manual Installation (Optional)

If you prefer to install the packages manually:

# Cellpose
pip install cellpose

# StarDist
pip install stardist

GPU Support

For GPU-accelerated segmentation:

• Cellpose: Install the GPU version of PyTorch. See Cellpose installation guide.
• StarDist: Requires TensorFlow with GPU support. If you encounter "libdevice not found" errors, install:

  pip install nvidia-cuda-nvcc-cu12

  This provides the libdevice.10.bc file that TensorFlow/XLA needs.

---

Common Parameters (All Algorithms)

Parameter	Description	Default
normalize	Normalize image intensity	True
norm_range_low	Lower percentile for normalization	auto
norm_range_high	Upper percentile for normalization	auto
min_segment_size	Remove segments smaller than this (pixels)	auto

---

2D and 3D Support

All three segmentation functions support both 2D and 3D images. Although Cellpose and StarDist were originally designed for 2D images, Omega's wrapper functions handle 3D images by processing them slice-by-slice or using built-in 3D support where available.

---

Tips

• If you are unsure which algorithm to use, try Cellpose first -- it is the most versatile.
• For round nuclei in fluorescence images, StarDist often gives the best results.
• For quick segmentation without deep learning dependencies, use the Classic method.
• Results are automatically added to the viewer as a labels layer named "segmented."