ComfyUI Canny ControlNet: Edge-Based Image Control Guide

What is Canny ControlNet?

Canny ControlNet uses the Canny edge detection algorithm to extract outlines from an image, then uses those outlines to guide AI generation. The result is an image that follows the structural contours of your reference while applying a completely different style, subject, or mood.

It's the most popular ControlNet type because it works well for nearly any subject — architecture, characters, products, landscapes.

Prerequisites

Models

File Placement

ComfyUI/
├── models/
│   ├── checkpoints/
│   │   └── dreamshaper_8.safetensors
│   ├── controlnet/
│   │   └── control_v11p_sd15_canny.pth
│   └── vae/
│       └── vae-ft-mse-840000-ema-pruned.safetensors

Building the Workflow

Node Setup

1. Load Image — load your reference photo
2. Canny — extracts edge lines from the image
3. Preview Image — (optional) preview the edge detection result before generating
4. Load Checkpoint — loads the SD1.5 model
5. Load ControlNet Model — loads control_v11p_sd15_canny.pth
6. Apply ControlNet (Advanced) — connects the Canny output to your prompt conditioning
7. CLIP Text Encode (x2) — positive and negative prompts
8. Empty Latent Image — sets the output resolution (512x512 for SD1.5)
9. KSampler — generates the image
10. VAE Decode → Save Image

Connections

Load Image → Canny → Apply ControlNet (image input)
Load ControlNet Model → Apply ControlNet (control_net input)
CLIP Text Encode (positive) → Apply ControlNet (positive input)
CLIP Text Encode (negative) → Apply ControlNet (negative input)
Apply ControlNet outputs → KSampler conditioning inputs

Key Parameters

Canny Node

Apply ControlNet (Advanced)

KSampler

Tips for Better Results

Prompt alignment — Write prompts that relate to the content of your reference image. If the reference is a building, prompt for architecture. Conflicting prompts and references produce poor results.

Threshold tuning — For detailed subjects (jewelry, machinery), use lower thresholds (0.2/0.5) to capture fine detail. For simple subjects (portraits, landscapes), use higher thresholds (0.4/0.7) for cleaner edges.

Strength adjustment — Start at 1.0, then lower to 0.7–0.8 if the result feels too constrained. Lower strength lets the AI interpret the edges more freely.

Common Issues and Fixes

Output looks like a traced outline with no creativity

• Lower strength to 0.6–0.7
• Set end_percent to 0.7 so the AI adds its own detail in later steps
• Use more descriptive prompts to give the AI room for stylistic interpretation

Too many edges — output is messy

• Increase both low_threshold and high_threshold on the Canny node
• Preview the edge map — it should show clean structure lines, not noise

Important edges are missing

• Lower the threshold values
• Use a higher-resolution input image — blurry photos produce poor edges

Output ignores the edges completely

• Verify you're using the correct ControlNet model (control_v11p_sd15_canny.pth for SD1.5)
• Check that the Apply ControlNet node is connected correctly to both the conditioning and the KSampler

Related Guides

• ControlNet Overview — All ControlNet types explained
• Depth ControlNet — Spatial depth control
• OpenPose ControlNet — Human pose control