Sign in

Nodes Browser

ComfyDeploy: How Mosaica works in ComfyUI?

What is Mosaica?

Create colorful mosaic images in ComfyUI by computing label images and applying lookup tables.

How to install it in ComfyDeploy?

Head over to the machine page

  1. Click on the "Create a new machine" button
  2. Select the Edit build steps
  3. Add a new step -> Custom Node
  4. Search for Mosaica and select it
  5. Close the build step dialig and then click on the "Save" button to rebuild the machine

🎨 ComfyUI-Mosaica

mosaica-banner

Create colorful mosaic images in ComfyUI by computing label images and applying lookup tables.

Workflow Examples

K-Means

Generate an image using a stable diffusion model and apply the k-means clustering algorithm to convert it to a label image. The average color of each cluster is applied to the image's labels and a colorized image is returned.

K-means is quick and easy to use, but you must specify the number of clusters (i.e. unique labels) that you intend to find.

kmeans-example

Mean Shift

Generate an image using a stable diffusion model and apply the mean shift clustering algorithm to convert it to a label image. The average color of each cluster is applied to the image's labels and a colorized image is returned.

Mean shift is much slower than k-means, especially for images greater than 512x512. However, you do not need to specify the number of clusters. Instead, you adjust the "bandwidth" parameter. From my experience, values in the range [0.0, 0.15] tend to produce the best results.

mean-shift-example

Watershed

Generate an image using a stable diffusion model and apply the watershed segmentation algorithm to convert it to a label image. The average color of each cluster is applied to the image's labels and a colorized image is returned.

Watershed is a fast region-based method and will only produce the best results on images with a lot of intensity variation. It does not account for the hue of the original image like k-means or mean shift.

watershed-example

Random LUT

Apply a randomly generated lookup table of RGB colors to colorize the label image from the mean shift clustering node.

random-lut-example

Load LUT from Matplotlib

Apply a lookup table from Matplotlib to colorize the label image.

load-lut-from-matplotlib-example

Label img2img

Apply an img2img with light denoising to the colorized label image.

label-img2img-example

Colorize an image with K-Means

This slightly more complex workflow uses a k-means label image and a Matplotlib LUT to colorize a generated image. The resulting image is then upscaled for a few additional denoising steps (similar to the hires fix technique) to smoothly blend the colors of the label image with the content from the generated image.

kmeans-with-hires-fix-example

Nodes

  • Mean Shift - Apply the Mean Shift clustering algorithm to an image.
  • Apply LUT To Label Image - Converts a label image into an RGB image by applying a RGB lookup table (LUT).
  • Random LUT - Randomly generate a LUT of RGB colors.
  • Load LUT From Matplotlib - Load an RGB LUT from Matplotlib.

To do

  • ⬜ implement LoadLUTFromFile node
  • ⬜ implement MedianFilter node
  • ✅ implement KMeans node
  • ✅ implement Watershed node
  • ⬜ implement Resize Label Image node
  • ⬜ add support for Segment Anything labels
  • ⬜ write unit tests
  • ⬜ use LAB space in RandomLUT for better perceptual uniformity
  • ⬜ add random seed option to RandomLUT