Sign in

Nodes Browser

ComfyDeploy: How ComfyUI Neural Network Latent Upscale works in ComfyUI?

What is ComfyUI Neural Network Latent Upscale?

Nodes:NNLatentUpscale, A custom ComfyUI node designed for rapid latent upscaling using a compact neural network, eliminating the need for VAE-based decoding and encoding.

How to install it in ComfyDeploy?

Head over to the machine page

  1. Click on the "Create a new machine" button
  2. Select the Edit build steps
  3. Add a new step -> Custom Node
  4. Search for ComfyUI Neural Network Latent Upscale and select it
  5. Close the build step dialig and then click on the "Save" button to rebuild the machine

ComfyUI Neural network latent upscale custom node

Example 1

This repository includes a custom node for ComfyUI for upscaling the latents quickly using a small neural network without needing to decode and encode with VAE. The node can be found in "Add Node -> latent -> NNLatentUpscale".

This node is meant to be used in a workflow where the initial image is generated in lower resolution, the latent is upscaled and the upscaled latent is fed back into the stable diffusion u-net for low noise diffusion pass (high-res fix).

Compared to VAE decode -> upscale -> encode, the neural net latent upscale is about 20 - 50 times faster depending on the image resolution with minimal quality loss. Compared to direct linear interpolation of the latent the neural net upscale is slower but has much better quality. Direct latent interpolation usually has very large artifacts.

Installation

Clone this repository in ComfyUI custom_nodes directory with: git clone https://github.com/Ttl/ComfyUi_NNLatentUpscale.git.

Evaluation

Example 2

Dataset: COCO 2017 validation images center cropped to 256x256 resolution. The comparison image is linear upscale of the input image. All tests are done with fp32 precision and batch size 4.

VAE Upscale: VAE decode -> Linear interpolation -> Encode.

NN Upscale: Neural network upscale (This repository).

Latent Upscale: Linear interpolation of latent.

SDXL, 2x upscale:

| | MSE ↓ | LPIPS ↓ | PSNR ↑ | Time (ms) ↓ | |----------------------|--------|---------|--------|-------------| | VAE Upscale | 0.009 | 0.22 | 26.9 | 832 | | NN Upscale | 0.010 | 0.28 | 26.3 | 36 | | Latent Upscale | 0.047 | 0.65 | 19.5 | 0.1 |

SDXL, 1.5x upscale:

| | MSE ↓ | LPIPS ↓ | PSNR ↑ | Time (ms) ↓ | |----------------------|--------|---------|--------|-------------| | VAE Upscale | 0.009 | 0.20 | 26.9 | 583 | | NN Upscale | 0.010 | 0.26 | 26.3 | 19 | | Latent Upscale | 0.038 | 0.58 | 20.4 | 0.1 |

SD 1.5, 2x upscale:

| | MSE ↓ | LPIPS ↓ | PSNR ↑ | Time (ms) ↓ | |----------------------|-------|---------|--------|-------------| | VAE Upscale | 0.009 | 0.21 | 26.7 | 822 | | NN Upscale | 0.008 | 0.24 | 27.0 | 36 | | Latent Upscale | 0.033 | 0.61 | 20.9 | 0.1 |

SD 1.5, 1.5x upscale:

| | MSE ↓ | LPIPS ↓ | PSNR ↑ | Time (ms) ↓ | |----------------------|-------|---------|--------|-------------| | VAE Upscale | 0.010 | 0.18 | 26.5 | 594 | | NN Upscale | 0.009 | 0.21 | 26.9 | 20 | | Latent Upscale | 0.031 | 0.52 | 21.3 | 0.1 |