Sign in

Nodes Browser

ComfyDeploy: How ComfyUI-Golden-Noise works in ComfyUI?

What is ComfyUI-Golden-Noise?

ComfyUI Custom Node for [a/'Golden Noise for Diffusion Models: A Learning Framework'](https://arxiv.org/abs/2411.09502) and most of the code is adapted from [a/here](https://github.com/xie-lab-ml/Golden-Noise-for-Diffusion-Models). This node refines the initial latent noise in the diffusion process, enhancing both image quality and semantic coherence.

How to install it in ComfyDeploy?

Head over to the machine page

  1. Click on the "Create a new machine" button
  2. Select the Edit build steps
  3. Add a new step -> Custom Node
  4. Search for ComfyUI-Golden-Noise and select it
  5. Close the build step dialig and then click on the "Save" button to rebuild the machine

ComfyUI-Golden-Noise

ComfyUI Custom Node for "Golden Noise for Diffusion Models: A Learning Framework". Most of the code for this node is adapted from here. This node refines the initial latent noise in the diffusion process, enhancing both image quality and semantic coherence.

Installation

  1. Navigate to your ComfyUI's custom_nodes directory:
cd ComfyUI/custom_nodes
  1. Clone this repository:
git clone https://github.com/LucipherDev/ComfyUI-Golden-Noise
  1. Install requirements:
cd ComfyUI-Golden-Noise
pip install -r requirements.txt

Or Install via ComfyUI Manager

Usage

Download the safetensors of the pre-trained NPNet weights of Stable Diffusion XL, DreamShaper-xl-v2-turbo, and Hunyuan-DiT from Huggingface LucipherDev/Golden-Noise-NPNets and put them in the models/npnets folder.

The node can be found in "sampling/custom_sampling/noise" category as "GoldenNoise".

Take a look at the example workflow for more info.

Inputs

  • noise: Noise output from RandomNoise node
  • conditioning: Prompt conditioning
  • model_id: "SDXL", "DreamShaper", "DiT"
  • npnet_model: Pretrained NPNet model
  • device: "cuda", "cpu"

Image Comparisons

comparison *Comparisons from the node

This is a small comparison with only 50 images for each prompt, with and without using the golden noise node. The images were generated using the provided workflow (the random noise used to generate the first image is fed into the GoldenNoise node to generate the golden noise), and then each image was scored using ImageReward. The mean scores for with and without the node are shown in the comparison. From this, we can see that when using the node, the overall image quality is increased. However, The sample size is admittedly small, but it's all I can do with the resources and time I have.

Another comparison. This is taken from the original paper. x1 *Comparisons from the original paper

Citation

Original Paper.

@misc{zhou2024goldennoisediffusionmodels,
      title={Golden Noise for Diffusion Models: A Learning Framework}, 
      author={Zikai Zhou and Shitong Shao and Lichen Bai and Zhiqiang Xu and Bo Han and Zeke Xie},
      year={2024},
      eprint={2411.09502},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2411.09502}, 
}