👁️ ComfyUI_Yvann-Nodes

Made with the help of Lilien <a href="https://x.com/Lilien_RIG"><img src="https://github.com/user-attachments/assets/26a483b9-cfe6-4666-af0d-52d40ee65dcf" alt="unnamed" width="35"></a>

With this pack of nodes, you can analyze audio, extract drums, bass, vocal tracks, and use the scheduled masks and weights to create AI-generated audio-reactive animations in ComfyUI

Works with IPAdapter, AnimateDiff, ControlNets, Prompts Schedules

• Workflows
• Nodes
• Installation

---

Workflows

Audio Reactive Images To Video :

Workflow File (click on the link, then on "download raw file" button at the right and drop the file into ComfyUI) 📜

AudioReactive_ImagesToVideo_Yvann.json

Youtube Tutorial (Click on the image)

Workflow Demo Render (ENABLE VIDEO SOUND) 🔊

https://github.com/user-attachments/assets/1e6590fc-e0d7-42d7-a205-433adf6c405c

<details> <summary><i>AudioReactive VideoToVideo</i></summary>

Workflow File (click on the link, then on "download raw file" button at the right and drop the file into ComfyUI) 📜

AudioReactive_VideoToVideo_Yvann.json

Workflow Demo Render (ENABLE VIDEO SOUND) 🔊

https://github.com/user-attachments/assets/6b0aa544-aa20-4257-b6be-28673082c7ef

Youtube Tutorial (Click on the image)

Workflow Preview

CIVITAI Workflow Page

</details>

---

Nodes

Audio Analysis 🔍

Analyzes audio to generate reactive weights and graph. Can extract specific elements like drums, vocals, bass. Parameters allow manual control over audio weights

<details> <summary><i>Node Parameters</i></summary>

• audio_sep_model: Loaded model from "Load Audio Separation Model"
• audio: Input audio file
• batch_size: Number of frames to associate with audio weights
• fps: Frames per second for processing audio weights

Parameters:

• analysis_mode: Select audio component to analyze
• threshold: Minimum weight value to pass through
• multiply: Amplification factor for weights before normalization

Outputs:

• graph_audio: Graph image of audio weights over frames
• processed_audio: Separated or processed audio (e.g., drums vocals)
• original_audio: Original unmodified audio input
• audio_weights: List of audio-reactive weights based on processed audio

</details>

---

Load Audio Separation Model 🎧

Load an audio separation model, If unavailable downloads to `ComfyUI/models/audio_separation_model/

<details> <summary><i>Node Parameters</i></summary>

• model: Audio separation model to load
• HybridDemucs: Most accurate fastest and lightweight
• OpenUnmix: Alternative model

Outputs:

• audio_sep_model: Loaded audio separation model<br> Connect it to "Audio Analysis" or "Audio Remixer"

</details>

---

Audio Peaks Detection 📈

Detects peaks in audio weights based on a threshold and minimum distance. Identifies significant audio events to trigger visual changes or actions.

<details> <summary><i>Node Parameters</i></summary>

• peaks_threshold: Threshold for peak detection
• min_peaks_distance: Minimum frames between consecutive peaks help remove close unwanted peaks around big peaks

Outputs:

• peaks_weights: Binary list indicating peak presence (1 for peak 0 otherwise)
• peaks_alternate_weights: Alternating binary list based on detected peaks
• peaks_index: String of peak indices
• peaks_count: Total number of detected peaks
• graph_peaks: Visualization image of detected peaks over audio weights

</details>

---

Audio IP Adapter Transitions 🔄

Uses "peaks_weights" from "Audio Peaks Detection" to control image transitions based on audio peaks. Outputs images and weights for two IPAdapter batches, logic from "IPAdapter Weights", IPAdapter_Plus

<details> <summary><i>Node Parameters</i></summary> - **images**: Batch of images for transitions, Loops images to match peak count - **peaks_weights**: List of audio peaks from "Audio Peaks Detection"

Parameters:

• blend_mode: transition method applied to weights
• transitions_length: Frames used to blend between images
• min_IPA_weight: Minimum weight applied by IPAdapter per frame
• max_IPA_weight: Maximum weight applied by IPAdapter per frame

Outputs:

• image_1: Starting image for transition Connect to first IPAdapter batch "image"
• weights: Blending weights for transitions Connect to first IPAdapter batch "weight"
• image_2: Ending image for transition Connect to second IPAdapter batch "image"
• weights_invert: Inversed weights Connect to second IPAdapter batch "weight"
• graph_transitions: Visualization of weight transitions over frames

</details>

---

Audio Prompt Schedule 📝

Associates "prompts" with "peaks_index" into a scheduled format. Connect output to "batch prompt schedule" of Fizz Nodes add an empty line between each individual prompts

<details> <summary><i>Node Parameters</i></summary> - **peaks_index**: frames where peaks occurs from "Audio Peaks Detections" - **prompts**: Multiline string of prompts for each index

Outputs:

• prompt_schedule: String mapping each audio index to a prompt

</details>

---

Audio Remixer 🎛️

Modify input audio by adjusting the intensity of drums bass vocals or others elements

<details> <summary><i>Node Parameters</i></summary> - **audio_sep_model**: Loaded model from "Load Audio Separation Model" - **audio**: Input audio file

Parameters:

• bass_volume: Adjusts bass volume
• drums_volume: Adjusts drums volume
• others_volume: Adjusts others elements' volume
• vocals_volume: Adjusts vocals volume

Outputs:

• merged_audio: Composition of separated tracks with applied modifications

</details>

---

Repeat Image To Count 🔁

Repeats images N times, Cycles inputs if N > images

<details> <summary><i>Node Parameters</i></summary> - **mask**: Mask input to convert

Outputs:

• float: Float value

</details>

---

Invert Floats 🔄

Inverts each value in a list of floats

<details> <summary><i>Node Parameters</i></summary>

• floats: List of float values to invert.

Outputs:

• inverted_floats: Inverted list of float values.

</details>

---

Floats Visualizer 📈

Generates a graph from floats for visual data comparison<br> Useful to compare audio weights

<details> <summary><i>Node Parameters</i></summary>

• floats: Primary list of floats to visualize
• floats_optional1: (Optional) Second list of floats
• floats_optional2: (Optional) Third list of floats

Parameters:

• title: Graph title
• x_label: Label for the x-axis
• y_label: Label for the y-axis

Outputs:

• visual_graph: Visual graph of provided floats

</details>

---

Mask To Float 🎭

Converts mask into float works with batch of mask

<details> <summary><i>Node Parameters</i></summary>

• mask: Mask input to convert.

Outputs:

• float: Float value representing the average value of the mask.

</details>

---

Floats To Weights Strategy 🏋️

Converts a list of floats into an IPAdapter weights strategy format. Use with "IPAdapter Weights From Strategy" or "Prompt Schedule From Weights Strategy" to integrate output into IPAdapter pipeline

<details> <summary><i>Node Parameters</i></summary>

Inputs:

• floats: List of float values to convert

Outputs:

• WEIGHTS_STRATEGY: Dictionary of the weights strategy

---

Installation

1. Install ComfyUI & ComfyUI-Manager
2. Launch ComfyUI
3. Click on "🧩 Manager" -> "Custom Nodes Manager"
4. Search for ComfyUI_Yvann-Nodes in the manager and install it

---

Giving a ⭐ to this repo is the best way to support us (: