Nodes Browser
ComfyDeploy: How ComfyUI-IF_AI_tools works in ComfyUI?
What is ComfyUI-IF_AI_tools?
Various AI tools to use in Comfy UI. Starting with VL and prompt making tools using Ollma as backend will evolve as I find time.
How to install it in ComfyDeploy?
Head over to the machine page
- Click on the "Create a new machine" button
- Select the
Edit
build steps - Add a new step -> Custom Node
- Search for
ComfyUI-IF_AI_tools
and select it - Close the build step dialig and then click on the "Save" button to rebuild the machine
ComfyUI-IF_AI_tools
ComfyUI-IF_AI_tools is a set of custom nodes to Run Local and API LLMs and LMMs, Features OCR-RAG (Bialdy), nanoGraphRAG, Supervision Object Detection, supports Ollama, LlamaCPP LMstudio, Koboldcpp, TextGen, Transformers or via APIs Anthropic, Groq, OpenAI, Google Gemini, Mistral, xAI and create your own charcters assistants (SystemPrompts) with custom presets and muchmore
Prerequisite Installation (Poppler)
To ensure compatibility and functionality with all tools, you may need poppler
for PDF-related operations. Use scoop
to install poppler
on Windows:
Step 1: Install scoop
(if not already installed)
If you haven't installed scoop
yet, run the following command in PowerShell:
iwr -useb get.scoop.sh | iex
Step 2: Install poppler with scoop Once scoop is installed, you can install poppler by running:
windows 10+ istall scoop and then
scoop install poppler
Debian/Ubuntu
sudo apt-get install poppler-utils
MacOS
brew install poppler
check is working
pdftotext -v
Install Ollama
You can technically use any LLM API that you want, but for the best expirience install Ollama and set it up.
- Visit ollama.com for more information.
To install Ollama models just open CMD or any terminal and type the run command follow by the model name such as
ollama run llama3.2-vision
If you want to use omost
ollama run impactframes/dolphin_llama3_omost
if you need a good smol model
ollama run ollama run llama3.2
Optionally Set enviromnet variables for any of your favourite LLM API keys "XAI_API_KEY", "GOOGLE_API_KEY", "ANTHROPIC_API_KEY", "MISTRAL_API_KEY", "OPENAI_API_KEY" or "GROQ_API_KEY" with those names or otherwise it won't pick it up you can also use .env file to store your keys
Features
[NEW] nanoGraphRAG,
[NEW] OCR-RAG ColPali & ColQwen (Bialdy),
[NEW] Supervision Object Detection Florence2
[NEW] Endpoints xAI, Transformers,
[NEW] IF_Assistants System Prompts with Reasoning/Reflection/Reward Templates and custom presets
- Gemini, Groq, Mistral, OpenAI, Anthropic, Google, xAI, Transformers, Koboldcpp, TextGen, LlamaCPP, LMstudio, Ollama
- Omost_tool the first tool
- Vision Models Haiku, Florence2
- [Ollama-Omost]https://ollama.com/impactframes/dolphin_llama3_omost can be 2x to 3x faster than other Omost Models LLama3 and Phi3 IF_AI Prompt mkr models released
ollama run impactframes/llama3_ifai_sd_prompt_mkr_q4km:latest
ollama run impactframes/ifai_promptmkr_dolphin_phi3:latest
https://huggingface.co/impactframes/llama3_if_ai_sdpromptmkr_q4km
https://huggingface.co/impactframes/ifai_promptmkr_dolphin_phi3_gguf
Installation
- Open the manager search for IF_AI_tools and install
Install ComfyUI-IF_AI_tools -hardest way
- Navigate to your ComfyUI
custom_nodes
folder, typeCMD
on the address bar to open a command prompt, and run the following command to clone the repository:git clone https://github.com/if-ai/ComfyUI-IF_AI_tools.git
OR
-
In ComfyUI protable version just dounle click
embedded_install.bat
or typeCMD
on the address bar on the newly createdcustom_nodes\ComfyUI-IF_AI_tools
folder typeH:\ComfyUI_windows_portable\python_embeded\python.exe -m pip install -r requirements.txt
replace
C:\
for your Drive letter where you have the ComfyUI_windows_portable directory -
On custom environment activate the environment and move to the newly created ComfyUI-IF_AI_tools
cd ComfyUI-IF_AI_tools python -m pip install -r requirements.txt
Related Tools
- IF_prompt_MKR
- A similar tool available for Stable Diffusion WebUI
Videos
AIFuzz made a great video usining ollama and IF_AI tools
Also Future thinker @ Benji Thankyou both for putting out this awesome videos
Example using normal Model
ancient Megastructure, small lone figure 'A dwarfed figure standing atop an ancient megastructure, worn stone towering overhead. Underneath the dim moonlight, intricate engravings adorn the crumbling walls. Overwhelmed by the sheer size and age of the structure, the small figure appears lost amidst the weathered stone behemoth. The background reveals a dark landscape, dotted with faint twinkles from other ancient structures, scattered across the horizon. The silent air is only filled with the soft echoes of distant whispers, carrying secrets of times long past. ethereal-fantasy-concept-art, magical-ambiance, magnificent, celestial, ethereal-lighting, painterly, epic, majestic, dreamy-atmosphere, otherworldly, mystic-elements, surreal, immersive-detail'
TODO
- Undergoing full refactor the next update will have three nodes one for general purpose chat, one for Images creation, one complementary carrying the embeddings and auxiliary tools
- ⬜ Fix Bugs and make it work on latest ComfyUI
- ⬜ Fix Graph Visualizer Node
- ⬜ Tweak IF_Assistants and Templates
- ⬜ FrontEnd for IF_Assistants and Chat
- ⬜ Node and workflow creator
- ⬜ two additional Endpoints one API and one Local
- ⬜ Add New workflows
- ⬜ Image Generation, Text 2 Image, Image to Image, Video Generation
Support
If you find this tool useful, please consider supporting my work by:
- Starring the repository on GitHub: ComfyUI-IF_AI_tools
- Subscribing to my YouTube channel: Impact Frames
- Follow me on X: Impact Frames X
- Supporting me on Ko-fi: Impact Frames Ko-fi
- Becoming a patron on Patreon: Impact Frames Patreon Thank You!