ComfyUI_CSGO_Wrapper

You can using InstantX's CSGO in comfyUI

CSGO From: link

Update

2024/09/07

• fix runway loader error, using single clip_vision /controlnet weight now；
• 修复runway跑路导致的diffuser加载报错，现在直接使用IP adapter的SDXL 图片解码单体模型（请放在clip_vision目录下）和单体controlnet SDLX模型（请放在controlnet目录下）；
• vae is not necessary,if you get black img ,please using vae
• vae并不是必须的，但是如果出现黑图或者解码不正确，才使用；

1.Installation

In the ./ComfyUI /custom_node directory, run the following:

git clone https://github.com/smthemex/ComfyUI_CSGO_Wrapper.git

2.requirements

For ComfyUI users, all libraries in the requirements file should be available. If not, please uncomment the # and reinstall

pip install -r requirements.txt

3 Need model

3.1 base SDXl ckpt and vae and clip_vision and controlnet
h94/IP-Adapter/sdxl_models link
TTPlanet/TTPLanet_SDXL_Controlnet_Tile_Realistic link no need config

├── ComfyUI/models/checkpoints/
|      ├──any SDXL weights  # for example: Jumpernaut XL_v9-RunDiffusionPhoto_v2.safetensors
├── ComfyUI/models/vae/
|      ├──any SDXL vae weights  # sdxl.vae.safetensors 
├── ComfyUI/models/clip_vision/
|      ├──model.safetensors  # h94/IP-Adapter/sdxl_models/model.safetensors
├── ComfyUI/models/controlnet /
|      ├──TTPLANET_Controlnet_Tile_realistic_v2_fp16.safetensors  # TTPlanet/TTPLanet_SDXL_Controlnet_Tile_Realistic

3.2 main ckpt
CSGO models link

├── ComfyUI/models/checkpoints/
|      ├──acsgo_4_32.bin  #need token 4/32
|      ├──acsgo.bin   #need token 4/16

3.3 if using LLM (unnecessary)
Salesforce/blip-image-captioning-large link

├── any path dir/
|             ├── model.safetensors
|             ├── config.json
|             ├── preprocessor_config.json
|             ├── special_tokens_map.json
|             ├── tokenizer.json
|             ├── tokenizer_config.json
|             ├── vocab.txt

or change prompt input using any other LLM.

4 Example

conternt+style img

style img + prompt
conternt+style img +llm new 最新

5 Function Description of Nodes

• content_tokens: keep in 4;
• style_tokens: if using csgo.bin keep num "16",if using csgo_4_32.bin keep num "32" ;
• text_only: if not using content img ;
• Blip is not necessary;

6 Citation

CSGO

@article{xing2024csgo,
       title={CSGO: Content-Style Composition in Text-to-Image Generation}, 
       author={Peng Xing and Haofan Wang and Yanpeng Sun and Qixun Wang and Xu Bai and Hao Ai and Renyuan Huang and Zechao Li},
       year={2024},
       journal = {arXiv 2408.16766},
}
}