Sign in

Nodes Browser

ComfyDeploy: How ComfyUI-DragNUWA works in ComfyUI?

What is ComfyUI-DragNUWA?

Nodes: Download the weights of DragNUWA [a/drag_nuwa_svd.pth](https://drive.google.com/file/d/1Z4JOley0SJCb35kFF4PCc6N6P1ftfX4i/view) and put it to ComfyUI/models/checkpoints/drag_nuwa_svd.pth [w/Due to changes in the torch package and versions of many other packages, it may disrupt your installation environment.]

How to install it in ComfyDeploy?

Head over to the machine page

  1. Click on the "Create a new machine" button
  2. Select the Edit build steps
  3. Add a new step -> Custom Node
  4. Search for ComfyUI-DragNUWA and select it
  5. Close the build step dialig and then click on the "Save" button to rebuild the machine

This is an implementation of DragNUWA for ComfyUI

DragNUWA: DragNUWA enables users to manipulate backgrounds or objects within images directly, and the model seamlessly translates these actions into camera movements or object motions, generating the corresponding video.

Install

  1. Clone this repo into custom_nodes directory of ComfyUI location

  2. Run pip install -r requirements.txt

  3. Download the weights of DragNUWA drag_nuwa_svd.pth and put it to ComfyUI/models/checkpoints/drag_nuwa_svd.pth

For chinese users:drag_nuwa_svd.pth

smaller and faster fp16 model: dragnuwa-svd-pruned.fp16.safetensors from https://github.com/painebenjamin/app.enfugue.ai

For chinese users: wget https://hf-mirror.com/benjamin-paine/dragnuwa-pruned-safetensors/resolve/main/dragnuwa-svd-pruned.fp16.safetensors 不能直接在浏览器下载，或者参照 https://hf-mirror.com/ 官方使用说明

Tools

Motion Traj Tool Generate motion trajectories

<img src="assets/multiline.png" raw=true> <img src="assets/multiline.gif" raw=true>

Examples

  1. basic workflow
<img src="assets/base_wf.png" raw=true>

https://github.com/chaojie/ComfyUI-DragNUWA/blob/main/workflow.json

  1. InstantCameraMotionBrush & InstantObjectMotionBrush

InstantCameraMotionBrush Node Generate zoomin/zoomout/left/right/up/bottom camera motion brush

InstantObjectMotionBrush Node Generate zoomin/zoomout/left/right/up/bottom object motion brush (by draw mask on object)

<img src="assets/instantmotionbrush.png" raw=true>

https://github.com/chaojie/ComfyUI-DragNUWA/blob/main/workflow_InstantMotionBrush.json

  1. optical flow workflow

Thanks for Fannovol16's Unimatch_ OptFlowPreprocessor Thanks for toyxyz's load optical flow from directory

<img src="assets/optical_flow.png" raw=true>

https://github.com/chaojie/ComfyUI-DragNUWA/blob/main/workflow_optical_flow.json

  1. motion brush

need nodes "https://github.com/chaojie/ComfyUI-RAFT"

<img src="assets/motionbrush.png" raw=true>

https://github.com/chaojie/ComfyUI-DragNUWA/blob/main/workflow_motionbrush.json