Discover and read the best of Twitter Threads about #controlnet
Most recents (7)
🧠 AI experiment comparing #ControlNet and #Gen1. Video goes in ➡ Minecraft comes out.
Results are wild, and it's only a matter of time till this tech runs at 60fps. Then it'll transform 3D and AR.
How soon until we're channel surfing realities layered on top of the world?🧵
Results are wild, and it's only a matter of time till this tech runs at 60fps. Then it'll transform 3D and AR.
How soon until we're channel surfing realities layered on top of the world?🧵
First ControlNet. Wow, this tool makes it very easy to get photorealistic results. I used the HED method for this scene and got some amazing results. I used EbSynth for smoother interpolation between ControlNet keyframes. Check out my prior posts for the end-to-end workflow.
Next up Gen 1: impressive is the word. The star of the show is the temporal consistency. Getting photoreal results is harder than ControlNet IMO. #Gen1 is almost its own stylized thing, so I advise leaning into that. But why does it matter - can't we just type text to get video?
Testing Multi-ControlNet on a scene with extended dialog, a range of facial expressions, and head turning. No EBSynth used. #AImusic from aiva.ai. Breakdown🧵1/15 #aicinema #controlnet #stablediffusion #aiia #aiartcommunity #aiva #machinelearning #deeplearning #ai
Overall flow: pre-process video > img seq > play with prompts > initial controlnet settings > control net batch render > upscale > clean up in post > key out background > deflicker > video post-production 2/15
The approach I used here was figuring out an initial workflow. There’s definitely a lot to play with and improve on. The orig vid is low-res and a little blurry so I started with a pre-process pass by bringing out the edges/brightness & upscaling. 3/15
Will scale pay dividends in diffusion models? Here is Alibaba's Composer with 5B parameters. Dall-e 2 has 3.5B parameters, and StableDiffusion has 890m parameters.
Browsing through the examples, there are only a few things that #stablediffusion with #controlnet cannot do. There aren't any Color palette and intensity ControlNet networks. But certainly this can be trained. 
The generality of modular networks appears to trump scale. medium.com/intuitionmachi…
👉#Blender + #ControlNet prototype. Pose any character (I'm using Mixamo) right in Blender, hit F12, Booooooom! You don't have to leave Blender now, using A1111 in the background. #AIart #stablediffusion #b3d #aigirls
just a quick share, will opensource when i finish these messy codes.😉
ControlNet experiment where I'm toggling through different styles of contemporary Indian décor, while keeping a consistent scene layout.
Loving how ControlNet is putting the artists back in control of AI image generation process.
🧵Thread
#ControlNet #StableDiffusion #EbSynth
Loving how ControlNet is putting the artists back in control of AI image generation process.
🧵Thread
#ControlNet #StableDiffusion #EbSynth
For the input sequence I use a short animation made from a photogrammetry 3d scan I did a few years back of my parents living room in India.
- Top: Output generated with ControlNet + EbSynth
- Bottom: Input video sequence from my 3D scan
- Top: Output generated with ControlNet + EbSynth
- Bottom: Input video sequence from my 3D scan
I used the ControlNet depth method for this experiment.
- Left: MiDaS depth map generated from a screen recording of my 3d scan
- Right: Awesome results reskinning the room! Pretty majestic if I say so myself
Next time I want to use the synthetic depth from my 3d scan
- Left: MiDaS depth map generated from a screen recording of my 3d scan
- Right: Awesome results reskinning the room! Pretty majestic if I say so myself
Next time I want to use the synthetic depth from my 3d scan
#stablediffusion
👁️如何一键安装 #Chilloutmix 到云端电脑？
完全免费！不用下载任何软件到本地？
而且完全在google云端运行🤓
我来教你快速使用Google的Colab网站去安装 #Chilloutmix 在上面生成各种ai美图🫵
👁️如何一键安装 #Chilloutmix 到云端电脑？
完全免费！不用下载任何软件到本地？
而且完全在google云端运行🤓
我来教你快速使用Google的Colab网站去安装 #Chilloutmix 在上面生成各种ai美图🫵
1⃣️首先你需要注册谷歌的Colab平台
colab.research.google.com
colab.research.google.com
Turns out #stablediffusion #controlnet openpose model works with a posable figurine. This is going to make getting the right pose super easy. Next step is hooking it up to a webcam!
Some more examples here
