Thu 06/03 | AI Tools
Thursday, March 6 2025
Room 3430, Experimental Studio 1
Realtime Stable Diffusion #
If you have a powerful enough graphics card, you can do almost realtime generation with Stable Diffusion using something called StreamDiffusion.
StreamDiffusionTD by dotsimulate #
dotsimulate has created a TouchDesigner component that allows you to integrate StreamDiffusion with TouchDesigner. In order to access the component, you would need to become a Patreon supporter though.
The latest version of it also integrates the ability to use ControlNets.
TouchDiffusion by olegchomp #
This one is released on GitHub without any paywall, but might be a little bit harder to install and setup. It gets better performance though in my tests, once you get it running.
OpenAI (GPT and Whisper) #
Note that you will need an API key to work with the OpenAI components. You can find the key in MyCourses and you can use it during this class as long as you use it moderately. The API key will be deleted after the course. If you plan to use OpenAI in your project, come talk to Matti and we will create a separate key for you.
GPT and Whisper (audio-to-text) #
For TouchDesigner there are nice components created by Torin Blankensmith. In order to access the component, you would need to become a Patreon supporter though. Ask Matti about possibility for reimbursement from the course budget, if you would like to use tools like this.
If you want to learn a little bit more in-depth how to work with the API, this tutorial goes through the process of writing the code from scratch.