Biggest problem for diffusion models were performance (as you need to iterate even at inference) But I'm not up to date with newest architectures maybe its already solved :P
Has anyone used any of the generative models mentioned in the article? Didn’t see any images or direct comparisons of the outputs with current diffusion models
Unlike Stable Diffusion, I don't stumble upon people who actually use it. Are there examples of the output this can generate? What happens once you manage to run the model?
Any pointers on getting up to speed on diffusion models? I haven't encountered them in my corner of the ML world, and googling around for a review paper didn't turn anything up.
There are already open source LLMs with comparable parameter counts (Facebook's OPT-175B, BLOOM), but you'll need ~10x A100 GPUs to run them (which would cost ~$100K+).
I suspect a big part of why stable diffusion managed to consume so much mindshare is that it can run on ordinary consumer hardware. On that point, I would be excited about an open-source RETRO (https://arxiv.org/pdf/2112.04426.pdf) model with comparable performance to GPT-3 that could run on consumer hardware with an NVMe SSD.
Diffusion is relatively compute intensive compared to transformers llms, and (in current implementation) doesn't quantize as well.
A 70B parameter model would be very slow and vram hungry, hence very expensive to run.
Also, image generation is more reliant on tooling surrounding the models than pure text prompting. I dont think even a 300B model would get things quite right through text prompting alone.
What sort of setup do you need to be able to fine tune Stable Diffusion models? Are there good tutorials out there for fine tuning with cloud or non-cloud GPUs?
A lot of HN has been having fun with stable diffusion. Do we really need 1 x GPU with 10GB of RAM? How do you distribute or "shard" a model you're training? Could we get this running on the raspberry pi clusters we all have? Hook it up to OpenFaaS too.
Yes. I created a course which uses implementing Stable Diffusion from scratch as the project, and goes through lots of architecture choices, hyperparam selection, and debugging. (But note that this isn't something that's fast or easy to learn - it'll take around a month full-time intensive study.)
https://course.fast.ai/Lessons/part2.html
Then you recall incorrectly. Stable diffusion models are capable of running on laptops with even modest discrete GPUs. I'm running it using the automatic GitHub repo on a laptop that's over four years old and a 50 step iteration only takes about 15 seconds.
You need zero technical acumen to be able to install it, just the ability to follow basic instructions. Maybe you should ask ChatGPT to help you.
Rest assured someone is working on a self-hosted (distilled) model. Stable Diffusion has shown there is a viable market for open, consumer-hardware inferencable models.
reply