Back to Blog
Vexx art invert5/24/2023 ![]() Take 5-8 high quality images of your dog.So, to fine-tune Stable Diffusion to create images of your dog, you would: This is a unique reference to the object you want Stable Diffusion to learn. a : The prompt for fine-tuning to a new object or style, generally a string that corresponds to a token the model is unfamiliar with.a : The prompt for the prior preservation loss is the category of the style or object in question, like dog or painting.The prior preservation loss is the mean squared error of the now generated images and the pre-training generated images for the category in the latent space. It does this by extending Stable Diffusion’s fine-tuning loss with a prior preservation loss to train the model to still generate diverse images for the category of that style (e.g. And Mona Lisa would still look like it came from Leonardo and Starry Night from Van Gogh. Fine-tune it for your favorite artist? A walk in the park. You want to train it for your dog? Piece of cake. Left: Generated image by Stable Diffusion with prompt a picasso painting, before fine-tuning Right: Generated image for prompt a picasso painting, after fine-tuning to Löwentraut DreamBooth: Fine-tune to your favorite artist (and remember!)ĭreamBooth fixes that. If you feed Stable Diffusion a Leon Löwentraut image it can learn his style (using, for example, text-to-image fine-tuning for Stable Diffusion.) Luckily, this can be fixed by fine-tuning (yeah, we’re dropping the Biblical speak). And Stable Diffusion did say “uh, what? lol I’ll give it my best. “Generate an image in the style of Leon Löwentraut” you proclaim. And verily an image of a woman with too many angles would be created. “Create an artwork in the style of Picasso” you would exclaim. “Create a Banksy picture” you would say, and verily a Banksy would be created. In the beginning there was Stable Diffusion and it was good. Stable Diffusion: Fine-tune to your favorite artist (but forget everyone else) In this blog post we’ll go over how we did that, and how well it works.īut first, let’s take a quick look at how we got here, by starting off with Stable Diffusion itself. We created BIG by taking the DreamBooth paper, which allows fine-tuning with one subject, and leveling it up it into a metamodel to learn multiple new objects without using up all your compute. That means you can take a picture of you and a picture of your pooch and combine them into a composite image in the style of Picasso, Pixar or pop art. In short, BIG lets you fine-tune Stable Diffusion to the next level, letting you create images of multiple subjects and in any style you want. If we ever get out of the game and get Back At the Game Again, we’ll have to go with BAGA.
0 Comments
Read More
Leave a Reply. |