How to create images from drawings and descriptions with Scribble Diffusion AI

A fun and unusual new website, based on an AI that generates realistic images through a hand-drawn drawing by the user and a short textual description, has emerged in recent weeks and has gained some attention from the public. Given the wave of multipurpose artificial intelligence around the world, it is not surprising that the Scribble Diffusion have made a fuss. Understand below what this site is about, how to use it and how it works.

The Scribble Diffusion

The amazing and creative Scribble Diffusion It is based on a neural network structure, with edge detection, similarity of poses and depth maps of images and concepts that are indexable and available on the internet. With these tools, it reads and understands your drawing – done by hand on the blank board – and what you write in the alphanumeric field, then creating new images based on these two parameters.

Its development was inspired by benchmarks (screen design inspired by an infinity of the same niche sites) made by vercel and with an AI facilitated by the Replicate, which we explain below. 

See some examples of images created on the site:

Who did? How it works?

You can see how the “machine” that runs behind the site works closely in the engine designed by the developer. This, as far as he is concerned, is in a domain of Replicate, a site that houses operating AI source codes, simplifying their use. Such codes are used in many different ways and fan ideas and inspirations for developers within the platform. Often, their pages even function as a kind of portfolio of these professionals, as well as the GitHub is today.

The site Scribble Diffusion was built by Computer Science (Ph.D.) student at Stanford University, lvmin zhang, and by Professor Maneesh Agrawala. It only took 3 weeks on the air for the open source repository to be used in many other studies and sites based on image generation by Artificial Intelligence. O dreambooth, still in the process of being created, is an example of this. It creates realistic photos based on text description and a user-customized human pose.

If you are a programming enthusiast, be sure to check out the demo video found at the last link in the sources of this text. In it, the complete and detailed explanation of how the site works is done on the Replicate channel.

How to use the website

Using the Scribble Diffusion site is very simple. Just follow the steps below:

1. Visit the site https://scribblediffusion.com/ in your browser;
2. There will be a drawing already made on the whiteboard on the canvas. You can use the same design to increment it or delete it completely on the button. Clear, right below this frame, to the phrase “draw something here" to appear;

1. Now, you can freely draw on the whiteboard using your touchpad, mouse, or graphics tablet (the device you use to move your cursor on your computer). When accessing the website through your smartphone, the drawing can simply be done by touching the screen as well. Remember to make well-defined strokes and a design that is minimally recognizable to Artificial Intelligence to reach a satisfactory result, but you can test this aspect of its functionality if you want;
2. Once the drawing is done, it's time to briefly describe it or write in detail in the alphanumeric text field below the whiteboard. Give the description that you think is best for your drawing or what you want to get in the final result;
3. Having the design and description ready, it's time to see the result: click on the button Go and wait for it to load. Your drawing will be shown in “Results” and the generated realistic image will appear just to the right of it. 
4. Ready! Now you can download the image and drawing made just by right-clicking on the images and saving. It is possible to save and/or share the generated link of your images in the button just below them, "Copy link“. You can create as many images as you like on the same site.

Did you like this content? Learn more about other AI-powered sites and features we talk about here:

8 tools to create presentations with artificial intelligence

YouTubers Create Animation Using Artificial Intelligence

How to take your selfie with artificial intelligence 

Sources: Analytics in India Mag, ControlNet (article), Website Github, Replicate's GitHub, Scribble Diffusion Demo

Text proofread by: Pedro Bomfim (20 / 03 / 23)