What is a sampler
As we all know, when we use Stable Diffusion, it first creates a raw image full of noise, and then gradually removes the noise from the image. The process is like engraving: the original noise map is like the raw material for engraving, the denoising process is like using a carving knife to remove waste, and the resulting image is like a carved work. This process of denoising is called sampling.
Choosing which sampler to denoise is like choosing which carving knife to engrave. The sampler is used to control the specific operation of denoising.
An important metric for samplers is convergence. When the sampler reaches a stage where further steps will no longer improve the result, we consider the image to have reached a convergent state.
Some samplers converge quickly, making them ideal for quickly validating ideas and ideas. While some other samplers may take longer or more steps to converge, they are generally able to provide higher quality results. There are also samplers that never converge because they don't set limits, giving more room for innovation and creativity.
What is a scheduler
If the sampler is responsible for controlling the specific operation of denoising, then the scheduler is responsible for regulating the degree of denoising, deciding how much noise is removed at each step to ensure that the whole process is efficient and accurate.
Continuing with the carving analogy, in the early stages of carving, it is beneficial to use greater force to quickly knock off large parts, which can speed up the overall progress of the carving. In the final stages of engraving, we need to be extremely meticulous and careful in order to carve out the details precisely and prevent the sculpture from breaking. So it's also important to choose the right scheduler.
Sampler in detail
Next, we will introduce the commonly used samplers in detail according to the order of samplers in ComfyUI:
euler
The simplest sampler, which does not add random noise during the sampling process, tells the sampler by noise plan how much noise should be in each step. and use the Euler method to reduce the appropriate amount of noise. to match the noise plan until the last step is 0.
euler_ancestral (简写成euler a)
A type of ancestral sampler, similar to Euler. The feature is that with each new processing step, they add random variables that were generated in the previous step. It is conceivable that after each step of the noise is removed, a portion of the previous noise is reintroduced. Since new random noise is added at each step, a sampler with this feature will never reach a convergent state.
heum
Heum is a more precise improvement of Euler, but the noise needs to be predicted twice in each step, so it is 2 times slower than Euler.
dpm_2
dpm_2 can be considered an upgraded version of DPM, which is equivalent to "version 2". It has been further improved and optimized on the basis of the original model.
dpm_2_ancestral (abbreviated as dpm_2 a)
A type of ancestral sampler that, using dpm_2 method, never reaches a convergent state.
lms
The sampling speed of LMS is the same as that of Euler.
dpmpp_2s_ancestral(简写成dpm++ 2s a)
DPM++ is another upgraded version of DPM. DPM++ uses a hybrid method that combines deterministic and probabilistic methods for sampling and subsequent noise reduction. "2" means they are second-order models, "s" stands for single step, and "a" means one of the ancestor samplers that will never converge.
dpmpp_sde(简写成dpm++ sde)
The SDE variant of dpm++, it never converges, so even if the number of steps increases, it does not provide higher quality, but rather more variation, just like the ancestral sampler.
dpmpp_2m(简写成dpm++ 2m)
Similar to dpm++ 2s, "m" stands for Multi step. This method performs multiple calculations at each step and takes into account the information obtained in the previous steps, resulting in more precise and higher-quality convergence, but correspondingly longer times.
dpmpp_2m_sde(简写成dpm++ 2m sde)
dpmpp_2m variant of SDE, it never converges, so even if the number of steps increases, it does not provide higher quality, but rather more variety, just like the ancestral sampler.
not
DDIM was one of the first diffuse model samplers to be proposed. Its core idea is that the picture of each step can be approximated by adding the following three things: the final picture, the image with the image orientation pointing to the current step, and random noise. But how do we know the final image before we get to the final step? The dDIM sampler uses the denoised image to estimate the final image.
uni_pc
The newly added sampler should be the fastest and most up-to-date sampling method available, achieving high-quality images in less than 10 steps.
How to choose a sampler
There is no such thing as the best sampler, only the best one.
- • If you want to stabilize a reproducible image, avoid selecting any ancestor sampler (names ending in A or SDE)
- • If you want a simple diagram, it is recommended to choose Euler, Heun (you can reduce the steps to save time)
- • If you want to quickly generate good quality images, it is recommended to choose dpmpp_2m (20-30 steps) and uni_pc (15-25 steps)
- • If you want to generate a different image each time, you can choose a non-convergent ancestor sampler (with a or sde in the name)
Comparative testing
Create an XY chart, with the abscissa being the number of steps, starting from step 5 and intervaling four steps, for a total of 8 values, and the ordinate being the various samplers described above.
From the above figure, it is not difficult to find the following rules:
- • Names ending in A or SDE do not converge, and the image changes with each iteration
- • The Euler, Heun, and Ddim samplers are already plotted in step 9, which is really good and fast for simple graphs.
- • The dpmpp_2m and uni_pc samplers plot at step 14, where the dpmpp_2m start to converge at step 18 with great results.
- • Don't be fooled by the dpm_adaptive sampler, which is very slow, although it has converged in step 5. I tested it separately and it was actually converged in step 2, but it was very slow.
- • The LMS sampler performs the worst.
Author: Code man Lao Gao
Source-WeChat public account: AI full-stack road
Source: https://mp.weixin.qq.com/s/MhErwK2ReRkRBWg733U6tw