Compressed Image Generation with
Denoising Diffusion Codebook Models

Guy Ohayon*
Hila Manor*
Tomer Michaeli
Michael Elad

* Equal contribution

Technion — Israel Institute of Technology

ArXiv
Code (Coming Soon)
🤗 Demo
Image Compression
Image Compression
Compressed Image Generation
Compressed Image Restoration
Input
0.030 BPP
Ours
Image Compression
Input
Ours
0.042 BPP
Compressed Image Generation
Input: A man travels down a path in the mountains.
0.010 BPP
Compressed Image Restoration
Input
0.046 BPP
Ours

Abstract

We present a novel generative approach based on Denoising Diffusion Models (DDMs), which produces high-quality image samples along with their losslessly compressed bit-stream representations. This is obtained by replacing the standard Gaussian noise sampling in the reverse diffusion with a selection of noise samples from pre-defined codebooks of fixed iid Gaussian vectors. Surprisingly, we find that our method, termed Denoising Diffusion Codebook Model (DDCM), retains sample quality and diversity of standard DDMs, even for extremely small codebooks. We leverage DDCM and pick the noises from the codebooks that best match a given image, converting our generative model into a highly effective lossy image codec achieving state-of-the-art perceptual image compression results. More generally, by setting other noise selections rules, we extend our compression method to any conditional image generation task (e.g., image restoration), where the generated images are produced jointly with their condensed bit-stream representations. Our work is accompanied by a mathematical interpretation of the proposed compressed conditional generation schemes, establishing a connection with score-based approximations of posterior samplers for the tasks considered.

DDCM can easily be leveraged for perceptual image compression, using any pre-trained DDM.

Original
0.046 BPP
Ours
K=128 K=256 K=1024 K=4096
Original
0.046 BPP
Ours
K=128 K=256 K=1024 K=4096
Original
0.046 BPP
Ours
K=128 K=256 K=1024 K=4096
Original
0.046 BPP
Ours
K=128 K=256 K=1024 K=4096

Original
0.046 BPP
Ours
K=128 K=256 K=1024 K=4096
Original
0.046 BPP
Ours
K=128 K=256 K=1024 K=4096
Original
0.046 BPP
Ours
K=128 K=256 K=1024 K=4096
Original
0.046 BPP
Ours
K=128 K=256 K=1024 K=4096
Original
0.046 BPP
Ours
K=128 K=256 K=1024 K=4096
Original
0.046 BPP
Ours
K=128 K=256 K=1024 K=4096
Original
0.046 BPP
Ours
K=128 K=256 K=1024 K=4096
Original
0.046 BPP
Ours
K=128 K=256 K=1024 K=4096

Compressed Conditional Image Generation

DDCM can be utilized to solve a variety of compressed conditional image generation tasks, where the generated images are produced jointly with their compressed bit-stream representations.

We use DDCM to solve zero-shot image restoration tasks with noiseless linear degradations, while producing their compressed bit-stream representations. We compare our results with DPS and DDNM, compressed using DDCM for a fair comparison, as well as their original uncompressed outputs.

*Comparisons available in full-screen display.

x4 Super Resolution

Ours
DDNM
DPS
0.183 BPP
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096
0.183 BPP
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096

Colorization

Ours
DDNM
DPS
0.183 BPP
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096
0.183 BPP
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096

x4 Super Resolution

Ours
DDNM
DPS
0.183 BPP
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096
0.183 BPP
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096
0.183 BPP
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096

Colorization

Ours
DDNM
DPS
0.183 BPP
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096
0.183 BPP
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096
0.183 BPP
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096
no comp.
K=128 K=512 K=1024 K=4096

DDCM can also be used to solve real-world image restoration tasks, again, while automatically producing the compressed bit-stream representations of the restored images. We compare our results against the compressed outputs of several state-of-the-art algorithms: PMRF, DifFace, and BFRffusion. We also compare with the original uncompressed outputs of these methods.

*Comparisons available in full-screen display.

Ours
PMRF
DifFace
BFRffusion
0.046 BPP
K=256 K=512 K=1024 K=4096
0.046 BPP
K=256 K=512 K=1024 K=4096
0.046 BPP
K=256 K=512 K=1024 K=4096
0.046 BPP
K=256 K=512 K=1024 K=4096
0.046 BPP
K=256 K=512 K=1024 K=4096
0.046 BPP
K=256 K=512 K=1024 K=4096
0.046 BPP
K=256 K=512 K=1024 K=4096
0.046 BPP
K=256 K=512 K=1024 K=4096

Ours
PMRF
DifFace
BFRffusion
0.046 BPP
K=256 K=512 K=1024 K=4096
0.046 BPP
K=256 K=512 K=1024 K=4096
0.046 BPP
K=256 K=512 K=1024 K=4096
0.046 BPP
K=256 K=512 K=1024 K=4096

We propose novel compressed classifier-based and classier-free guidance methods (CG & CFG) based on DDCM, as well as a preliminary algorithm for compressed text-based image editing. For both guidance methods demonstrated below, we set K̃=2 and only alter K. Here, K̃ behaves like a guidance scale, and K controls the bitrate.

Compressed CG


Class: lorikeet

0.015 BPP
K=2 K=4 K=8 K=16

Compressed CFG


"Rainbow over the mountains."

0.042 BPP
K=2 K=4 K=8 K=16

Compressed Editing

"a sculpture of a cat"
→ "a graffiti of a cat"

Original
0.049 BPP

BibTeX

@article{ohayon2025compressedimagegenerationdenoising,
    title         = {Compressed Image Generation with Denoising Diffusion Codebook Models},
    author        = {Ohayon, Guy and Manor, Hila and Michaeli, Tomer and Elad, Michael},
    year          = {2025},
    eprint        = {2502.01189},
    archivePrefix = {arXiv},
    primaryClass  = {eess.IV},
    url           = {https://arxiv.org/abs/2502.01189}, 
}

Acknowledgements

This webpage was originally made by Matan Kleiner with the help of Hila Manor. The code for the original template can be found here.
Icons are taken from font awesome or from Academicons.