ahmedgh970 / convnext-charm

Official Tensorflow implementation of ConvNeXt-ChARM: ConvNeXt-based Transform for Efficient Neural Image Compression.

ConvNeXt-ChARM: ConvNeXt-based Transform for Efficient Neural Image Compression

Official TensorFlow implementation of ConvNeXt-ChARM: ConvNeXt-based Transform for Efficient Neural Image Compression.

• ConvNeXt-ChARM
  • Tags
  • Overall ConvNeXt-ChARM Framework
  • Disclaimer
  • Documentation
  • Requirements
  • Folder Structure
  • CLI-Usage
  • Rate-Distortion Coding Performance
  • Citation
  • License

Tags

Swin Transformer ConvNeXt Learning-based Codecs Image Compression TensorFlow

Overall ConvNeXt-ChARM Framework

Disclaimer

Please do not hesitate to open an issue to inform of any problem you may find within this repository. Also, you can email me for questions or comments.

Documentation

• This repository is built upon the official TensorFlow implementation of Channel-Wise Autoregressive Entropy Models for Learned Image Compression. This baseline is referred to as Conv-ChARM
• We provide lightweight versions of the models by removing the latent residual prediction (LRP) transform and slicing latent means and scales, as done in the Tensorflow reimplementation of SwinT-ChARM from the original paper TRANSFORMER-BASED TRANSFORM CODING.
• Refer to the TensorFlow Compression (TFC) library to build your own ML models with end-to-end optimized data compression built in.
• Refer to the API documentation for a complete classes and functions description of the TensorFlow Compression (TFC) library.

Requirements

Python >= 3.6 tensorflow_compression tensorflow_datasets tensorflow_addons einops

All packages used in this repository are listed in requirements.txt. To install those, run:

pip install -r requirements.txt

Folder Structure

ConvNeXt-ChARM
│
├── conv-charm.py                 # Conv-ChARM Model
├── conv-charm_lrp.py             # Conv-ChARM Model with latent residual prediction (LRP) 
├── convnext-charm.py             # ConvNeXt-ChARM Model
├── convnext-charm_lrp.py         # ConvNeXt-ChARM Model with latent residual prediction (LRP) 
├── swint-charm.py                # SwinT-ChARM Model
├── swint-charm_lrp.py            # SwinT-ChARM Model with latent residual prediction (LRP) 
├── utils.py                      # Utility scripts
│
├── layers/
│   └── convNext.py/              # ConvNeXt block layers
│   └── swinTransformer.py/       # Swin Transformer block layers
|
└── figures/                      # Documentation figures

CLI Usage

Every model can be trained and tested individually using:

python convnext-charm.py train

python convnext-charm.py evaluate

Rate-Distortion coding performance

Table 1. BD-rate↓ performance of BPG (4:4:4), SwinT-ChARM, and ConvNeXt-ChARM compared to the VTM-18.0 for the four considered datasets.

Dataset	BPG444	SwinT-ChARM	ConvNeXt-ChARM
Kodak	20.73%	-3.47%	-4.90%
Tecnick	27.03%	-6.52%	-7.56%
JPEG-AI	28.14%	-0.23%	-1.17%
CLIC21	26.54%	-5.86%	-7.36%
Average	25.61%	-4.02%	-5.24%

Citation

If you use this library for research purposes, please cite:

@inproceedings{ghorbel2023convnextcharm,
  title={ConvNeXt-ChARM: ConvNeXt-based Transform for Efficient Neural Image Compression},
  author={Ghorbel, Ahmed and Hamidouche, Wassim and Luce, Morin},
  booktitle={},
  year={2023}
}

License

This project is licensed under the MIT License. See LICENSE for more details