image-super-resolution

Image Super-Resolution (ISR)

The Image Super-Resolution (ISR) project is designed to transform low-resolution images into high-resolution ones, enhancing their quality and details. This task is crucial in various applications like improving the quality of old photographs, medical imaging, and more.

Overview

The ISR project offers Keras implementations of various Residual Dense Networks specifically crafted for Single Image Super-Resolution (ISR). It provides scripts for training these networks, utilizing both content and adversarial loss components to achieve superior resolution outputs.

Implemented Networks

1. Residual Dense Network (RDN): This network, referenced from the paper Residual Dense Network for Image Super-Resolution by Zhang et al., is crucial for scaling up the image size while maintaining clarity.

2. Residual in Residual Dense Network (RRDN): Derived from ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks by Wang et al., this network is a further improvement adding layers for better output.

3. VGG19 Multi-output Network: Used for deep feature extraction when computing perceptual loss, helping improve image quality.

4. Custom Discriminator Network: Based on the SRGANs model described in Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network (SRGANS by Ledig et al.), this aids in training the networks by evaluating the realism of generated images.

Detailed documentation is available at https://idealo.github.io/image-super-resolution/.

Pre-trained Models

The ISR provides four pre-trained models for immediate use:

• RDN Models: Available in variations such as "psnr-large," "psnr-small," and "noise-cancel" for different use cases.
• RRDN Model: Trained with adversarial and VGG features losses, named "gans."

Example usage of RRDN model:

model = RRDN(weights='gans')

Installation

There are two primary ways to install the ISR package:

• Via PyPI (recommended):

pip install ISR

• From GitHub Source:

git clone https://github.com/idealo/image-super-resolution
cd image-super-resolution
python setup.py install

Usage Instructions

Predicting with Pre-trained Models

To upscale an image:

1. Load the image and prepare it:

   from PIL import Image
   import numpy as np
   
   img = Image.open('path_to_your_image.jpg')
   lr_img = np.array(img)
   

2. Use a pre-trained model to get the high-resolution image:

   from ISR.models import RDN
   
   rdn = RDN(weights='psnr-small')
   sr_img = rdn.predict(lr_img)
   Image.fromarray(sr_img)
   

For larger images, use the by_patch_of_size option to avoid memory errors:

sr_img = model.predict(lr_img, by_patch_of_size=50)

Training Custom Models

Train a model using custom inputs:

1. Initialize the models:

   from ISR.models import RRDN, Discriminator, Cut_VGG19
   
   rrdn = RRDN(...)
   f_ext = Cut_VGG19(...)
   discr = Discriminator(...)
   

2. Set up the Trainer and launch the training process:

   from ISR.train import Trainer
   
   trainer = Trainer(generator=rrdn, ...)
   trainer.train(epochs=80, ...)
   

Additional Information

Explore insightful articles to understand the training methodologies behind ISR in parts 1 and 2 published on Medium, providing deep dives into model efficacy and improvements over time.

Contributing

ISR thrives on community contributions ranging from new datasets and models to innovative architectures. Contributors are encouraged to improve model performance or add new capabilities.

Citation

To acknowledge usage of ISR in any research or publications:

@misc{cardinale2018isr,
  title={ISR},
  author={Francesco Cardinale et al.},
  year={2018},
  howpublished={\url{https://github.com/idealo/image-super-resolution}},
}

Maintainers

• Francesco Cardinale (cfrancesco)
• Dat Tran (datitran)

The ISR project, licensed under Apache 2.0, is a powerful tool for anyone looking to enhance image quality, backed by thorough documentation and a supportive community encouraging continuous improvement and collaboration.