Official PyTorch implementation of the paper Learning Texture Transformer Network for Image Super-Resolution accepted in CVPR 2020.

• Introduction
  • Contribution
  • Approach overview
  • Main results
• Requirements and dependencies
• Model
• Quick test
• Dataset prepare
• Evaluation
• Train
• Citation
• Contact

We proposed an approach named TTSR for RefSR task. Compared to SISR, RefSR has an extra high-resolution reference image whose textures can be utilized to help super-resolve low-resolution input.

1. We are one of the first to introduce the transformer architecture into image generation tasks. More specifically, we propose a texture transformer with four closely-related modules for image SR which achieves significant improvements over SOTA approaches.
2. We propose a novel cross-scale feature integration module for image generation tasks which enables our approach to learn a more powerful feature representation by stacking multiple texture transformers.

• python 3.7 (recommend to use Anaconda)
• python packages: pip install opencv-python imageio
• pytorch >= 1.1.0
• torchvision >= 0.4.0

Pre-trained models can be downloaded from onedrive, baidu cloud(0u6i), google drive.

• TTSR-rec.pt: trained with only reconstruction loss
• TTSR.pt: trained with all losses

1. Clone this github repo

git clone https://github.com/FuzhiYang/TTSR.git
cd TTSR

2. Download pre-trained models and modify "model_path" in test.sh
3. Run test

sh test.sh

4. The results are in "save_dir" (default: ./test/demo/output)

1. Download CUFED train set and CUFED test set
2. Make dataset structure be:

• CUFED
  • train
    • input
    • ref
  • test
    • CUFED5

1. Prepare CUFED dataset and modify "dataset_dir" in eval.sh
2. Download pre-trained models and modify "model_path" in eval.sh
3. Run evaluation

sh eval.sh

4. The results are in "save_dir" (default: ./eval/CUFED/TTSR)

1. Prepare CUFED dataset and modify "dataset_dir" in train.sh
2. Run training

sh train.sh

3. The training results are in "save_dir" (default: ./train/CUFED/TTSR)

We also sincerely recommend some other excellent works related to us. ✨

• FTVSR: Learning Spatiotemporal Frequency-Transformer for Compressed Video Super-Resolution
• TTVSR: Learning Trajectory-Aware Transformer for Video Super-Resolution
• CKDN: Learning Conditional Knowledge Distillation for Degraded-Reference Image Quality Assessment

@InProceedings{yang2020learning,
author = {Yang, Fuzhi and Yang, Huan and Fu, Jianlong and Lu, Hongtao and Guo, Baining},
title = {Learning Texture Transformer Network for Image Super-Resolution},
booktitle = {CVPR},
year = {2020},
month = {June}
}

If you meet any problems, please describe them in issues or contact:

• Fuzhi Yang: yfzcopy0702@sjtu.edu.cn