Abstract
Developing and integrating advanced image sensors with novel algorithms in camera systems is prevalent with the increasing demand for computational photography and imaging on mobile platforms. However, the lack of high-quality data for research and the rare opportunity for in-depth exchange of views from industry and academia constrain the development of mobile intelligent photography and imaging (MIPI). To bridge the gap, we introduce the first MIPI challenge including five tracks focusing on novel image sensors and imaging algorithms. In this paper, Quad Joint Remosaic and Denoise, one of the five tracks, working on the interpolation of Quad CFA to Bayer at full-resolution is introduced. The participants were provided with a new dataset including 70 (training) and 15 (validation) scenes of high-quality Quad and Bayer pair. In addition, for each scene, Quad of different noise level were provided at 0 dB, 24 dB and 42 dB. All the data were captured using a Quad sensor in both outdoor and indoor conditions. The final results are evaluated using objective metrics including PSNR, SSIM [6], LPIPS [10] and KLD. A detailed description of all models developed in this challenge is provided in this paper. More details of this challenge and the link to the dataset can be found in https://github.com/mipi-challenge/MIPI2022.
Q. Yang, J. Jiang, C. Li, S. Zhou, R. Feng, W. Sun, Q. Zhu, C. C. Loz, J. Gu are the MIPI 2022 challenge organizers.
The other authors participated in the challenge. Please refer to Appendix A for details.
MIPI 2022 challenge website: http://mipi-challenge.org/.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abdelhamed, A., Afifi, M., Timofte, R., Brown, M.S.: NTIRE 2020 challenge on real image denoising: dataset, methods and results. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops (2020)
Kim, B.-H., Song, J., Ye, J.C., Baek, J.H.: PyNET-CA: enhanced PyNET with channel attention for end-to-end mobile image signal processing. In: Bartoli, A., Fusiello, A. (eds.) ECCV 2020. LNCS, vol. 12537, pp. 202–212. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-67070-2_12
Shi, W., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1874–1883 (2016)
Sun, B., Zhang, Y., Jiang, S., Fu, Y.: Hybrid pixel-unshuffled network for lightweight image super-resolution. arXiv preprint arXiv:2203.08921 (2022)
Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., Li, H.: Uformer: a general U-shaped transformer for image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17683–17693 (2022)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Zamir, S.W., et al.: Learning enriched features for real image restoration and enhancement. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12370, pp. 492–511. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58595-2_30
Zhang, K., et al.: Practical blind denoising via swin-conv-unet and data synthesis. arXiv preprint (2022)
Zhang, K., Li, Y., Zuo, W., Zhang, L., Van Gool, L., Timofte, R.: Plug-and-play image restoration with deep denoiser prior. IEEE Trans. Pattern Anal. Mach. Intell. (2021)
Zhang, R., Isola, P., Efros, A.A., Shechtman, E., Wang, O.: The unreasonable effectiveness of deep features as a perceptual metric. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2018)
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., Fu, Y.: Image super-resolution using very deep residual channel attention networks. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 294–310. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_18
Acknowledgements
We thank Shanghai Artificial Intelligence Laboratory, Sony, and Nanyang Technological University to sponsor this MIPI 2022 challenge. We thank all the organizers and participants for their great work.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
A Teams and Affiliations
A Teams and Affiliations
BITSpectral
Title: Cross-Patch Attention Network for Quad Joint Remosaic and Denoise
Members: Zhen Wang, (wzhstruggle@bit.edu.cn), Daoyu Li, Yuzhe Zhang, Lintao Peng, Xuyang Chang, Yinuo Zhang
Affiliations: Beijing Institute of Technology
HITZST01
Title: Multi-scale convolution network (JDRMCANet) with joint denoise and remosaic.
Members: \(^{1}\)Yaqi Wu (titimasta@163.com), \(^{2}\)Xun Wu, \(^{3}\)Zhihao Fan, \(^{4}\)Chengjie Xia, Feng Zhang,
Affiliations: \(^{1}\)Harbin Institute of Technology, Harbin, 150001, China, \(^{2}\)Tsinghua University, Beijing, 100084, China, \(^{3}\)University of Shanghai for Science and Technology, Shanghai, 200093, China, \(^{4}\)Zhejiang University, Hangzhou, 310027, China
IMEC-IPI & NPU-MPI
Title: Hard Datasets and Invariance guided Parallel Swin-Conv-Attention Network for Quad Joint Remosaic and Denoise.
Members: \(^{1}\)Haijin Zeng (Haijin.Zeng@imec.be), \(^{2}\)Kai Feng, \(^{2}\)Yongqiang Zhao, \(^{1}\)Hiep Quang Luong, \(^{1}\)Jan Aelterman, \(^{1}\)Anh Minh Truong, and \(^{1}\)Wilfried Philips.
Affiliations: \(^{1}\)IMEC & Ghent University \(^{2}\)Northwestern Polytechnical University
JHC-SJTU
Title: Jointly Remosaicing and Denoising for Quad Bayer with Multi-Resolution Feature Extraction and Fusion.
Members: \(^{1}\)Xiaohong Liu (xiaohongliu@sjtu.edu.cn), \(^{1}\)Jun Jia, \(^{1}\)Hanchi Sun, \(^{1}\)Guangtao Zhai, \(^{2}\)Anlong Xiao, \(^{2}\)Qihang Xu
Affiliations: \(^{1}\)Shanghai Jiao Tong University, \(^{2}\)Transsion
MegNR
Title: HAUformer: Hybrid Attention-guided U-shaped Transformer for Quad Remosaic Image Restoration.
Members: Ting Jiang (jiangting@megvii.com), Qi Wu, Chengzhi Jiang, Mingyan Han, Xinpeng Li, Wenjie Lin, Youwei Li, Haoqiang Fan and Shuaicheng Liu
Affiliations: Megvii Technology
op-summer-po
Title: Two LPIPS Functions in Raw and RGB domains for Quad-Bayer Joint Remosaic and Denoise.
Members: \(^{1}\)Rongyuan Wu (1104138645@qq.com), \(^{1}\)Lingchen Sun, \(^{1,2}\)Qiaosi Yi
Affiliations: \(^{1}\)OPPO Research Institute, \(^{2}\)East China Normal University
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yang, Q. et al. (2023). MIPI 2022 Challenge on Quad-Bayer Re-mosaic: Dataset and Report. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision – ECCV 2022 Workshops. ECCV 2022. Lecture Notes in Computer Science, vol 13805. Springer, Cham. https://doi.org/10.1007/978-3-031-25072-9_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-25072-9_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25071-2
Online ISBN: 978-3-031-25072-9
eBook Packages: Computer ScienceComputer Science (R0)