Provably scale-covariant continuous hierarchical networks based on scale-normalized differential expressions coupled in cascade. (English) Zbl 1435.68291
Summary: This article presents a theory for constructing hierarchical networks in such a way that the networks are guaranteed to be provably scale covariant. We first present a general sufficiency argument for obtaining scale covariance, which holds for a wide class of networks defined from linear and nonlinear differential expressions expressed in terms of scale-normalized scale-space derivatives. Then, we present a more detailed development of one example of such a network constructed from a combination of mathematically derived models of receptive fields and biologically inspired computations. Based on a functional model of complex cells in terms of an oriented quasi quadrature combination of first- and second-order directional Gaussian derivatives, we couple such primitive computations in cascade over combinatorial expansions over image orientations. Scale-space properties of the computational primitives are analysed, and we give explicit proofs of how the resulting representation allows for scale and rotation covariance. A prototype application to texture analysis is developed, and it is demonstrated that a simplified mean-reduced representation of the resulting QuasiQuadNet leads to promising experimental results on three texture datasets.
MSC:
| 68T07 | Artificial neural networks and deep learning |
| 68T45 | Machine vision and scene understanding |
| 92B20 | Neural networks for/in biological studies, artificial life and related topics |
| 92C20 | Neural biology |
Keywords:
hierarchical network; scale covariance; scale invariance; differential expression; quasi quadrature; complex cell; feature detection; texture analysis; texture classification; scale space; deep learning; computer visionSoftware:
ScatNet; AlexNet; ESPNet; SURF; CUReT; DeepLab; Faster R-CNN; PCANet; Mask R-CNN; ImageNet; SIFT; LIBSVM; RotEqNetReferences:
| [1] | Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097-1105 (2012) |
| [2] | Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (ICLR 2015) (2015). arXiv:1409.1556 |
| [3] | Lecun, Y.; Bengio, Y.; Hinton, G., Deep learning, Nature, 521, 436-444 (2015) |
| [4] | Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2015), pp. 1-9 (2015) |
| [5] | He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2016), pp. 770-778 (2016) |
| [6] | Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2016), pp. 779-788 (2016) |
| [7] | Huang, G., Liu, Z., van der Maaten, L., Weinberger, K.Q.: Densely connected convolutional networks. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2017), pp. 4700-4708 (2017) |
| [8] | Xie, S., Girshick, R., Dollár, P., Tu, Z., He, K.: Aggregated residual transformations for deep neural networks. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2017), pp. 5987-5995 (2017) |
| [9] | Ren, S.; He, K.; Girshick, R.; Sun, J., Faster R-CNN: towards real-time object detection with region proposal networks, IEEE Trans. Pattern Anal. Mach. Intell., 39, 1137-1149 (2017) |
| [10] | Sabour, S., Frosst, N., Hinton, G.E.: Dynamic routing between capsules. In: Advances in Neural Information Processing Systems (NIPS 2017), pp. 3856-3866 (2017) |
| [11] | Cohen, N., Sharir, O., Shashua, A.: On the expressive power of deep learning: a tensor analysis. In: Conference on Learning Theory, pp. 698-728 (2015) |
| [12] | Mahendran, A., Vedaldi, A.: Understanding deep image representations by inverting them. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2015), pp. 5188-5196 (2015) |
| [13] | Tishby, N., Zaslavsky, N.: Deep learning and the information bottleneck principle. In: Information Theory Workshop (ITW 2015), pp. 1-5 (2015) |
| [14] | Mallat, S., Understanding deep convolutional networks, Philos. Trans. R. Soc. A, 374, 20150203 (2016) |
| [15] | Lin, Hw; Tegmark, M.; Rolnick, D., Why does deep and cheap learning work so well?, J. Stat. Phys., 168, 1223-1247 (2017) · Zbl 1373.82061 |
| [16] | Vidal, R., Bruna, J., Giryes, R., Soatto, S.: Mathematics of deep learning (2017). arXiv:1712.04741 |
| [17] | Wiatowski, T.; Bölcskei, H., A mathematical theory of deep convolutional neural networks for feature extraction, IEEE Trans. Inf. Theory, 64, 1845-1866 (2018) · Zbl 1390.94053 |
| [18] | Goldfeld, Z., van den Berg, E., Greenewald, K., Melnyk, I., Nguyen, N., Kingsbury, B., Polyanskiy, Y.: Estimating information flow in neural networks (2018). arXiv:1810.05728 |
| [19] | Achille, A., Rovere, M., Soatto, S.: Critical learning periods in deep neural networks (2017). arXiv:1711.08856 |
| [20] | Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., Fergus, B.: Intriguing properties of neural networks (2013). arXiv:1312.6199 |
| [21] | Nguyen, A., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: high confidence predictions for unrecognizable images. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2015), pp. 427-436 (2015) |
| [22] | Tanay, T., Griffin, L.: A boundary tilting persepective on the phenomenon of adversarial examples (2016). arXiv:1608.07690 |
| [23] | Athalye, A., Sutskever, I.: Synthesizing robust adversarial examples (2017). arXiv:1707.07397 |
| [24] | Su, J., Vargas, D.V., Kouichi, S.: One pixel attack for fooling deep neural networks (2017). arXiv:1710.08864 |
| [25] | Moosavi-Dezfooli, S.M., Fawzi, A., Fawzi, O., Frossard, P.: Universal adversarial perturbations. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2017) (2017) |
| [26] | Baker, N.; Lu, H.; Erlikhman, G.; Kellman, Pj, Deep convolutional networks do not classify based on global object shape, PLoS Comput. Biol., 14, e1006613 (2018) |
| [27] | Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K.: Spatial transformer networks. In: Advances in Neural Information Processing Systems, pp. 2017-2025 (2015) |
| [28] | Cai, Zhaowei; Fan, Quanfu; Feris, Rogerio S.; Vasconcelos, Nuno, A Unified Multi-scale Deep Convolutional Neural Network for Fast Object Detection, Computer Vision - ECCV 2016, 354-370 (2016), Cham: Springer International Publishing, Cham |
| [29] | Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2017) (2017) |
| [30] | Koenderink, Jj; Van Doorn, Aj, Generic neighborhood operators, IEEE Trans. Pattern Anal. Mach. Intell., 14, 597-605 (1992) |
| [31] | Lindeberg, T., Generalized Gaussian scale-space axiomatics comprising linear scale-space, affine scale-space and spatio-temporal scale-space, J. Math. Imaging Vis., 40, 36-81 (2011) · Zbl 1255.68250 |
| [32] | Lindeberg, T., A computational theory of visual receptive fields, Biol. Cybern., 107, 589-635 (2013) · Zbl 1294.92009 |
| [33] | Lindeberg, T., Invariance of visual operations at the level of receptive fields, PLoS ONE, 8, e66990 (2013) |
| [34] | Adelson, E.; Bergen, J., Spatiotemporal energy models for the perception of motion, J. Opt. Soc. Am.A, 2, 284-299 (1985) |
| [35] | Heeger, Dj, Normalization of cell responses in cat striate cortex, Vis. Neurosci., 9, 181-197 (1992) |
| [36] | Lindeberg, T., Feature detection with automatic scale selection, Int. J. Comput. Vis., 30, 77-116 (1998) |
| [37] | Lindeberg, T., Dense scale selection over space, time and space-time, SIAM J. Imaging Sci., 11, 407-441 (2018) · Zbl 1401.65024 |
| [38] | Fukushima, K., Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position, Biol. Cybern., 36, 193-202 (1980) · Zbl 0419.92009 |
| [39] | Riesenhuber, M.; Poggio, T., Hierarchical models of object recognition in cortex, Nature, 2, 1019-1025 (1999) |
| [40] | Serre, T.; Wolf, L.; Bileschi, S.; Riesenhuber, M.; Poggio, T., Robust object recognition with cortex-like mechanisms, IEEE Trans. Pattern Anal. Mach. Intell., 29, 411-426 (2007) |
| [41] | Bruna, J.; Mallat, S., Invariant scattering convolution networks, IEEE Trans. Pattern Anal. Mach. Intell., 35, 1872-1886 (2013) |
| [42] | Poggio, Ta; Anselmi, F., Visual Cortex and Deep Networks: Learning Invariant Representations (2016), Cambridge: MIT Press, Cambridge |
| [43] | Touryan, J.; Felsen, G.; Dan, Y., Spatial structure of complex cell receptive fields measured with natural images, Neuron, 45, 781-791 (2005) |
| [44] | Carandini, M.; Heeger, Dj, Normalization as a canonical neural computation, Nat. Rev. Neurosci., 13, 51-62 (2012) |
| [45] | Goris, Rlt; Simoncelli, Ep; Movshon, Ja, Origin and function of tuning diversity in Macaque visual cortex, Neuron, 88, 819-831 (2015) |
| [46] | Hubel, Dh; Wiesel, Tn, Receptive fields of single neurones in the cat’s striate cortex, J Physiol, 147, 226-238 (1959) |
| [47] | Hubel, Dh; Wiesel, Tn, Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex, J Physiol, 160, 106-154 (1962) |
| [48] | Hubel, Dh; Wiesel, Tn, Brain and Visual Perception: The Story of a 25-Year Collaboration (2005), Oxford: Oxford University Press, Oxford |
| [49] | Lindeberg, Tony, Provably Scale-Covariant Networks from Oriented Quasi Quadrature Measures in Cascade, Lecture Notes in Computer Science, 328-340 (2019), Cham: Springer International Publishing, Cham |
| [50] | Iijima, T., Basic theory on normalization of pattern (in case of typical one-dimensional pattern), Bull. Electrotech. Lab., 26, 368-388 (1962) |
| [51] | Iijima, T., Basic theory on normalization of two-dimensional pattern, Stud. Inf. Control, 1, 15-22 (1963) |
| [52] | Witkin, A.P.: Scale-space filtering. In: Proceedings of 8th International Joint Conference of Artificial intelligence, Karlsruhe, Germany, pp. 1019-1022 (1983) |
| [53] | Koenderink, Jj, The structure of images, Biol. Cybern., 50, 363-370 (1984) · Zbl 0537.92011 |
| [54] | Babaud, J.; Witkin, Ap; Baudin, M.; Duda, Ro, Uniqueness of the Gaussian kernel for scale-space filtering, IEEE Trans. Pattern Anal. Mach. Intell., 8, 26-33 (1986) · Zbl 0574.93054 |
| [55] | Yuille, Al; Poggio, Ta, Scaling theorems for zero-crossings, IEEE Trans. Pattern Anal. Mach. Intell., 8, 15-25 (1986) · Zbl 0575.94001 |
| [56] | Lindeberg, T., Scale-Space Theory in Computer Vision (1993), Berlin: Springer, Berlin · Zbl 0812.68040 |
| [57] | Lindeberg, T., Scale-space theory: a basic tool for analysing structures at different scales, J. Appl. Stat., 21, 225-270 (1994) |
| [58] | Lindeberg, T.: On the axiomatic foundations of linear scale-space. In: Sporring, J., Nielsen, M., Florack, L., Johansen, P. (eds.) Gaussian Scale-Space Theory: Proceedings of Ph.D. School on Scale-Space Theory, pp. 75-97. Copenhagen, Denmark, Springer (1996) |
| [59] | Florack, Lmj, Image Structure. Series in Mathematical Imaging and Vision (1997), Berlin: Springer, Berlin |
| [60] | Weickert, J.; Ishikawa, S.; Imiya, A., Linear scale-space has first been proposed in Japan, J. Math. Imaging Vis., 10, 237-252 (1999) · Zbl 1002.68177 |
| [61] | Ter Haar Romeny, B., Front-End Vision and Multi-Scale Image Analysis (2003), Berlin: Springer, Berlin |
| [62] | Duits, R.; Florack, L.; De Graaf, J.; Ter Haar Romeny, B., On the axioms of scale space theory, J. Math. Imaging Vis., 22, 267-298 (2004) · Zbl 1435.94064 |
| [63] | Lindeberg, T., Edge detection and ridge detection with automatic scale selection, Int. J. Comput. Vis., 30, 117-154 (1998) |
| [64] | Bretzner, L.; Lindeberg, T., Feature tracking with automatic selection of spatial scales, Comput. Vis. Image Underst., 71, 385-392 (1998) |
| [65] | Chomat, Olivier; De Verdiere, Vincent Colin; Hall, Daniela; Crowley, James L., Local Scale Selection for Gaussian Based Description Techniques, Computer Vision - ECCV 2000, 117-134 (2000), Berlin, Heidelberg: Springer Berlin Heidelberg, Berlin, Heidelberg |
| [66] | Mikolajczyk, K.; Schmid, C., Scale and affine invariant interest point detectors, Int. J. Comput. Vis., 60, 63-86 (2004) |
| [67] | Lowe, Dg, Distinctive image features from scale-invariant keypoints, Int. J. Comput. Vis., 60, 91-110 (2004) |
| [68] | Bay, H.; Ess, A.; Tuytelaars, T.; Van Gool, L., Speeded up robust features (SURF), Comput. Vis. Image Underst., 110, 346-359 (2008) |
| [69] | Tuytelaars, T.; Mikolajczyk, K., A Survey on Local Invariant Features. Volume 3(3) of Foundations and Trends in Computer Graphics and Vision (2008), Boston: Now Publishers, Boston |
| [70] | Lindeberg, T.; Ikeuchi, K., Scale selection, Computer Vision: A Reference Guide, 701-713 (2014), Berlin: Springer, Berlin |
| [71] | Lindeberg, T., Image matching using generalized scale-space interest points, J. Math. Imaging Vis., 52, 3-36 (2015) · Zbl 1357.94023 |
| [72] | Lindeberg, T.; Gårding, J., Shape-adapted smoothing in estimation of 3-D depth cues from affine distortions of local 2-D structure, Image Vis. Comput., 15, 415-434 (1997) |
| [73] | Baumberg, A.: Reliable feature matching across widely separated views. In: Proceedings of Computer Vision and Pattern Recognition (CVPR’00), Hilton Head, SC, vol. I, pp. 1774-1781 (2000) |
| [74] | Barnard, E.; Casasent, D., Invariance and neural nets, IEEE Trans. Neural Netw., 2, 498-508 (1991) |
| [75] | Lin, C.H., Lucey, S.: Inverse compositional spatial transformer networks. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2017), pp. 2568-2576 (2017) |
| [76] | Lin, T.Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of International Conference on Computer Vision (ICCV 2017), pp. 2980-2988 (2017) |
| [77] | He, K., Gkioxari, G., Dollár, P., Girshick, R.: Mask R-CNN. In: Proceedings of International Conference on Computer Vision (ICCV 2017), pp. 2961-2969 (2017) |
| [78] | Hu, P., Ramanan, D.: Finding tiny faces. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2017), pp. 951-959 (2017) |
| [79] | Ren, S.; He, K.; Girshick, R.; Zhang, X.; Sun, J., Object detection networks on convolutional feature maps, IEEE Trans. Pattern Anal. Mach. Intell., 39, 1476-1481 (2016) |
| [80] | Nah, S., Kim, T.H., Lee, K.M.: Deep multi-scale convolutional neural network for dynamic scene deblurring. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2017), pp. 3883-3891 (2017) |
| [81] | Chen, Lc; Papandreou, G.; Kokkinos, I.; Murphy, K.; Yuille, Al, Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs, IEEE Trans. Pattern Anal. Mach. Intell., 40, 834-848 (2017) |
| [82] | Singh, B., Davis, L.S.: An analysis of scale invariance in object detection—SNIP. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2018), pp. 3578-3587 (2018) |
| [83] | Yang, F., Choi, W., Lin, Y.: Exploit all the layers: Fast and accurate CNN object detector with scale dependent pooling and cascaded rejection classifiers. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2016), pp. 2129-2137 (2016) |
| [84] | Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions (2015). arXiv:1511.07122 |
| [85] | Yu, F., Koltun, V., Funkhouser, T.: Dilated residual networks. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2017), pp. 472-480 (2017) |
| [86] | Mehta, Sachin; Rastegari, Mohammad; Caspi, Anat; Shapiro, Linda; Hajishirzi, Hannaneh, ESPNet: Efficient Spatial Pyramid of Dilated Convolutions for Semantic Segmentation, Computer Vision - ECCV 2018, 561-580 (2018), Cham: Springer International Publishing, Cham |
| [87] | Zhang, R., Tang, S., Zhang, Y., Li, J., Yan, S.: Scale-adaptive convolutions for scene parsing. In: Proceedings of International Conference on Computer Vision (ICCV 2017), pp. 2031-2039 (2017) |
| [88] | Wang, H., Kembhavi, A., Farhadi, A., Yuille, A.L., Rastegari, M.: ELASTIC: improving CNNs with dynamic scaling policies. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2019), pp. 2258-2267 (2019) |
| [89] | Chen, Y., Fang, H., Xu, B., Yan, Z., Kalantidis, Y., Rohrbach, M., Yan, S., Feng, J.: Drop an octave: reducing spatial redundancy in convolutional neural networks with octave convolution (2019). arXiv:1904.05049 |
| [90] | Henriques, Jf; Vedaldi, A., Warped convolutions: efficient invariance to spatial transformations, Int. Conf. Mach. Learn., 70, 1461-1469 (2017) |
| [91] | Esteves, C., Allen-Blanchette, C., Zhou, X., Daniilidis, K.: Polar transformer networks. In: International Conference on Learning Representations (ICLR 2018) (2018) |
| [92] | Lenc, Karel; Vedaldi, Andrea, Learning Covariant Feature Detectors, Lecture Notes in Computer Science, 100-117 (2016), Cham: Springer International Publishing, Cham |
| [93] | Zhang, X., Yu, F.X., Karaman, S., Chang, S.F.: Learning discriminative and transformation covariant local feature detectors. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2017), pp. 6818-6826 (2017) |
| [94] | Dieleman, S., Fauw, J.D., Kavukcuoglu, K.: Exploiting cyclic symmetry in convolutional neural networks. In: International Conference on Machine Learning (ICML 2016) (2016) |
| [95] | Laptev, D., Savinov, N., Buhmann, J.M., Pollefeys, M.: TI-pooling: transformation-invariant pooling for feature learning in convolutional neural networks. In: Proceedings Computer Vision and Pattern Recognition (CVPR 2016), pp. 289-297 (2016) |
| [96] | Worrall, D.E., Garbin, S.J., Turmukhambetov, D., Brostow, G.J.: Harmonic networks: deep translation and rotation equivariance. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2017), pp. 5028-5037 (2017) |
| [97] | Zhou, Y., Ye, Q., Qiu, Q., Jiao, J.: Oriented response networks. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2017), pp. 519-528 (2017) |
| [98] | Marcos, D., Volpi, M., Komodakis, N., Tuia, D.: Rotation equivariant vector field networks. In: Proceedings of International Conference on Computer Vision (ICCV 2017), pp. 5048-5057 (2017) |
| [99] | Cohen, T.S., Welling, M.: Steerable CNNs. In: International Conference on Learning Representations (ICLR 2017) (2017) |
| [100] | Weiler, M., Geiger, M., Welling, M., Boomsma, W., Cohen, T.: 3D steerable CNNs: learning rotationally equivariant features in volumetric data. In: Advances in Neural Information Processing Systems (NIPS 2018), pp. 10381-10392 (2018) |
| [101] | Weiler, M., Hamprecht, F.A., Storath, M.: Learning steerable filters for rotation equivariant CNNs. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2018), pp. 849-858 (2018) |
| [102] | Worrall, Daniel; Brostow, Gabriel, CubeNet: Equivariance to 3D Rotation and Translation, Computer Vision - ECCV 2018, 585-602 (2018), Cham: Springer International Publishing, Cham |
| [103] | Cheng, G.; Han, J.; Zhou, P.; Xu, D., Learning rotation-invariant and Fisher discriminative convolutional neural networks for object detection, IEEE Trans. Image Process., 28, 265-278 (2018) · Zbl 1409.94825 |
| [104] | Dieleman, S.; Willett, Kw; Dambre, J., Rotation-invariant convolutional neural networks for galaxy morphology prediction, Mon. Not. R. Astron. Soc., 450, 1441-1459 (2015) |
| [105] | Cheng, G.; Zhou, P.; Han, J., Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images, IEEE Trans. Geosci. Remote Sens., 54, 7405-7415 (2016) |
| [106] | Wang, Q.; Zheng, Y.; Yang, G.; Jin, W.; Chen, X.; Yin, Y., Multiscale rotation-invariant convolutional neural networks for lung texture classification, IEEE J. Biomed. Health Inform., 22, 184-195 (2017) |
| [107] | Bekkers, Erik J.; Lafarge, Maxime W.; Veta, Mitko; Eppenhof, Koen A. J.; Pluim, Josien P. W.; Duits, Remco, Roto-Translation Covariant Convolutional Networks for Medical Image Analysis, Medical Image Computing and Computer Assisted Intervention - MICCAI 2018, 440-448 (2018), Cham: Springer International Publishing, Cham |
| [108] | Andrearczyk, V., Depeursinge, A.: Rotational 3D texture classification using group equivariant CNNs (2018). arXiv:1810.06889 |
| [109] | Cohen, T., Welling, M.: Group equivariant convolutional networks. In: International Conference on Machine Learning (ICML 2016), pp. 2990-2999 (2016) |
| [110] | Kondor, R., Trivedi, S.: On the generalization of equivariance and convolution in neural networks to the action of compact groups (2018). arXiv:1802.03690 |
| [111] | Lindeberg, T.; Hawkes, P., Generalized axiomatic scale-space theory, Advances in Imaging and Electron Physics, 1-96 (2013), Amsterdam: Elsevier, Amsterdam |
| [112] | Lindeberg, T.: Normative theory of visual receptive fields (2017). arXiv:1701.06333 · Zbl 1294.92009 |
| [113] | Lindeberg, T., Time-causal and time-recursive spatio-temporal receptive fields, J. Math. Imaging Vis., 55, 50-88 (2016) · Zbl 1334.94034 |
| [114] | Jhuang, H., Serre, T., Wolf, L., Poggio, T.: A biologically inspired system for action recognition. In: International Conference on Computer Vision (ICCV’07), pp. 1-8 (2007) |
| [115] | Sifre, L., Mallat, S.: Rotation, scaling and deformation invariant scattering for texture discrimination. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2013), pp. 1233-1240 (2013) |
| [116] | Oyallon, E., Mallat, S.: Deep roto-translation scattering for object classification. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2015), pp. 2865-2873 (2015) |
| [117] | Chan, Th; Jia, K.; Gao, S.; Lu, J.; Zeng, Z.; Ma, Y., PCANet: a simple deep learning baseline for image classification?, IEEE Trans. Image Process., 24, 5017-5032 (2015) · Zbl 1408.94080 |
| [118] | Luan, S.; Chen, C.; Zhang, B.; Han, J.; Liu, J., Gabor convolutional networks, IEEE Trans. Pattern Anal. Mach. Intell., 27, 4357-4366 (2018) |
| [119] | Jacobsen, J.J., van Gemert, J., Lou, Z., Smeulders, A.W.M.: Structured receptive fields in CNNs. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2016), pp. 2610-2619 (2016) |
| [120] | Shelhamer, E., Wang, D., Darrell, T.: Blurring the line between structure and learning to optimize and adapt receptive fields (2019). arXiv:1904.11487 |
| [121] | Roux, N.L., Bengio, Y.: Continuous neural networks. In: Artificial Intelligence and Statistics (AISTATS 2007). Volume 2 of Proceedings of Machine Learning Research, pp. 404-411 (2007) |
| [122] | Liu, L.; Chen, J.; Fieguth, P.; Zhao, G.; Chellappa, R.; Pietikäinen, M., From BoW to CNN: two decades of texture representation for texture classification, Int. J. Comput. Vis., 127, 74-109 (2019) |
| [123] | Gabor, D., Theory of communication, IEE J., 93, 429-457 (1946) |
| [124] | Bracewell, Rn, The Fourier Transform and Its Applications (1999), New York: McGraw-Hill, New York |
| [125] | Loog, M.: The jet metric. In: International Conference on Scale Space and Variational Methods in Computer Vision (SSVM 2007). Volume 4485 of Springer LNCS, pp. 25-31 (2007) |
| [126] | Griffin, Ld, The second order local-image-structure solid, IEEE Trans. Pattern Anal. Mach. Intell., 29, 1355-1366 (2007) |
| [127] | Johnson, En; Hawken, Mj; Shapley, R., The orientation selectivity of color-responsive neurons in Macaque V1, J. Neurosci., 28, 8096-8106 (2008) |
| [128] | Koenderink, Jj; Van Doorn, Aj, Receptive field families, Biol. Cybern., 63, 291-298 (1990) · Zbl 0696.92009 |
| [129] | Valois, Rld; Cottaris, Np; Mahon, Le; Elfer, Sd; Wilson, Ja, Spatial and temporal receptive fields of geniculate and cortical cells and directional selectivity, Vis. Res., 40, 3685-3702 (2000) |
| [130] | Westö, J.; May, Pjc, Describing complex cells in primary visual cortex: a comparison of context and multi-filter LN models, J. Neurophysiol., 120, 703-719 (2018) |
| [131] | Movshon, Ja; Thompson, Ed; Tolhurst, Dj, Receptive field organization of complex cells in the cat’s striate cortex, J. Physiol., 283, 79-99 (1978) |
| [132] | Emerson, Rc; Citron, Mc; Vaughn, Wj; Klein, Sa, Nonlinear directionally selective subunits in complex cells of cat striate cortex, J. Neurophysiol., 58, 33-65 (1987) |
| [133] | Touryan, J.; Lau, B.; Dan, Y., Isolation of relevant visual features from random stimuli for cortical complex cells, J. Neurosci., 22, 10811-10818 (2002) |
| [134] | Rust, Nc; Schwartz, O.; Movshon, Ja; Simoncelli, Ep, Spatiotemporal elements of macaque V1 receptive fields, Neuron, 46, 945-956 (2005) |
| [135] | Serre, T., Riesenhuber, M.: Realistic modeling of simple and complex cell tuning in the HMAX model, and implications for invariant object recognition in cortex. Technical Report AI Memo 2004-017, MIT Computer Science and Artifical Intelligence Laboratory (2004) |
| [136] | Einhäuser, W.; Kayser, C.; König, P.; Körding, Kp, Learning the invariance properties of complex cells from their responses to natural stimuli, Eur. J. Neurosci., 15, 475-486 (2002) |
| [137] | Kording, Kp; Kayser, C.; Einhäuser, W.; Konig, P., How are complex cell properties adapted to the statistics of natural stimuli?, J. Neurophysiol., 91, 206-212 (2004) |
| [138] | Merolla, P., Boahn, K.: A recurrent model of orientation maps with simple and complex cells. In: Advances in Neural Information Processing Systems (NIPS 2004), pp. 995-1002 (2004) |
| [139] | Berkes, P.; Wiskott, L., Slow feature analysis yields a rich repertoire of complex cell properties, J. Vis., 5, 579-602 (2005) |
| [140] | Carandini, M., What simple and complex cells compute, J. Physiol., 577, 463-466 (2006) |
| [141] | Hansard, M.; Horaud, R., A differential model of the complex cell, Neural Comput., 23, 2324-2357 (2011) · Zbl 1231.92022 |
| [142] | Yamins, Dlk; Dicarlo, Jj, Using goal-driven deep learning models to understand sensory cortex, Nat. Neurosci., 19, 356-365 (2016) |
| [143] | Hadji, I., Wildes, R.P.: A spatiotemporal oriented energy network for dynamic texture recognition. In: Proceedings of International Conference on Computer Vision (ICCV 2017), pp. 3066-3074 (2017) |
| [144] | Mallikarjuna, P.; Targhi, At; Fritz, M.; Hayman, E.; Caputo, B.; Eklundh, Jo, The KTH-TIPS2 database (2006), Stockholm: KTH Royal Institute of Technology, Stockholm |
| [145] | Lindeberg, T., Scale-space for discrete signals, IEEE Trans. Pattern Anal. Mach. Intell., 12, 234-254 (1990) |
| [146] | Abramowitz, M.; Stegun, Ia, Handbook of Mathematical Functions. Applied Mathematics Series (1964), Gaithersburg: National Bureau of Standards, Gaithersburg · Zbl 0171.38503 |
| [147] | Lindeberg, T., Discrete derivative approximations with scale-space properties: a basis for low-level feature extraction, J. Math. Imaging Vis., 3, 349-376 (1993) |
| [148] | Dana, Kj; Van Ginneken, B.; Nayar, Sk; Koenderink, Jj, Reflectance and texture of real-world surfaces, ACM Trans. Graph., 18, 1-34 (1999) |
| [149] | Varma, M.; Zisserman, A., A statistical approach to material classification using image patch exemplars, IEEE Trans. Pattern Anal. Mach. Intell., 31, 2032-2047 (2009) |
| [150] | Xu, Y., Yang, X., Ling, H., Ji, H.: A new texture descriptor using multifractal analysis in multi-orientation wavelet pyramid. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2010), pp. 161-168 (2010) |
| [151] | Cimpoi, M., Maji, S., Vedaldi, A.: Deep filter banks for texture recognition and segmentation. In: Proceedings of Computer Vision and Pattern Recognition (CVPR 2015), pp. 3828-3836 (2015) |
| [152] | Liu, L.; Lao, S.; Fieguth, Pw; Guo, Y.; Wang, X.; Pietikäinen, M., Median robust extended local binary pattern for texture classification, IEEE Trans. Image Process., 25, 1368-1381 (2016) · Zbl 1408.94395 |
| [153] | Liu, L.; Long, Y.; Fieguth, Pw; Lao, S.; Zhao, G., BRINT: binary rotation invariant and noise tolerant texture classification, IEEE Trans. Image Process., 23, 3071-3084 (2014) · Zbl 1374.94226 |
| [154] | Schaefer, G., Doshi, N.P.: Multi-dimensional local binary pattern descriptors for improved texture analysis. In: Proceedings of International Conference on Pattern Recognition (ICPR 2012), pp. 2500-2503 (2012) |
| [155] | Ojala, T.; Pietikäinen, M.; Maenpaa, T., Multiresolution gray-scale and rotation invariant texture classification with local binary patterns, IEEE Trans. Pattern Anal. Mach. Intell., 24, 971-987 (2002) |
| [156] | Chang, Cc; Lin, Cj, LIBSVM: a library for support vector machines, ACM Trans. Intell. Syst. Technol., 2, 27 (2011) |
| [157] | Liu, L.; Fieguth, P.; Guo, Y.; Wang, Z.; Pietikäinen, M., Local binary features for texture classification: taxonomy and experimental study, Pattern Recognit., 62, 135-160 (2017) |
| [158] | Crosier, M.; Griffin, Ld, Using basic image features for texture classification, Int. J. Comput. Vis., 88, 447-460 (2010) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
