-
A Joint Cross-Attention Model for Audio-Visual Fusion in Dimensional Emotion Recognition
Authors:
R. Gnana Praveen,
Wheidima Carneiro de Melo,
Nasib Ullah,
Haseeb Aslam,
Osama Zeeshan,
Théo Denorme,
Marco Pedersoli,
Alessandro Koerich,
Simon Bacon,
Patrick Cardinal,
Eric Granger
Abstract:
Multimodal emotion recognition has recently gained much attention since it can leverage diverse and complementary relationships over multiple modalities (e.g., audio, visual, biosignals, etc.), and can provide some robustness to noisy modalities. Most state-of-the-art methods for audio-visual (A-V) fusion rely on recurrent networks or conventional attention mechanisms that do not effectively lever…
▽ More
Multimodal emotion recognition has recently gained much attention since it can leverage diverse and complementary relationships over multiple modalities (e.g., audio, visual, biosignals, etc.), and can provide some robustness to noisy modalities. Most state-of-the-art methods for audio-visual (A-V) fusion rely on recurrent networks or conventional attention mechanisms that do not effectively leverage the complementary nature of A-V modalities. In this paper, we focus on dimensional emotion recognition based on the fusion of facial and vocal modalities extracted from videos. Specifically, we propose a joint cross-attention model that relies on the complementary relationships to extract the salient features across A-V modalities, allowing for accurate prediction of continuous values of valence and arousal. The proposed fusion model efficiently leverages the inter-modal relationships, while reducing the heterogeneity between the features. In particular, it computes the cross-attention weights based on correlation between the combined feature representation and individual modalities. By deploying the combined A-V feature representation into the cross-attention module, the performance of our fusion module improves significantly over the vanilla cross-attention module. Experimental results on validation-set videos from the AffWild2 dataset indicate that our proposed A-V fusion model provides a cost-effective solution that can outperform state-of-the-art approaches. The code is available on GitHub: https://github.com/praveena2j/JointCrossAttentional-AV-Fusion.
△ Less
Submitted 6 July, 2024; v1 submitted 28 March, 2022;
originally announced March 2022.
-
Facial Expression Analysis Using Decomposed Multiscale Spatiotemporal Networks
Authors:
Wheidima Carneiro de Melo,
Eric Granger,
Miguel Bordallo Lopez
Abstract:
Video-based analysis of facial expressions has been increasingly applied to infer health states of individuals, such as depression and pain. Among the existing approaches, deep learning models composed of structures for multiscale spatiotemporal processing have shown strong potential for encoding facial dynamics. However, such models have high computational complexity, making for a difficult deplo…
▽ More
Video-based analysis of facial expressions has been increasingly applied to infer health states of individuals, such as depression and pain. Among the existing approaches, deep learning models composed of structures for multiscale spatiotemporal processing have shown strong potential for encoding facial dynamics. However, such models have high computational complexity, making for a difficult deployment of these solutions. To address this issue, we introduce a new technique to decompose the extraction of multiscale spatiotemporal features. Particularly, a building block structure called Decomposed Multiscale Spatiotemporal Network (DMSN) is presented along with three variants: DMSN-A, DMSN-B, and DMSN-C blocks. The DMSN-A block generates multiscale representations by analyzing spatiotemporal features at multiple temporal ranges, while the DMSN-B block analyzes spatiotemporal features at multiple ranges, and the DMSN-C block analyzes spatiotemporal features at multiple spatial sizes. Using these variants, we design our DMSN architecture which has the ability to explore a variety of multiscale spatiotemporal features, favoring the adaptation to different facial behaviors. Our extensive experiments on challenging datasets show that the DMSN-C block is effective for depression detection, whereas the DMSN-A block is efficient for pain estimation. Results also indicate that our DMSN architecture provides a cost-effective solution for expressions that range from fewer facial variations over time, as in depression detection, to greater variations, as in pain estimation.
△ Less
Submitted 21 March, 2022;
originally announced March 2022.
-
Rigidity of critical circle maps
Authors:
Pablo Guarino,
Marco Martens,
Welington de Melo
Abstract:
We prove that any two $C^4$ critical circle maps with the same irrational rotation number and the same odd criticality are conjugate to each other by a $C^1$ circle diffeomorphism. The conjugacy is $C^{1+α}$ for Lebesgue almost every rotation number.
We prove that any two $C^4$ critical circle maps with the same irrational rotation number and the same odd criticality are conjugate to each other by a $C^1$ circle diffeomorphism. The conjugacy is $C^{1+α}$ for Lebesgue almost every rotation number.
△ Less
Submitted 9 November, 2015;
originally announced November 2015.
-
Rigidity of smooth critical circle maps
Authors:
Pablo Guarino,
Welington de Melo
Abstract:
We prove that any two $C^3$ critical circle maps with the same irrational rotation number of bounded type and the same odd criticality are conjugate to each other by a $C^{1+α}$ circle diffeomorphism, for some universal $α>0$.
We prove that any two $C^3$ critical circle maps with the same irrational rotation number of bounded type and the same odd criticality are conjugate to each other by a $C^{1+α}$ circle diffeomorphism, for some universal $α>0$.
△ Less
Submitted 14 March, 2013;
originally announced March 2013.
-
Chaotic Period Doubling
Authors:
V. V. M. S. Chandramouli,
M. Martens,
W. de Melo,
C. P. Tresser
Abstract:
The period doubling renormalization operator was introduced by M. Feigenbaum and by P. Coullet and C. Tresser in the nineteen-seventieth to study the asymptotic small scale geometry of the attractor of one-dimensional systems which are at the transition from simple to chaotic dynamics. This geometry turns out to not depend on the choice of the map under rather mild smoothness conditions. The exi…
▽ More
The period doubling renormalization operator was introduced by M. Feigenbaum and by P. Coullet and C. Tresser in the nineteen-seventieth to study the asymptotic small scale geometry of the attractor of one-dimensional systems which are at the transition from simple to chaotic dynamics. This geometry turns out to not depend on the choice of the map under rather mild smoothness conditions. The existence of a unique renormalization fixed point which is also hyperbolic among generic smooth enough maps plays a crucial role in the corresponding renormalization theory. The uniqueness and hyperbolicity of the renormalization fixed point were first shown in the holomorphic context, by means that generalize to other renormalization operators. It was then proved that in the space of $C^{2+α}$ unimodal maps, for $α$ close to one, the period doubling renormalization fixed point is hyperbolic as well. In this paper we study what happens when one approaches from below the minimal smoothness thresholds for the uniqueness and for the hyperbolicity of the period doubling renormalization generic fixed point. Indeed, our main results states that in the space of $C^2$ unimodal maps the analytic fixed point is not hyperbolic and that the same remains true when adding enough smoothness to get a priori bounds. In this smoother class, called $C^{2+|\cdot|}$ the failure of hyperbolicity is tamer than in $C^2$. Things get much worse with just a bit less of smoothness than $C^2$ as then even the uniqueness is lost and other asymptotic behavior become possible. We show that the period doubling renormalization operator acting on the space of $C^{1+Lip}$ unimodal maps has infinite topological entropy.
△ Less
Submitted 2 October, 2007;
originally announced October 2007.
-
Global hyperbolicity of renormalization for C^r unimodal mappings
Authors:
Edson de Faria,
Welington de Melo,
Alberto Pinto
Abstract:
In this paper we extend M. Lyubich's recent results on the global hyperbolicity of renormalization of quadratic-like germs to the space of C^r unimodal maps with quadratic critical point. We show that in this space the bounded-type limit sets of the renormalization operator have an invariant hyperbolic structure provided r ge 2+ alpha with alpha close to one. As an intermediate step between Lyub…
▽ More
In this paper we extend M. Lyubich's recent results on the global hyperbolicity of renormalization of quadratic-like germs to the space of C^r unimodal maps with quadratic critical point. We show that in this space the bounded-type limit sets of the renormalization operator have an invariant hyperbolic structure provided r ge 2+ alpha with alpha close to one. As an intermediate step between Lyubich's results and ours, we prove that the renormalization operator is hyperbolic in a Banach space of real analytic maps. We construct the local stable manifolds and prove that they form a continuous lamination whose leaves are C^1 codimension one, Banach submanifolds of the ambient space, and whose holonomy is C^{1+β} for some beta >0. We also prove that the global stable sets are C^1 immersed (codimension one) submanifolds as well, provided r ge 3+ alpha with alpha close to one. As a corollary, we deduce that in generic, one-parameter families of C^r unimodal maps, the set of parameters corresponding to infinitely renormalizable maps of bounded combinatorial type is a Cantor set with Hausdorff dimension less than one.
△ Less
Submitted 23 March, 2007; v1 submitted 27 February, 2007;
originally announced February 2007.
-
Rigidity of $C^2$ Infinitely Renormalizable Unimodal Maps
Authors:
Wellington de Melo,
Alberto Pinto
Abstract:
Given $C^2$ infinitely renormalizable unimodal maps $f$ and $g$ with a quadratic critical point and the same bounded combinatorial type, we prove that they are $C^{1+α}$ conjugate along the closure of the corresponding forward orbits of the critical points, for some $α>0$.
Given $C^2$ infinitely renormalizable unimodal maps $f$ and $g$ with a quadratic critical point and the same bounded combinatorial type, we prove that they are $C^{1+α}$ conjugate along the closure of the corresponding forward orbits of the critical points, for some $α>0$.
△ Less
Submitted 26 May, 1999;
originally announced May 1999.
-
Rigidity of critical circle mappings, II
Authors:
Edson de Faria,
Welington de Melo
Abstract:
We prove that any two real-analytic critical circle maps with cubic critical point and the same irrational rotation number of bounded type are $C^{1+α}$ conjugate for some $α>0$.
We prove that any two real-analytic critical circle maps with cubic critical point and the same irrational rotation number of bounded type are $C^{1+α}$ conjugate for some $α>0$.
△ Less
Submitted 14 November, 1997;
originally announced November 1997.
-
Rigidity of critical circle mappings, I
Authors:
Edson de Faria,
Welington de Melo
Abstract:
We prove that two $C^r$ critical circle maps with the same rotation number of bounded type are $C^{1+α}$ conjugate for some $α>0$ provided their successive renormalizations converge together at an exponential rate in the $C^0$ sense. The number $α$ depends only on the rate of convergence. We also give examples of $C^\infty$ critical circle maps with the same rotation number that are not…
▽ More
We prove that two $C^r$ critical circle maps with the same rotation number of bounded type are $C^{1+α}$ conjugate for some $α>0$ provided their successive renormalizations converge together at an exponential rate in the $C^0$ sense. The number $α$ depends only on the rate of convergence. We also give examples of $C^\infty$ critical circle maps with the same rotation number that are not $C^{1+β}$ conjugate for any $β>0$.
△ Less
Submitted 14 November, 1997;
originally announced November 1997.
-
The multipliers of periodic points in one-dimensional dynamics
Authors:
Marco Martens,
Welington de Melo
Abstract:
It will be shown that the smooth conjugacy class of an $S-$unimodal map which does not have a periodic attractor neither a Cantor attractor is determined by the multipliers of the periodic orbits. This generalizes a result by M.Shub and D.Sullivan for smooth expanding maps of the circle.
It will be shown that the smooth conjugacy class of an $S-$unimodal map which does not have a periodic attractor neither a Cantor attractor is determined by the multipliers of the periodic orbits. This generalizes a result by M.Shub and D.Sullivan for smooth expanding maps of the circle.
△ Less
Submitted 11 August, 1998; v1 submitted 14 October, 1997;
originally announced October 1997.
-
Universal models for Lorenz maps
Authors:
Marco Martens,
Welington de Melo
Abstract:
The existence of smooth families of Lorenz maps exhibiting all possible dynamical behavior is established and the structure of the parameter space of these families is described.
The existence of smooth families of Lorenz maps exhibiting all possible dynamical behavior is established and the structure of the parameter space of these families is described.
△ Less
Submitted 14 October, 1996;
originally announced October 1996.
