-
Seamless Multimodal Biometrics for Continuous Personalised Wellbeing Monitoring
Authors:
João Ribeiro Pinto
Abstract:
Artificially intelligent perception is increasingly present in the lives of every one of us. Vehicles are no exception, (...) In the near future, pattern recognition will have an even stronger role in vehicles, as self-driving cars will require automated ways to understand what is happening around (and within) them and act accordingly. (...) This doctoral work focused on advancing in-vehicle sensi…
▽ More
Artificially intelligent perception is increasingly present in the lives of every one of us. Vehicles are no exception, (...) In the near future, pattern recognition will have an even stronger role in vehicles, as self-driving cars will require automated ways to understand what is happening around (and within) them and act accordingly. (...) This doctoral work focused on advancing in-vehicle sensing through the research of novel computer vision and pattern recognition methodologies for both biometrics and wellbeing monitoring. The main focus has been on electrocardiogram (ECG) biometrics, a trait well-known for its potential for seamless driver monitoring. Major efforts were devoted to achieving improved performance in identification and identity verification in off-the-person scenarios, well-known for increased noise and variability. Here, end-to-end deep learning ECG biometric solutions were proposed and important topics were addressed such as cross-database and long-term performance, waveform relevance through explainability, and interlead conversion. Face biometrics, a natural complement to the ECG in seamless unconstrained scenarios, was also studied in this work. The open challenges of masked face recognition and interpretability in biometrics were tackled in an effort to evolve towards algorithms that are more transparent, trustworthy, and robust to significant occlusions. Within the topic of wellbeing monitoring, improved solutions to multimodal emotion recognition in groups of people and activity/violence recognition in in-vehicle scenarios were proposed. At last, we also proposed a novel way to learn template security within end-to-end models, dismissing additional separate encryption processes, and a self-supervised learning approach tailored to sequential data, in order to ensure data security and optimal performance. (...)
△ Less
Submitted 19 January, 2023; v1 submitted 8 January, 2023;
originally announced January 2023.
-
ATP-induced reconfiguration of the micro-viscoelasticity of cardiac and skeletal myosin solutions
Authors:
Pablo Domínguez-García,
Jose R. Pinto,
Ana Akrap,
Sylvia Jeney
Abstract:
We study the high-frequency micro-mechanical response of suspensions composed by cardiac and skeletal muscle myosin by optical trapping interferometry. We observe that in low ionic strength solutions upon the addition of magnesium adenosine triphosphate (\ch{MgATP^2-}), myosin suspensions radically change their micro-mechanics properties, generating a viscoelastic fluid characterized by a complex…
▽ More
We study the high-frequency micro-mechanical response of suspensions composed by cardiac and skeletal muscle myosin by optical trapping interferometry. We observe that in low ionic strength solutions upon the addition of magnesium adenosine triphosphate (\ch{MgATP^2-}), myosin suspensions radically change their micro-mechanics properties, generating a viscoelastic fluid characterized by a complex modulus similar to a suspension of worm-like micelles. This transduction of energy, from chemical to mechanical, may be related to the relaxed states of myosin, which regulate muscle contractility and can be involved in the etiology of many myopathies. Within an analogous generic mechanical response, cardiac and skeletal myosin suspensions provide different stress relaxation times, elastic modulus values, and characteristic lengths. These discrepancies probably rely on the dissimilar physiological functions of cardiac and skeletal muscle, on the different MgATPase hydrolysis rates of cardiac and skeletal myosins, and on the observed distinct cooperative behavior of their myosin heads in the super-relaxed state. \textit{In vitro} studies like these allow to understand the foundations of muscle cells mechanics on the micro-scale, and may contribute to the engineering of biological materials whose micro-mechanics can be activated by energy regulators.
△ Less
Submitted 12 October, 2024; v1 submitted 15 September, 2022;
originally announced September 2022.
-
Micro-mechanical response and power-law exponents from the longitudinal fluctuations of F-actin solutions
Authors:
Pablo Domínguez-García,
Jose R. Pinto,
Ana Akrap,
Sylvia Jeney
Abstract:
We investigate the local fluctuations of filamentous actin (F-actin), with focus on the skeletal thin filament, using single-particle optical trapping interferometry. This experimental technique allows us to detect the Brownian motion of a tracer bead immersed in a complex fluid with nanometric resolution at the microsecond time-scale. The mean square displacement, loss modulus, and velocity autoc…
▽ More
We investigate the local fluctuations of filamentous actin (F-actin), with focus on the skeletal thin filament, using single-particle optical trapping interferometry. This experimental technique allows us to detect the Brownian motion of a tracer bead immersed in a complex fluid with nanometric resolution at the microsecond time-scale. The mean square displacement, loss modulus, and velocity autocorrelation function (VAF) of the trapped microprobes in the fluid follow power-law behaviors, whose exponents can be determined in the short-time/high-frequency regime along several decades. We obtain 7/8 subdiffusive power-law exponents for polystyrene depleted microtracers at low optical trapping forces. Microrheologically, the elastic modulus of these suspensions is observed to be constant up to the limit of high frequencies, confirming the origin of this subdiffusive exponent on the local longitudinal fluctuations of the polymers. Deviations from this value are measured and discussed in relation to the characteristic lengths scales of these F-actin networks and probes' properties, and also in connection with the different power-law exponents detected in the VAFs. Finally, we observe that the thin filament, composed by tropomyosin (Tm) and troponin (Tn) coupled to F-actin in the presence of Ca$^{2+}$, returns exponent values less dispersed than F-actin alone, which we interpret as a micro-measurement of the filament stabilization.
△ Less
Submitted 24 May, 2024; v1 submitted 15 September, 2022;
originally announced September 2022.
-
Causality-Inspired Taxonomy for Explainable Artificial Intelligence
Authors:
Pedro C. Neto,
Tiago Gonçalves,
João Ribeiro Pinto,
Wilson Silva,
Ana F. Sequeira,
Arun Ross,
Jaime S. Cardoso
Abstract:
As two sides of the same coin, causality and explainable artificial intelligence (xAI) were initially proposed and developed with different goals. However, the latter can only be complete when seen through the lens of the causality framework. As such, we propose a novel causality-inspired framework for xAI that creates an environment for the development of xAI approaches. To show its applicability…
▽ More
As two sides of the same coin, causality and explainable artificial intelligence (xAI) were initially proposed and developed with different goals. However, the latter can only be complete when seen through the lens of the causality framework. As such, we propose a novel causality-inspired framework for xAI that creates an environment for the development of xAI approaches. To show its applicability, biometrics was used as case study. For this, we have analysed 81 research papers on a myriad of biometric modalities and different tasks. We have categorised each of these methods according to our novel xAI Ladder and discussed the future directions of the field.
△ Less
Submitted 4 March, 2024; v1 submitted 19 August, 2022;
originally announced August 2022.
-
OCFR 2022: Competition on Occluded Face Recognition From Synthetically Generated Structure-Aware Occlusions
Authors:
Pedro C. Neto,
Fadi Boutros,
Joao Ribeiro Pinto,
Naser Damer,
Ana F. Sequeira,
Jaime S. Cardoso,
Messaoud Bengherabi,
Abderaouf Bousnat,
Sana Boucheta,
Nesrine Hebbadj,
Mustafa Ekrem Erakın,
Uğur Demir,
Hazım Kemal Ekenel,
Pedro Beber de Queiroz Vidal,
David Menotti
Abstract:
This work summarizes the IJCB Occluded Face Recognition Competition 2022 (IJCB-OCFR-2022) embraced by the 2022 International Joint Conference on Biometrics (IJCB 2022). OCFR-2022 attracted a total of 3 participating teams, from academia. Eventually, six valid submissions were submitted and then evaluated by the organizers. The competition was held to address the challenge of face recognition in th…
▽ More
This work summarizes the IJCB Occluded Face Recognition Competition 2022 (IJCB-OCFR-2022) embraced by the 2022 International Joint Conference on Biometrics (IJCB 2022). OCFR-2022 attracted a total of 3 participating teams, from academia. Eventually, six valid submissions were submitted and then evaluated by the organizers. The competition was held to address the challenge of face recognition in the presence of severe face occlusions. The participants were free to use any training data and the testing data was built by the organisers by synthetically occluding parts of the face images using a well-known dataset. The submitted solutions presented innovations and performed very competitively with the considered baseline. A major output of this competition is a challenging, realistic, and diverse, and publicly available occluded face recognition benchmark with well defined evaluation protocols.
△ Less
Submitted 15 August, 2022; v1 submitted 4 August, 2022;
originally announced August 2022.
-
FocusFace: Multi-task Contrastive Learning for Masked Face Recognition
Authors:
Pedro C. Neto,
Fadi Boutros,
João Ribeiro Pinto,
Naser Damer,
Ana F. Sequeira,
Jaime S. Cardoso
Abstract:
SARS-CoV-2 has presented direct and indirect challenges to the scientific community. One of the most prominent indirect challenges advents from the mandatory use of face masks in a large number of countries. Face recognition methods struggle to perform identity verification with similar accuracy on masked and unmasked individuals. It has been shown that the performance of these methods drops consi…
▽ More
SARS-CoV-2 has presented direct and indirect challenges to the scientific community. One of the most prominent indirect challenges advents from the mandatory use of face masks in a large number of countries. Face recognition methods struggle to perform identity verification with similar accuracy on masked and unmasked individuals. It has been shown that the performance of these methods drops considerably in the presence of face masks, especially if the reference image is unmasked. We propose FocusFace, a multi-task architecture that uses contrastive learning to be able to accurately perform masked face recognition. The proposed architecture is designed to be trained from scratch or to work on top of state-of-the-art face recognition methods without sacrificing the capabilities of a existing models in conventional face recognition tasks. We also explore different approaches to design the contrastive learning module. Results are presented in terms of masked-masked (M-M) and unmasked-masked (U-M) face verification performance. For both settings, the results are on par with published methods, but for M-M specifically, the proposed method was able to outperform all the solutions that it was compared to. We further show that when using our method on top of already existing methods the training computational costs decrease significantly while retaining similar performances. The implementation and the trained models are available at GitHub.
△ Less
Submitted 1 November, 2021; v1 submitted 28 October, 2021;
originally announced October 2021.
-
My Eyes Are Up Here: Promoting Focus on Uncovered Regions in Masked Face Recognition
Authors:
Pedro C. Neto,
Fadi Boutros,
João Ribeiro Pinto,
Mohsen Saffari,
Naser Damer,
Ana F. Sequeira,
Jaime S. Cardoso
Abstract:
The recent Covid-19 pandemic and the fact that wearing masks in public is now mandatory in several countries, created challenges in the use of face recognition systems (FRS). In this work, we address the challenge of masked face recognition (MFR) and focus on evaluating the verification performance in FRS when verifying masked vs unmasked faces compared to verifying only unmasked faces. We propose…
▽ More
The recent Covid-19 pandemic and the fact that wearing masks in public is now mandatory in several countries, created challenges in the use of face recognition systems (FRS). In this work, we address the challenge of masked face recognition (MFR) and focus on evaluating the verification performance in FRS when verifying masked vs unmasked faces compared to verifying only unmasked faces. We propose a methodology that combines the traditional triplet loss and the mean squared error (MSE) intending to improve the robustness of an MFR system in the masked-unmasked comparison mode. The results obtained by our proposed method show improvements in a detailed step-wise ablation study. The conducted study showed significant performance gains induced by our proposed training paradigm and modified triplet loss on two evaluation databases.
△ Less
Submitted 18 August, 2021; v1 submitted 2 August, 2021;
originally announced August 2021.
-
MFR 2021: Masked Face Recognition Competition
Authors:
Fadi Boutros,
Naser Damer,
Jan Niklas Kolf,
Kiran Raja,
Florian Kirchbuchner,
Raghavendra Ramachandra,
Arjan Kuijper,
Pengcheng Fang,
Chao Zhang,
Fei Wang,
David Montero,
Naiara Aginako,
Basilio Sierra,
Marcos Nieto,
Mustafa Ekrem Erakin,
Ugur Demir,
Hazim Kemal,
Ekenel,
Asaki Kataoka,
Kohei Ichikawa,
Shizuma Kubo,
Jie Zhang,
Mingjie He,
Dan Han,
Shiguang Shan
, et al. (10 additional authors not shown)
Abstract:
This paper presents a summary of the Masked Face Recognition Competitions (MFR) held within the 2021 International Joint Conference on Biometrics (IJCB 2021). The competition attracted a total of 10 participating teams with valid submissions. The affiliations of these teams are diverse and associated with academia and industry in nine different countries. These teams successfully submitted 18 vali…
▽ More
This paper presents a summary of the Masked Face Recognition Competitions (MFR) held within the 2021 International Joint Conference on Biometrics (IJCB 2021). The competition attracted a total of 10 participating teams with valid submissions. The affiliations of these teams are diverse and associated with academia and industry in nine different countries. These teams successfully submitted 18 valid solutions. The competition is designed to motivate solutions aiming at enhancing the face recognition accuracy of masked faces. Moreover, the competition considered the deployability of the proposed solutions by taking the compactness of the face recognition models into account. A private dataset representing a collaborative, multi-session, real masked, capture scenario is used to evaluate the submitted solutions. In comparison to one of the top-performing academic face recognition solutions, 10 out of the 18 submitted solutions did score higher masked face verification accuracy.
△ Less
Submitted 29 June, 2021;
originally announced June 2021.
-
The Southern Photometric Local Universe Survey (S-PLUS): improved SEDs, morphologies and redshifts with 12 optical filters
Authors:
C. Mendes de Oliveira,
T. Ribeiro,
W. Schoenell,
A. Kanaan,
R. A. Overzier,
A. Molino,
L. Sampedro,
P. Coelho,
C. E. Barbosa,
A. Cortesi,
M. V. Costa-Duarte,
F. R. Herpich,
J. A. Hernandez-Jimenez,
V. M. Placco,
H. S. Xavier,
L. R. Abramo,
R. K. Saito,
A. L. Chies-Santos,
A. Ederoclite,
R. Lopes de Oliveira,
D. R. Gonçalves,
S. Akras,
L. A. Almeida,
F. Almeida-Fernandes,
T. C. Beers
, et al. (120 additional authors not shown)
Abstract:
The Southern Photometric Local Universe Survey (S-PLUS) is imaging ~9300 deg^2 of the celestial sphere in twelve optical bands using a dedicated 0.8 m robotic telescope, the T80-South, at the Cerro Tololo Inter-American Observatory, Chile. The telescope is equipped with a 9.2k by 9.2k e2v detector with 10 um pixels, resulting in a field-of-view of 2 deg^2 with a plate scale of 0.55"/pixel. The sur…
▽ More
The Southern Photometric Local Universe Survey (S-PLUS) is imaging ~9300 deg^2 of the celestial sphere in twelve optical bands using a dedicated 0.8 m robotic telescope, the T80-South, at the Cerro Tololo Inter-American Observatory, Chile. The telescope is equipped with a 9.2k by 9.2k e2v detector with 10 um pixels, resulting in a field-of-view of 2 deg^2 with a plate scale of 0.55"/pixel. The survey consists of four main subfields, which include two non-contiguous fields at high Galactic latitudes (8000 deg^2 at |b| > 30 deg) and two areas of the Galactic plane and bulge (for an additional 1300 deg^2). S-PLUS uses the Javalambre 12-band magnitude system, which includes the 5 u, g, r, i, z broad-band filters and 7 narrow-band filters centered on prominent stellar spectral features: the Balmer jump/[OII], Ca H+K, H-delta, G-band, Mg b triplet, H-alpha, and the Ca triplet. S-PLUS delivers accurate photometric redshifts (delta_z/(1+z) = 0.02 or better) for galaxies with r < 20 AB mag and redshift < 0.5, thus producing a 3D map of the local Universe over a volume of more than 1 (Gpc/h)^3. The final S-PLUS catalogue will also enable the study of star formation and stellar populations in and around the Milky Way and nearby galaxies, as well as searches for quasars, variable sources, and low-metallicity stars. In this paper we introduce the main characteristics of the survey, illustrated with science verification data highlighting the unique capabilities of S-PLUS. We also present the first public data release of ~336 deg^2 of the Stripe-82 area, which is available at http://datalab.noao.edu/splus.
△ Less
Submitted 2 September, 2019; v1 submitted 2 July, 2019;
originally announced July 2019.
-
Target Selection for the Apache Point Observatory Galactic Evolution Experiment (APOGEE)
Authors:
G. Zasowski,
Jennifer A. Johnson,
P. M. Frinchaboy,
S. R. Majewski,
D. L. Nidever,
H. J. Rocha Pinto,
L. Girardi,
B. Andrews,
S. D. Chojnowski,
K. M. Cudworth,
K. Jackson,
J. Munn,
M. F. Skrutskie,
R. L. Beaton,
C. H. Blake,
K. Covey,
R. Deshpande,
C. Epstein,
D. Fabbian,
S. W. Fleming,
A. Garcia Hernandez,
A. Herrero,
S. Mahadevan,
Sz. Meszaros,
M. Schultheis
, et al. (21 additional authors not shown)
Abstract:
The Apache Point Observatory Galactic Evolution Experiment (APOGEE) is a high-resolution infrared spectroscopic survey spanning all Galactic environments (i.e., bulge, disk, and halo), with the principal goal of constraining dynamical and chemical evolution models of the Milky Way. APOGEE takes advantage of the reduced effects of extinction at infrared wavelengths to observe the inner Galaxy and b…
▽ More
The Apache Point Observatory Galactic Evolution Experiment (APOGEE) is a high-resolution infrared spectroscopic survey spanning all Galactic environments (i.e., bulge, disk, and halo), with the principal goal of constraining dynamical and chemical evolution models of the Milky Way. APOGEE takes advantage of the reduced effects of extinction at infrared wavelengths to observe the inner Galaxy and bulge at an unprecedented level of detail. The survey's broad spatial and wavelength coverage enables users of APOGEE data to address numerous Galactic structure and stellar populations issues. In this paper we describe the APOGEE targeting scheme and document its various target classes to provide the necessary background and reference information to analyze samples of APOGEE data with awareness of the imposed selection criteria and resulting sample properties. APOGEE's primary sample consists of ~100,000 red giant stars, selected to minimize observational biases in age and metallicity. We present the methodology and considerations that drive the selection of this sample and evaluate the accuracy, efficiency, and caveats of the selection and sampling algorithms. We also describe additional target classes that contribute to the APOGEE sample, including numerous ancillary science programs, and we outline the targeting data that will be included in the public data releases.
△ Less
Submitted 1 August, 2013;
originally announced August 2013.
-
Line and Word Matching in Old Documents
Authors:
A. Marcolino,
Vitorino Ramos,
Mario Ramalho,
J. R. Caldas Pinto
Abstract:
This paper is concerned with the problem of establishing an index based on word matching. It is assumed that the book was digitised as better as possible and some pre-processing techniques were already applied as line orientation correction and some noise removal. However two main factor are responsible for being not possible to apply ordinary optical character recognition techniques (OCR): the…
▽ More
This paper is concerned with the problem of establishing an index based on word matching. It is assumed that the book was digitised as better as possible and some pre-processing techniques were already applied as line orientation correction and some noise removal. However two main factor are responsible for being not possible to apply ordinary optical character recognition techniques (OCR): the presence of antique fonts and the degraded state of many characters due to unrecoverable original time degradation. In this paper we make a short introduction to word segmentation that involves finding the lines that characterise a word. After we discuss different approaches for word matching and how they can be combined to obtain an ordered list for candidate words for the matching. This discussion will be illustrated by examples.
△ Less
Submitted 17 December, 2004;
originally announced December 2004.
