-
Widespread occurrence of lava lakes on Io observed from Juno
Authors:
Alessandro Mura,
Federico Tosi,
Francesca Zambon,
Rosaly M. C. Lopes,
Pete J. Mouginis-Mark,
Jani Radebaugh,
Alberto Adriani,
Scott Bolton,
Julie Rathbun,
Andrea Cicchetti,
Davide Grassi,
Raffaella Noschese,
Giuseppe Piccioni,
Christina Plainaki,
Roberto Sordini,
Giuseppe Sindoni
Abstract:
We report recent observations of lava lakes within patera on Io made by the JIRAM imager/spectrometer on board the Juno spacecraft, taken during close observation occurred in the extended mission. At least 40 lava lakes have been identified from JIRAM observations. The majority (>50%) of paterae have elevated thermal signatures when imaged at sufficiently high spatial resolution (a few km/pixel),…
▽ More
We report recent observations of lava lakes within patera on Io made by the JIRAM imager/spectrometer on board the Juno spacecraft, taken during close observation occurred in the extended mission. At least 40 lava lakes have been identified from JIRAM observations. The majority (>50%) of paterae have elevated thermal signatures when imaged at sufficiently high spatial resolution (a few km/pixel), implying that lava lakes are ubiquitous on Io. The annular width of the spattering region around the margins, a characteristic of lava lakes, is of the order of few meters to tens of meters, the diameter of the observed lava lakes ranges from 10 to 100 km. The thickness of the crust in the center of some lava lakes is of the order of 5-10 m; we estimate that this crust is a few years old. Also, the bulk of the thermal emission comes from the much larger crust and not from the smaller exposed lava, so the total power output cannot be calculated from the 5-um radiance alone. Eight of the proposed lava lakes have never been reported previously as active hotspots.
△ Less
Submitted 14 October, 2024;
originally announced October 2024.
-
A patchy CO$_2$ exosphere on Ganymede revealed by the James Webb Space Telescope
Authors:
Dominique Bockelée-Morvan,
Olivier Poch,
Françcois Leblanc,
Vladimir Zakharov,
Emmanuel Lellouch,
Eric Quirico,
Imke de Pater,
Thierry Fouchet,
Pablo Rodriguez-Ovalle,
Lorenz Roth,
Frédéric Merlin,
Stefan Duling,
Joachim Saur,
Adrien Masson,
Patrick Fry,
Samantha Trumbo,
Michael Brown,
Richard Cartwright,
Stéphanie Cazaux,
Katherine de Kleer,
Leigh N. Fletcher,
Zachariah Milby,
Audrey Moingeon,
Alessandro Mura,
Glenn S. Orton
, et al. (3 additional authors not shown)
Abstract:
Jupiter's icy moon Ganymede has a tenuous exosphere produced by sputtering and possibly sublimation of water ice. To date, only atomic hydrogen and oxygen have been directly detected in this exosphere. Here, we present observations of Ganymede's CO$_2$ exosphere obtained with the James Webb Space Telescope. CO$_2$ gas is observed over different terrain types, mainly over those exposed to intense J…
▽ More
Jupiter's icy moon Ganymede has a tenuous exosphere produced by sputtering and possibly sublimation of water ice. To date, only atomic hydrogen and oxygen have been directly detected in this exosphere. Here, we present observations of Ganymede's CO$_2$ exosphere obtained with the James Webb Space Telescope. CO$_2$ gas is observed over different terrain types, mainly over those exposed to intense Jovian plasma irradiation, as well as over some bright or dark terrains. Despite warm surface temperatures, the CO$_2$ abundance over equatorial subsolar regions is low. CO$_2$ vapor has the highest abundance over the north polar cap of the leading hemisphere, reaching a surface pressure of 1 pbar. From modeling we show that the local enhancement observed near 12 h local time in this region can be explained by the presence of cold traps enabling CO$_2$ adsorption. However, whether the release mechanism in this high-latitude region is sputtering or sublimation remains unclear. The north polar cap of the leading hemisphere also has unique surface-ice properties, probably linked to the presence of the large atmospheric CO2 excess over this region. These CO2 molecules might have been initially released in the atmosphere after the radiolysis of CO$_2$ precursors, or from the sputtering of CO$_2$ embedded in the H$_2$O ice bedrock. Dark terrains (regiones), more widespread on the north versus south polar regions, possibly harbor CO$_2$ precursors. CO$_2$ molecules would then be redistributed via cold trapping on ice-rich terrains of the polar cap and be diurnally released and redeposited on these terrains. Ganymede's CO$_2$ exosphere highlights the complexity of surface-atmosphere interactions on Jupiter's icy Galilean moons.
△ Less
Submitted 20 September, 2024;
originally announced September 2024.
-
Self-Evolving Depth-Supervised 3D Gaussian Splatting from Rendered Stereo Pairs
Authors:
Sadra Safadoust,
Fabio Tosi,
Fatma Güney,
Matteo Poggi
Abstract:
3D Gaussian Splatting (GS) significantly struggles to accurately represent the underlying 3D scene geometry, resulting in inaccuracies and floating artifacts when rendering depth maps. In this paper, we address this limitation, undertaking a comprehensive analysis of the integration of depth priors throughout the optimization process of Gaussian primitives, and present a novel strategy for this pu…
▽ More
3D Gaussian Splatting (GS) significantly struggles to accurately represent the underlying 3D scene geometry, resulting in inaccuracies and floating artifacts when rendering depth maps. In this paper, we address this limitation, undertaking a comprehensive analysis of the integration of depth priors throughout the optimization process of Gaussian primitives, and present a novel strategy for this purpose. This latter dynamically exploits depth cues from a readily available stereo network, processing virtual stereo pairs rendered by the GS model itself during training and achieving consistent self-improvement of the scene representation. Experimental results on three popular datasets, breaking ground as the first to assess depth accuracy for these models, validate our findings.
△ Less
Submitted 11 September, 2024;
originally announced September 2024.
-
Diffusion Models for Monocular Depth Estimation: Overcoming Challenging Conditions
Authors:
Fabio Tosi,
Pierluigi Zama Ramirez,
Matteo Poggi
Abstract:
We present a novel approach designed to address the complexities posed by challenging, out-of-distribution data in the single-image depth estimation task. Starting with images that facilitate depth prediction due to the absence of unfavorable factors, we systematically generate new, user-defined scenes with a comprehensive set of challenges and associated depth information. This is achieved by lev…
▽ More
We present a novel approach designed to address the complexities posed by challenging, out-of-distribution data in the single-image depth estimation task. Starting with images that facilitate depth prediction due to the absence of unfavorable factors, we systematically generate new, user-defined scenes with a comprehensive set of challenges and associated depth information. This is achieved by leveraging cutting-edge text-to-image diffusion models with depth-aware control, known for synthesizing high-quality image content from textual prompts while preserving the coherence of 3D structure between generated and source imagery. Subsequent fine-tuning of any monocular depth network is carried out through a self-distillation protocol that takes into account images generated using our strategy and its own depth predictions on simple, unchallenging scenes. Experiments on benchmarks tailored for our purposes demonstrate the effectiveness and versatility of our proposal.
△ Less
Submitted 23 July, 2024;
originally announced July 2024.
-
A Survey on Deep Stereo Matching in the Twenties
Authors:
Fabio Tosi,
Luca Bartolomei,
Matteo Poggi
Abstract:
Stereo matching is close to hitting a half-century of history, yet witnessed a rapid evolution in the last decade thanks to deep learning. While previous surveys in the late 2010s covered the first stage of this revolution, the last five years of research brought further ground-breaking advancements to the field. This paper aims to fill this gap in a two-fold manner: first, we offer an in-depth ex…
▽ More
Stereo matching is close to hitting a half-century of history, yet witnessed a rapid evolution in the last decade thanks to deep learning. While previous surveys in the late 2010s covered the first stage of this revolution, the last five years of research brought further ground-breaking advancements to the field. This paper aims to fill this gap in a two-fold manner: first, we offer an in-depth examination of the latest developments in deep stereo matching, focusing on the pioneering architectural designs and groundbreaking paradigms that have redefined the field in the 2020s; second, we present a thorough analysis of the critical challenges that have emerged alongside these advances, providing a comprehensive taxonomy of these issues and exploring the state-of-the-art techniques proposed to address them. By reviewing both the architectural innovations and the key challenges, we offer a holistic view of deep stereo matching and highlight the specific areas that require further investigation. To accompany this survey, we maintain a regularly updated project page that catalogs papers on deep stereo matching in our Awesome-Deep-Stereo-Matching (https://github.com/fabiotosi92/Awesome-Deep-Stereo-Matching) repository.
△ Less
Submitted 10 July, 2024;
originally announced July 2024.
-
Stereo-Depth Fusion through Virtual Pattern Projection
Authors:
Luca Bartolomei,
Matteo Poggi,
Fabio Tosi,
Andrea Conti,
Stefano Mattoccia
Abstract:
This paper presents a novel general-purpose stereo and depth data fusion paradigm that mimics the active stereo principle by replacing the unreliable physical pattern projector with a depth sensor. It works by projecting virtual patterns consistent with the scene geometry onto the left and right images acquired by a conventional stereo camera, using the sparse hints obtained from a depth sensor, t…
▽ More
This paper presents a novel general-purpose stereo and depth data fusion paradigm that mimics the active stereo principle by replacing the unreliable physical pattern projector with a depth sensor. It works by projecting virtual patterns consistent with the scene geometry onto the left and right images acquired by a conventional stereo camera, using the sparse hints obtained from a depth sensor, to facilitate the visual correspondence. Purposely, any depth sensing device can be seamlessly plugged into our framework, enabling the deployment of a virtual active stereo setup in any possible environment and overcoming the severe limitations of physical pattern projection, such as the limited working range and environmental conditions. Exhaustive experiments on indoor and outdoor datasets featuring both long and close range, including those providing raw, unfiltered depth hints from off-the-shelf depth sensors, highlight the effectiveness of our approach in notably boosting the robustness and accuracy of algorithms and deep stereo without any code modification and even without re-training. Additionally, we assess the performance of our strategy on active stereo evaluation datasets with conventional pattern projection. Indeed, in all these scenarios, our virtual pattern projection paradigm achieves state-of-the-art performance. The source code is available at: https://github.com/bartn8/vppstereo.
△ Less
Submitted 6 June, 2024;
originally announced June 2024.
-
Calibration of MAJIS (Moons And Jupiter Imaging Spectrometer): III. Spectral Calibration
Authors:
Paolo Haffoud,
François Poulet,
Mathieu Vincendon,
Gianrico Filacchione,
Alessandra Barbis,
Pierre Guiot,
Benoit Lecomte,
Yves Langevin,
Giuseppe Piccioni,
Cydalise Dumesnil,
Sébastien Rodriguez,
John Carter,
Stefani Stefania,
Leonardo Tommasi,
Federico Tosi,
Cédric Pilorget
Abstract:
The Moons And Jupiter Imaging Spectrometer (MAJIS) is the visible and near-infrared imaging spectrometer onboard ESA s Jupiter Icy Moons Explorer (JUICE) mission. Before its integration into the spacecraft, the instrument undergoes an extensive ground calibration to establish its baseline performances. This process prepares the imaging spectrometer for flight operations by characterizing the behav…
▽ More
The Moons And Jupiter Imaging Spectrometer (MAJIS) is the visible and near-infrared imaging spectrometer onboard ESA s Jupiter Icy Moons Explorer (JUICE) mission. Before its integration into the spacecraft, the instrument undergoes an extensive ground calibration to establish its baseline performances. This process prepares the imaging spectrometer for flight operations by characterizing the behavior of the instrument under various operative conditions and uncovering instrumental distortions that may depend on instrumental commands. Two steps of the on-ground calibration campaigns were held at the instrument level to produce the data. Additional in-flight measurements have recently been obtained after launch during the Near-Earth Commissioning Phase. In this article, we present the analyses of these datasets, focusing on the characterization of the spectral performances. First, we describe and analyze the spectral calibration datasets obtained using both monochromatic sources and polychromatic sources coupled with solid and gas samples. Then, we derive the spectral sampling and the spectral response function over the entire field of view. These spectral characteristics are quantified for various operational parameters of MAJIS, such as temperature and spectral binning. The derived on-ground performances are then compared with in-flight measurements obtained after launch and presented in the framework of the MAJIS performance requirements.
△ Less
Submitted 29 May, 2024;
originally announced May 2024.
-
Federated Online Adaptation for Deep Stereo
Authors:
Matteo Poggi,
Fabio Tosi
Abstract:
We introduce a novel approach for adapting deep stereo networks in a collaborative manner. By building over principles of federated learning, we develop a distributed framework allowing for demanding the optimization process to a number of clients deployed in different environments. This makes it possible, for a deep stereo network running on resourced-constrained devices, to capitalize on the ada…
▽ More
We introduce a novel approach for adapting deep stereo networks in a collaborative manner. By building over principles of federated learning, we develop a distributed framework allowing for demanding the optimization process to a number of clients deployed in different environments. This makes it possible, for a deep stereo network running on resourced-constrained devices, to capitalize on the adaptation process carried out by other instances of the same architecture, and thus improve its accuracy in challenging environments even when it cannot carry out adaptation on its own. Experimental results show how federated adaptation performs equivalently to on-device adaptation, and even better when dealing with challenging environments.
△ Less
Submitted 23 May, 2024;
originally announced May 2024.
-
The Third Monocular Depth Estimation Challenge
Authors:
Jaime Spencer,
Fabio Tosi,
Matteo Poggi,
Ripudaman Singh Arora,
Chris Russell,
Simon Hadfield,
Richard Bowden,
GuangYuan Zhou,
ZhengXin Li,
Qiang Rao,
YiPing Bao,
Xiao Liu,
Dohyeong Kim,
Jinseong Kim,
Myunghyun Kim,
Mykola Lavreniuk,
Rui Li,
Qing Mao,
Jiang Wu,
Yu Zhu,
Jinqiu Sun,
Yanning Zhang,
Suraj Patni,
Aradhye Agarwal,
Chetan Arora
, et al. (16 additional authors not shown)
Abstract:
This paper discusses the results of the third edition of the Monocular Depth Estimation Challenge (MDEC). The challenge focuses on zero-shot generalization to the challenging SYNS-Patches dataset, featuring complex scenes in natural and indoor settings. As with the previous edition, methods can use any form of supervision, i.e. supervised or self-supervised. The challenge received a total of 19 su…
▽ More
This paper discusses the results of the third edition of the Monocular Depth Estimation Challenge (MDEC). The challenge focuses on zero-shot generalization to the challenging SYNS-Patches dataset, featuring complex scenes in natural and indoor settings. As with the previous edition, methods can use any form of supervision, i.e. supervised or self-supervised. The challenge received a total of 19 submissions outperforming the baseline on the test set: 10 among them submitted a report describing their approach, highlighting a diffused use of foundational models such as Depth Anything at the core of their method. The challenge winners drastically improved 3D F-Score performance, from 17.51% to 23.72%.
△ Less
Submitted 27 April, 2024; v1 submitted 25 April, 2024;
originally announced April 2024.
-
How NeRFs and 3D Gaussian Splatting are Reshaping SLAM: a Survey
Authors:
Fabio Tosi,
Youmin Zhang,
Ziren Gong,
Erik Sandström,
Stefano Mattoccia,
Martin R. Oswald,
Matteo Poggi
Abstract:
Over the past two decades, research in the field of Simultaneous Localization and Mapping (SLAM) has undergone a significant evolution, highlighting its critical role in enabling autonomous exploration of unknown environments. This evolution ranges from hand-crafted methods, through the era of deep learning, to more recent developments focused on Neural Radiance Fields (NeRFs) and 3D Gaussian Spla…
▽ More
Over the past two decades, research in the field of Simultaneous Localization and Mapping (SLAM) has undergone a significant evolution, highlighting its critical role in enabling autonomous exploration of unknown environments. This evolution ranges from hand-crafted methods, through the era of deep learning, to more recent developments focused on Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS) representations. Recognizing the growing body of research and the absence of a comprehensive survey on the topic, this paper aims to provide the first comprehensive overview of SLAM progress through the lens of the latest advancements in radiance fields. It sheds light on the background, evolutionary path, inherent strengths and limitations, and serves as a fundamental reference to highlight the dynamic progress and specific challenges.
△ Less
Submitted 11 April, 2024; v1 submitted 20 February, 2024;
originally announced February 2024.
-
Revisiting Depth Completion from a Stereo Matching Perspective for Cross-domain Generalization
Authors:
Luca Bartolomei,
Matteo Poggi,
Andrea Conti,
Fabio Tosi,
Stefano Mattoccia
Abstract:
This paper proposes a new framework for depth completion robust against domain-shifting issues. It exploits the generalization capability of modern stereo networks to face depth completion, by processing fictitious stereo pairs obtained through a virtual pattern projection paradigm. Any stereo network or traditional stereo matcher can be seamlessly plugged into our framework, allowing for the depl…
▽ More
This paper proposes a new framework for depth completion robust against domain-shifting issues. It exploits the generalization capability of modern stereo networks to face depth completion, by processing fictitious stereo pairs obtained through a virtual pattern projection paradigm. Any stereo network or traditional stereo matcher can be seamlessly plugged into our framework, allowing for the deployment of a virtual stereo setup that is future-proof against advancement in the stereo field. Exhaustive experiments on cross-domain generalization support our claims. Hence, we argue that our framework can help depth completion to reach new deployment scenarios.
△ Less
Submitted 14 December, 2023;
originally announced December 2023.
-
Identification of Ammonium Salts on Comet 67P/C-G Surface from Infrared VIRTIS/Rosetta Data Based on Laboratory Experiments. Implications and Perspectives
Authors:
Olivier Poch,
Istiqomah Istiqomah,
Eric Quirico,
Pierre Beck,
Bernard Schmitt,
Patrice Theulé,
Alexandre Faure,
Pierre Hily-Blant,
Lydie Bonal,
Andrea Raponi,
Mauro Ciarniello,
Batiste Rousseau,
Sandra Potin,
Olivier Brissaud,
Laurène Flandinet,
Gianrico Filacchione,
Antoine Pommerol,
Nicolas Thomas,
David Kappel,
Vito Mennella,
Lyuba Moroz,
Vassilissa Vinogradoff,
Gabriele Arnold,
Stéphane Erard,
Dominique Bockelée-Morvan
, et al. (7 additional authors not shown)
Abstract:
The nucleus of comet 67P/Churyumov-Gerasimenko exhibits a broad spectral reflectance feature around 3.2 $μ$m, which is omnipresent in all spectra of the surface, and whose attribution has remained elusive since its discovery. Based on laboratory experiments, we have shown that most of this absorption feature is due to ammonium (NH4+) salts mixed with the dark surface material. The depth of the ban…
▽ More
The nucleus of comet 67P/Churyumov-Gerasimenko exhibits a broad spectral reflectance feature around 3.2 $μ$m, which is omnipresent in all spectra of the surface, and whose attribution has remained elusive since its discovery. Based on laboratory experiments, we have shown that most of this absorption feature is due to ammonium (NH4+) salts mixed with the dark surface material. The depth of the band is compatible with semi-volatile ammonium salts being a major reservoir of nitrogen in the comet, which could dominate over refractory organic matter and volatile species. These salts may thus represent the long-sought reservoir of nitrogen in comets, possibly bringing their nitrogen-to-carbon ratio in agreement with the solar value. Moreover, the reflectance spectra of several asteroids are compatible with the presence of NH4+ salts at their surfaces. The presence of such salts, and other NH4+-bearing compounds on asteroids, comets, and possibly in proto-stellar environments, suggests that NH4+ may be a tracer of the incorporation and transformation of nitrogen in ices, minerals and organics, at different phases of the formation of the Solar System.
△ Less
Submitted 20 November, 2023;
originally announced November 2023.
-
Composition and thermal properties of Ganymede's surface from JWST/NIRSpec and MIRI observations
Authors:
D. Bockelee-Morvan,
E. Lellouch,
O. Poch,
E. Quirico,
S. Cazaux,
I. de Pater,
T. Fouchet,
P. M. Fry,
P. Rodriguez-Ovalle,
F. Tosi,
M. H. Wong,
I. Boshuizen,
K. de Kleer,
L. N. Fletcher,
L. Meunier,
A. Mura,
L. Roth,
J. Saur,
B. Schmitt,
S. K. Trumbo,
M. E. Brown,
J. O'Donoghue,
G. S. Orton,
M. R. Showalter
Abstract:
JWST NIRSpec IFU (2.9-5.3 mu) and MIRI MRS (4.9-28.5 mu) observations were performed on both the leading and trailing hemispheres of Ganymede with a spectral resolution of ~2700. Reflectance spectra show signatures of water ice, CO2 and H2O2. An absorption feature at 5.9 mu is revealed and is tentatively assigned to sulfuric acid hydrates. The CO2 4.26-mu band shows latitudinal and longitudinal va…
▽ More
JWST NIRSpec IFU (2.9-5.3 mu) and MIRI MRS (4.9-28.5 mu) observations were performed on both the leading and trailing hemispheres of Ganymede with a spectral resolution of ~2700. Reflectance spectra show signatures of water ice, CO2 and H2O2. An absorption feature at 5.9 mu is revealed and is tentatively assigned to sulfuric acid hydrates. The CO2 4.26-mu band shows latitudinal and longitudinal variations in depth, shape and position over the two hemispheres, unveiling different CO2 physical states. In the ice-rich polar regions, which are the most exposed to Jupiter's plasma irradiation, the CO2 band is redshifted with respect to other terrains. In the leading northern polar cap, the CO2 band is dominated by a high wavelength component at ~4.27 mu, consistent with CO2 trapped in amorphous water ice. At equatorial latitudes (and especially on dark terrains) the observed band is broader and shifted towards the blue, suggesting CO2 adsorbed on non-icy materials. Amorphous ice is detected in the ice-rich polar regions, and is especially abundant on the leading northern polar cap. In both hemispheres the north polar cap ice appears to be more processed than the south polar cap. A longitudinal modification of the H2O ice molecular structure and/or nano/micrometre-scale texture, of diurnal or geographic origin, is observed in both hemispheres. Ice frost is observed on the morning limb of the trailing hemisphere, possibly formed during the night from the recondensation of water subliming from the warmer subsurface. Reflectance spectra of the dark terrains are compatible with the presence of Na-/Mg-sulfate salts, sulfuric acid hydrates, and possibly phyllosilicates mixed with fine-grained opaque minerals, having an highly porous texture. Mid-IR brightness temperatures indicate a rough surface and a very low thermal inertia of 20-40 J m-2 s-0.5 K-1, consistent with a porous surface.
△ Less
Submitted 21 October, 2023;
originally announced October 2023.
-
GasMono: Geometry-Aided Self-Supervised Monocular Depth Estimation for Indoor Scenes
Authors:
Chaoqiang Zhao,
Matteo Poggi,
Fabio Tosi,
Lei Zhou,
Qiyu Sun,
Yang Tang,
Stefano Mattoccia
Abstract:
This paper tackles the challenges of self-supervised monocular depth estimation in indoor scenes caused by large rotation between frames and low texture. We ease the learning process by obtaining coarse camera poses from monocular sequences through multi-view geometry to deal with the former. However, we found that limited by the scale ambiguity across different scenes in the training dataset, a n…
▽ More
This paper tackles the challenges of self-supervised monocular depth estimation in indoor scenes caused by large rotation between frames and low texture. We ease the learning process by obtaining coarse camera poses from monocular sequences through multi-view geometry to deal with the former. However, we found that limited by the scale ambiguity across different scenes in the training dataset, a naïve introduction of geometric coarse poses cannot play a positive role in performance improvement, which is counter-intuitive. To address this problem, we propose to refine those poses during training through rotation and translation/scale optimization. To soften the effect of the low texture, we combine the global reasoning of vision transformers with an overfitting-aware, iterative self-distillation mechanism, providing more accurate depth guidance coming from the network itself. Experiments on NYUv2, ScanNet, 7scenes, and KITTI datasets support the effectiveness of each component in our framework, which sets a new state-of-the-art for indoor self-supervised monocular depth estimation, as well as outstanding generalization ability. Code and models are available at https://github.com/zxcqlf/GasMono
△ Less
Submitted 26 September, 2023;
originally announced September 2023.
-
Active Stereo Without Pattern Projector
Authors:
Luca Bartolomei,
Matteo Poggi,
Fabio Tosi,
Andrea Conti,
Stefano Mattoccia
Abstract:
This paper proposes a novel framework integrating the principles of active stereo in standard passive camera systems without a physical pattern projector. We virtually project a pattern over the left and right images according to the sparse measurements obtained from a depth sensor. Any such devices can be seamlessly plugged into our framework, allowing for the deployment of a virtual active stere…
▽ More
This paper proposes a novel framework integrating the principles of active stereo in standard passive camera systems without a physical pattern projector. We virtually project a pattern over the left and right images according to the sparse measurements obtained from a depth sensor. Any such devices can be seamlessly plugged into our framework, allowing for the deployment of a virtual active stereo setup in any possible environment, overcoming the limitation of pattern projectors, such as limited working range or environmental conditions. Experiments on indoor/outdoor datasets, featuring both long and close-range, support the seamless effectiveness of our approach, boosting the accuracy of both stereo algorithms and deep networks.
△ Less
Submitted 21 September, 2023;
originally announced September 2023.
-
GO-SLAM: Global Optimization for Consistent 3D Instant Reconstruction
Authors:
Youmin Zhang,
Fabio Tosi,
Stefano Mattoccia,
Matteo Poggi
Abstract:
Neural implicit representations have recently demonstrated compelling results on dense Simultaneous Localization And Mapping (SLAM) but suffer from the accumulation of errors in camera tracking and distortion in the reconstruction. Purposely, we present GO-SLAM, a deep-learning-based dense visual SLAM framework globally optimizing poses and 3D reconstruction in real-time. Robust pose estimation is…
▽ More
Neural implicit representations have recently demonstrated compelling results on dense Simultaneous Localization And Mapping (SLAM) but suffer from the accumulation of errors in camera tracking and distortion in the reconstruction. Purposely, we present GO-SLAM, a deep-learning-based dense visual SLAM framework globally optimizing poses and 3D reconstruction in real-time. Robust pose estimation is at its core, supported by efficient loop closing and online full bundle adjustment, which optimize per frame by utilizing the learned global geometry of the complete history of input frames. Simultaneously, we update the implicit and continuous surface representation on-the-fly to ensure global consistency of 3D reconstruction. Results on various synthetic and real-world datasets demonstrate that GO-SLAM outperforms state-of-the-art approaches at tracking robustness and reconstruction accuracy. Furthermore, GO-SLAM is versatile and can run with monocular, stereo, and RGB-D input.
△ Less
Submitted 5 September, 2023;
originally announced September 2023.
-
JIRAM Observations of Volcanic Flux on Io: Distribution and Comparison to Tidal Heat Flow Models
Authors:
M. Pettine,
S. Imbeah,
J. Rathbun,
A. Hayes,
R. Lopes-Gautier,
A. Mura,
F. Tosi,
F. Zambon,
S. Bertolino
Abstract:
Juno has allowed clear, high-resolution imaging of Io's polar volcanoes using the Jovian Infrared Auroral Mapper (JIRAM) instrument. We have used data from JIRAM's M-band (4.78 um) imager from eleven Juno orbits to construct a global map of volcanic flux. This map provides short-term insight into the spatial distribution of volcanoes and the ways in which high- and low-latitude volcanoes differ. U…
▽ More
Juno has allowed clear, high-resolution imaging of Io's polar volcanoes using the Jovian Infrared Auroral Mapper (JIRAM) instrument. We have used data from JIRAM's M-band (4.78 um) imager from eleven Juno orbits to construct a global map of volcanic flux. This map provides short-term insight into the spatial distribution of volcanoes and the ways in which high- and low-latitude volcanoes differ. Using spherical harmonic analysis, we quantitatively compare our volcanic flux map to the surface heat flow distribution expected from models of Io's tidal heat deposition (summarized in de Kleer et al. 2019). Our observations confirm previously detected systems of bright volcanoes at high latitudes. Our study finds that both poles are comparably active and that the observed flux distribution is inconsistent with an asthenospheric heating model, although the south pole is viewed too infrequently to establish reliable trends.
△ Less
Submitted 24 September, 2024; v1 submitted 10 August, 2023;
originally announced August 2023.
-
Learning Depth Estimation for Transparent and Mirror Surfaces
Authors:
Alex Costanzino,
Pierluigi Zama Ramirez,
Matteo Poggi,
Fabio Tosi,
Stefano Mattoccia,
Luigi Di Stefano
Abstract:
Inferring the depth of transparent or mirror (ToM) surfaces represents a hard challenge for either sensors, algorithms, or deep networks. We propose a simple pipeline for learning to estimate depth properly for such surfaces with neural networks, without requiring any ground-truth annotation. We unveil how to obtain reliable pseudo labels by in-painting ToM objects in images and processing them wi…
▽ More
Inferring the depth of transparent or mirror (ToM) surfaces represents a hard challenge for either sensors, algorithms, or deep networks. We propose a simple pipeline for learning to estimate depth properly for such surfaces with neural networks, without requiring any ground-truth annotation. We unveil how to obtain reliable pseudo labels by in-painting ToM objects in images and processing them with a monocular depth estimation model. These labels can be used to fine-tune existing monocular or stereo networks, to let them learn how to deal with ToM surfaces. Experimental results on the Booster dataset show the dramatic improvements enabled by our remarkably simple proposal.
△ Less
Submitted 27 July, 2023;
originally announced July 2023.
-
The Second Monocular Depth Estimation Challenge
Authors:
Jaime Spencer,
C. Stella Qian,
Michaela Trescakova,
Chris Russell,
Simon Hadfield,
Erich W. Graf,
Wendy J. Adams,
Andrew J. Schofield,
James Elder,
Richard Bowden,
Ali Anwar,
Hao Chen,
Xiaozhi Chen,
Kai Cheng,
Yuchao Dai,
Huynh Thai Hoa,
Sadat Hossain,
Jianmian Huang,
Mohan Jing,
Bo Li,
Chao Li,
Baojun Li,
Zhiwen Liu,
Stefano Mattoccia,
Siegfried Mercelis
, et al. (18 additional authors not shown)
Abstract:
This paper discusses the results for the second edition of the Monocular Depth Estimation Challenge (MDEC). This edition was open to methods using any form of supervision, including fully-supervised, self-supervised, multi-task or proxy depth. The challenge was based around the SYNS-Patches dataset, which features a wide diversity of environments with high-quality dense ground-truth. This includes…
▽ More
This paper discusses the results for the second edition of the Monocular Depth Estimation Challenge (MDEC). This edition was open to methods using any form of supervision, including fully-supervised, self-supervised, multi-task or proxy depth. The challenge was based around the SYNS-Patches dataset, which features a wide diversity of environments with high-quality dense ground-truth. This includes complex natural environments, e.g. forests or fields, which are greatly underrepresented in current benchmarks.
The challenge received eight unique submissions that outperformed the provided SotA baseline on any of the pointcloud- or image-based metrics. The top supervised submission improved relative F-Score by 27.62%, while the top self-supervised improved it by 16.61%. Supervised submissions generally leveraged large collections of datasets to improve data diversity. Self-supervised submissions instead updated the network architecture and pretrained backbones. These results represent a significant progress in the field, while highlighting avenues for future research, such as reducing interpolation artifacts at depth boundaries, improving self-supervised indoor performance and overall natural image accuracy.
△ Less
Submitted 26 April, 2023; v1 submitted 14 April, 2023;
originally announced April 2023.
-
NeRF-Supervised Deep Stereo
Authors:
Fabio Tosi,
Alessio Tonioni,
Daniele De Gregorio,
Matteo Poggi
Abstract:
We introduce a novel framework for training deep stereo networks effortlessly and without any ground-truth. By leveraging state-of-the-art neural rendering solutions, we generate stereo training data from image sequences collected with a single handheld camera. On top of them, a NeRF-supervised training procedure is carried out, from which we exploit rendered stereo triplets to compensate for occl…
▽ More
We introduce a novel framework for training deep stereo networks effortlessly and without any ground-truth. By leveraging state-of-the-art neural rendering solutions, we generate stereo training data from image sequences collected with a single handheld camera. On top of them, a NeRF-supervised training procedure is carried out, from which we exploit rendered stereo triplets to compensate for occlusions and depth maps as proxy labels. This results in stereo networks capable of predicting sharp and detailed disparity maps. Experimental results show that models trained under this regime yield a 30-40% improvement over existing self-supervised methods on the challenging Middlebury dataset, filling the gap to supervised models and, most times, outperforming them at zero-shot generalization.
△ Less
Submitted 30 March, 2023;
originally announced March 2023.
-
Depth Super-Resolution from Explicit and Implicit High-Frequency Features
Authors:
Xin Qiao,
Chenyang Ge,
Youmin Zhang,
Yanhui Zhou,
Fabio Tosi,
Matteo Poggi,
Stefano Mattoccia
Abstract:
We propose a novel multi-stage depth super-resolution network, which progressively reconstructs high-resolution depth maps from explicit and implicit high-frequency features. The former are extracted by an efficient transformer processing both local and global contexts, while the latter are obtained by projecting color images into the frequency domain. Both are combined together with depth feature…
▽ More
We propose a novel multi-stage depth super-resolution network, which progressively reconstructs high-resolution depth maps from explicit and implicit high-frequency features. The former are extracted by an efficient transformer processing both local and global contexts, while the latter are obtained by projecting color images into the frequency domain. Both are combined together with depth features by means of a fusion strategy within a multi-stage and multi-scale framework. Experiments on the main benchmarks, such as NYUv2, Middlebury, DIML and RGBDD, show that our approach outperforms existing methods by a large margin (~20% on NYUv2 and DIML against the contemporary work DADA, with 16x upsampling), establishing a new state-of-the-art in the guided depth super-resolution task.
△ Less
Submitted 30 May, 2023; v1 submitted 16 March, 2023;
originally announced March 2023.
-
Booster: a Benchmark for Depth from Images of Specular and Transparent Surfaces
Authors:
Pierluigi Zama Ramirez,
Alex Costanzino,
Fabio Tosi,
Matteo Poggi,
Samuele Salti,
Stefano Mattoccia,
Luigi Di Stefano
Abstract:
Estimating depth from images nowadays yields outstanding results, both in terms of in-domain accuracy and generalization. However, we identify two main challenges that remain open in this field: dealing with non-Lambertian materials and effectively processing high-resolution images. Purposely, we propose a novel dataset that includes accurate and dense ground-truth labels at high resolution, featu…
▽ More
Estimating depth from images nowadays yields outstanding results, both in terms of in-domain accuracy and generalization. However, we identify two main challenges that remain open in this field: dealing with non-Lambertian materials and effectively processing high-resolution images. Purposely, we propose a novel dataset that includes accurate and dense ground-truth labels at high resolution, featuring scenes containing several specular and transparent surfaces. Our acquisition pipeline leverages a novel deep space-time stereo framework, enabling easy and accurate labeling with sub-pixel precision. The dataset is composed of 606 samples collected in 85 different scenes, each sample includes both a high-resolution pair (12 Mpx) as well as an unbalanced stereo pair (Left: 12 Mpx, Right: 1.1 Mpx), typical of modern mobile devices that mount sensors with different resolutions. Additionally, we provide manually annotated material segmentation masks and 15K unlabeled samples. The dataset is composed of a train set and two test sets, the latter devoted to the evaluation of stereo and monocular depth estimation networks. Our experiments highlight the open challenges and future research directions in this field.
△ Less
Submitted 30 January, 2024; v1 submitted 19 January, 2023;
originally announced January 2023.
-
The Monocular Depth Estimation Challenge
Authors:
Jaime Spencer,
C. Stella Qian,
Chris Russell,
Simon Hadfield,
Erich Graf,
Wendy Adams,
Andrew J. Schofield,
James Elder,
Richard Bowden,
Heng Cong,
Stefano Mattoccia,
Matteo Poggi,
Zeeshan Khan Suri,
Yang Tang,
Fabio Tosi,
Hao Wang,
Youmin Zhang,
Yusheng Zhang,
Chaoqiang Zhao
Abstract:
This paper summarizes the results of the first Monocular Depth Estimation Challenge (MDEC) organized at WACV2023. This challenge evaluated the progress of self-supervised monocular depth estimation on the challenging SYNS-Patches dataset. The challenge was organized on CodaLab and received submissions from 4 valid teams. Participants were provided a devkit containing updated reference implementati…
▽ More
This paper summarizes the results of the first Monocular Depth Estimation Challenge (MDEC) organized at WACV2023. This challenge evaluated the progress of self-supervised monocular depth estimation on the challenging SYNS-Patches dataset. The challenge was organized on CodaLab and received submissions from 4 valid teams. Participants were provided a devkit containing updated reference implementations for 16 State-of-the-Art algorithms and 4 novel techniques. The threshold for acceptance for novel techniques was to outperform every one of the 16 SotA baselines. All participants outperformed the baseline in traditional metrics such as MAE or AbsRel. However, pointcloud reconstruction metrics were challenging to improve upon. We found predictions were characterized by interpolation artefacts at object boundaries and errors in relative object positioning. We hope this challenge is a valuable contribution to the community and encourage authors to participate in future editions.
△ Less
Submitted 22 November, 2022;
originally announced November 2022.
-
Cross-Spectral Neural Radiance Fields
Authors:
Matteo Poggi,
Pierluigi Zama Ramirez,
Fabio Tosi,
Samuele Salti,
Stefano Mattoccia,
Luigi Di Stefano
Abstract:
We propose X-NeRF, a novel method to learn a Cross-Spectral scene representation given images captured from cameras with different light spectrum sensitivity, based on the Neural Radiance Fields formulation. X-NeRF optimizes camera poses across spectra during training and exploits Normalized Cross-Device Coordinates (NXDC) to render images of different modalities from arbitrary viewpoints, which a…
▽ More
We propose X-NeRF, a novel method to learn a Cross-Spectral scene representation given images captured from cameras with different light spectrum sensitivity, based on the Neural Radiance Fields formulation. X-NeRF optimizes camera poses across spectra during training and exploits Normalized Cross-Device Coordinates (NXDC) to render images of different modalities from arbitrary viewpoints, which are aligned and at the same resolution. Experiments on 16 forward-facing scenes, featuring color, multi-spectral and infrared images, confirm the effectiveness of X-NeRF at modeling Cross-Spectral scene representations.
△ Less
Submitted 1 September, 2022;
originally announced September 2022.
-
MonoViT: Self-Supervised Monocular Depth Estimation with a Vision Transformer
Authors:
Chaoqiang Zhao,
Youmin Zhang,
Matteo Poggi,
Fabio Tosi,
Xianda Guo,
Zheng Zhu,
Guan Huang,
Yang Tang,
Stefano Mattoccia
Abstract:
Self-supervised monocular depth estimation is an attractive solution that does not require hard-to-source depth labels for training. Convolutional neural networks (CNNs) have recently achieved great success in this task. However, their limited receptive field constrains existing network architectures to reason only locally, dampening the effectiveness of the self-supervised paradigm. In the light…
▽ More
Self-supervised monocular depth estimation is an attractive solution that does not require hard-to-source depth labels for training. Convolutional neural networks (CNNs) have recently achieved great success in this task. However, their limited receptive field constrains existing network architectures to reason only locally, dampening the effectiveness of the self-supervised paradigm. In the light of the recent successes achieved by Vision Transformers (ViTs), we propose MonoViT, a brand-new framework combining the global reasoning enabled by ViT models with the flexibility of self-supervised monocular depth estimation. By combining plain convolutions with Transformer blocks, our model can reason locally and globally, yielding depth prediction at a higher level of detail and accuracy, allowing MonoViT to achieve state-of-the-art performance on the established KITTI dataset. Moreover, MonoViT proves its superior generalization capacities on other datasets such as Make3D and DrivingStereo.
△ Less
Submitted 6 August, 2022;
originally announced August 2022.
-
RGB-Multispectral Matching: Dataset, Learning Methodology, Evaluation
Authors:
Fabio Tosi,
Pierluigi Zama Ramirez,
Matteo Poggi,
Samuele Salti,
Stefano Mattoccia,
Luigi Di Stefano
Abstract:
We address the problem of registering synchronized color (RGB) and multi-spectral (MS) images featuring very different resolution by solving stereo matching correspondences. Purposely, we introduce a novel RGB-MS dataset framing 13 different scenes in indoor environments and providing a total of 34 image pairs annotated with semi-dense, high-resolution ground-truth labels in the form of disparity…
▽ More
We address the problem of registering synchronized color (RGB) and multi-spectral (MS) images featuring very different resolution by solving stereo matching correspondences. Purposely, we introduce a novel RGB-MS dataset framing 13 different scenes in indoor environments and providing a total of 34 image pairs annotated with semi-dense, high-resolution ground-truth labels in the form of disparity maps. To tackle the task, we propose a deep learning architecture trained in a self-supervised manner by exploiting a further RGB camera, required only during training data acquisition. In this setup, we can conveniently learn cross-modal matching in the absence of ground-truth labels by distilling knowledge from an easier RGB-RGB matching task based on a collection of about 11K unlabeled image triplets. Experiments show that the proposed pipeline sets a good performance bar (1.16 pixels average registration error) for future research on this novel, challenging task.
△ Less
Submitted 14 June, 2022;
originally announced June 2022.
-
Open Challenges in Deep Stereo: the Booster Dataset
Authors:
Pierluigi Zama Ramirez,
Fabio Tosi,
Matteo Poggi,
Samuele Salti,
Stefano Mattoccia,
Luigi Di Stefano
Abstract:
We present a novel high-resolution and challenging stereo dataset framing indoor scenes annotated with dense and accurate ground-truth disparities. Peculiar to our dataset is the presence of several specular and transparent surfaces, i.e. the main causes of failures for state-of-the-art stereo networks. Our acquisition pipeline leverages a novel deep space-time stereo framework which allows for ea…
▽ More
We present a novel high-resolution and challenging stereo dataset framing indoor scenes annotated with dense and accurate ground-truth disparities. Peculiar to our dataset is the presence of several specular and transparent surfaces, i.e. the main causes of failures for state-of-the-art stereo networks. Our acquisition pipeline leverages a novel deep space-time stereo framework which allows for easy and accurate labeling with sub-pixel precision. We release a total of 419 samples collected in 64 different scenes and annotated with dense ground-truth disparities. Each sample include a high-resolution pair (12 Mpx) as well as an unbalanced pair (Left: 12 Mpx, Right: 1.1 Mpx). Additionally, we provide manually annotated material segmentation masks and 15K unlabeled samples. We evaluate state-of-the-art deep networks based on our dataset, highlighting their limitations in addressing the open challenges in stereo and drawing hints for future research.
△ Less
Submitted 9 June, 2022;
originally announced June 2022.
-
On the origin of molecular oxygen on the surface of Ganymede
Authors:
A. Migliorini,
Z. Kanuchova,
S. Ioppolo,
M. Barbieri,
N. C. Jones,
S. V. Hoffmann,
G. Strazzulla,
F. Tosi,
G. Piccioni
Abstract:
Since its first identification on the surface of Ganymede in 1995, molecular oxygen (O2) ice has been at the center of a scientific debate as the surface temperature of the Jovian moon is on average well above the freezing point of O2. Laboratory evidence suggested that solid O2 may either exist in a cold (<50 K) subsurface layer of the icy surface of Ganymede, or it is in an atmospheric haze of t…
▽ More
Since its first identification on the surface of Ganymede in 1995, molecular oxygen (O2) ice has been at the center of a scientific debate as the surface temperature of the Jovian moon is on average well above the freezing point of O2. Laboratory evidence suggested that solid O2 may either exist in a cold (<50 K) subsurface layer of the icy surface of Ganymede, or it is in an atmospheric haze of the moon. Alternatively, O2 is constantly replenished at the surface through ion irradiation of water-containing ices. A conclusive answer on the existence of solid O2 on the surface of Ganymede is hampered by the lack of detailed, extensive observational datasets. We present new ground-based, high-resolution spectroscopic observations of Ganymede's surface obtained at the Telescopio Nazionale Galileo. These are combined with dedicated laboratory measurements of ultraviolet-visible (UV-vis) photoabsorption spectra of O2 ice, both pure and mixed with other species of potential interest for the Galilean satellites. Our study confirms that the two bands identified in the visible spectra of Ganymede's surface are due to the (1,0) and (0,0) transition bands of O2 ice. Oxygen-rich ice mixtures including water (H2O) and carbon dioxide (CO2) can reproduce observational reflectance data of the Ganymede's surface better than pure O2 ice in the temperature range 20-35 K. Solid H2O and CO2 also provide an environment where O2 ice can be trapped at higher temperatures than its pure ice desorption under vacuum space conditions. Our experiments at different temperatures show also that the (1,0)/(0,0) ratio in case of the CO2:O2=1:2 ice mixture at 35 K has the closest value to observations, while at 30 K the (1,0)/(0,0) ratio seems to be mixture independent with the exception of the N2:O2=1:2 ice mixture. The present work will support the ESA/JUICE mission to the Jovian system.
△ Less
Submitted 3 May, 2022;
originally announced May 2022.
-
Neural Disparity Refinement for Arbitrary Resolution Stereo
Authors:
Filippo Aleotti,
Fabio Tosi,
Pierluigi Zama Ramirez,
Matteo Poggi,
Samuele Salti,
Stefano Mattoccia,
Luigi Di Stefano
Abstract:
We introduce a novel architecture for neural disparity refinement aimed at facilitating deployment of 3D computer vision on cheap and widespread consumer devices, such as mobile phones. Our approach relies on a continuous formulation that enables to estimate a refined disparity map at any arbitrary output resolution. Thereby, it can handle effectively the unbalanced camera setup typical of nowaday…
▽ More
We introduce a novel architecture for neural disparity refinement aimed at facilitating deployment of 3D computer vision on cheap and widespread consumer devices, such as mobile phones. Our approach relies on a continuous formulation that enables to estimate a refined disparity map at any arbitrary output resolution. Thereby, it can handle effectively the unbalanced camera setup typical of nowadays mobile phones, which feature both high and low resolution RGB sensors within the same device. Moreover, our neural network can process seamlessly the output of a variety of stereo methods and, by refining the disparity maps computed by a traditional matching algorithm like SGM, it can achieve unpaired zero-shot generalization performance compared to state-of-the-art end-to-end stereo models.
△ Less
Submitted 28 October, 2021;
originally announced October 2021.
-
The surface of (4) Vesta in visible light as seen by Dawn/VIR
Authors:
B. Rousseau,
M. C. De Sanctis,
A. Raponi,
M. Ciarniello,
E. Ammannito,
A. Frigeri,
F. G. Carrozzo,
F. Tosi,
P. Scarica,
S. Fonte,
C. A. Raymond,
C. T. Russel
Abstract:
We analyzed the surface of Vesta at visible wavelengths, using the data of the Visible and InfraRed mapping spectrometer (VIR) on board the Dawn spacecraft. We mapped the variations of various spectral parameters on the entire surface of the asteroid, and also derived a map of the lithology. We took advantage of the recent corrected VIR visible data to map the radiance factor at 550 nm, three colo…
▽ More
We analyzed the surface of Vesta at visible wavelengths, using the data of the Visible and InfraRed mapping spectrometer (VIR) on board the Dawn spacecraft. We mapped the variations of various spectral parameters on the entire surface of the asteroid, and also derived a map of the lithology. We took advantage of the recent corrected VIR visible data to map the radiance factor at 550 nm, three color composites, two spectral slopes, and a band area parameter relative to the 930 nm crystal field signature in pyroxene. Using the howardite-eucrite-diogenite (HED) meteorites data as a reference, we derived the lithology of Vesta using the variations of the 930 nm and 506 nm (spin-forbidden) band centers observed in the VIR dataset. Our spectral parameters highlight a significant spectral diversity at the surface of Vesta. This diversity is mainly evidenced by impact craters and illustrates the heterogeneous subsurface and upper crust of Vesta. Impact craters also participate directly in this spectral diversity by bringing dark exogenous material to an almost entire hemisphere. Our derived lithology agrees with previous results obtained using a combination of infrared and visible data. We therefore demonstrate that it is possible to obtain crucial mineralogical information from visible wavelengths alone. In addition to the 506 nm band, we identified the 550 nm spin-forbidden one. As reported by a laboratory study for synthetic pyroxenes, we also do not observe any shift of the band center of this feature across the surface of Vesta, and thus across different mineralogies, preventing use of the 550 nm spin-forbidden band for the lithology derivation. Finally, the largest previously identified olivine rich-spot shows a peculiar behavior in two color composites but not in the other spectral parameters.
△ Less
Submitted 20 September, 2021;
originally announced September 2021.
-
SMD-Nets: Stereo Mixture Density Networks
Authors:
Fabio Tosi,
Yiyi Liao,
Carolin Schmitt,
Andreas Geiger
Abstract:
Despite stereo matching accuracy has greatly improved by deep learning in the last few years, recovering sharp boundaries and high-resolution outputs efficiently remains challenging. In this paper, we propose Stereo Mixture Density Networks (SMD-Nets), a simple yet effective learning framework compatible with a wide class of 2D and 3D architectures which ameliorates both issues. Specifically, we e…
▽ More
Despite stereo matching accuracy has greatly improved by deep learning in the last few years, recovering sharp boundaries and high-resolution outputs efficiently remains challenging. In this paper, we propose Stereo Mixture Density Networks (SMD-Nets), a simple yet effective learning framework compatible with a wide class of 2D and 3D architectures which ameliorates both issues. Specifically, we exploit bimodal mixture densities as output representation and show that this allows for sharp and precise disparity estimates near discontinuities while explicitly modeling the aleatoric uncertainty inherent in the observations. Moreover, we formulate disparity estimation as a continuous problem in the image domain, allowing our model to query disparities at arbitrary spatial precision. We carry out comprehensive experiments on a new high-resolution and highly realistic synthetic stereo dataset, consisting of stereo pairs at 8Mpx resolution, as well as on real-world stereo datasets. Our experiments demonstrate increased depth accuracy near object boundaries and prediction of ultra high-resolution disparity maps on standard GPUs. We demonstrate the flexibility of our technique by improving the performance of a variety of stereo backbones.
△ Less
Submitted 8 April, 2021;
originally announced April 2021.
-
On the confidence of stereo matching in a deep-learning era: a quantitative evaluation
Authors:
Matteo Poggi,
Seungryong Kim,
Fabio Tosi,
Sunok Kim,
Filippo Aleotti,
Dongbo Min,
Kwanghoon Sohn,
Stefano Mattoccia
Abstract:
Stereo matching is one of the most popular techniques to estimate dense depth maps by finding the disparity between matching pixels on two, synchronized and rectified images. Alongside with the development of more accurate algorithms, the research community focused on finding good strategies to estimate the reliability, i.e. the confidence, of estimated disparity maps. This information proves to b…
▽ More
Stereo matching is one of the most popular techniques to estimate dense depth maps by finding the disparity between matching pixels on two, synchronized and rectified images. Alongside with the development of more accurate algorithms, the research community focused on finding good strategies to estimate the reliability, i.e. the confidence, of estimated disparity maps. This information proves to be a powerful cue to naively find wrong matches as well as to improve the overall effectiveness of a variety of stereo algorithms according to different strategies. In this paper, we review more than ten years of developments in the field of confidence estimation for stereo matching. We extensively discuss and evaluate existing confidence measures and their variants, from hand-crafted ones to the most recent, state-of-the-art learning based methods. We study the different behaviors of each measure when applied to a pool of different stereo algorithms and, for the first time in literature, when paired with a state-of-the-art deep stereo network. Our experiments, carried out on five different standard datasets, provide a comprehensive overview of the field, highlighting in particular both strengths and limitations of learning-based strategies.
△ Less
Submitted 30 March, 2021; v1 submitted 2 January, 2021;
originally announced January 2021.
-
Correction of the VIR-visible data set from the Dawn mission at Vesta
Authors:
B. Rousseau,
M. C De Sanctis,
A. Raponi,
M. Ciarniello,
E. Ammannito,
P. Scarica,
S. Fonte,
A. Frigeri,
F. G Carrozzo,
F. Tosi
Abstract:
This work describes the correction method applied to the dataset acquired at the asteroid (4) Vesta by the visible channel of the visible and infrared mapping spectrometer. The rising detector temperature during data acquisitions in the visible wavelengths leads to a spectral slope increase over the whole spectral range. This limits the accuracy of the studies of the Vesta surface in this waveleng…
▽ More
This work describes the correction method applied to the dataset acquired at the asteroid (4) Vesta by the visible channel of the visible and infrared mapping spectrometer. The rising detector temperature during data acquisitions in the visible wavelengths leads to a spectral slope increase over the whole spectral range. This limits the accuracy of the studies of the Vesta surface in this wavelength range. Here, we detail an empirical method to correct for the visible detector temperature dependency while taking into account the specificity of the Vesta dataset.
△ Less
Submitted 26 November, 2020;
originally announced November 2020.
-
The surface of (1) Ceres in visible light as seen by Dawn/VIR
Authors:
B. Rousseau,
M. C. De Sanctis,
A. Raponi,
M. Ciarniello,
E. Ammannito,
A. Frigeri,
M. Ferrari,
S. De Angelis,
F. Tosi,
S. E. Schröder,
C. A. Raymond,
C. T. Russell
Abstract:
We study the surface of Ceres at visible wavelengths, as observed by the Visible and InfraRed mapping spectrometer (VIR) onboard the Dawn spacecraft, and analyze the variations of various spectral parameters across the whole surface. We also focus on several noteworthy areas of the surface of this dwarf planet. We made use of the newly corrected VIR visible data to build global maps of a calibrate…
▽ More
We study the surface of Ceres at visible wavelengths, as observed by the Visible and InfraRed mapping spectrometer (VIR) onboard the Dawn spacecraft, and analyze the variations of various spectral parameters across the whole surface. We also focus on several noteworthy areas of the surface of this dwarf planet. We made use of the newly corrected VIR visible data to build global maps of a calibrated radiance factor at 550 nm, with two color composites and three spectral slopes between 400 and 950 nm. We have made these maps available for the community via the Aladin Desktop software. Ceres surface shows diverse spectral behaviors in the visible range. The color composite and the spectral slope between 480 and 800 nm highlight fresh impact craters and young geologic formations of endogenous origin, which appear bluer than the rest of the surface. The steep slope before 465 nm displays very distinct variations and may be a proxy for the absorptions caused by the $O_2^{-}$ -> $Fe^{3+}$ or the $2Fe^{3+}$ -> $Fe^{2+} + Fe^{4+}$ charge transfer, if the latter are found to be responsible for the drop in this spectral range. We notice several similarities between the spectral slopes and the abundance of phyllosilicates detected in the infrared by the VIR, whereas no correlation can be clearly established with carbonate species. The region of the Dantu impact crater presents a peculiar spectral behavior (especially through the color and the spectral slope before 465 nm) suggesting a change in composition or in the surface physical properties that is not observed elsewhere on Ceres.
△ Less
Submitted 7 October, 2020;
originally announced October 2020.
-
Infrared detection of aliphatic organics on a cometary nucleus
Authors:
A. Raponi,
M. Ciarniello,
F. Capaccioni,
V. Mennella,
G. Filacchione,
V. Vinogradoff,
O. Poch,
P. Beck,
E. Quirico,
M. C. De Sanctis,
L. Moroz,
D. Kappel,
S. Erard,
D. Bockelée-Morvan,
A. Longobardo,
F. Tosi,
E. Palomba,
J. -P. Combe,
B. Rousseau,
G. Arnold,
R. W. Carlson,
A. Pommerol,
C. Pilorget,
S. Fornasier,
G. Bellucci
, et al. (6 additional authors not shown)
Abstract:
The ESA Rosetta mission has acquired unprecedented measurements of comet 67/P-Churyumov-Gerasimenko (hereafter 67P) nucleus surface, whose composition, as determined by in situ and remote sensing instruments including VIRTIS (Visible, InfraRed and Thermal Imaging Spectrometer) appears to be made by an assemblage of ices, minerals, and organic material. We performed a refined analysis of infrared o…
▽ More
The ESA Rosetta mission has acquired unprecedented measurements of comet 67/P-Churyumov-Gerasimenko (hereafter 67P) nucleus surface, whose composition, as determined by in situ and remote sensing instruments including VIRTIS (Visible, InfraRed and Thermal Imaging Spectrometer) appears to be made by an assemblage of ices, minerals, and organic material. We performed a refined analysis of infrared observations of the nucleus of comet 67P carried out by the VIRTIS-M hyperspectral imager. We found that the overall shape of the 67P infrared spectrum is similar to that of other carbon-rich outer solar system objects suggesting a possible genetic link with them. More importantly, we are also able to confirm the complex spectral structure of the wide 2.8-3.6 micron absorption feature populated by fainter bands. Among these, we unambiguously identified the presence of aliphatic organics by their ubiquitous 3.38, 3.42 and 3.47 micron bands. This novel infrared detection of aliphatic species on a cometary surface has strong implications for the evolutionary history of the primordial solar system and give evidence that comets provide an evolutionary link between interstellar material and solar system bodies.
△ Less
Submitted 30 September, 2020;
originally announced September 2020.
-
Reversing the cycle: self-supervised deep stereo through enhanced monocular distillation
Authors:
Filippo Aleotti,
Fabio Tosi,
Li Zhang,
Matteo Poggi,
Stefano Mattoccia
Abstract:
In many fields, self-supervised learning solutions are rapidly evolving and filling the gap with supervised approaches. This fact occurs for depth estimation based on either monocular or stereo, with the latter often providing a valid source of self-supervision for the former. In contrast, to soften typical stereo artefacts, we propose a novel self-supervised paradigm reversing the link between th…
▽ More
In many fields, self-supervised learning solutions are rapidly evolving and filling the gap with supervised approaches. This fact occurs for depth estimation based on either monocular or stereo, with the latter often providing a valid source of self-supervision for the former. In contrast, to soften typical stereo artefacts, we propose a novel self-supervised paradigm reversing the link between the two. Purposely, in order to train deep stereo networks, we distill knowledge through a monocular completion network. This architecture exploits single-image clues and few sparse points, sourced by traditional stereo algorithms, to estimate dense yet accurate disparity maps by means of a consensus mechanism over multiple estimations. We thoroughly evaluate with popular stereo datasets the impact of different supervisory signals showing how stereo networks trained with our paradigm outperform existing self-supervised frameworks. Finally, our proposal achieves notable generalization capabilities dealing with domain shift issues. Code available at https://github.com/FilippoAleotti/Reversing
△ Less
Submitted 17 August, 2020;
originally announced August 2020.
-
Self-adapting confidence estimation for stereo
Authors:
Matteo Poggi,
Filippo Aleotti,
Fabio Tosi,
Giulio Zaccaroni,
Stefano Mattoccia
Abstract:
Estimating the confidence of disparity maps inferred by a stereo algorithm has become a very relevant task in the years, due to the increasing number of applications leveraging such cue. Although self-supervised learning has recently spread across many computer vision tasks, it has been barely considered in the field of confidence estimation. In this paper, we propose a flexible and lightweight so…
▽ More
Estimating the confidence of disparity maps inferred by a stereo algorithm has become a very relevant task in the years, due to the increasing number of applications leveraging such cue. Although self-supervised learning has recently spread across many computer vision tasks, it has been barely considered in the field of confidence estimation. In this paper, we propose a flexible and lightweight solution enabling self-adapting confidence estimation agnostic to the stereo algorithm or network. Our approach relies on the minimum information available in any stereo setup (i.e., the input stereo pair and the output disparity map) to learn an effective confidence measure. This strategy allows us not only a seamless integration with any stereo system, including consumer and industrial devices equipped with undisclosed stereo perception methods, but also, due to its self-adapting capability, for its out-of-the-box deployment in the field. Exhaustive experimental results with different standard datasets support our claims, showing how our solution is the first-ever enabling online learning of accurate confidence estimation for any stereo system and without any requirement for the end-user.
△ Less
Submitted 24 November, 2020; v1 submitted 14 August, 2020;
originally announced August 2020.
-
Continual Adaptation for Deep Stereo
Authors:
Matteo Poggi,
Alessio Tonioni,
Fabio Tosi,
Stefano Mattoccia,
Luigi Di Stefano
Abstract:
Depth estimation from stereo images is carried out with unmatched results by convolutional neural networks trained end-to-end to regress dense disparities. Like for most tasks, this is possible if large amounts of labelled samples are available for training, possibly covering the whole data distribution encountered at deployment time. Being such an assumption systematically unmet in real applicati…
▽ More
Depth estimation from stereo images is carried out with unmatched results by convolutional neural networks trained end-to-end to regress dense disparities. Like for most tasks, this is possible if large amounts of labelled samples are available for training, possibly covering the whole data distribution encountered at deployment time. Being such an assumption systematically unmet in real applications, the capacity of adapting to any unseen setting becomes of paramount importance. Purposely, we propose a continual adaptation paradigm for deep stereo networks designed to deal with challenging and ever-changing environments. We design a lightweight and modular architecture, Modularly ADaptive Network (MADNet), and formulate Modular ADaptation algorithms (MAD, MAD++) which permit efficient optimization of independent sub-portions of the entire network. In our paradigm, the learning signals needed to continuously adapt models online can be sourced from self-supervision via right-to-left image warping or from traditional stereo algorithms. With both sources, no other data than the input images being gathered at deployment time are needed. Thus, our network architecture and adaptation algorithms realize the first real-time self-adaptive deep stereo system and pave the way for a new paradigm that can facilitate practical deployment of end-to-end architectures for dense disparity regression.
△ Less
Submitted 3 May, 2021; v1 submitted 10 July, 2020;
originally announced July 2020.
-
Real-time single image depth perception in the wild with handheld devices
Authors:
Filippo Aleotti,
Giulio Zaccaroni,
Luca Bartolomei,
Matteo Poggi,
Fabio Tosi,
Stefano Mattoccia
Abstract:
Depth perception is paramount to tackle real-world problems, ranging from autonomous driving to consumer applications. For the latter, depth estimation from a single image represents the most versatile solution, since a standard camera is available on almost any handheld device. Nonetheless, two main issues limit its practical deployment: i) the low reliability when deployed in-the-wild and ii) th…
▽ More
Depth perception is paramount to tackle real-world problems, ranging from autonomous driving to consumer applications. For the latter, depth estimation from a single image represents the most versatile solution, since a standard camera is available on almost any handheld device. Nonetheless, two main issues limit its practical deployment: i) the low reliability when deployed in-the-wild and ii) the demanding resource requirements to achieve real-time performance, often not compatible with such devices. Therefore, in this paper, we deeply investigate these issues showing how they are both addressable adopting appropriate network design and training strategies -- also outlining how to map the resulting networks on handheld devices to achieve real-time performance. Our thorough evaluation highlights the ability of such fast networks to generalize well to new environments, a crucial feature required to tackle the extremely varied contexts faced in real applications. Indeed, to further support this evidence, we report experimental results concerning real-time depth-aware augmented reality and image blurring with smartphones in-the-wild.
△ Less
Submitted 10 June, 2020;
originally announced June 2020.
-
On the uncertainty of self-supervised monocular depth estimation
Authors:
Matteo Poggi,
Filippo Aleotti,
Fabio Tosi,
Stefano Mattoccia
Abstract:
Self-supervised paradigms for monocular depth estimation are very appealing since they do not require ground truth annotations at all. Despite the astonishing results yielded by such methodologies, learning to reason about the uncertainty of the estimated depth maps is of paramount importance for practical applications, yet uncharted in the literature. Purposely, we explore for the first time how…
▽ More
Self-supervised paradigms for monocular depth estimation are very appealing since they do not require ground truth annotations at all. Despite the astonishing results yielded by such methodologies, learning to reason about the uncertainty of the estimated depth maps is of paramount importance for practical applications, yet uncharted in the literature. Purposely, we explore for the first time how to estimate the uncertainty for this task and how this affects depth accuracy, proposing a novel peculiar technique specifically designed for self-supervised approaches. On the standard KITTI dataset, we exhaustively assess the performance of each method with different self-supervised paradigms. Such evaluation highlights that our proposal i) always improves depth accuracy significantly and ii) yields state-of-the-art results concerning uncertainty estimation when training on sequences and competitive results uniquely deploying stereo pairs.
△ Less
Submitted 13 May, 2020;
originally announced May 2020.
-
On the Synergies between Machine Learning and Binocular Stereo for Depth Estimation from Images: a Survey
Authors:
Matteo Poggi,
Fabio Tosi,
Konstantinos Batsos,
Philippos Mordohai,
Stefano Mattoccia
Abstract:
Stereo matching is one of the longest-standing problems in computer vision with close to 40 years of studies and research. Throughout the years the paradigm has shifted from local, pixel-level decision to various forms of discrete and continuous optimization to data-driven, learning-based methods. Recently, the rise of machine learning and the rapid proliferation of deep learning enhanced stereo m…
▽ More
Stereo matching is one of the longest-standing problems in computer vision with close to 40 years of studies and research. Throughout the years the paradigm has shifted from local, pixel-level decision to various forms of discrete and continuous optimization to data-driven, learning-based methods. Recently, the rise of machine learning and the rapid proliferation of deep learning enhanced stereo matching with new exciting trends and applications unthinkable until a few years ago. Interestingly, the relationship between these two worlds is two-way. While machine, and especially deep, learning advanced the state-of-the-art in stereo matching, stereo itself enabled new ground-breaking methodologies such as self-supervised monocular depth estimation based on deep networks. In this paper, we review recent research in the field of learning-based depth estimation from single and binocular images highlighting the synergies, the successes achieved so far and the open challenges the community is going to face in the immediate future.
△ Less
Submitted 31 March, 2021; v1 submitted 18 April, 2020;
originally announced April 2020.
-
Distilled Semantics for Comprehensive Scene Understanding from Videos
Authors:
Fabio Tosi,
Filippo Aleotti,
Pierluigi Zama Ramirez,
Matteo Poggi,
Samuele Salti,
Luigi Di Stefano,
Stefano Mattoccia
Abstract:
Whole understanding of the surroundings is paramount to autonomous systems. Recent works have shown that deep neural networks can learn geometry (depth) and motion (optical flow) from a monocular video without any explicit supervision from ground truth annotations, particularly hard to source for these two tasks. In this paper, we take an additional step toward holistic scene understanding with mo…
▽ More
Whole understanding of the surroundings is paramount to autonomous systems. Recent works have shown that deep neural networks can learn geometry (depth) and motion (optical flow) from a monocular video without any explicit supervision from ground truth annotations, particularly hard to source for these two tasks. In this paper, we take an additional step toward holistic scene understanding with monocular cameras by learning depth and motion alongside with semantics, with supervision for the latter provided by a pre-trained network distilling proxy ground truth images. We address the three tasks jointly by a) a novel training protocol based on knowledge distillation and self-supervision and b) a compact network architecture which enables efficient scene understanding on both power hungry GPUs and low-power embedded platforms. We thoroughly assess the performance of our framework and show that it yields state-of-the-art results for monocular depth estimation, optical flow and motion segmentation.
△ Less
Submitted 31 March, 2020;
originally announced March 2020.
-
Ammonium salts are a reservoir of nitrogen on a cometary nucleus and possibly on some asteroids
Authors:
O. Poch,
I. Istiqomah,
E. Quirico,
P. Beck,
B. Schmitt,
P. Theulé,
A. Faure,
P. Hily-Blant,
L. Bonal,
A. Raponi,
M. Ciarniello,
B. Rousseau,
S. Potin,
O. Brissaud,
L. Flandinet,
G. Filacchione,
A. Pommerol,
N. Thomas,
D. Kappel,
V. Mennella,
L. Moroz,
V. Vinogradoff,
G. Arnold,
S. Erard,
D. Bockelée-Morvan
, et al. (7 additional authors not shown)
Abstract:
The measured nitrogen-to-carbon ratio in comets is lower than for the Sun, a discrepancy which could be alleviated if there is an unknown reservoir of nitrogen in comets. The nucleus of comet 67P/Churyumov-Gerasimenko exhibits an unidentified broad spectral reflectance feature around 3.2 micrometers, which is ubiquitous across its surface. On the basis of laboratory experiments, we attribute this…
▽ More
The measured nitrogen-to-carbon ratio in comets is lower than for the Sun, a discrepancy which could be alleviated if there is an unknown reservoir of nitrogen in comets. The nucleus of comet 67P/Churyumov-Gerasimenko exhibits an unidentified broad spectral reflectance feature around 3.2 micrometers, which is ubiquitous across its surface. On the basis of laboratory experiments, we attribute this absorption band to ammonium salts mixed with dust on the surface. The depth of the band indicates that semivolatile ammonium salts are a substantial reservoir of nitrogen in the comet, potentially dominating over refractory organic matter and more volatile species. Similar absorption features appear in the spectra of some asteroids, implying a compositional link between asteroids, comets, and the parent interstellar cloud.
△ Less
Submitted 12 March, 2020;
originally announced March 2020.
-
Correction of the VIR-Visible data set from the Dawn mission
Authors:
B. Rousseau,
A. Raponi,
M. Ciarniello,
E. Ammannito,
F. G. Carrozzo,
M. C. De Sanctis,
S. Fonte,
A. Frigeri,
F. Tosi
Abstract:
Data acquired at Ceres by the visible channel of the Visible and InfraRed mapping spectrometer (VIR) on board the NASA Dawn spacecraft are affected by the temperatures of both the visible (VIS) and the infrared (IR) sensors, which are respectively a CCD and a HgCdTe array. The variations of the visible channel temperatures measured during the sessions of acquisitions are correlated with variations…
▽ More
Data acquired at Ceres by the visible channel of the Visible and InfraRed mapping spectrometer (VIR) on board the NASA Dawn spacecraft are affected by the temperatures of both the visible (VIS) and the infrared (IR) sensors, which are respectively a CCD and a HgCdTe array. The variations of the visible channel temperatures measured during the sessions of acquisitions are correlated with variations in the spectral slope and shape for all the mission phases. The infrared channel (IR) temperature is more stable during the acquisitions, nonetheless it is characterized by a bi-modal distribution whether the cryocooler (and therefore the IR channel) is used or not during the visible channel operations. When the infrared channel temperature is high (175K, i.e. not in use and with crycooler off), an additional negative slope and a distortion are observed in the spectra of the visible channel. We developed an empirical correction based on a reference spectrum for the whole data set; it is designed to correct the two issues related to the sensor temperatures that we have identified. The reference spectrum is calculated to be representative of the global Ceres' surface. It is also made of data acquired when the visible and infrared channel temperatures are equal to the ones measured during an observation of the Arcturus star by VIR, which is consistent with several ground-based observations. The developed correction allows reliable analysis and mapping to be performed by minimizing the artifacts induced by fluctuations of the VIS temperature. Thanks to this correction, a direct comparison between different mission phases during which VIR experienced different visible and infrared channel temperatures is now possible.
△ Less
Submitted 19 December, 2019;
originally announced December 2019.
-
Learning End-To-End Scene Flow by Distilling Single Tasks Knowledge
Authors:
Filippo Aleotti,
Matteo Poggi,
Fabio Tosi,
Stefano Mattoccia
Abstract:
Scene flow is a challenging task aimed at jointly estimating the 3D structure and motion of the sensed environment. Although deep learning solutions achieve outstanding performance in terms of accuracy, these approaches divide the whole problem into standalone tasks (stereo and optical flow) addressing them with independent networks. Such a strategy dramatically increases the complexity of the tra…
▽ More
Scene flow is a challenging task aimed at jointly estimating the 3D structure and motion of the sensed environment. Although deep learning solutions achieve outstanding performance in terms of accuracy, these approaches divide the whole problem into standalone tasks (stereo and optical flow) addressing them with independent networks. Such a strategy dramatically increases the complexity of the training procedure and requires power-hungry GPUs to infer scene flow barely at 1 FPS. Conversely, we propose DWARF, a novel and lightweight architecture able to infer full scene flow jointly reasoning about depth and optical flow easily and elegantly trainable end-to-end from scratch. Moreover, since ground truth images for full scene flow are scarce, we propose to leverage on the knowledge learned by networks specialized in stereo or flow, for which much more data are available, to distill proxy annotations. Exhaustive experiments show that i) DWARF runs at about 10 FPS on a single high-end GPU and about 1 FPS on NVIDIA Jetson TX2 embedded at KITTI resolution, with moderate drop in accuracy compared to 10x deeper models, ii) learning from many distilled samples is more effective than from the few, annotated ones available. Code available at: https://github.com/FilippoAleotti/Dwarf-Tensorflow
△ Less
Submitted 22 November, 2019;
originally announced November 2019.
-
GAUSS -- A Sample Return Mission to Ceres
Authors:
Xian Shi,
Julie Castillo-Rogez,
Henry Hsieh,
Hejiu Hui,
Wing-Huen Ip,
Hanlun Lei,
Jian-Yang Li,
Federico Tosi,
Liyong Zhou,
Jessica Agarwal,
Antonella Barucci,
Pierre Beck,
Adriano Campo Bagatin,
Fabrizio Capaccioni,
Andrew Coates,
Gabriele Cremonese,
Rene Duffard,
Ralf Jaumann,
Geraint Jones,
Manuel Grande,
Esa Kallio,
Yangting Lin,
Olivier Mousis,
Andreas Nathues,
Jürgen Oberst
, et al. (4 additional authors not shown)
Abstract:
The goal of Project GAUSS is to return samples from the dwarf planet Ceres. Ceres is the most accessible ocean world candidate and the largest reservoir of water in the inner solar system. It shows active cryovolcanism and hydrothermal activities in recent history that resulted in minerals not found in any other planets to date except for Earth's upper crust. The possible occurrence of recent subs…
▽ More
The goal of Project GAUSS is to return samples from the dwarf planet Ceres. Ceres is the most accessible ocean world candidate and the largest reservoir of water in the inner solar system. It shows active cryovolcanism and hydrothermal activities in recent history that resulted in minerals not found in any other planets to date except for Earth's upper crust. The possible occurrence of recent subsurface ocean on Ceres and the complex geochemistry suggest possible past habitability and even the potential for ongoing habitability. Aiming to answer a broad spectrum of questions about the origin and evolution of Ceres and its potential habitability, GAUSS will return samples from this possible ocean world for the first time. The project will address the following top-level scientific questions: 1) What is the origin of Ceres and the origin and transfer of water and other volatiles in the inner solar system? 2) What are the physical properties and internal structure of Ceres? What do they tell us about the evolutionary and aqueous alteration history of icy dwarf planets? 3) What are the astrobiological implications of Ceres? Was it habitable in the past and is it still today? 4) What are the mineralogical connections between Ceres and our current collections of primitive meteorites? GAUSS will first perform a high-resolution global remote sensing investigation, characterizing the geophysical and geochemical properties of Ceres. Candidate sampling sites will then be identified, and observation campaigns will be run for an in-depth assessment of the candidate sites. Once the sampling site is selected, a lander will be deployed on the surface to collect samples and return them to Earth in cryogenic conditions that preserves the volatile and organic composition as well as the original physical status as much as possible.
△ Less
Submitted 19 March, 2020; v1 submitted 21 August, 2019;
originally announced August 2019.
-
Ice Giant Systems: The Scientific Potential of Orbital Missions to Uranus and Neptune
Authors:
Leigh N. Fletcher,
Ravit Helled,
Elias Roussos,
Geraint Jones,
Sébastien Charnoz,
Nicolas André,
David Andrews,
Michele Bannister,
Emma Bunce,
Thibault Cavalié,
Francesca Ferri,
Jonathan Fortney,
Davide Grassi,
Léa Griton,
Paul Hartogh,
Ricardo Hueso,
Yohai Kaspi,
Laurent Lamy,
Adam Masters,
Henrik Melin,
Julianne Moses,
Olivier Mousis,
Nadine Nettleman,
Christina Plainaki,
Jürgen Schmidt
, et al. (5 additional authors not shown)
Abstract:
Uranus and Neptune, and their diverse satellite and ring systems, represent the least explored environments of our Solar System, and yet may provide the archetype for the most common outcome of planetary formation throughout our galaxy. Ice Giants will be the last remaining class of Solar System planet to have a dedicated orbital explorer, and international efforts are under way to realise such an…
▽ More
Uranus and Neptune, and their diverse satellite and ring systems, represent the least explored environments of our Solar System, and yet may provide the archetype for the most common outcome of planetary formation throughout our galaxy. Ice Giants will be the last remaining class of Solar System planet to have a dedicated orbital explorer, and international efforts are under way to realise such an ambitious mission in the coming decades. In 2019, the European Space Agency released a call for scientific themes for its strategic science planning process for the 2030s and 2040s, known as Voyage 2050. We used this opportunity to review our present-day knowledge of the Uranus and Neptune systems, producing a revised and updated set of scientific questions and motivations for their exploration. This review article describes how such a mission could explore their origins, ice-rich interiors, dynamic atmospheres, unique magnetospheres, and myriad icy satellites, to address questions at the heart of modern planetary science. These two worlds are superb examples of how planets with shared origins can exhibit remarkably different evolutionary paths: Neptune as the archetype for Ice Giants, whereas Uranus may be atypical. Exploring Uranus' natural satellites and Neptune's captured moon Triton could reveal how Ocean Worlds form and remain active, redefining the extent of the habitable zone in our Solar System. For these reasons and more, we advocate that an Ice Giant System explorer should become a strategic cornerstone mission within ESA's Voyage 2050 programme, in partnership with international collaborators, and targeting launch opportunities in the early 2030s.
△ Less
Submitted 11 June, 2020; v1 submitted 4 July, 2019;
originally announced July 2019.
-
Guided Stereo Matching
Authors:
Matteo Poggi,
Davide Pallotti,
Fabio Tosi,
Stefano Mattoccia
Abstract:
Stereo is a prominent technique to infer dense depth maps from images, and deep learning further pushed forward the state-of-the-art, making end-to-end architectures unrivaled when enough data is available for training. However, deep networks suffer from significant drops in accuracy when dealing with new environments. Therefore, in this paper, we introduce Guided Stereo Matching, a novel paradigm…
▽ More
Stereo is a prominent technique to infer dense depth maps from images, and deep learning further pushed forward the state-of-the-art, making end-to-end architectures unrivaled when enough data is available for training. However, deep networks suffer from significant drops in accuracy when dealing with new environments. Therefore, in this paper, we introduce Guided Stereo Matching, a novel paradigm leveraging a small amount of sparse, yet reliable depth measurements retrieved from an external source enabling to ameliorate this weakness. The additional sparse cues required by our method can be obtained with any strategy (e.g., a LiDAR) and used to enhance features linked to corresponding disparity hypotheses. Our formulation is general and fully differentiable, thus enabling to exploit the additional sparse inputs in pre-trained deep stereo networks as well as for training a new instance from scratch. Extensive experiments on three standard datasets and two state-of-the-art deep architectures show that even with a small set of sparse input cues, i) the proposed paradigm enables significant improvements to pre-trained networks. Moreover, ii) training from scratch notably increases accuracy and robustness to domain shifts. Finally, iii) it is suited and effective even with traditional stereo algorithms such as SGM.
△ Less
Submitted 24 May, 2019;
originally announced May 2019.
-
The thermal, mechanical, structural, and dielectric properties of cometary nuclei after Rosetta
Authors:
O. Groussin,
N. Attree,
Y. Brouet,
V. Ciarletti,
B. Davidsson,
G. Filacchione,
H. H. Fischer,
B. Gundlach,
M. Knapmeyer,
J. Knollenberg,
R. Kokotanekova,
E. Kührt,
C. Leyrat,
D. Marshall,
I. Pelivan,
Y. Skorov,
C. Snodgrass,
T. Spohn,
F. Tosi
Abstract:
The physical properties of cometary nuclei observed today relate to their complex history and help to constrain their formation and evolution. In this article, we review some of the main physical properties of cometary nuclei and focus in particular on the thermal, mechanical, structural and dielectric properties, emphasizing the progress made during the Rosetta mission. Comets have a low density…
▽ More
The physical properties of cometary nuclei observed today relate to their complex history and help to constrain their formation and evolution. In this article, we review some of the main physical properties of cometary nuclei and focus in particular on the thermal, mechanical, structural and dielectric properties, emphasizing the progress made during the Rosetta mission. Comets have a low density of 480 $\pm$ 220 kg m-3 and a low permittivity of 1.9 - 2.0, consistent with a high porosity of 70 - 80 %, are weak with a very low global tensile strength $<$100 Pa, and have a low bulk thermal inertia of 0 - 60 J K-1 m-2 s-1/2 that allowed them to preserve highly volatiles species (e.g. CO, CO2, CH4, N2) into their interior since their formation. As revealed by 67P/Churyumov-Gerasimenko, the above physical properties vary across the nucleus, spatially at its surface but also with depth. The broad picture is that the bulk of the nucleus consists of a weakly bonded, rather homogeneous material that preserved primordial properties under a thin shell of processed material, and possibly covered by a granular material; this cover might in places reach a thickness of several meters. The properties of the top layer (the first meter) are not representative of that of the bulk nucleus. More globally, strong nucleus heterogeneities at a scale of a few meters are ruled out on 67P small lobe.
△ Less
Submitted 3 May, 2019;
originally announced May 2019.
-
Learning monocular depth estimation infusing traditional stereo knowledge
Authors:
Fabio Tosi,
Filippo Aleotti,
Matteo Poggi,
Stefano Mattoccia
Abstract:
Depth estimation from a single image represents a fascinating, yet challenging problem with countless applications. Recent works proved that this task could be learned without direct supervision from ground truth labels leveraging image synthesis on sequences or stereo pairs. Focusing on this second case, in this paper we leverage stereo matching in order to improve monocular depth estimation. To…
▽ More
Depth estimation from a single image represents a fascinating, yet challenging problem with countless applications. Recent works proved that this task could be learned without direct supervision from ground truth labels leveraging image synthesis on sequences or stereo pairs. Focusing on this second case, in this paper we leverage stereo matching in order to improve monocular depth estimation. To this aim we propose monoResMatch, a novel deep architecture designed to infer depth from a single input image by synthesizing features from a different point of view, horizontally aligned with the input image, performing stereo matching between the two cues. In contrast to previous works sharing this rationale, our network is the first trained end-to-end from scratch. Moreover, we show how obtaining proxy ground truth annotation through traditional stereo algorithms, such as Semi-Global Matching, enables more accurate monocular depth estimation still countering the need for expensive depth labels by keeping a self-supervised approach. Exhaustive experimental results prove how the synergy between i) the proposed monoResMatch architecture and ii) proxy-supervision attains state-of-the-art for self-supervised monocular depth estimation. The code is publicly available at https://github.com/fabiotosi92/monoResMatch-Tensorflow.
△ Less
Submitted 8 April, 2019;
originally announced April 2019.