-
Consensus Learning with Deep Sets for Essential Matrix Estimation
Authors:
Dror Moran,
Yuval Margalit,
Guy Trostianetsky,
Fadi Khatib,
Meirav Galun,
Ronen Basri
Abstract:
Robust estimation of the essential matrix, which encodes the relative position and orientation of two cameras, is a fundamental step in structure from motion pipelines. Recent deep-based methods achieved accurate estimation by using complex network architectures that involve graphs, attention layers, and hard pruning steps. Here, we propose a simpler network architecture based on Deep Sets. Given…
▽ More
Robust estimation of the essential matrix, which encodes the relative position and orientation of two cameras, is a fundamental step in structure from motion pipelines. Recent deep-based methods achieved accurate estimation by using complex network architectures that involve graphs, attention layers, and hard pruning steps. Here, we propose a simpler network architecture based on Deep Sets. Given a collection of point matches extracted from two images, our method identifies outlier point matches and models the displacement noise in inlier matches. A weighted DLT module uses these predictions to regress the essential matrix. Our network achieves accurate recovery that is superior to existing networks with significantly more complex architectures.
△ Less
Submitted 25 June, 2024;
originally announced June 2024.
-
RESFM: Robust Equivariant Multiview Structure from Motion
Authors:
Fadi Khatib,
Yoni Kasten,
Dror Moran,
Meirav Galun,
Ronen Basri
Abstract:
Multiview Structure from Motion is a fundamental and challenging computer vision problem. A recent deep-based approach was proposed utilizing matrix equivariant architectures for the simultaneous recovery of camera pose and 3D scene structure from large image collections. This work however made the unrealistic assumption that the point tracks given as input are clean of outliers. Here we propose a…
▽ More
Multiview Structure from Motion is a fundamental and challenging computer vision problem. A recent deep-based approach was proposed utilizing matrix equivariant architectures for the simultaneous recovery of camera pose and 3D scene structure from large image collections. This work however made the unrealistic assumption that the point tracks given as input are clean of outliers. Here we propose an architecture suited to dealing with outliers by adding an inlier/outlier classifying module that respects the model equivariance and by adding a robust bundle adjustment step. Experiments demonstrate that our method can be successfully applied in realistic settings that include large image collections and point tracks extracted with common heuristics and include many outliers.
△ Less
Submitted 22 April, 2024;
originally announced April 2024.
-
Leveraging Image Matching Toward End-to-End Relative Camera Pose Regression
Authors:
Fadi Khatib,
Yuval Margalit,
Meirav Galun,
Ronen Basri
Abstract:
This paper proposes a generalizable, end-to-end deep learning-based method for relative pose regression between two images. Given two images of the same scene captured from different viewpoints, our method predicts the relative rotation and translation (including direction and scale) between the two respective cameras. Inspired by the classical pipeline, our method leverages Image Matching (IM) as…
▽ More
This paper proposes a generalizable, end-to-end deep learning-based method for relative pose regression between two images. Given two images of the same scene captured from different viewpoints, our method predicts the relative rotation and translation (including direction and scale) between the two respective cameras. Inspired by the classical pipeline, our method leverages Image Matching (IM) as a pre-trained task for relative pose regression. Specifically, we use LoFTR, an architecture that utilizes an attention-based network pre-trained on Scannet, to extract semi-dense feature maps, which are then warped and fed into a pose regression network. Notably, we use a loss function that utilizes separate terms to account for the translation direction and scale. We believe such a separation is important because translation direction is determined by point correspondences while the scale is inferred from prior on shape sizes. Our ablations further support this choice. We evaluate our method on several datasets and show that it outperforms previous end-to-end methods. The method also generalizes well to unseen datasets.
△ Less
Submitted 16 April, 2024; v1 submitted 27 November, 2022;
originally announced November 2022.
-
Guided macro-mutation in a graded energy based genetic algorithm for protein structure prediction
Authors:
Mahmood A. Rashid,
Sumaiya Iqbal,
Firas Khatib,
Md Tamjidul Hoque,
Abdul Sattar
Abstract:
Protein structure prediction is considered as one of the most challenging and computationally intractable combinatorial problem. Thus, the efficient modeling of convoluted search space, the clever use of energy functions, and more importantly, the use of effective sampling algorithms become crucial to address this problem. For protein structure modeling, an off-lattice model provides limited scope…
▽ More
Protein structure prediction is considered as one of the most challenging and computationally intractable combinatorial problem. Thus, the efficient modeling of convoluted search space, the clever use of energy functions, and more importantly, the use of effective sampling algorithms become crucial to address this problem. For protein structure modeling, an off-lattice model provides limited scopes to exercise and evaluate the algorithmic developments due to its astronomically large set of data-points. In contrast, an on-lattice model widens the scopes and permits studying the relatively larger proteins because of its finite set of data-points. In this work, we took the full advantage of an on-lattice model by using a face-centered-cube lattice that has the highest packing density with the maximum degree of freedom. We proposed a graded energy-strategically mixes the Miyazawa-Jernigan (MJ) energy with the hydrophobic-polar (HP) energy-based genetic algorithm (GA) for conformational search. In our application, we introduced a 2x2 HP energy guided macro-mutation operator within the GA to explore the best possible local changes exhaustively. Conversely, the 20x20 MJ energy model-the ultimate objective function of our GA that needs to be minimized-considers the impacts amongst the 20 different amino acids and allow searching the globally acceptable conformations. On a set of benchmark proteins, our proposed approach outperformed state-of-the-art approaches in terms of the free energy levels and the root-mean-square deviations.
△ Less
Submitted 7 March, 2016;
originally announced July 2016.
-
Protein preliminaries and structure prediction fundamentals for computer scientists
Authors:
Mahmood A. Rashid,
Firas Khatib,
Abdul Sattar
Abstract:
Protein structure prediction is a challenging and unsolved problem in computer science. Proteins are the sequence of amino acids connected together by single peptide bond. The combinations of the twenty primary amino acids are the constituents of all proteins. In-vitro laboratory methods used in this problem are very time-consuming, cost-intensive, and failure-prone. Thus, alternative computationa…
▽ More
Protein structure prediction is a challenging and unsolved problem in computer science. Proteins are the sequence of amino acids connected together by single peptide bond. The combinations of the twenty primary amino acids are the constituents of all proteins. In-vitro laboratory methods used in this problem are very time-consuming, cost-intensive, and failure-prone. Thus, alternative computational methods come into play. The protein structure prediction problem is to find the three-dimensional native structure of a protein, from its amino acid sequence. The native structure of a protein has the minimum free energy possible and arguably determines the function of the protein. In this study, we present the preliminaries of proteins and their structures, protein structure prediction problem, and protein models. We also give a brief overview on experimental and computational methods used in protein structure prediction. This study will provide a fundamental knowledge to the computer scientists who are intending to pursue their future research on protein structure prediction problem.
△ Less
Submitted 9 October, 2015;
originally announced October 2015.
