Showing 1–50 of 64 results for author: Guerrero, P

Hybrid methods in reaction-diffusion equations

Authors: Tomás Alarcón, Natalia Briñas-Pascual, Juan Calvo, Pilar Guerrero, Daria Stepanova

Abstract: Simulation of stochastic spatially-extended systems is a challenging problem. The fundamental quantities in these models are individual entities such as molecules, cells, or animals, which move and react in a random manner. In big systems, accounting for each individual is inefficient. If the number of entities is large enough, random effects are negligible, and often partial differential equation… ▽ More Simulation of stochastic spatially-extended systems is a challenging problem. The fundamental quantities in these models are individual entities such as molecules, cells, or animals, which move and react in a random manner. In big systems, accounting for each individual is inefficient. If the number of entities is large enough, random effects are negligible, and often partial differential equations (PDEs) are used in which the fluctuations are neglected. When the system is heterogeneous, so that the number of individuals is large in certain regions and small in others, the PDE description becomes inaccurate in certain regions. To overcome this problem, the so-called hybrid schemes have been proposed that couple a stochastic description in parts of the domain with its mean field limit in the others. In this chapter, we review the different formulations of this approach and our recent contributions to overcome several of the limitations of previous schemes, including the extension of the concept to multiscale models of cell populations. △ Less

Submitted 20 September, 2024; originally announced September 2024.

Comments: 40 pages, 9 figures

Resonant analogue configurations in atomic condensates

Authors: Juan Ramón Muñoz de Nova, Pablo Fernández Palacios, Pedro Alcázar Guerrero, Ivar Zapata, Fernando Sols

Abstract: As a contribution to a memorial volume, we provide a comprehensive discussion of resonant configurations in analogue gravity, focusing on its implementation in atomic condensates and combining review features with original insights and calculations. In particular, we analyze the analogues of the Andreev and Hawking effects using a microscopic description based on the Bogoliubov approximation. We c… ▽ More As a contribution to a memorial volume, we provide a comprehensive discussion of resonant configurations in analogue gravity, focusing on its implementation in atomic condensates and combining review features with original insights and calculations. In particular, we analyze the analogues of the Andreev and Hawking effects using a microscopic description based on the Bogoliubov approximation. We contemplate several resonant scenarios whose efficiency to enhance anomalous scattering processes is compared to that of non-resonant setups. The presence of quantum signatures in analogue configurations, such as the violation of Cauchy-Schwarz inequalities or entanglement, is analyzed, observing that resonant configurations highly increase the entanglement signal, especially for the Andreev effect. We also discuss how these results have served as inspiration for the rapidly expanding field of quantum information in high-energy colliders. Finally, we study the physics of black-hole lasers as further examples of resonant analogue structures, distinguishing three stages in its time evolution. For short times, we compute the linear and non-linear spectrum for different models. For intermediate times, we generalize the current analysis of the BHL-BCL crossover. For long times, we discuss the emerging concept of spontaneous Floquet state and its potential implications. △ Less

Submitted 14 June, 2024; originally announced June 2024.

Comments: 40 pages, 10 figures

SuperGaussian: Repurposing Video Models for 3D Super Resolution

Authors: Yuan Shen, Duygu Ceylan, Paul Guerrero, Zexiang Xu, Niloy J. Mitra, Shenlong Wang, Anna Frühstück

Abstract: We present a simple, modular, and generic method that upsamples coarse 3D models by adding geometric and appearance details. While generative 3D models now exist, they do not yet match the quality of their counterparts in image and video domains. We demonstrate that it is possible to directly repurpose existing (pretrained) video models for 3D super-resolution and thus sidestep the problem of the… ▽ More We present a simple, modular, and generic method that upsamples coarse 3D models by adding geometric and appearance details. While generative 3D models now exist, they do not yet match the quality of their counterparts in image and video domains. We demonstrate that it is possible to directly repurpose existing (pretrained) video models for 3D super-resolution and thus sidestep the problem of the shortage of large repositories of high-quality 3D training models. We describe how to repurpose video upsampling models, which are not 3D consistent, and combine them with 3D consolidation to produce 3D-consistent results. As output, we produce high quality Gaussian Splat models, which are object centric and effective. Our method is category agnostic and can be easily incorporated into existing 3D workflows. We evaluate our proposed SuperGaussian on a variety of 3D inputs, which are diverse both in terms of complexity and representation (e.g., Gaussian Splats or NeRFs), and demonstrate that our simple method significantly improves the fidelity of the final 3D models. Check our project website for details: supergaussian.github.io △ Less

Submitted 16 July, 2024; v1 submitted 1 June, 2024; originally announced June 2024.

Comments: Accepted at ECCV 2024, project website with interactive demo: https://supergaussian.github.io

Definable separability and second-countability in o-minimal structures

Authors: Pablo Andújar Guerrero

Abstract: We show that separability and second-countability are first-order properties among topological spaces definable in o-minimal expansions of $(\mathbb{R},<)$. We do so by introducing first-order characterizations -- definable separability and definable second-countability -- which make sense in a wider model-theoretic context. We prove that within o-minimality these notions have the desired properti… ▽ More We show that separability and second-countability are first-order properties among topological spaces definable in o-minimal expansions of $(\mathbb{R},<)$. We do so by introducing first-order characterizations -- definable separability and definable second-countability -- which make sense in a wider model-theoretic context. We prove that within o-minimality these notions have the desired properties, including their equivalence among definable metric spaces, and conjecture a definable version of Urysohn's Metrization Theorem. △ Less

Submitted 11 May, 2024; originally announced May 2024.

MSC Class: 03C64 (Primary); 54A05; 54D65; 54D70 (Secondary)

Definable compactness in o-minimal structures

Authors: Pablo Andújar Guerrero

Abstract: We characterize the notion of definable compactness for topological spaces definable in o-minimal structures, answering questions of Peterzil and Steinhorn (1999) and Johnson (2018). Specifically, we prove the equivalence of various definitions of definable compactness in the literature, including those in terms of definable curves, definable types and definable downward directed families of close… ▽ More We characterize the notion of definable compactness for topological spaces definable in o-minimal structures, answering questions of Peterzil and Steinhorn (1999) and Johnson (2018). Specifically, we prove the equivalence of various definitions of definable compactness in the literature, including those in terms of definable curves, definable types and definable downward directed families of closed sets. △ Less

Submitted 11 May, 2024; originally announced May 2024.

Comments: This is a round-up of the topological content of arXiv:2111.03802

MSC Class: 03C64 (Primary); 54A05; 54D30 (Secondary)

TexSliders: Diffusion-Based Texture Editing in CLIP Space

Authors: Julia Guerrero-Viu, Milos Hasan, Arthur Roullier, Midhun Harikumar, Yiwei Hu, Paul Guerrero, Diego Gutierrez, Belen Masia, Valentin Deschaintre

Abstract: Generative models have enabled intuitive image creation and manipulation using natural language. In particular, diffusion models have recently shown remarkable results for natural image editing. In this work, we propose to apply diffusion techniques to edit textures, a specific class of images that are an essential part of 3D content creation pipelines. We analyze existing editing methods and show… ▽ More Generative models have enabled intuitive image creation and manipulation using natural language. In particular, diffusion models have recently shown remarkable results for natural image editing. In this work, we propose to apply diffusion techniques to edit textures, a specific class of images that are an essential part of 3D content creation pipelines. We analyze existing editing methods and show that they are not directly applicable to textures, since their common underlying approach, manipulating attention maps, is unsuitable for the texture domain. To address this, we propose a novel approach that instead manipulates CLIP image embeddings to condition the diffusion generation. We define editing directions using simple text prompts (e.g., "aged wood" to "new wood") and map these to CLIP image embedding space using a texture prior, with a sampling-based approach that gives us identity-preserving directions in CLIP space. To further improve identity preservation, we project these directions to a CLIP subspace that minimizes identity variations resulting from entangled texture attributes. Our editing pipeline facilitates the creation of arbitrary sliders using natural language prompts only, with no ground-truth annotated data necessary. △ Less

Submitted 1 May, 2024; originally announced May 2024.

Comments: SIGGRAPH 2024 Conference Proceedings

New fractional classifications of papers based on two generations of references and on the ASJC Scopus scheme

Authors: Jesus M. Alvarez Llorente, Vicente P. Guerrero-Bote, Felix de Moya-Anegon

Abstract: This paper presents and evaluates a set of methods to classify individual Scopus publications using their references back to the second generation, where each publication can be assigned fractionally into up to five ASJC (All Science Journal Classifications) categories, excluding the Multidisciplinary area and the miscellaneous categories. Based on proposals by Glanzel et al. (1999a, 1999b, 2021),… ▽ More This paper presents and evaluates a set of methods to classify individual Scopus publications using their references back to the second generation, where each publication can be assigned fractionally into up to five ASJC (All Science Journal Classifications) categories, excluding the Multidisciplinary area and the miscellaneous categories. Based on proposals by Glanzel et al. (1999a, 1999b, 2021), some additional parameters are established that allow different results to be obtained depending on how category membership is weighted or how the acceptance thresholds for multiple assignments are established. Various classifications are obtained, and then compared with each other, with the original ASJC Scopus journal classification, and with the AAC (Authors Assignation Collection) classification of a previous study (Alvarez-Llorente et al., 2023) in which the papers corresponding authors assign them the most appropriate categories. Classifications in which a high threshold is set for allowing assignments to multiple categories, combined with the use of first- and second-generation references and averaging over the number of references, provide the most promising results, improving over other reference-based reclassification proposals in terms of granularity, and over the Scopus classification itself in such aspects as the homogeneity of the publications assigned to a category. They also show greater coincidence with the AAC classification. △ Less

Submitted 4 April, 2024; originally announced April 2024.

Persistent Homology analysis for solar magnetograms

Authors: Pablo Santamarina Guerrero, Yukio Katsukawa, Shin Toriumi, David Orozco Suárez

Abstract: Understanding the magnetic fields of the Sun is essential for unraveling the underlying mechanisms driving solar activity. Integrating topological data analysis techniques into these investigations can provide valuable insights into the intricate structures of magnetic fields, enhancing our comprehension of solar activity and its implications. In this study, we explore what persistent homology can… ▽ More Understanding the magnetic fields of the Sun is essential for unraveling the underlying mechanisms driving solar activity. Integrating topological data analysis techniques into these investigations can provide valuable insights into the intricate structures of magnetic fields, enhancing our comprehension of solar activity and its implications. In this study, we explore what persistent homology can offer in the analysis of solar magnetograms, with the objective of introducing a novel tool that will serve as the foundation for further studies of magnetic structures at the solar surface. By combining various filtration methods of the persistent homology analysis, we conduct an analysis of solar magnetograms that captures the broad magnetic scene, involving a mixture of positive and negative polarities. This analysis is applied to observations of both quiet Sun and active regions, taken with Hinode/SOT and SDO/HMI, respectively. Our primary focus is on analyzing the properties of the spatial structures and features of the magnetic fields identified through these techniques. The results show that persistent diagrams can encode the spatial structural complexity of the magnetic flux of active regions by identifying the isolated, connected, and interacting features. They facilitate the classification of active regions based on their morphology and the detection and quantification of interacting structures of opposing polarities, such as $δ$-spots. The small-scale events in the quiet Sun, such as magnetic flux cancellation and emergence, are also revealed in persistent diagrams and can be studied by observing the evolution of the plots and tracking the relevant features. △ Less

Submitted 30 January, 2024; originally announced January 2024.

PPSURF: Combining Patches and Point Convolutions for Detailed Surface Reconstruction

Authors: Philipp Erler, Lizeth Fuentes, Pedro Hermosilla, Paul Guerrero, Renato Pajarola, Michael Wimmer

Abstract: 3D surface reconstruction from point clouds is a key step in areas such as content creation, archaeology, digital cultural heritage, and engineering. Current approaches either try to optimize a non-data-driven surface representation to fit the points, or learn a data-driven prior over the distribution of commonly occurring surfaces and how they correlate with potentially noisy point clouds. Data-d… ▽ More 3D surface reconstruction from point clouds is a key step in areas such as content creation, archaeology, digital cultural heritage, and engineering. Current approaches either try to optimize a non-data-driven surface representation to fit the points, or learn a data-driven prior over the distribution of commonly occurring surfaces and how they correlate with potentially noisy point clouds. Data-driven methods enable robust handling of noise and typically either focus on a global or a local prior, which trade-off between robustness to noise on the global end and surface detail preservation on the local end. We propose PPSurf as a method that combines a global prior based on point convolutions and a local prior based on processing local point cloud patches. We show that this approach is robust to noise while recovering surface details more accurately than the current state-of-the-art. Our source code, pre-trained model and dataset are available at: https://github.com/cg-tuwien/ppsurf △ Less

Submitted 8 February, 2024; v1 submitted 16 January, 2024; originally announced January 2024.

Comments: Published in Computer Graphics Forum (Jan 2024): https://onlinelibrary.wiley.com/doi/10.1111/cgf.15000

ACM Class: I.4.5

Journal ref: Computer Graphics Forum e15000, 2024

Disorder-Induced Delocalization in Magic-Angle Twisted Bilayer Graphene

Authors: Pedro Alcázar Guerrero, Viet-Hung Nguyen, Jorge Martínez Romeral, Aron W. Cummings, José-Hugo Garcia, Jean-Christophe Charlier, Stephan Roche

Abstract: Flat bands in moiré systems are exciting new playgrounds for the generation and study of exotic many-body physics phenomena in low-dimensional materials. Such physics is attributed to the vanishing kinetic energy and strong spatial localization of the flat-band states. Here we use numerical simulations to examine the electronic transport properties of such flat bands in magic-angle twisted bilayer… ▽ More Flat bands in moiré systems are exciting new playgrounds for the generation and study of exotic many-body physics phenomena in low-dimensional materials. Such physics is attributed to the vanishing kinetic energy and strong spatial localization of the flat-band states. Here we use numerical simulations to examine the electronic transport properties of such flat bands in magic-angle twisted bilayer graphene in the presence of disorder. We find that while a conventional downscaling of the mean free path with increasing disorder strength occurs at higher energies, in the flat bands the mean free path can actually increase with increasing disorder strength.This phenomenon is also captured by the disorder-dependent quantum metric, which is directly linked to the ground state localization.This disorder-induced delocalization suggests that weak disorder may have a strong impact on the exotic physics of magic-angle bilayer graphene and other related moiré systems. △ Less

Submitted 3 June, 2024; v1 submitted 16 January, 2024; originally announced January 2024.

Comments: Main text: 7 pages, 4 figures. Sup .Mat.: 4 pages, 2 figures

Diffusion Handles: Enabling 3D Edits for Diffusion Models by Lifting Activations to 3D

Authors: Karran Pandey, Paul Guerrero, Matheus Gadelha, Yannick Hold-Geoffroy, Karan Singh, Niloy Mitra

Abstract: Diffusion Handles is a novel approach to enabling 3D object edits on diffusion images. We accomplish these edits using existing pre-trained diffusion models, and 2D image depth estimation, without any fine-tuning or 3D object retrieval. The edited results remain plausible, photo-real, and preserve object identity. Diffusion Handles address a critically missing facet of generative image based creat… ▽ More Diffusion Handles is a novel approach to enabling 3D object edits on diffusion images. We accomplish these edits using existing pre-trained diffusion models, and 2D image depth estimation, without any fine-tuning or 3D object retrieval. The edited results remain plausible, photo-real, and preserve object identity. Diffusion Handles address a critically missing facet of generative image based creative design, and significantly advance the state-of-the-art in generative image editing. Our key insight is to lift diffusion activations for an object to 3D using a proxy depth, 3D-transform the depth and associated activations, and project them back to image space. The diffusion process applied to the manipulated activations with identity control, produces plausible edited images showing complex 3D occlusion and lighting effects. We evaluate Diffusion Handles: quantitatively, on a large synthetic data benchmark; and qualitatively by a user study, showing our output to be more plausible, and better than prior art at both, 3D editing and identity control. Project Webpage: https://diffusionhandles.github.io/ △ Less

Submitted 6 December, 2023; v1 submitted 1 December, 2023; originally announced December 2023.

Comments: Project Webpage: https://diffusionhandles.github.io/

Automatic and Computationally Efficient Alignment in Fan- and Cone-beam Tomography

Authors: Patricio Guerrero, Simon Bellens, Ricardo Santander, Wim Dewulf

Abstract: This work is concerned with fan- and cone-beam computed tomography with circular source trajectory, where the reconstruction inverse problem requires an accurate knowledge of source, detector and rotational axis relative positions and orientations. We address this additional inverse problem as a preceding step of the reconstruction process directly from the acquired projections. In the cone-beam c… ▽ More This work is concerned with fan- and cone-beam computed tomography with circular source trajectory, where the reconstruction inverse problem requires an accurate knowledge of source, detector and rotational axis relative positions and orientations. We address this additional inverse problem as a preceding step of the reconstruction process directly from the acquired projections. In the cone-beam case, we present a method that estimates both the detector shift (orthogonal to both focal and rotational axes) and the in-plane detector rotation (over the focal axis) based on the variable projection optimization approach. In addition and for the fan-beam case, two new strategies with low computational cost are presented to estimate the detector shift based on a fan-beam symmetry condition. The methods are validated with simulated and experimental industrial tomographic data with code examples available for both fan- and cone-beam geometries. △ Less

Submitted 7 May, 2024; v1 submitted 14 October, 2023; originally announced October 2023.

Explorable Mesh Deformation Subspaces from Unstructured Generative Models

Authors: Arman Maesumi, Paul Guerrero, Vladimir G. Kim, Matthew Fisher, Siddhartha Chaudhuri, Noam Aigerman, Daniel Ritchie

Abstract: Exploring variations of 3D shapes is a time-consuming process in traditional 3D modeling tools. Deep generative models of 3D shapes often feature continuous latent spaces that can, in principle, be used to explore potential variations starting from a set of input shapes. In practice, doing so can be problematic: latent spaces are high dimensional and hard to visualize, contain shapes that are not… ▽ More Exploring variations of 3D shapes is a time-consuming process in traditional 3D modeling tools. Deep generative models of 3D shapes often feature continuous latent spaces that can, in principle, be used to explore potential variations starting from a set of input shapes. In practice, doing so can be problematic: latent spaces are high dimensional and hard to visualize, contain shapes that are not relevant to the input shapes, and linear paths through them often lead to sub-optimal shape transitions. Furthermore, one would ideally be able to explore variations in the original high-quality meshes used to train the generative model, not its lower-quality output geometry. In this paper, we present a method to explore variations among a given set of landmark shapes by constructing a mapping from an easily-navigable 2D exploration space to a subspace of a pre-trained generative model. We first describe how to find a mapping that spans the set of input landmark shapes and exhibits smooth variations between them. We then show how to turn the variations in this subspace into deformation fields, to transfer those variations to high-quality meshes for the landmark shapes. Our results show that our method can produce visually-pleasing and easily-navigable 2D exploration spaces for several different shape categories, especially as compared to prior work on learning deformation spaces for 3D shapes. △ Less

Submitted 11 October, 2023; originally announced October 2023.

Comments: SIGGRAPH Asia 2023, 15 pages

One-dimensional definable topological spaces in o-minimal structures

Authors: Pablo Andújar Guerrero, Margaret E. M. Thomas

Abstract: We study the properties of topological spaces $(X,τ)$, where $X$ is a definable set in an o-minimal structure and the topology $τ$ on $X$ has a basis that is (uniformly) definable. Examples of such spaces include the canonical euclidean topology on definable sets, definable order topologies, definable quotient spaces and definable metric spaces. We use o-minimality to undertake their study in topo… ▽ More We study the properties of topological spaces $(X,τ)$, where $X$ is a definable set in an o-minimal structure and the topology $τ$ on $X$ has a basis that is (uniformly) definable. Examples of such spaces include the canonical euclidean topology on definable sets, definable order topologies, definable quotient spaces and definable metric spaces. We use o-minimality to undertake their study in topological terms, focussing here in particular on spaces of dimension one. We present several results, given in terms of piecewise decompositions and existence of definable embeddings and homeomorphisms, for various classes of spaces that are described in terms of classical separation axioms and definable analogues of properties such as separability, compactness and metrizability. For example, we prove that all Hausdorff one-dimensional definable topologies are piecewise the euclidean, discrete, or upper or lower limit topology; we give a characterization of all one-dimensional, regular, Hausdorff definable topologies in terms of spaces that have a lexicographic ordering or a topology generalizing the Alexandrov double of the euclidean topology; and we show that, if the underlying structure expands an ordered field, then any one-dimensional Hausdorff definable topology that is piecewise euclidean is definably homeomorphic to a euclidean space. As applications of these results, we prove definable versions of several open conjectures from set-theoretic topology, due to Gruenhage and Fremlin, on the existence of a 3-element basis for regular, Hausdorff topologies and on the nature of perfectly normal, compact, Hausdorff spaces; we obtain universality results for some classes of Hausdorff and regular topologies; and we characterize when certain metrizable definable topologies admit a definable metric. △ Less

Submitted 6 October, 2023; originally announced October 2023.

MSC Class: 03C64(Primary); 54A05; 54F65; 54D10; 54F50; 54D15; 54A35 (Secondary)

A shorter proof of the Marker-Steinhorn Theorem

Authors: Pablo Andújar Guerrero

Abstract: By analyzing o-minimal definable preorders we give a proof of the Marker-Steinhorn Theorem [MS94] that shortens the original proof. By analyzing o-minimal definable preorders we give a proof of the Marker-Steinhorn Theorem [MS94] that shortens the original proof. △ Less

Submitted 16 June, 2023; originally announced June 2023.

Comments: I have two family names: Andújar Guerrero. The BibTeX citation generated by arXiv outputs "Andújar" as a middle name. arXiv admin note: substantial text overlap with arXiv:2111.03802

MSC Class: 03C64

PhotoMat: A Material Generator Learned from Single Flash Photos

Authors: Xilong Zhou, Miloš Hašan, Valentin Deschaintre, Paul Guerrero, Yannick Hold-Geoffroy, Kalyan Sunkavalli, Nima Khademi Kalantari

Abstract: Authoring high-quality digital materials is key to realism in 3D rendering. Previous generative models for materials have been trained exclusively on synthetic data; such data is limited in availability and has a visual gap to real materials. We circumvent this limitation by proposing PhotoMat: the first material generator trained exclusively on real photos of material samples captured using a cel… ▽ More Authoring high-quality digital materials is key to realism in 3D rendering. Previous generative models for materials have been trained exclusively on synthetic data; such data is limited in availability and has a visual gap to real materials. We circumvent this limitation by proposing PhotoMat: the first material generator trained exclusively on real photos of material samples captured using a cell phone camera with flash. Supervision on individual material maps is not available in this setting. Instead, we train a generator for a neural material representation that is rendered with a learned relighting module to create arbitrarily lit RGB images; these are compared against real photos using a discriminator. We then train a material maps estimator to decode material reflectance properties from the neural material representation. We train PhotoMat with a new dataset of 12,000 material photos captured with handheld phone cameras under flash lighting. We demonstrate that our generated materials have better visual quality than previous material generators trained on synthetic data. Moreover, we can fit analytical material models to closely match these generated neural materials, thus allowing for further editing and use in 3D rendering. △ Less

Submitted 23 May, 2023; v1 submitted 20 May, 2023; originally announced May 2023.

Journal ref: Siggraph 2023

Cardinality of definable families of sets in o-minimal structures

Authors: Pablo Andújar Guerrero

Abstract: We prove that any definable family of subsets of a definable infinite set $A$ in an o-minimal structure has cardinality at most $|A|$. We derive some consequences in terms of counting definable types and existence of definable topological spaces. We prove that any definable family of subsets of a definable infinite set $A$ in an o-minimal structure has cardinality at most $|A|$. We derive some consequences in terms of counting definable types and existence of definable topological spaces. △ Less

Submitted 20 May, 2023; originally announced May 2023.

Comments: I have two surnames: Andújar Guerrero. ArXiV is unable to represent this and instead considers Andújar my middle name when generating citations etc

MSC Class: 03C64 (Primary); 03C45 (Secondary)

ShapeCoder: Discovering Abstractions for Visual Programs from Unstructured Primitives

Authors: R. Kenny Jones, Paul Guerrero, Niloy J. Mitra, Daniel Ritchie

Abstract: Programs are an increasingly popular representation for visual data, exposing compact, interpretable structure that supports manipulation. Visual programs are usually written in domain-specific languages (DSLs). Finding "good" programs, that only expose meaningful degrees of freedom, requires access to a DSL with a "good" library of functions, both of which are typically authored by domain experts… ▽ More Programs are an increasingly popular representation for visual data, exposing compact, interpretable structure that supports manipulation. Visual programs are usually written in domain-specific languages (DSLs). Finding "good" programs, that only expose meaningful degrees of freedom, requires access to a DSL with a "good" library of functions, both of which are typically authored by domain experts. We present ShapeCoder, the first system capable of taking a dataset of shapes, represented with unstructured primitives, and jointly discovering (i) useful abstraction functions and (ii) programs that use these abstractions to explain the input shapes. The discovered abstractions capture common patterns (both structural and parametric) across the dataset, so that programs rewritten with these abstractions are more compact, and expose fewer degrees of freedom. ShapeCoder improves upon previous abstraction discovery methods, finding better abstractions, for more complex inputs, under less stringent input assumptions. This is principally made possible by two methodological advancements: (a) a shape to program recognition network that learns to solve sub-problems and (b) the use of e-graphs, augmented with a conditional rewrite scheme, to determine when abstractions with complex parametric expressions can be applied, in a tractable manner. We evaluate ShapeCoder on multiple datasets of 3D shapes, where primitive decompositions are either parsed from manual annotations or produced by an unsupervised cuboid abstraction method. In all domains, ShapeCoder discovers a library of abstractions that capture high-level relationships, remove extraneous degrees of freedom, and achieve better dataset compression compared with alternative approaches. Finally, we investigate how programs rewritten to use discovered abstractions prove useful for downstream tasks. △ Less

Submitted 9 May, 2023; originally announced May 2023.

Comments: SIGGRAPH 2023

Generating Procedural Materials from Text or Image Prompts

Authors: Yiwei Hu, Paul Guerrero, Miloš Hašan, Holly Rushmeier, Valentin Deschaintre

Abstract: Node graph systems are used ubiquitously for material design in computer graphics. They allow the use of visual programming to achieve desired effects without writing code. As high-level design tools they provide convenience and flexibility, but mastering the creation of node graphs usually requires professional training. We propose an algorithm capable of generating multiple node graphs from diff… ▽ More Node graph systems are used ubiquitously for material design in computer graphics. They allow the use of visual programming to achieve desired effects without writing code. As high-level design tools they provide convenience and flexibility, but mastering the creation of node graphs usually requires professional training. We propose an algorithm capable of generating multiple node graphs from different types of prompts, significantly lowering the bar for users to explore a specific design space. Previous work was limited to unconditional generation of random node graphs, making the generation of an envisioned material challenging. We propose a multi-modal node graph generation neural architecture for high-quality procedural material synthesis which can be conditioned on different inputs (text or image prompts), using a CLIP-based encoder. We also create a substantially augmented material graph dataset, key to improving the generation quality. Finally, we generate high-quality graph samples using a regularized sampling process and improve the matching quality by differentiable optimization for top-ranked samples. We compare our methods to CLIP-based database search baselines (which are themselves novel) and achieve superior or similar performance without requiring massive data storage. We further show that our model can produce a set of material graphs unconditionally, conditioned on images, text prompts or partial graphs, serving as a tool for automatic visual programming completion. △ Less

Submitted 25 April, 2023; originally announced April 2023.

Journal ref: SIGGRAPH 2023

Neurosymbolic Models for Computer Graphics

Authors: Daniel Ritchie, Paul Guerrero, R. Kenny Jones, Niloy J. Mitra, Adriana Schulz, Karl D. D. Willis, Jiajun Wu

Abstract: Procedural models (i.e. symbolic programs that output visual data) are a historically-popular method for representing graphics content: vegetation, buildings, textures, etc. They offer many advantages: interpretable design parameters, stochastic variations, high-quality outputs, compact representation, and more. But they also have some limitations, such as the difficulty of authoring a procedural… ▽ More Procedural models (i.e. symbolic programs that output visual data) are a historically-popular method for representing graphics content: vegetation, buildings, textures, etc. They offer many advantages: interpretable design parameters, stochastic variations, high-quality outputs, compact representation, and more. But they also have some limitations, such as the difficulty of authoring a procedural model from scratch. More recently, AI-based methods, and especially neural networks, have become popular for creating graphic content. These techniques allow users to directly specify desired properties of the artifact they want to create (via examples, constraints, or objectives), while a search, optimization, or learning algorithm takes care of the details. However, this ease of use comes at a cost, as it's often hard to interpret or manipulate these representations. In this state-of-the-art report, we summarize research on neurosymbolic models in computer graphics: methods that combine the strengths of both AI and symbolic programs to represent, generate, and manipulate visual data. We survey recent work applying these techniques to represent 2D shapes, 3D shapes, and materials & textures. Along the way, we situate each prior work in a unified design space for neurosymbolic models, which helps reveal underexplored areas and opportunities for future research. △ Less

Submitted 20 April, 2023; originally announced April 2023.

Comments: Eurographics 2023 State-of-the-art report (STAR)

Unsupervised 3D Shape Reconstruction by Part Retrieval and Assembly

Authors: Xianghao Xu, Paul Guerrero, Matthew Fisher, Siddhartha Chaudhuri, Daniel Ritchie

Abstract: Representing a 3D shape with a set of primitives can aid perception of structure, improve robotic object manipulation, and enable editing, stylization, and compression of 3D shapes. Existing methods either use simple parametric primitives or learn a generative shape space of parts. Both have limitations: parametric primitives lead to coarse approximations, while learned parts offer too little cont… ▽ More Representing a 3D shape with a set of primitives can aid perception of structure, improve robotic object manipulation, and enable editing, stylization, and compression of 3D shapes. Existing methods either use simple parametric primitives or learn a generative shape space of parts. Both have limitations: parametric primitives lead to coarse approximations, while learned parts offer too little control over the decomposition. We instead propose to decompose shapes using a library of 3D parts provided by the user, giving full control over the choice of parts. The library can contain parts with high-quality geometry that are suitable for a given category, resulting in meaningful decompositions with clean geometry. The type of decomposition can also be controlled through the choice of parts in the library. Our method works via a self-supervised approach that iteratively retrieves parts from the library and refines their placements. We show that this approach gives higher reconstruction accuracy and more desirable decompositions than existing approaches. Additionally, we show how the decomposition can be controlled through the part library by using different part libraries to reconstruct the same shapes. △ Less

Submitted 3 March, 2023; originally announced March 2023.

Comments: CVPR 2023

3D-LDM: Neural Implicit 3D Shape Generation with Latent Diffusion Models

Authors: Gimin Nam, Mariem Khlifi, Andrew Rodriguez, Alberto Tono, Linqi Zhou, Paul Guerrero

Abstract: Diffusion models have shown great promise for image generation, beating GANs in terms of generation diversity, with comparable image quality. However, their application to 3D shapes has been limited to point or voxel representations that can in practice not accurately represent a 3D surface. We propose a diffusion model for neural implicit representations of 3D shapes that operates in the latent s… ▽ More Diffusion models have shown great promise for image generation, beating GANs in terms of generation diversity, with comparable image quality. However, their application to 3D shapes has been limited to point or voxel representations that can in practice not accurately represent a 3D surface. We propose a diffusion model for neural implicit representations of 3D shapes that operates in the latent space of an auto-decoder. This allows us to generate diverse and high quality 3D surfaces. We additionally show that we can condition our model on images or text to enable image-to-3D generation and text-to-3D generation using CLIP embeddings. Furthermore, adding noise to the latent codes of existing shapes allows us to explore shape variations. △ Less

Submitted 15 December, 2022; v1 submitted 1 December, 2022; originally announced December 2022.

RenderDiffusion: Image Diffusion for 3D Reconstruction, Inpainting and Generation

Authors: Titas Anciukevičius, Zexiang Xu, Matthew Fisher, Paul Henderson, Hakan Bilen, Niloy J. Mitra, Paul Guerrero

Abstract: Diffusion models currently achieve state-of-the-art performance for both conditional and unconditional image generation. However, so far, image diffusion models do not support tasks required for 3D understanding, such as view-consistent 3D generation or single-view object reconstruction. In this paper, we present RenderDiffusion, the first diffusion model for 3D generation and inference, trained u… ▽ More Diffusion models currently achieve state-of-the-art performance for both conditional and unconditional image generation. However, so far, image diffusion models do not support tasks required for 3D understanding, such as view-consistent 3D generation or single-view object reconstruction. In this paper, we present RenderDiffusion, the first diffusion model for 3D generation and inference, trained using only monocular 2D supervision. Central to our method is a novel image denoising architecture that generates and renders an intermediate three-dimensional representation of a scene in each denoising step. This enforces a strong inductive structure within the diffusion process, providing a 3D consistent representation while only requiring 2D supervision. The resulting 3D representation can be rendered from any view. We evaluate RenderDiffusion on FFHQ, AFHQ, ShapeNet and CLEVR datasets, showing competitive performance for generation of 3D scenes and inference of 3D scenes from 2D images. Additionally, our diffusion-based approach allows us to use 2D inpainting to edit 3D scenes. △ Less

Submitted 20 February, 2024; v1 submitted 17 November, 2022; originally announced November 2022.

Comments: Accepted at CVPR 2023. Project page: https://github.com/Anciukevicius/RenderDiffusion

Search for Concepts: Discovering Visual Concepts Using Direct Optimization

Authors: Pradyumna Reddy, Paul Guerrero, Niloy J. Mitra

Abstract: Finding an unsupervised decomposition of an image into individual objects is a key step to leverage compositionality and to perform symbolic reasoning. Traditionally, this problem is solved using amortized inference, which does not generalize beyond the scope of the training data, may sometimes miss correct decompositions, and requires large amounts of training data. We propose finding a decomposi… ▽ More Finding an unsupervised decomposition of an image into individual objects is a key step to leverage compositionality and to perform symbolic reasoning. Traditionally, this problem is solved using amortized inference, which does not generalize beyond the scope of the training data, may sometimes miss correct decompositions, and requires large amounts of training data. We propose finding a decomposition using direct, unamortized optimization, via a combination of a gradient-based optimization for differentiable object properties and global search for non-differentiable properties. We show that using direct optimization is more generalizable, misses fewer correct decompositions, and typically requires less data than methods based on amortized inference. This highlights a weakness of the current prevalent practice of using amortized inference that can potentially be improved by integrating more direct optimization elements. △ Less

Submitted 25 October, 2022; originally announced October 2022.

Around definable types in $p$-adically closed fields

Authors: Pablo Andujar Guerrero, Will Johnson

Abstract: We prove some technical results on definable types in $p$-adically closed fields, with consequences for definable groups and definable topological spaces. First, the code of a definable $n$-type (in the field sort) can be taken to be a real tuple (in the field sort) rather than an imaginary tuple (in the geometric sorts). Second, any definable type in the real or imaginary sorts is generated by a… ▽ More We prove some technical results on definable types in $p$-adically closed fields, with consequences for definable groups and definable topological spaces. First, the code of a definable $n$-type (in the field sort) can be taken to be a real tuple (in the field sort) rather than an imaginary tuple (in the geometric sorts). Second, any definable type in the real or imaginary sorts is generated by a countable union of chains parameterized by the value group. Third, if $X$ is an interpretable set, then the space of global definable types on $X$ is strictly pro-interpretable, building off work of Cubides Kovacsics, Hils, and Ye. Fourth, global definable types can be lifted (in a non-canonical way) along interpretable surjections. Fifth, if $G$ is a definable group with definable f-generics ($dfg$), and $G$ acts on a definable set $X$, then the quotient space $X/G$ is definable, not just interpretable. This explains some phenomena observed by Pillay and Yao. Lastly, we show that interpretable topological spaces satisfy analogues of first-countability and curve selection. Using this, we show that all reasonable notions of definable compactness agree on interpretable topological spaces, and that definable compactness is definable in families. △ Less

Submitted 17 July, 2024; v1 submitted 11 August, 2022; originally announced August 2022.

Comments: 39 pages; fixed two minor typos

MSC Class: 03C60 (Primary); 12L12 (Secondary)

Journal ref: Annals of Pure and Applied Logic 175(10): 103484 (December 2024)

In-orbit Commissioning of the Near-Infrared Spectrograph on the James Webb Space Telescope

Authors: Torsten Böker, Yasin Abul-Huda, Martin Altenburg, Catarina Alves de Oliveira, Katie Bechtold, Tracy Beck, Stephan M. Birkmann, Nina Bonaventura, Ralf Ehrenwinkler, Pierre Ferruit, David E. Franz, Giovanna Giardino, Peter Jakobsen, Peter Jensen, Delphine Jollet, Diane Karakla, Hermann Karl, Charles Keyes, Nimisha Kumari, Matthew Lander, Marcos López-Caniego, Nora Lützgendorf, Elena Manjavacas, Anthony Marston, Marc Maschmann , et al. (19 additional authors not shown)

Abstract: The Near-Infrared Spectrograph (NIRSpec) is one of the four focal plane instruments on the James Webb Space Telescope which was launched on Dec. 25, 2021. We present an overview of the as-run NIRSpec commissioning campaign, with particular emphasis on the sequence of activities that led to the verification of all hardware components of NIRSpec. We also discuss the mechanical, thermal, and operatio… ▽ More The Near-Infrared Spectrograph (NIRSpec) is one of the four focal plane instruments on the James Webb Space Telescope which was launched on Dec. 25, 2021. We present an overview of the as-run NIRSpec commissioning campaign, with particular emphasis on the sequence of activities that led to the verification of all hardware components of NIRSpec. We also discuss the mechanical, thermal, and operational performance of NIRSpec, as well as the readiness of all NIRSpec observing modes for use in the upcoming JWST science program. △ Less

Submitted 4 August, 2022; originally announced August 2022.

Comments: 10 pages, to appear in Proceedings of SPIE conference "Space Telescopes and Instrumentation 2022: Optical, Infrared, and Millimeter Wave"

NeuForm: Adaptive Overfitting for Neural Shape Editing

Authors: Connor Z. Lin, Niloy J. Mitra, Gordon Wetzstein, Leonidas Guibas, Paul Guerrero

Abstract: Neural representations are popular for representing shapes, as they can be learned form sensor data and used for data cleanup, model completion, shape editing, and shape synthesis. Current neural representations can be categorized as either overfitting to a single object instance, or representing a collection of objects. However, neither allows accurate editing of neural scene representations: on… ▽ More Neural representations are popular for representing shapes, as they can be learned form sensor data and used for data cleanup, model completion, shape editing, and shape synthesis. Current neural representations can be categorized as either overfitting to a single object instance, or representing a collection of objects. However, neither allows accurate editing of neural scene representations: on the one hand, methods that overfit objects achieve highly accurate reconstructions, but do not generalize to unseen object configurations and thus cannot support editing; on the other hand, methods that represent a family of objects with variations do generalize but produce only approximate reconstructions. We propose NEUFORM to combine the advantages of both overfitted and generalizable representations by adaptively using the one most appropriate for each shape region: the overfitted representation where reliable data is available, and the generalizable representation everywhere else. We achieve this with a carefully designed architecture and an approach that blends the network weights of the two representations, avoiding seams and other artifacts. We demonstrate edits that successfully reconfigure parts of human-designed shapes, such as chairs, tables, and lamps, while preserving semantic integrity and the accuracy of an overfitted shape representation. We compare with two state-of-the-art competitors and demonstrate clear improvements in terms of plausibility and fidelity of the resultant edits. △ Less

Submitted 18 July, 2022; originally announced July 2022.

Node Graph Optimization Using Differentiable Proxies

Authors: Yiwei Hu, Paul Guerrero, Miloš Hašan, Holly Rushmeier, Valentin Deschaintre

Abstract: Graph-based procedural materials are ubiquitous in content production industries. Procedural models allow the creation of photorealistic materials with parametric control for flexible editing of appearance. However, designing a specific material is a time-consuming process in terms of building a model and fine-tuning parameters. Previous work [Hu et al. 2022; Shi et al. 2020] introduced material g… ▽ More Graph-based procedural materials are ubiquitous in content production industries. Procedural models allow the creation of photorealistic materials with parametric control for flexible editing of appearance. However, designing a specific material is a time-consuming process in terms of building a model and fine-tuning parameters. Previous work [Hu et al. 2022; Shi et al. 2020] introduced material graph optimization frameworks for matching target material samples. However, these previous methods were limited to optimizing differentiable functions in the graphs. In this paper, we propose a fully differentiable framework which enables end-to-end gradient based optimization of material graphs, even if some functions of the graph are non-differentiable. We leverage the Differentiable Proxy, a differentiable approximator of a non-differentiable black-box function. We use our framework to match structure and appearance of an output material to a target material, through a multi-stage differentiable optimization. Differentiable Proxies offer a more general optimization solution to material appearance matching than previous work. △ Less

Submitted 15 July, 2022; originally announced July 2022.

Journal ref: SIGGRAPH '22 Conference Proceedings, Vancouver, BC, Canada, 2022

MatFormer: A Generative Model for Procedural Materials

Authors: Paul Guerrero, Miloš Hašan, Kalyan Sunkavalli, Radomír Měch, Tamy Boubekeur, Niloy J. Mitra

Abstract: Procedural material graphs are a compact, parameteric, and resolution-independent representation that are a popular choice for material authoring. However, designing procedural materials requires significant expertise and publicly accessible libraries contain only a few thousand such graphs. We present MatFormer, a generative model that can produce a diverse set of high-quality procedural material… ▽ More Procedural material graphs are a compact, parameteric, and resolution-independent representation that are a popular choice for material authoring. However, designing procedural materials requires significant expertise and publicly accessible libraries contain only a few thousand such graphs. We present MatFormer, a generative model that can produce a diverse set of high-quality procedural materials with complex spatial patterns and appearance. While procedural materials can be modeled as directed (operation) graphs, they contain arbitrary numbers of heterogeneous nodes with unstructured, often long-range node connections, and functional constraints on node parameters and connections. MatFormer addresses these challenges with a multi-stage transformer-based model that sequentially generates nodes, node parameters, and edges, while ensuring the semantic validity of the graph. In addition to generation, MatFormer can be used for the auto-completion and exploration of partial material graphs. We qualitatively and quantitatively demonstrate that our method outperforms alternative approaches, in both generated graph and material quality. △ Less

Submitted 15 August, 2022; v1 submitted 3 July, 2022; originally announced July 2022.

Journal ref: ACM Transactions on Graphics, Volume 41, Issue 4 (Proceedings of Siggraph 2022)

Controlling Material Appearance by Examples

Authors: Yiwei Hu, Miloš Hašan, Paul Guerrero, Holly Rushmeier, Valentin Deschaintre

Abstract: Despite the ubiquitousness of materials maps in modern rendering pipelines, their editing and control remains a challenge. In this paper, we present an example-based material control method to augment input material maps based on user-provided material photos. We train a tileable version of MaterialGAN and leverage its material prior to guide the appearance transfer, optimizing its latent space us… ▽ More Despite the ubiquitousness of materials maps in modern rendering pipelines, their editing and control remains a challenge. In this paper, we present an example-based material control method to augment input material maps based on user-provided material photos. We train a tileable version of MaterialGAN and leverage its material prior to guide the appearance transfer, optimizing its latent space using differentiable rendering. Our method transfers the micro and meso-structure textures of user provided target(s) photographs, while preserving the structure of the input and quality of the input material. We show our methods can control existing material maps, increasing realism or generating new, visually appealing materials. △ Less

Submitted 29 June, 2022; originally announced June 2022.

Journal ref: Computer Graphics Forum (Proc. of Eurographics Symposium on Rendering 2022), vol. 41, no. 4, 2022

TileGen: Tileable, Controllable Material Generation and Capture

Authors: Xilong Zhou, Miloš Hašan, Valentin Deschaintre, Paul Guerrero, Kalyan Sunkavalli, Nima Kalantari

Abstract: Recent methods (e.g. MaterialGAN) have used unconditional GANs to generate per-pixel material maps, or as a prior to reconstruct materials from input photographs. These models can generate varied random material appearance, but do not have any mechanism to constrain the generated material to a specific category or to control the coarse structure of the generated material, such as the exact brick l… ▽ More Recent methods (e.g. MaterialGAN) have used unconditional GANs to generate per-pixel material maps, or as a prior to reconstruct materials from input photographs. These models can generate varied random material appearance, but do not have any mechanism to constrain the generated material to a specific category or to control the coarse structure of the generated material, such as the exact brick layout on a brick wall. Furthermore, materials reconstructed from a single input photo commonly have artifacts and are generally not tileable, which limits their use in practical content creation pipelines. We propose TileGen, a generative model for SVBRDFs that is specific to a material category, always tileable, and optionally conditional on a provided input structure pattern. TileGen is a variant of StyleGAN whose architecture is modified to always produce tileable (periodic) material maps. In addition to the standard "style" latent code, TileGen can optionally take a condition image, giving a user direct control over the dominant spatial (and optionally color) features of the material. For example, in brick materials, the user can specify a brick layout and the brick color, or in leather materials, the locations of wrinkles and folds. Our inverse rendering approach can find a material perceptually matching a single target photograph by optimization. This reconstruction can also be conditional on a user-provided pattern. The resulting materials are tileable, can be larger than the target image, and are editable by varying the condition. △ Less

Submitted 19 June, 2022; v1 submitted 11 June, 2022; originally announced June 2022.

Comments: 18 pages, 19 figures

COFS: Controllable Furniture layout Synthesis

Authors: Wamiq Reyaz Para, Paul Guerrero, Niloy Mitra, Peter Wonka

Abstract: Scalable generation of furniture layouts is essential for many applications in virtual reality, augmented reality, game development and synthetic data generation. Many existing methods tackle this problem as a sequence generation problem which imposes a specific ordering on the elements of the layout making such methods impractical for interactive editing or scene completion. Additionally, most me… ▽ More Scalable generation of furniture layouts is essential for many applications in virtual reality, augmented reality, game development and synthetic data generation. Many existing methods tackle this problem as a sequence generation problem which imposes a specific ordering on the elements of the layout making such methods impractical for interactive editing or scene completion. Additionally, most methods focus on generating layouts unconditionally and offer minimal control over the generated layouts. We propose COFS, an architecture based on standard transformer architecture blocks from language modeling. The proposed model is invariant to object order by design, removing the unnatural requirement of specifying an object generation order. Furthermore, the model allows for user interaction at multiple levels enabling fine grained control over the generation process. Our model consistently outperforms other methods which we verify by performing quantitative evaluations. Our method is also faster to train and sample from, compared to existing methods. △ Less

Submitted 29 May, 2022; originally announced May 2022.

Comments: Initial Version

Definable $(ω, 2)$-theorem for families with VC-codensity less than $2$

Authors: Pablo Andújar-Guerrero

Abstract: Let $\mathcal{S}$ be a family of sets with VC-codensity less than $2$. We prove that, if $\mathcal{S}$ has the $(ω, 2)$-property (for any infinitely many sets in $\mathcal{S}$, at least $2$ among them intersect), then $\mathcal{S}$ can be partitioned into finitely many subfamilies, each with the finite intersection property. If $\mathcal{S}$ is definable in some first-order structure, then these s… ▽ More Let $\mathcal{S}$ be a family of sets with VC-codensity less than $2$. We prove that, if $\mathcal{S}$ has the $(ω, 2)$-property (for any infinitely many sets in $\mathcal{S}$, at least $2$ among them intersect), then $\mathcal{S}$ can be partitioned into finitely many subfamilies, each with the finite intersection property. If $\mathcal{S}$ is definable in some first-order structure, then these subfamilies can be chosen definable too. This is a strengthening of the case $q=2$ of the definable $(p,q)$- conjecture in model theory and of the Alon-Kleitman-Matoušek $(p,q)$-theorem in combinatorics. △ Less

Submitted 20 May, 2023; v1 submitted 26 May, 2022; originally announced May 2022.

Comments: Proof streamlined after referee comments. I have two last names: Andújar Guerrero. ArXiV is currently unable to represent this and considers Andújar a middle name (when creating the BibTeX citation etc)

MSC Class: 03C45; 52A35

Neural Convolutional Surfaces

Authors: Luca Morreale, Noam Aigerman, Paul Guerrero, Vladimir G. Kim, Niloy J. Mitra

Abstract: This work is concerned with a representation of shapes that disentangles fine, local and possibly repeating geometry, from global, coarse structures. Achieving such disentanglement leads to two unrelated advantages: i) a significant compression in the number of parameters required to represent a given geometry; ii) the ability to manipulate either global geometry, or local details, without harming… ▽ More This work is concerned with a representation of shapes that disentangles fine, local and possibly repeating geometry, from global, coarse structures. Achieving such disentanglement leads to two unrelated advantages: i) a significant compression in the number of parameters required to represent a given geometry; ii) the ability to manipulate either global geometry, or local details, without harming the other. At the core of our approach lies a novel pipeline and neural architecture, which are optimized to represent one specific atlas, representing one 3D surface. Our pipeline and architecture are designed so that disentanglement of global geometry from local details is accomplished through optimization, in a completely unsupervised manner. We show that this approach achieves better neural shape compression than the state of the art, as well as enabling manipulation and transfer of shape details. Project page at http://geometry.cs.ucl.ac.uk/projects/2022/cnnmaps/ . △ Less

Submitted 5 April, 2022; originally announced April 2022.

Journal ref: CVPR 2022

LayoutEnhancer: Generating Good Indoor Layouts from Imperfect Data

Authors: Kurt Leimer, Paul Guerrero, Tomer Weiss, Przemyslaw Musialski

Abstract: We address the problem of indoor layout synthesis, which is a topic of continuing research interest in computer graphics. The newest works made significant progress using data-driven generative methods; however, these approaches rely on suitable datasets. In practice, desirable layout properties may not exist in a dataset, for instance, specific expert knowledge can be missing in the data. We prop… ▽ More We address the problem of indoor layout synthesis, which is a topic of continuing research interest in computer graphics. The newest works made significant progress using data-driven generative methods; however, these approaches rely on suitable datasets. In practice, desirable layout properties may not exist in a dataset, for instance, specific expert knowledge can be missing in the data. We propose a method that combines expert knowledge, for example, knowledge about ergonomics, with a data-driven generator based on the popular Transformer architecture. The knowledge is given as differentiable scalar functions, which can be used both as weights or as additional terms in the loss function. Using this knowledge, the synthesized layouts can be biased to exhibit desirable properties, even if these properties are not present in the dataset. Our approach can also alleviate problems of lack of data and imperfections in the data. Our work aims to improve generative machine learning for modeling and provide novel tools for designers and amateurs for the problem of interior layout creation. △ Less

Submitted 5 October, 2022; v1 submitted 31 January, 2022; originally announced February 2022.

Comments: preprint of ACM SIGGRAPH Asia 2022 Conference Paper, 14 pages including appendix and supplementary figures, 16 figures

ACM Class: I.3.6; I.2.1; I.4.9

The Shape Part Slot Machine: Contact-based Reasoning for Generating 3D Shapes from Parts

Authors: Kai Wang, Paul Guerrero, Vladimir Kim, Siddhartha Chaudhuri, Minhyuk Sung, Daniel Ritchie

Abstract: We present the Shape Part Slot Machine, a new method for assembling novel 3D shapes from existing parts by performing contact-based reasoning. Our method represents each shape as a graph of ``slots,'' where each slot is a region of contact between two shape parts. Based on this representation, we design a graph-neural-network-based model for generating new slot graphs and retrieving compatible par… ▽ More We present the Shape Part Slot Machine, a new method for assembling novel 3D shapes from existing parts by performing contact-based reasoning. Our method represents each shape as a graph of ``slots,'' where each slot is a region of contact between two shape parts. Based on this representation, we design a graph-neural-network-based model for generating new slot graphs and retrieving compatible parts, as well as a gradient-descent-based optimization scheme for assembling the retrieved parts into a complete shape that respects the generated slot graph. This approach does not require any semantic part labels; interestingly, it also does not require complete part geometries -- reasoning about the slots proves sufficient to generate novel, high-quality 3D shapes. We demonstrate that our method generates shapes that outperform existing modeling-by-assembly approaches regarding quality, diversity, and structural complexity. △ Less

Submitted 21 July, 2022; v1 submitted 1 December, 2021; originally announced December 2021.

Comments: European Conference on Computer Vision (ECCV) 2022

Types, transversals and definable compactness in o-minimal structures

Authors: Pablo Andújar Guerrero

Abstract: Through careful analysis of types inspired by [AGTW21] we characterize a notion of definable compactness for definable topologies in general o-minimal structures, generalizing results from [PP07] about closed and bounded definable sets in o-minimal expansions of ordered groups. Along the way we prove a parameter version for o-minimal theories of the connection between dividing and definable types… ▽ More Through careful analysis of types inspired by [AGTW21] we characterize a notion of definable compactness for definable topologies in general o-minimal structures, generalizing results from [PP07] about closed and bounded definable sets in o-minimal expansions of ordered groups. Along the way we prove a parameter version for o-minimal theories of the connection between dividing and definable types known in the more general dp-minimal context [SS14], through an elementary proof that avoids the use of existing forking and VC literature. In particular we show that, if an $A$-definable family of sets has the $(p,q)$-property, for some $p\geq q$ with $q$ large enough, then the family admits a partition into finitely many subfamilies, each of which extends to an $A$-definable type. △ Less

Submitted 6 November, 2021; originally announced November 2021.

MSC Class: 03C64 (Primary); 54A05; 54D30 (Secondary)

Differentiable Surface Triangulation

Authors: Marie-Julie Rakotosaona, Noam Aigerman, Niloy Mitra, Maks Ovsjanikov, Paul Guerrero

Abstract: Triangle meshes remain the most popular data representation for surface geometry. This ubiquitous representation is essentially a hybrid one that decouples continuous vertex locations from the discrete topological triangulation. Unfortunately, the combinatorial nature of the triangulation prevents taking derivatives over the space of possible meshings of any given surface. As a result, to date, me… ▽ More Triangle meshes remain the most popular data representation for surface geometry. This ubiquitous representation is essentially a hybrid one that decouples continuous vertex locations from the discrete topological triangulation. Unfortunately, the combinatorial nature of the triangulation prevents taking derivatives over the space of possible meshings of any given surface. As a result, to date, mesh processing and optimization techniques have been unable to truly take advantage of modular gradient descent components of modern optimization frameworks. In this work, we present a differentiable surface triangulation that enables optimization for any per-vertex or per-face differentiable objective function over the space of underlying surface triangulations. Our method builds on the result that any 2D triangulation can be achieved by a suitably perturbed weighted Delaunay triangulation. We translate this result into a computational algorithm by proposing a soft relaxation of the classical weighted Delaunay triangulation and optimizing over vertex weights and vertex locations. We extend the algorithm to 3D by decomposing shapes into developable sets and differentiably meshing each set with suitable boundary constraints. We demonstrate the efficacy of our method on various planar and surface meshes on a range of difficult-to-optimize objective functions. Our code can be found online: https://github.com/mrakotosaon/diff-surface-triangulation. △ Less

Submitted 22 September, 2021; originally announced September 2021.

SketchGen: Generating Constrained CAD Sketches

Authors: Wamiq Reyaz Para, Shariq Farooq Bhat, Paul Guerrero, Tom Kelly, Niloy Mitra, Leonidas Guibas, Peter Wonka

Abstract: Computer-aided design (CAD) is the most widely used modeling approach for technical design. The typical starting point in these designs is 2D sketches which can later be extruded and combined to obtain complex three-dimensional assemblies. Such sketches are typically composed of parametric primitives, such as points, lines, and circular arcs, augmented with geometric constraints linking the primit… ▽ More Computer-aided design (CAD) is the most widely used modeling approach for technical design. The typical starting point in these designs is 2D sketches which can later be extruded and combined to obtain complex three-dimensional assemblies. Such sketches are typically composed of parametric primitives, such as points, lines, and circular arcs, augmented with geometric constraints linking the primitives, such as coincidence, parallelism, or orthogonality. Sketches can be represented as graphs, with the primitives as nodes and the constraints as edges. Training a model to automatically generate CAD sketches can enable several novel workflows, but is challenging due to the complexity of the graphs and the heterogeneity of the primitives and constraints. In particular, each type of primitive and constraint may require a record of different size and parameter types. We propose SketchGen as a generative model based on a transformer architecture to address the heterogeneity problem by carefully designing a sequential language for the primitives and constraints that allows distinguishing between different primitive or constraint types and their parameters, while encouraging our model to re-use information across related parameters, encoding shared structure. A particular highlight of our work is the ability to produce primitives linked via constraints that enables the final output to be further regularized via a constraint solver. We evaluate our model by demonstrating constraint prediction for given sets of primitives and full sketch generation from scratch, showing that our approach significantly out performs the state-of-the-art in CAD sketch generation. △ Less

Submitted 4 June, 2021; originally announced June 2021.

Comments: 21 pages, 12 figures, 8 tables

ShapeMOD: Macro Operation Discovery for 3D Shape Programs

Authors: R. Kenny Jones, David Charatan, Paul Guerrero, Niloy J. Mitra, Daniel Ritchie

Abstract: A popular way to create detailed yet easily controllable 3D shapes is via procedural modeling, i.e. generating geometry using programs. Such programs consist of a series of instructions along with their associated parameter values. To fully realize the benefits of this representation, a shape program should be compact and only expose degrees of freedom that allow for meaningful manipulation of out… ▽ More A popular way to create detailed yet easily controllable 3D shapes is via procedural modeling, i.e. generating geometry using programs. Such programs consist of a series of instructions along with their associated parameter values. To fully realize the benefits of this representation, a shape program should be compact and only expose degrees of freedom that allow for meaningful manipulation of output geometry. One way to achieve this goal is to design higher-level macro operators that, when executed, expand into a series of commands from the base shape modeling language. However, manually authoring such macros, much like shape programs themselves, is difficult and largely restricted to domain experts. In this paper, we present ShapeMOD, an algorithm for automatically discovering macros that are useful across large datasets of 3D shape programs. ShapeMOD operates on shape programs expressed in an imperative, statement-based language. It is designed to discover macros that make programs more compact by minimizing the number of function calls and free parameters required to represent an input shape collection. We run ShapeMOD on multiple collections of programs expressed in a domain-specific language for 3D shape structures. We show that it automatically discovers a concise set of macros that abstract out common structural and parametric patterns that generalize over large shape collections. We also demonstrate that the macros found by ShapeMOD improve performance on downstream tasks including shape generative modeling and inferring programs from point clouds. Finally, we conduct a user study that indicates that ShapeMOD's discovered macros make interactive shape editing more efficient. △ Less

Submitted 22 March, 2022; v1 submitted 13 April, 2021; originally announced April 2021.

Comments: SIGGRAPH 2021. Project Page: https://rkjones4.github.io/shapeMOD.html

Learning Delaunay Surface Elements for Mesh Reconstruction

Authors: Marie-Julie Rakotosaona, Paul Guerrero, Noam Aigerman, Niloy Mitra, Maks Ovsjanikov

Abstract: We present a method for reconstructing triangle meshes from point clouds. Existing learning-based methods for mesh reconstruction mostly generate triangles individually, making it hard to create manifold meshes. We leverage the properties of 2D Delaunay triangulations to construct a mesh from manifold surface elements. Our method first estimates local geodesic neighborhoods around each point. We t… ▽ More We present a method for reconstructing triangle meshes from point clouds. Existing learning-based methods for mesh reconstruction mostly generate triangles individually, making it hard to create manifold meshes. We leverage the properties of 2D Delaunay triangulations to construct a mesh from manifold surface elements. Our method first estimates local geodesic neighborhoods around each point. We then perform a 2D projection of these neighborhoods using a learned logarithmic map. A Delaunay triangulation in this 2D domain is guaranteed to produce a manifold patch, which we call a Delaunay surface element. We synchronize the local 2D projections of neighboring elements to maximize the manifoldness of the reconstructed mesh. Our results show that we achieve better overall manifoldness of our reconstructed meshes than current methods to reconstruct meshes with arbitrary topology. Our code, data and pretrained models can be found online: https://github.com/mrakotosaon/dse-meshing △ Less

Submitted 6 May, 2021; v1 submitted 2 December, 2020; originally announced December 2020.

Journal ref: CVPR 2021

Generative Layout Modeling using Constraint Graphs

Authors: Wamiq Para, Paul Guerrero, Tom Kelly, Leonidas Guibas, Peter Wonka

Abstract: We propose a new generative model for layout generation. We generate layouts in three steps. First, we generate the layout elements as nodes in a layout graph. Second, we compute constraints between layout elements as edges in the layout graph. Third, we solve for the final layout using constrained optimization. For the first two steps, we build on recent transformer architectures. The layout opti… ▽ More We propose a new generative model for layout generation. We generate layouts in three steps. First, we generate the layout elements as nodes in a layout graph. Second, we compute constraints between layout elements as edges in the layout graph. Third, we solve for the final layout using constrained optimization. For the first two steps, we build on recent transformer architectures. The layout optimization implements the constraints efficiently. We show three practical contributions compared to the state of the art: our work requires no user input, produces higher quality layouts, and enables many novel capabilities for conditional layout generation. △ Less

Submitted 26 November, 2020; originally announced November 2020.

Discovering Pattern Structure Using Differentiable Compositing

Authors: Pradyumna Reddy, Paul Guerrero, Matt Fisher, Wilmot Li, Miloy J. Mitra

Abstract: Patterns, which are collections of elements arranged in regular or near-regular arrangements, are an important graphic art form and widely used due to their elegant simplicity and aesthetic appeal. When a pattern is encoded as a flat image without the underlying structure, manually editing the pattern is tedious and challenging as one has to both preserve the individual element shapes and their or… ▽ More Patterns, which are collections of elements arranged in regular or near-regular arrangements, are an important graphic art form and widely used due to their elegant simplicity and aesthetic appeal. When a pattern is encoded as a flat image without the underlying structure, manually editing the pattern is tedious and challenging as one has to both preserve the individual element shapes and their original relative arrangements. State-of-the-art deep learning frameworks that operate at the pixel level are unsuitable for manipulating such patterns. Specifically, these methods can easily disturb the shapes of the individual elements or their arrangement, and thus fail to preserve the latent structures of the input patterns. We present a novel differentiable compositing operator using pattern elements and use it to discover structures, in the form of a layered representation of graphical objects, directly from raw pattern images. This operator allows us to adapt current deep learning based image methods to effectively handle patterns. We evaluate our method on a range of patterns and demonstrate superiority in the context of pattern manipulations when compared against state-of-the-art △ Less

Submitted 17 October, 2020; originally announced October 2020.

ShapeAssembly: Learning to Generate Programs for 3D Shape Structure Synthesis

Authors: R. Kenny Jones, Theresa Barton, Xianghao Xu, Kai Wang, Ellen Jiang, Paul Guerrero, Niloy J. Mitra, Daniel Ritchie

Abstract: Manually authoring 3D shapes is difficult and time consuming; generative models of 3D shapes offer compelling alternatives. Procedural representations are one such possibility: they offer high-quality and editable results but are difficult to author and often produce outputs with limited diversity. On the other extreme are deep generative models: given enough data, they can learn to generate any c… ▽ More Manually authoring 3D shapes is difficult and time consuming; generative models of 3D shapes offer compelling alternatives. Procedural representations are one such possibility: they offer high-quality and editable results but are difficult to author and often produce outputs with limited diversity. On the other extreme are deep generative models: given enough data, they can learn to generate any class of shape but their outputs have artifacts and the representation is not editable. In this paper, we take a step towards achieving the best of both worlds for novel 3D shape synthesis. We propose ShapeAssembly, a domain-specific "assembly-language" for 3D shape structures. ShapeAssembly programs construct shapes by declaring cuboid part proxies and attaching them to one another, in a hierarchical and symmetrical fashion. Its functions are parameterized with free variables, so that one program structure is able to capture a family of related shapes. We show how to extract ShapeAssembly programs from existing shape structures in the PartNet dataset. Then we train a deep generative model, a hierarchical sequence VAE, that learns to write novel ShapeAssembly programs. The program captures the subset of variability that is interpretable and editable. The deep model captures correlations across shape collections that are hard to express procedurally. We evaluate our approach by comparing shapes output by our generated programs to those from other recent shape structure synthesis models. We find that our generated shapes are more plausible and physically-valid than those of other methods. Additionally, we assess the latent spaces of these models, and find that ours is better structured and produces smoother interpolations. As an application, we use our generative model and differentiable program interpreter to infer and fit shape programs to unstructured geometry, such as point clouds. △ Less

Submitted 16 September, 2020; originally announced September 2020.

Comments: Accepted to Siggraph Asia 2020; project page: https://rkjones4.github.io/shapeAssembly.html

Pix2Surf: Learning Parametric 3D Surface Models of Objects from Images

Authors: Jiahui Lei, Srinath Sridhar, Paul Guerrero, Minhyuk Sung, Niloy Mitra, Leonidas J. Guibas

Abstract: We investigate the problem of learning to generate 3D parametric surface representations for novel object instances, as seen from one or more views. Previous work on learning shape reconstruction from multiple views uses discrete representations such as point clouds or voxels, while continuous surface generation approaches lack multi-view consistency. We address these issues by designing neural ne… ▽ More We investigate the problem of learning to generate 3D parametric surface representations for novel object instances, as seen from one or more views. Previous work on learning shape reconstruction from multiple views uses discrete representations such as point clouds or voxels, while continuous surface generation approaches lack multi-view consistency. We address these issues by designing neural networks capable of generating high-quality parametric 3D surfaces which are also consistent between views. Furthermore, the generated 3D surfaces preserve accurate image pixel to 3D surface point correspondences, allowing us to lift texture information to reconstruct shapes with rich geometry and appearance. Our method is supervised and trained on a public dataset of shapes from common object categories. Quantitative results indicate that our method significantly outperforms previous work, while qualitative results demonstrate the high quality of our reconstructions. △ Less

Submitted 18 August, 2020; originally announced August 2020.

Comments: ECCV 2020

Points2Surf: Learning Implicit Surfaces from Point Cloud Patches

Authors: Philipp Erler, Paul Guerrero, Stefan Ohrhallinger, Michael Wimmer, Niloy J. Mitra

Abstract: A key step in any scanning-based asset creation workflow is to convert unordered point clouds to a surface. Classical methods (e.g., Poisson reconstruction) start to degrade in the presence of noisy and partial scans. Hence, deep learning based methods have recently been proposed to produce complete surfaces, even from partial scans. However, such data-driven methods struggle to generalize to new… ▽ More A key step in any scanning-based asset creation workflow is to convert unordered point clouds to a surface. Classical methods (e.g., Poisson reconstruction) start to degrade in the presence of noisy and partial scans. Hence, deep learning based methods have recently been proposed to produce complete surfaces, even from partial scans. However, such data-driven methods struggle to generalize to new shapes with large geometric and topological variations. We present Points2Surf, a novel patch-based learning framework that produces accurate surfaces directly from raw scans without normals. Learning a prior over a combination of detailed local patches and coarse global information improves generalization performance and reconstruction accuracy. Our extensive comparison on both synthetic and real data demonstrates a clear advantage of our method over state-of-the-art alternatives on previously unseen classes (on average, Points2Surf brings down reconstruction error by 30% over SPR and by 270%+ over deep learning based SotA methods) at the cost of longer computation times and a slight increase in small-scale topological noise in some cases. Our source code, pre-trained model, and dataset are available on: https://github.com/ErlerPhilipp/points2surf △ Less

Submitted 13 February, 2024; v1 submitted 20 July, 2020; originally announced July 2020.

Comments: To be published at ECCV 2020 Repository: https://github.com/ErlerPhilipp/points2surf

ACM Class: I.4.5

Journal ref: Computer Vision -- ECCV 2020, 108--124

StructEdit: Learning Structural Shape Variations

Authors: Kaichun Mo, Paul Guerrero, Li Yi, Hao Su, Peter Wonka, Niloy Mitra, Leonidas J. Guibas

Abstract: Learning to encode differences in the geometry and (topological) structure of the shapes of ordinary objects is key to generating semantically plausible variations of a given shape, transferring edits from one shape to another, and many other applications in 3D content creation. The common approach of encoding shapes as points in a high-dimensional latent feature space suggests treating shape diff… ▽ More Learning to encode differences in the geometry and (topological) structure of the shapes of ordinary objects is key to generating semantically plausible variations of a given shape, transferring edits from one shape to another, and many other applications in 3D content creation. The common approach of encoding shapes as points in a high-dimensional latent feature space suggests treating shape differences as vectors in that space. Instead, we treat shape differences as primary objects in their own right and propose to encode them in their own latent space. In a setting where the shapes themselves are encoded in terms of fine-grained part hierarchies, we demonstrate that a separate encoding of shape deltas or differences provides a principled way to deal with inhomogeneities in the shape space due to different combinatorial part structures, while also allowing for compactness in the representation, as well as edit abstraction and transfer. Our approach is based on a conditional variational autoencoder for encoding and decoding shape deltas, conditioned on a source shape. We demonstrate the effectiveness and robustness of our approach in multiple shape modification and generation tasks, and provide comparison and ablation studies on the PartNet dataset, one of the largest publicly available 3D datasets. △ Less

Submitted 25 November, 2019; originally announced November 2019.

Directed sets and topological spaces definable in o-minimal structures

Authors: Pablo Andujar Guerrero, Margaret E. M. Thomas, Erik Walsberg

Abstract: We study directed sets definable in o-minimal structures, showing that in expansions of ordered fields these admit cofinal definable curves, as well as a suitable analogue in expansions of ordered groups, and furthermore that no analogue holds in full generality. We use the theory of tame pairs to extend the results in the field case to definable families of sets with the finite intersection prope… ▽ More We study directed sets definable in o-minimal structures, showing that in expansions of ordered fields these admit cofinal definable curves, as well as a suitable analogue in expansions of ordered groups, and furthermore that no analogue holds in full generality. We use the theory of tame pairs to extend the results in the field case to definable families of sets with the finite intersection property. We then apply our results to the study of definable topologies. We prove that all definable topological spaces display properties akin to first countability, and give several characterizations of a notion of definable compactness due to Peterzil and Steinhorn generalized to this setting. △ Less

Submitted 4 February, 2021; v1 submitted 14 November, 2019; originally announced November 2019.

Comments: Improved presentation and new Remark 33

MSC Class: 03C64 (Primary); 54A20; 54A05; 54D30 (Secondary)

StructureNet: Hierarchical Graph Networks for 3D Shape Generation

Authors: Kaichun Mo, Paul Guerrero, Li Yi, Hao Su, Peter Wonka, Niloy Mitra, Leonidas J. Guibas

Abstract: The ability to generate novel, diverse, and realistic 3D shapes along with associated part semantics and structure is central to many applications requiring high-quality 3D assets or large volumes of realistic training data. A key challenge towards this goal is how to accommodate diverse shape variations, including both continuous deformations of parts as well as structural or discrete alterations… ▽ More The ability to generate novel, diverse, and realistic 3D shapes along with associated part semantics and structure is central to many applications requiring high-quality 3D assets or large volumes of realistic training data. A key challenge towards this goal is how to accommodate diverse shape variations, including both continuous deformations of parts as well as structural or discrete alterations which add to, remove from, or modify the shape constituents and compositional structure. Such object structure can typically be organized into a hierarchy of constituent object parts and relationships, represented as a hierarchy of n-ary graphs. We introduce StructureNet, a hierarchical graph network which (i) can directly encode shapes represented as such n-ary graphs; (ii) can be robustly trained on large and complex shape families; and (iii) can be used to generate a great diversity of realistic structured shape geometries. Technically, we accomplish this by drawing inspiration from recent advances in graph neural networks to propose an order-invariant encoding of n-ary graphs, considering jointly both part geometry and inter-part relations during network training. We extensively evaluate the quality of the learned latent spaces for various shape families and show significant advantages over baseline and competing methods. The learned latent spaces enable several structure-aware geometry processing applications, including shape generation and interpolation, shape editing, or shape structure discovery directly from un-annotated images, point clouds, or partial scans. △ Less

Submitted 1 August, 2019; originally announced August 2019.

Comments: Conditionally Accepted to Siggraph Asia 2019

PointCleanNet: Learning to Denoise and Remove Outliers from Dense Point Clouds

Authors: Marie-Julie Rakotosaona, Vittorio La Barbera, Paul Guerrero, Niloy J. Mitra, Maks Ovsjanikov

Abstract: Point clouds obtained with 3D scanners or by image-based reconstruction techniques are often corrupted with significant amount of noise and outliers. Traditional methods for point cloud denoising largely rely on local surface fitting (e.g., jets or MLS surfaces), local or non-local averaging, or on statistical assumptions about the underlying noise model. In contrast, we develop a simple data-driv… ▽ More Point clouds obtained with 3D scanners or by image-based reconstruction techniques are often corrupted with significant amount of noise and outliers. Traditional methods for point cloud denoising largely rely on local surface fitting (e.g., jets or MLS surfaces), local or non-local averaging, or on statistical assumptions about the underlying noise model. In contrast, we develop a simple data-driven method for removing outliers and reducing noise in unordered point clouds. We base our approach on a deep learning architecture adapted from PCPNet, which was recently proposed for estimating local 3D shape properties in point clouds. Our method first classifies and discards outlier samples, and then estimates correction vectors that project noisy points onto the original clean surfaces. The approach is efficient and robust to varying amounts of noise and outliers, while being able to handle large densely-sampled point clouds. In our extensive evaluation, both on synthesic and real data, we show an increased robustness to strong noise levels compared to various state-of-the-art methods, enabling accurate surface reconstruction from extremely noisy real data obtained by range scans. Finally, the simplicity and universality of our approach makes it very easy to integrate in any existing geometry processing pipeline. △ Less

Submitted 28 June, 2019; v1 submitted 4 January, 2019; originally announced January 2019.

Journal ref: Computer Graphics Forum, 2020