-
Convergence of Manifold Filter-Combine Networks
Authors:
David R. Johnson,
Joyce Chew,
Siddharth Viswanath,
Edward De Brouwer,
Deanna Needell,
Smita Krishnaswamy,
Michael Perlmutter
Abstract:
In order to better understand manifold neural networks (MNNs), we introduce Manifold Filter-Combine Networks (MFCNs). The filter-combine framework parallels the popular aggregate-combine paradigm for graph neural networks (GNNs) and naturally suggests many interesting families of MNNs which can be interpreted as the manifold analog of various popular GNNs. We then propose a method for implementing…
▽ More
In order to better understand manifold neural networks (MNNs), we introduce Manifold Filter-Combine Networks (MFCNs). The filter-combine framework parallels the popular aggregate-combine paradigm for graph neural networks (GNNs) and naturally suggests many interesting families of MNNs which can be interpreted as the manifold analog of various popular GNNs. We then propose a method for implementing MFCNs on high-dimensional point clouds that relies on approximating the manifold by a sparse graph. We prove that our method is consistent in the sense that it converges to a continuum limit as the number of data points tends to infinity.
△ Less
Submitted 18 October, 2024;
originally announced October 2024.
-
Geometry-Aware Generative Autoencoders for Warped Riemannian Metric Learning and Generative Modeling on Data Manifolds
Authors:
Xingzhi Sun,
Danqi Liao,
Kincaid MacDonald,
Yanlei Zhang,
Chen Liu,
Guillaume Huguet,
Guy Wolf,
Ian Adelstein,
Tim G. J. Rudner,
Smita Krishnaswamy
Abstract:
Rapid growth of high-dimensional datasets in fields such as single-cell RNA sequencing and spatial genomics has led to unprecedented opportunities for scientific discovery, but it also presents unique computational and statistical challenges. Traditional methods struggle with geometry-aware data generation, interpolation along meaningful trajectories, and transporting populations via feasible path…
▽ More
Rapid growth of high-dimensional datasets in fields such as single-cell RNA sequencing and spatial genomics has led to unprecedented opportunities for scientific discovery, but it also presents unique computational and statistical challenges. Traditional methods struggle with geometry-aware data generation, interpolation along meaningful trajectories, and transporting populations via feasible paths. To address these issues, we introduce Geometry-Aware Generative Autoencoder (GAGA), a novel framework that combines extensible manifold learning with generative modeling. GAGA constructs a neural network embedding space that respects the intrinsic geometries discovered by manifold learning and learns a novel warped Riemannian metric on the data space. This warped metric is derived from both the points on the data manifold and negative samples off the manifold, allowing it to characterize a meaningful geometry across the entire latent space. Using this metric, GAGA can uniformly sample points on the manifold, generate points along geodesics, and interpolate between populations across the learned manifold using geodesic-guided flows. GAGA shows competitive performance in simulated and real-world datasets, including a 30% improvement over the state-of-the-art methods in single-cell population-level trajectory inference.
△ Less
Submitted 18 October, 2024; v1 submitted 16 October, 2024;
originally announced October 2024.
-
DiffKillR: Killing and Recreating Diffeomorphisms for Cell Annotation in Dense Microscopy Images
Authors:
Chen Liu,
Danqi Liao,
Alejandro Parada-Mayorga,
Alejandro Ribeiro,
Marcello DiStasio,
Smita Krishnaswamy
Abstract:
The proliferation of digital microscopy images, driven by advances in automated whole slide scanning, presents significant opportunities for biomedical research and clinical diagnostics. However, accurately annotating densely packed information in these images remains a major challenge. To address this, we introduce DiffKillR, a novel framework that reframes cell annotation as the combination of a…
▽ More
The proliferation of digital microscopy images, driven by advances in automated whole slide scanning, presents significant opportunities for biomedical research and clinical diagnostics. However, accurately annotating densely packed information in these images remains a major challenge. To address this, we introduce DiffKillR, a novel framework that reframes cell annotation as the combination of archetype matching and image registration tasks. DiffKillR employs two complementary neural networks: one that learns a diffeomorphism-invariant feature space for robust cell matching and another that computes the precise warping field between cells for annotation mapping. Using a small set of annotated archetypes, DiffKillR efficiently propagates annotations across large microscopy images, reducing the need for extensive manual labeling. More importantly, it is suitable for any type of pixel-level annotation. We will discuss the theoretical properties of DiffKillR and validate it on three microscopy tasks, demonstrating its advantages over existing supervised, semi-supervised, and unsupervised methods.
△ Less
Submitted 3 October, 2024;
originally announced October 2024.
-
Looking through the mind's eye via multimodal encoder-decoder networks
Authors:
Arman Afrasiyabi,
Erica Busch,
Rahul Singh,
Dhananjay Bhaskar,
Laurent Caplette,
Nicholas Turk-Browne,
Smita Krishnaswamy
Abstract:
In this work, we explore the decoding of mental imagery from subjects using their fMRI measurements. In order to achieve this decoding, we first created a mapping between a subject's fMRI signals elicited by the videos the subjects watched. This mapping associates the high dimensional fMRI activation states with visual imagery. Next, we prompted the subjects textually, primarily with emotion label…
▽ More
In this work, we explore the decoding of mental imagery from subjects using their fMRI measurements. In order to achieve this decoding, we first created a mapping between a subject's fMRI signals elicited by the videos the subjects watched. This mapping associates the high dimensional fMRI activation states with visual imagery. Next, we prompted the subjects textually, primarily with emotion labels which had no direct reference to visual objects. Then to decode visual imagery that may have been in a person's mind's eye, we align a latent representation of these fMRI measurements with a corresponding video-fMRI based on textual labels given to the videos themselves. This alignment has the effect of overlapping the video fMRI embedding with the text-prompted fMRI embedding, thus allowing us to use our fMRI-to-video mapping to decode. Additionally, we enhance an existing fMRI dataset, initially consisting of data from five subjects, by including recordings from three more subjects gathered by our team. We demonstrate the efficacy of our model on this augmented dataset both in accurately creating a mapping, as well as in plausibly decoding mental imagery.
△ Less
Submitted 27 September, 2024;
originally announced October 2024.
-
Latent Representation Learning for Multimodal Brain Activity Translation
Authors:
Arman Afrasiyabi,
Dhananjay Bhaskar,
Erica L. Busch,
Laurent Caplette,
Rahul Singh,
Guillaume Lajoie,
Nicholas B. Turk-Browne,
Smita Krishnaswamy
Abstract:
Neuroscience employs diverse neuroimaging techniques, each offering distinct insights into brain activity, from electrophysiological recordings such as EEG, which have high temporal resolution, to hemodynamic modalities such as fMRI, which have increased spatial precision. However, integrating these heterogeneous data sources remains a challenge, which limits a comprehensive understanding of brain…
▽ More
Neuroscience employs diverse neuroimaging techniques, each offering distinct insights into brain activity, from electrophysiological recordings such as EEG, which have high temporal resolution, to hemodynamic modalities such as fMRI, which have increased spatial precision. However, integrating these heterogeneous data sources remains a challenge, which limits a comprehensive understanding of brain function. We present the Spatiotemporal Alignment of Multimodal Brain Activity (SAMBA) framework, which bridges the spatial and temporal resolution gaps across modalities by learning a unified latent space free of modality-specific biases. SAMBA introduces a novel attention-based wavelet decomposition for spectral filtering of electrophysiological recordings, graph attention networks to model functional connectivity between functional brain units, and recurrent layers to capture temporal autocorrelations in brain signal. We show that the training of SAMBA, aside from achieving translation, also learns a rich representation of brain information processing. We showcase this classify external stimuli driving brain activity from the representation learned in hidden layers of SAMBA, paving the way for broad downstream applications in neuroscience research and clinical contexts.
△ Less
Submitted 27 September, 2024;
originally announced September 2024.
-
Hyperedge Representations with Hypergraph Wavelets: Applications to Spatial Transcriptomics
Authors:
Xingzhi Sun,
Charles Xu,
João F. Rocha,
Chen Liu,
Benjamin Hollander-Bodie,
Laney Goldman,
Marcello DiStasio,
Michael Perlmutter,
Smita Krishnaswamy
Abstract:
In many data-driven applications, higher-order relationships among multiple objects are essential in capturing complex interactions. Hypergraphs, which generalize graphs by allowing edges to connect any number of nodes, provide a flexible and powerful framework for modeling such higher-order relationships. In this work, we introduce hypergraph diffusion wavelets and describe their favorable spectr…
▽ More
In many data-driven applications, higher-order relationships among multiple objects are essential in capturing complex interactions. Hypergraphs, which generalize graphs by allowing edges to connect any number of nodes, provide a flexible and powerful framework for modeling such higher-order relationships. In this work, we introduce hypergraph diffusion wavelets and describe their favorable spectral and spatial properties. We demonstrate their utility for biomedical discovery in spatially resolved transcriptomics by applying the method to represent disease-relevant cellular niches for Alzheimer's disease.
△ Less
Submitted 14 September, 2024;
originally announced September 2024.
-
Surface-Functionalization of Oleate-Capped Nano-Emitters for Stable Dispersion in 3D-Printable Polymers
Authors:
Akhilesh Kumar Pathak,
Sachin Prashant Kulkarni,
Rachel R. Chan,
Chad A. Mirkin,
Koray Aydin,
Sridhar Krishnaswamy
Abstract:
Two-photon polymerization (2PP) 3D printing is a well-known technique for fabricating passive micro/nanoscale structures, such as microlenses and inversely designed polarization splitters. The integration of light emitting nanoparticle (NP) dopants, such as quantum dots (QDs) and rare-earth doped nanoparticles (RENPs), into a polymer resist would enable 3D printing of active polymer micro-photonic…
▽ More
Two-photon polymerization (2PP) 3D printing is a well-known technique for fabricating passive micro/nanoscale structures, such as microlenses and inversely designed polarization splitters. The integration of light emitting nanoparticle (NP) dopants, such as quantum dots (QDs) and rare-earth doped nanoparticles (RENPs), into a polymer resist would enable 3D printing of active polymer micro-photonic devices, including sensors, lasers, and solid-state displays. Many NPs are stabilized with oleic acid ligands to prevent degradation, but oleate-capped NPs (oc-NPs) tend to agglomerate in nonpolar media despite the hydrophobicity of the ligand. This results in an uneven distribution of NPs in polymers and increased optical extinction properties. In this work, we propose a general approach for dispersing various oc-NPs in commercial 3D printable polymers. We achieve controlled growth of small carbon chains around the oc-NPs by functionalizing the NPs with methyl-methacrylate monomers. The proposed approach is validated on RENPs (~65 nm) and CdSe/ZnS quantum dots (~12 nm) using different commercial polymer resists (IP-Dip and IP-Visio). Dispersions of functionalized NPs (f-NPs) have improved NP density by an order of magnitude and are shown to be stable for several weeks with minimal impact on printing quality. Our approach is generalizable to a variety of oc-NPs and ultimately leads to higher quality polymer-based optical and electronic devices.
△ Less
Submitted 5 July, 2024;
originally announced July 2024.
-
ImageFlowNet: Forecasting Multiscale Image-Level Trajectories of Disease Progression with Irregularly-Sampled Longitudinal Medical Images
Authors:
Chen Liu,
Ke Xu,
Liangbo L. Shen,
Guillaume Huguet,
Zilong Wang,
Alexander Tong,
Danilo Bzdok,
Jay Stewart,
Jay C. Wang,
Lucian V. Del Priore,
Smita Krishnaswamy
Abstract:
Advances in medical imaging technologies have enabled the collection of longitudinal images, which involve repeated scanning of the same patients over time, to monitor disease progression. However, predictive modeling of such data remains challenging due to high dimensionality, irregular sampling, and data sparsity. To address these issues, we propose ImageFlowNet, a novel model designed to foreca…
▽ More
Advances in medical imaging technologies have enabled the collection of longitudinal images, which involve repeated scanning of the same patients over time, to monitor disease progression. However, predictive modeling of such data remains challenging due to high dimensionality, irregular sampling, and data sparsity. To address these issues, we propose ImageFlowNet, a novel model designed to forecast disease trajectories from initial images while preserving spatial details. ImageFlowNet first learns multiscale joint representation spaces across patients and time points, then optimizes deterministic or stochastic flow fields within these spaces using a position-parameterized neural ODE/SDE framework. The model leverages a UNet architecture to create robust multiscale representations and mitigates data scarcity by combining knowledge from all patients. We provide theoretical insights that support our formulation of ODEs, and motivate our regularizations involving high-level visual features, latent space organization, and trajectory smoothness. We validate ImageFlowNet on three longitudinal medical image datasets depicting progression in geographic atrophy, multiple sclerosis, and glioblastoma, demonstrating its ability to effectively forecast disease progression and outperform existing methods. Our contributions include the development of ImageFlowNet, its theoretical underpinnings, and empirical validation on real-world datasets. The official implementation is available at https://github.com/KrishnaswamyLab/ImageFlowNet.
△ Less
Submitted 16 September, 2024; v1 submitted 20 June, 2024;
originally announced June 2024.
-
Information Security and Privacy in the Digital World: Some Selected Topics
Authors:
Jaydip Sen,
Joceli Mayer,
Subhasis Dasgupta,
Subrata Nandi,
Srinivasan Krishnaswamy,
Pinaki Mitra,
Mahendra Pratap Singh,
Naga Prasanthi Kundeti,
Chandra Sekhara Rao MVP,
Sudha Sree Chekuri,
Seshu Babu Pallapothu,
Preethi Nanjundan,
Jossy P. George,
Abdelhadi El Allahi,
Ilham Morino,
Salma AIT Oussous,
Siham Beloualid,
Ahmed Tamtaoui,
Abderrahim Bajit
Abstract:
In the era of generative artificial intelligence and the Internet of Things, while there is explosive growth in the volume of data and the associated need for processing, analysis, and storage, several new challenges are faced in identifying spurious and fake information and protecting the privacy of sensitive data. This has led to an increasing demand for more robust and resilient schemes for aut…
▽ More
In the era of generative artificial intelligence and the Internet of Things, while there is explosive growth in the volume of data and the associated need for processing, analysis, and storage, several new challenges are faced in identifying spurious and fake information and protecting the privacy of sensitive data. This has led to an increasing demand for more robust and resilient schemes for authentication, integrity protection, encryption, non-repudiation, and privacy-preservation of data. The chapters in this book present some of the state-of-the-art research works in the field of cryptography and security in computing and communications.
△ Less
Submitted 29 March, 2024;
originally announced April 2024.
-
Experimental retrieval of photon statistics from click detection
Authors:
Suchitra Krishnaswamy,
Fabian Schule,
Laura Ares,
Vladyslav Dyachuk,
Michael Stefszky,
Benjamin Brecht,
Christine Silberhorn,
Jan Sperling
Abstract:
We utilize click-counting theory for the reconstruction of photon statistics. Our approach employs an analytic pseudo-inversion method to estimate photon counts from measured click counts. A reconfigurable time-bin multiplexing, click-counting detector is set up that renders it possible to alter the photon-number resolution as needed. A detector tomography is carried out, yielding vital measuremen…
▽ More
We utilize click-counting theory for the reconstruction of photon statistics. Our approach employs an analytic pseudo-inversion method to estimate photon counts from measured click counts. A reconfigurable time-bin multiplexing, click-counting detector is set up that renders it possible to alter the photon-number resolution as needed. A detector tomography is carried out, yielding vital measurement features, such as quantum efficiencies, cross-talk rates, etc. We gauge the success of the pseudo-inversion by applying the Mandel and binomial parameters, resulting in an additional interpretation of these parameters for the discrimination of distinct quantum statistics. In addition, we apply a loss deconvolution technique to account for detection losses.
△ Less
Submitted 11 September, 2024; v1 submitted 18 March, 2024;
originally announced March 2024.
-
Assessing Neural Network Representations During Training Using Noise-Resilient Diffusion Spectral Entropy
Authors:
Danqi Liao,
Chen Liu,
Benjamin W. Christensen,
Alexander Tong,
Guillaume Huguet,
Guy Wolf,
Maximilian Nickel,
Ian Adelstein,
Smita Krishnaswamy
Abstract:
Entropy and mutual information in neural networks provide rich information on the learning process, but they have proven difficult to compute reliably in high dimensions. Indeed, in noisy and high-dimensional data, traditional estimates in ambient dimensions approach a fixed entropy and are prohibitively hard to compute. To address these issues, we leverage data geometry to access the underlying m…
▽ More
Entropy and mutual information in neural networks provide rich information on the learning process, but they have proven difficult to compute reliably in high dimensions. Indeed, in noisy and high-dimensional data, traditional estimates in ambient dimensions approach a fixed entropy and are prohibitively hard to compute. To address these issues, we leverage data geometry to access the underlying manifold and reliably compute these information-theoretic measures. Specifically, we define diffusion spectral entropy (DSE) in neural representations of a dataset as well as diffusion spectral mutual information (DSMI) between different variables representing data. First, we show that they form noise-resistant measures of intrinsic dimensionality and relationship strength in high-dimensional simulated data that outperform classic Shannon entropy, nonparametric estimation, and mutual information neural estimation (MINE). We then study the evolution of representations in classification networks with supervised learning, self-supervision, or overfitting. We observe that (1) DSE of neural representations increases during training; (2) DSMI with the class label increases during generalizable learning but stays stagnant during overfitting; (3) DSMI with the input signal shows differing trends: on MNIST it increases, while on CIFAR-10 and STL-10 it decreases. Finally, we show that DSE can be used to guide better network initialization and that DSMI can be used to predict downstream classification accuracy across 962 models on ImageNet. The official implementation is available at https://github.com/ChenLiu-1996/DiffusionSpectralEntropy.
△ Less
Submitted 3 December, 2023;
originally announced December 2023.
-
Bayesian Formulations for Graph Spectral Denoising
Authors:
Sam Leone,
Xingzhi Sun,
Michael Perlmutter,
Smita Krishnaswamy
Abstract:
Here we consider the problem of denoising features associated to complex data, modeled as signals on a graph, via a smoothness prior. This is motivated in part by settings such as single-cell RNA where the data is very high-dimensional, but its structure can be captured via an affinity graph. This allows us to utilize ideas from graph signal processing. In particular, we present algorithms for the…
▽ More
Here we consider the problem of denoising features associated to complex data, modeled as signals on a graph, via a smoothness prior. This is motivated in part by settings such as single-cell RNA where the data is very high-dimensional, but its structure can be captured via an affinity graph. This allows us to utilize ideas from graph signal processing. In particular, we present algorithms for the cases where the signal is perturbed by Gaussian noise, dropout, and uniformly distributed noise. The signals are assumed to follow a prior distribution defined in the frequency domain which favors signals which are smooth across the edges of the graph. By pairing this prior distribution with our three models of noise generation, we propose Maximum A Posteriori (M.A.P.) estimates of the true signal in the presence of noisy data and provide algorithms for computing the M.A.P. Finally, we demonstrate the algorithms' ability to effectively restore signals from white noise on image data and from severe dropout in single-cell RNA sequence data.
△ Less
Submitted 8 December, 2023; v1 submitted 27 November, 2023;
originally announced November 2023.
-
Mechanical Characterization and Inverse Design of Stochastic Architected Metamaterials Using Neural Operators
Authors:
Hanxun Jin,
Enrui Zhang,
Boyu Zhang,
Sridhar Krishnaswamy,
George Em Karniadakis,
Horacio D. Espinosa
Abstract:
Machine learning (ML) is emerging as a transformative tool for the design of architected materials, offering properties that far surpass those achievable through lab-based trial-and-error methods. However, a major challenge in current inverse design strategies is their reliance on extensive computational and/or experimental datasets, which becomes particularly problematic for designing micro-scale…
▽ More
Machine learning (ML) is emerging as a transformative tool for the design of architected materials, offering properties that far surpass those achievable through lab-based trial-and-error methods. However, a major challenge in current inverse design strategies is their reliance on extensive computational and/or experimental datasets, which becomes particularly problematic for designing micro-scale stochastic architected materials that exhibit nonlinear mechanical behaviors. Here, we introduce a new end-to-end scientific ML framework, leveraging deep neural operators (DeepONet), to directly learn the relationship between the complete microstructure and mechanical response of architected metamaterials from sparse but high-quality in situ experimental data. The approach facilitates the inverse design of structures tailored to specific nonlinear mechanical behaviors. Results obtained from spinodal microstructures, printed using two-photon lithography, reveal that the prediction error for mechanical responses is within a range of 5 - 10%. Our work underscores that by employing neural operators with advanced micro-mechanics experimental techniques, the design of complex micro-architected materials with desired properties becomes feasible, even in scenarios constrained by data scarcity. Our work marks a significant advancement in the field of materials-by-design, potentially heralding a new era in the discovery and development of next-generation metamaterials with unparalleled mechanical characteristics derived directly from experimental insights.
△ Less
Submitted 10 December, 2023; v1 submitted 23 November, 2023;
originally announced November 2023.
-
BLIS-Net: Classifying and Analyzing Signals on Graphs
Authors:
Charles Xu,
Laney Goldman,
Valentina Guo,
Benjamin Hollander-Bodie,
Maedee Trank-Greene,
Ian Adelstein,
Edward De Brouwer,
Rex Ying,
Smita Krishnaswamy,
Michael Perlmutter
Abstract:
Graph neural networks (GNNs) have emerged as a powerful tool for tasks such as node classification and graph classification. However, much less work has been done on signal classification, where the data consists of many functions (referred to as signals) defined on the vertices of a single graph. These tasks require networks designed differently from those designed for traditional GNN tasks. Inde…
▽ More
Graph neural networks (GNNs) have emerged as a powerful tool for tasks such as node classification and graph classification. However, much less work has been done on signal classification, where the data consists of many functions (referred to as signals) defined on the vertices of a single graph. These tasks require networks designed differently from those designed for traditional GNN tasks. Indeed, traditional GNNs rely on localized low-pass filters, and signals of interest may have intricate multi-frequency behavior and exhibit long range interactions. This motivates us to introduce the BLIS-Net (Bi-Lipschitz Scattering Net), a novel GNN that builds on the previously introduced geometric scattering transform. Our network is able to capture both local and global signal structure and is able to capture both low-frequency and high-frequency information. We make several crucial changes to the original geometric scattering architecture which we prove increase the ability of our network to capture information about the input signal and show that BLIS-Net achieves superior performance on both synthetic and real-world data sets based on traffic flow and fMRI data.
△ Less
Submitted 26 October, 2023;
originally announced October 2023.
-
Learning graph geometry and topology using dynamical systems based message-passing
Authors:
Dhananjay Bhaskar,
Yanlei Zhang,
Charles Xu,
Xingzhi Sun,
Oluwadamilola Fasina,
Guy Wolf,
Maximilian Nickel,
Michael Perlmutter,
Smita Krishnaswamy
Abstract:
In this paper we introduce DYMAG: a message passing paradigm for GNNs built on the expressive power of continuous, multiscale graph-dynamics. Standard discrete-time message passing algorithms implicitly make use of simplistic graph dynamics and aggregation schemes which limit their ability to capture fundamental graph topological properties. By contrast, DYMAG makes use of complex graph dynamics b…
▽ More
In this paper we introduce DYMAG: a message passing paradigm for GNNs built on the expressive power of continuous, multiscale graph-dynamics. Standard discrete-time message passing algorithms implicitly make use of simplistic graph dynamics and aggregation schemes which limit their ability to capture fundamental graph topological properties. By contrast, DYMAG makes use of complex graph dynamics based on the heat and wave equation as well as a more complex equation which admits chaotic solutions. The continuous nature of the dynamics are leveraged to generate multiscale (dynamic-time snapshot) representations which we prove are linked to various graph topological and spectral properties. We demonstrate experimentally that DYMAG achieves superior performance in recovering the generating parameters of Erdös-Renyi and stochastic block model random graphs and the persistent homology of synthetic graphs and citation network. Since the behavior of proteins and biomolecules is sensitive to graph topology and exhibits important structure at multiple scales, we find that DYMAG outperforms other methods at predicting salient features of various biomolecules.
△ Less
Submitted 7 July, 2024; v1 submitted 18 September, 2023;
originally announced September 2023.
-
Directed Scattering for Knowledge Graph-based Cellular Signaling Analysis
Authors:
Aarthi Venkat,
Joyce Chew,
Ferran Cardoso Rodriguez,
Christopher J. Tape,
Michael Perlmutter,
Smita Krishnaswamy
Abstract:
Directed graphs are a natural model for many phenomena, in particular scientific knowledge graphs such as molecular interaction or chemical reaction networks that define cellular signaling relationships. In these situations, source nodes typically have distinct biophysical properties from sinks. Due to their ordered and unidirectional relationships, many such networks also have hierarchical and mu…
▽ More
Directed graphs are a natural model for many phenomena, in particular scientific knowledge graphs such as molecular interaction or chemical reaction networks that define cellular signaling relationships. In these situations, source nodes typically have distinct biophysical properties from sinks. Due to their ordered and unidirectional relationships, many such networks also have hierarchical and multiscale structure. However, the majority of methods performing node- and edge-level tasks in machine learning do not take these properties into account, and thus have not been leveraged effectively for scientific tasks such as cellular signaling network inference. We propose a new framework called Directed Scattering Autoencoder (DSAE) which uses a directed version of a geometric scattering transform, combined with the non-linear dimensionality reduction properties of an autoencoder and the geometric properties of the hyperbolic space to learn latent hierarchies. We show this method outperforms numerous others on tasks such as embedding directed graphs and learning cellular signaling networks.
△ Less
Submitted 14 September, 2023;
originally announced September 2023.
-
An iterative scheme for finite horizon model reduction of continuous-time linear time-varying systems
Authors:
Kasturi Das,
Srinivasan Krishnaswamy,
Somanath Majhi
Abstract:
In this paper, we obtain the functional derivatives of a finite horizon error norm between a full-order and a reduced-order continuous-time linear time-varying (LTV) system. Based on the functional derivatives, first-order necessary conditions for optimality of the error norm are derived, and a projection-based iterative scheme for model reduction is proposed. The iterative scheme upon convergence…
▽ More
In this paper, we obtain the functional derivatives of a finite horizon error norm between a full-order and a reduced-order continuous-time linear time-varying (LTV) system. Based on the functional derivatives, first-order necessary conditions for optimality of the error norm are derived, and a projection-based iterative scheme for model reduction is proposed. The iterative scheme upon convergence produces reduced-order models satisfying the optimality conditions. Finally, through a numerical example, we demonstrate the better performance of the proposed model reduction scheme in comparison to the finite horizon balanced truncation algorithm for continuous-time LTV systems.
△ Less
Submitted 2 September, 2023;
originally announced September 2023.
-
A Flow Artist for High-Dimensional Cellular Data
Authors:
Kincaid MacDonald,
Dhananjay Bhaskar,
Guy Thampakkul,
Nhi Nguyen,
Joia Zhang,
Michael Perlmutter,
Ian Adelstein,
Smita Krishnaswamy
Abstract:
We consider the problem of embedding point cloud data sampled from an underlying manifold with an associated flow or velocity. Such data arises in many contexts where static snapshots of dynamic entities are measured, including in high-throughput biology such as single-cell transcriptomics. Existing embedding techniques either do not utilize velocity information or embed the coordinates and veloci…
▽ More
We consider the problem of embedding point cloud data sampled from an underlying manifold with an associated flow or velocity. Such data arises in many contexts where static snapshots of dynamic entities are measured, including in high-throughput biology such as single-cell transcriptomics. Existing embedding techniques either do not utilize velocity information or embed the coordinates and velocities independently, i.e., they either impose velocities on top of an existing point embedding or embed points within a prescribed vector field. Here we present FlowArtist, a neural network that embeds points while jointly learning a vector field around the points. The combination allows FlowArtist to better separate and visualize velocity-informed structures. Our results, on toy datasets and single-cell RNA velocity data, illustrate the value of utilizing coordinate and velocity information in tandem for embedding and visualizing high-dimensional data.
△ Less
Submitted 31 July, 2023;
originally announced August 2023.
-
Manifold Filter-Combine Networks
Authors:
Joyce Chew,
Edward De Brouwer,
Smita Krishnaswamy,
Deanna Needell,
Michael Perlmutter
Abstract:
We introduce a class of manifold neural networks (MNNs) that we call Manifold Filter-Combine Networks (MFCNs), that aims to further our understanding of MNNs, analogous to how the aggregate-combine framework helps with the understanding of graph neural networks (GNNs). This class includes a wide variety of subclasses that can be thought of as the manifold analog of various popular GNNs. We then co…
▽ More
We introduce a class of manifold neural networks (MNNs) that we call Manifold Filter-Combine Networks (MFCNs), that aims to further our understanding of MNNs, analogous to how the aggregate-combine framework helps with the understanding of graph neural networks (GNNs). This class includes a wide variety of subclasses that can be thought of as the manifold analog of various popular GNNs. We then consider a method, based on building a data-driven graph, for implementing such networks when one does not have global knowledge of the manifold, but merely has access to finitely many sample points. We provide sufficient conditions for the network to provably converge to its continuum limit as the number of sample points tends to infinity. Unlike previous work (which focused on specific graph constructions), our rate of convergence does not directly depend on the number of filters used. Moreover, it exhibits linear dependence on the depth of the network rather than the exponential dependence obtained previously. Additionally, we provide several examples of interesting subclasses of MFCNs and of the rates of convergence that are obtained under specific graph constructions.
△ Less
Submitted 5 September, 2023; v1 submitted 8 July, 2023;
originally announced July 2023.
-
A Scheme to resist Fast Correlation Attack for Word Oriented LFSR based Stream Cipher
Authors:
Subrata Nandi,
Srinivasan Krishnaswamy,
Pinaki Mitra
Abstract:
In LFSR-based stream ciphers, the knowledge of the feedback equation of the LFSR plays a critical role in most attacks. In word-based stream ciphers such as those in the SNOW series, even if the feedback configuration is hidden, knowing the characteristic polynomial of the state transition matrix of the LFSR enables the attacker to create a feedback equation over $GF(2)$. This, in turn, can be use…
▽ More
In LFSR-based stream ciphers, the knowledge of the feedback equation of the LFSR plays a critical role in most attacks. In word-based stream ciphers such as those in the SNOW series, even if the feedback configuration is hidden, knowing the characteristic polynomial of the state transition matrix of the LFSR enables the attacker to create a feedback equation over $GF(2)$. This, in turn, can be used to launch fast correlation attacks. In this work, we propose a method for hiding both the feedback equation of a word-based LFSR and the characteristic polynomial of the state transition matrix. Here, we employ a $z$-primitive $σ$-LFSR whose characteristic polynomial is randomly sampled from the distribution of primitive polynomials over $GF(2)$ of the appropriate degree. We propose an algorithm for locating $z$-primitive $σ$-LFSR configurations of a given degree. Further, an invertible matrix is generated from the key. This is then employed to generate a public parameter which is used to retrieve the feedback configuration using the key. If the key size is $n$- bits, the process of retrieving the feedback equation from the public parameter has a average time complexity $\mathbb{O}(2^{n-1})$. The proposed method has been tested on SNOW 2.0 and SNOW 3G for resistance to fast correlation attacks. We have demonstrated that the security of SNOW 2.0 and SNOW 3G increases from 128 bits to 256 bits.
△ Less
Submitted 5 July, 2023;
originally announced July 2023.
-
Inferring dynamic regulatory interaction graphs from time series data with perturbations
Authors:
Dhananjay Bhaskar,
Sumner Magruder,
Edward De Brouwer,
Aarthi Venkat,
Frederik Wenkel,
Guy Wolf,
Smita Krishnaswamy
Abstract:
Complex systems are characterized by intricate interactions between entities that evolve dynamically over time. Accurate inference of these dynamic relationships is crucial for understanding and predicting system behavior. In this paper, we propose Regulatory Temporal Interaction Network Inference (RiTINI) for inferring time-varying interaction graphs in complex systems using a novel combination o…
▽ More
Complex systems are characterized by intricate interactions between entities that evolve dynamically over time. Accurate inference of these dynamic relationships is crucial for understanding and predicting system behavior. In this paper, we propose Regulatory Temporal Interaction Network Inference (RiTINI) for inferring time-varying interaction graphs in complex systems using a novel combination of space-and-time graph attentions and graph neural ordinary differential equations (ODEs). RiTINI leverages time-lapse signals on a graph prior, as well as perturbations of signals at various nodes in order to effectively capture the dynamics of the underlying system. This approach is distinct from traditional causal inference networks, which are limited to inferring acyclic and static graphs. In contrast, RiTINI can infer cyclic, directed, and time-varying graphs, providing a more comprehensive and accurate representation of complex systems. The graph attention mechanism in RiTINI allows the model to adaptively focus on the most relevant interactions in time and space, while the graph neural ODEs enable continuous-time modeling of the system's dynamics. We evaluate RiTINI's performance on various simulated and real-world datasets, demonstrating its state-of-the-art capability in inferring interaction graphs compared to previous methods.
△ Less
Submitted 13 June, 2023;
originally announced June 2023.
-
Neural FIM for learning Fisher Information Metrics from point cloud data
Authors:
Oluwadamilola Fasina,
Guillaume Huguet,
Alexander Tong,
Yanlei Zhang,
Guy Wolf,
Maximilian Nickel,
Ian Adelstein,
Smita Krishnaswamy
Abstract:
Although data diffusion embeddings are ubiquitous in unsupervised learning and have proven to be a viable technique for uncovering the underlying intrinsic geometry of data, diffusion embeddings are inherently limited due to their discrete nature. To this end, we propose neural FIM, a method for computing the Fisher information metric (FIM) from point cloud data - allowing for a continuous manifol…
▽ More
Although data diffusion embeddings are ubiquitous in unsupervised learning and have proven to be a viable technique for uncovering the underlying intrinsic geometry of data, diffusion embeddings are inherently limited due to their discrete nature. To this end, we propose neural FIM, a method for computing the Fisher information metric (FIM) from point cloud data - allowing for a continuous manifold model for the data. Neural FIM creates an extensible metric space from discrete point cloud data such that information from the metric can inform us of manifold characteristics such as volume and geodesics. We demonstrate Neural FIM's utility in selecting parameters for the PHATE visualization method as well as its ability to obtain information pertaining to local volume illuminating branching points and cluster centers embeddings of a toy dataset and two single-cell datasets of IPSC reprogramming and PBMCs (immune cells).
△ Less
Submitted 11 June, 2023; v1 submitted 1 June, 2023;
originally announced June 2023.
-
Graph Fourier MMD for Signals on Graphs
Authors:
Samuel Leone,
Aarthi Venkat,
Guillaume Huguet,
Alexander Tong,
Guy Wolf,
Smita Krishnaswamy
Abstract:
While numerous methods have been proposed for computing distances between probability distributions in Euclidean space, relatively little attention has been given to computing such distances for distributions on graphs. However, there has been a marked increase in data that either lies on graph (such as protein interaction networks) or can be modeled as a graph (single cell data), particularly in…
▽ More
While numerous methods have been proposed for computing distances between probability distributions in Euclidean space, relatively little attention has been given to computing such distances for distributions on graphs. However, there has been a marked increase in data that either lies on graph (such as protein interaction networks) or can be modeled as a graph (single cell data), particularly in the biomedical sciences. Thus, it becomes important to find ways to compare signals defined on such graphs. Here, we propose Graph Fourier MMD (GFMMD), a novel distance between distributions and signals on graphs. GFMMD is defined via an optimal witness function that is both smooth on the graph and maximizes difference in expectation between the pair of distributions on the graph. We find an analytical solution to this optimization problem as well as an embedding of distributions that results from this method. We also prove several properties of this method including scale invariance and applicability to disconnected graphs. We showcase it on graph benchmark datasets as well on single cell RNA-sequencing data analysis. In the latter, we use the GFMMD-based gene embeddings to find meaningful gene clusters. We also propose a novel type of score for gene selection called "gene localization score" which helps select genes for cellular state space characterization.
△ Less
Submitted 4 June, 2023;
originally announced June 2023.
-
A Heat Diffusion Perspective on Geodesic Preserving Dimensionality Reduction
Authors:
Guillaume Huguet,
Alexander Tong,
Edward De Brouwer,
Yanlei Zhang,
Guy Wolf,
Ian Adelstein,
Smita Krishnaswamy
Abstract:
Diffusion-based manifold learning methods have proven useful in representation learning and dimensionality reduction of modern high dimensional, high throughput, noisy datasets. Such datasets are especially present in fields like biology and physics. While it is thought that these methods preserve underlying manifold structure of data by learning a proxy for geodesic distances, no specific theoret…
▽ More
Diffusion-based manifold learning methods have proven useful in representation learning and dimensionality reduction of modern high dimensional, high throughput, noisy datasets. Such datasets are especially present in fields like biology and physics. While it is thought that these methods preserve underlying manifold structure of data by learning a proxy for geodesic distances, no specific theoretical links have been established. Here, we establish such a link via results in Riemannian geometry explicitly connecting heat diffusion to manifold distances. In this process, we also formulate a more general heat kernel based manifold embedding method that we call heat geodesic embeddings. This novel perspective makes clearer the choices available in manifold learning and denoising. Results show that our method outperforms existing state of the art in preserving ground truth manifold distances, and preserving cluster structure in toy datasets. We also showcase our method on single cell RNA-sequencing datasets with both continuum and cluster structure, where our method enables interpolation of withheld timepoints of data. Finally, we show that parameters of our more general method can be configured to give results similar to PHATE (a state-of-the-art diffusion based manifold learning method) as well as SNE (an attraction/repulsion neighborhood based method that forms the basis of t-SNE).
△ Less
Submitted 30 May, 2023;
originally announced May 2023.
-
Geodesic Sinkhorn for Fast and Accurate Optimal Transport on Manifolds
Authors:
Guillaume Huguet,
Alexander Tong,
María Ramos Zapatero,
Christopher J. Tape,
Guy Wolf,
Smita Krishnaswamy
Abstract:
Efficient computation of optimal transport distance between distributions is of growing importance in data science. Sinkhorn-based methods are currently the state-of-the-art for such computations, but require $O(n^2)$ computations. In addition, Sinkhorn-based methods commonly use an Euclidean ground distance between datapoints. However, with the prevalence of manifold structured scientific data, i…
▽ More
Efficient computation of optimal transport distance between distributions is of growing importance in data science. Sinkhorn-based methods are currently the state-of-the-art for such computations, but require $O(n^2)$ computations. In addition, Sinkhorn-based methods commonly use an Euclidean ground distance between datapoints. However, with the prevalence of manifold structured scientific data, it is often desirable to consider geodesic ground distance. Here, we tackle both issues by proposing Geodesic Sinkhorn -- based on diffusing a heat kernel on a manifold graph. Notably, Geodesic Sinkhorn requires only $O(n\log n)$ computation, as we approximate the heat kernel with Chebyshev polynomials based on the sparse graph Laplacian. We apply our method to the computation of barycenters of several distributions of high dimensional single cell data from patient samples undergoing chemotherapy. In particular, we define the barycentric distance as the distance between two such barycenters. Using this definition, we identify an optimal transport distance and path associated with the effect of treatment on cellular data.
△ Less
Submitted 26 September, 2023; v1 submitted 1 November, 2022;
originally announced November 2022.
-
CUTS: A Deep Learning and Topological Framework for Multigranular Unsupervised Medical Image Segmentation
Authors:
Chen Liu,
Matthew Amodio,
Liangbo L. Shen,
Feng Gao,
Arman Avesta,
Sanjay Aneja,
Jay C. Wang,
Lucian V. Del Priore,
Smita Krishnaswamy
Abstract:
Segmenting medical images is critical to facilitating both patient diagnoses and quantitative research. A major limiting factor is the lack of labeled data, as obtaining expert annotations for each new set of imaging data and task can be labor intensive and inconsistent among annotators. We present CUTS, an unsupervised deep learning framework for medical image segmentation. CUTS operates in two s…
▽ More
Segmenting medical images is critical to facilitating both patient diagnoses and quantitative research. A major limiting factor is the lack of labeled data, as obtaining expert annotations for each new set of imaging data and task can be labor intensive and inconsistent among annotators. We present CUTS, an unsupervised deep learning framework for medical image segmentation. CUTS operates in two stages. For each image, it produces an embedding map via intra-image contrastive learning and local patch reconstruction. Then, these embeddings are partitioned at dynamic granularity levels that correspond to the data topology. CUTS yields a series of coarse-to-fine-grained segmentations that highlight features at various granularities. We applied CUTS to retinal fundus images and two types of brain MRI images to delineate structures and patterns at different scales. When evaluated against predefined anatomical masks, CUTS improved the dice coefficient and Hausdorff distance by at least 10% compared to existing unsupervised methods. Finally, CUTS showed performance on par with Segment Anything Models (SAM, MedSAM, SAM-Med2D) pre-trained on gigantic labeled datasets.
△ Less
Submitted 25 June, 2024; v1 submitted 22 September, 2022;
originally announced September 2022.
-
Geometric Scattering on Measure Spaces
Authors:
Joyce Chew,
Matthew Hirn,
Smita Krishnaswamy,
Deanna Needell,
Michael Perlmutter,
Holly Steach,
Siddharth Viswanath,
Hau-Tieng Wu
Abstract:
The scattering transform is a multilayered, wavelet-based transform initially introduced as a model of convolutional neural networks (CNNs) that has played a foundational role in our understanding of these networks' stability and invariance properties. Subsequently, there has been widespread interest in extending the success of CNNs to data sets with non-Euclidean structure, such as graphs and man…
▽ More
The scattering transform is a multilayered, wavelet-based transform initially introduced as a model of convolutional neural networks (CNNs) that has played a foundational role in our understanding of these networks' stability and invariance properties. Subsequently, there has been widespread interest in extending the success of CNNs to data sets with non-Euclidean structure, such as graphs and manifolds, leading to the emerging field of geometric deep learning. In order to improve our understanding of the architectures used in this new field, several papers have proposed generalizations of the scattering transform for non-Euclidean data structures such as undirected graphs and compact Riemannian manifolds without boundary.
In this paper, we introduce a general, unified model for geometric scattering on measure spaces. Our proposed framework includes previous work on geometric scattering as special cases but also applies to more general settings such as directed graphs, signed graphs, and manifolds with boundary. We propose a new criterion that identifies to which groups a useful representation should be invariant and show that this criterion is sufficient to guarantee that the scattering transform has desirable stability and invariance properties. Additionally, we consider finite measure spaces that are obtained from randomly sampling an unknown manifold. We propose two methods for constructing a data-driven graph on which the associated graph scattering transform approximates the scattering transform on the underlying manifold. Moreover, we use a diffusion-maps based approach to prove quantitative estimates on the rate of convergence of one of these approximations as the number of sample points tends to infinity. Lastly, we showcase the utility of our method on spherical images, directed graphs, and on high-dimensional single-cell data.
△ Less
Submitted 13 October, 2022; v1 submitted 17 August, 2022;
originally announced August 2022.
-
Learnable Filters for Geometric Scattering Modules
Authors:
Alexander Tong,
Frederik Wenkel,
Dhananjay Bhaskar,
Kincaid Macdonald,
Jackson Grady,
Michael Perlmutter,
Smita Krishnaswamy,
Guy Wolf
Abstract:
We propose a new graph neural network (GNN) module, based on relaxations of recently proposed geometric scattering transforms, which consist of a cascade of graph wavelet filters. Our learnable geometric scattering (LEGS) module enables adaptive tuning of the wavelets to encourage band-pass features to emerge in learned representations. The incorporation of our LEGS-module in GNNs enables the lear…
▽ More
We propose a new graph neural network (GNN) module, based on relaxations of recently proposed geometric scattering transforms, which consist of a cascade of graph wavelet filters. Our learnable geometric scattering (LEGS) module enables adaptive tuning of the wavelets to encourage band-pass features to emerge in learned representations. The incorporation of our LEGS-module in GNNs enables the learning of longer-range graph relations compared to many popular GNNs, which often rely on encoding graph structure via smoothness or similarity between neighbors. Further, its wavelet priors result in simplified architectures with significantly fewer learned parameters compared to competing GNNs. We demonstrate the predictive performance of LEGS-based networks on graph classification benchmarks, as well as the descriptive quality of their learned features in biochemical graph data exploration tasks. Our results show that LEGS-based networks match or outperforms popular GNNs, as well as the original geometric scattering construction, on many datasets, in particular in biochemical domains, while retaining certain mathematical properties of handcrafted (non-learned) geometric scattering.
△ Less
Submitted 15 August, 2022;
originally announced August 2022.
-
Manifold Interpolating Optimal-Transport Flows for Trajectory Inference
Authors:
Guillaume Huguet,
D. S. Magruder,
Alexander Tong,
Oluwadamilola Fasina,
Manik Kuchroo,
Guy Wolf,
Smita Krishnaswamy
Abstract:
We present a method called Manifold Interpolating Optimal-Transport Flow (MIOFlow) that learns stochastic, continuous population dynamics from static snapshot samples taken at sporadic timepoints. MIOFlow combines dynamic models, manifold learning, and optimal transport by training neural ordinary differential equations (Neural ODE) to interpolate between static population snapshots as penalized b…
▽ More
We present a method called Manifold Interpolating Optimal-Transport Flow (MIOFlow) that learns stochastic, continuous population dynamics from static snapshot samples taken at sporadic timepoints. MIOFlow combines dynamic models, manifold learning, and optimal transport by training neural ordinary differential equations (Neural ODE) to interpolate between static population snapshots as penalized by optimal transport with manifold ground distance. Further, we ensure that the flow follows the geometry by operating in the latent space of an autoencoder that we call a geodesic autoencoder (GAE). In GAE the latent space distance between points is regularized to match a novel multiscale geodesic distance on the data manifold that we define. We show that this method is superior to normalizing flows, Schrödinger bridges and other generative models that are designed to flow from noise to data in terms of interpolating between populations. Theoretically, we link these trajectories with dynamic optimal transport. We evaluate our method on simulated data with bifurcations and merges, as well as scRNA-seq data from embryoid body differentiation, and acute myeloid leukemia treatment.
△ Less
Submitted 3 November, 2022; v1 submitted 29 June, 2022;
originally announced June 2022.
-
The Manifold Scattering Transform for High-Dimensional Point Cloud Data
Authors:
Joyce Chew,
Holly R. Steach,
Siddharth Viswanath,
Hau-Tieng Wu,
Matthew Hirn,
Deanna Needell,
Smita Krishnaswamy,
Michael Perlmutter
Abstract:
The manifold scattering transform is a deep feature extractor for data defined on a Riemannian manifold. It is one of the first examples of extending convolutional neural network-like operators to general manifolds. The initial work on this model focused primarily on its theoretical stability and invariance properties but did not provide methods for its numerical implementation except in the case…
▽ More
The manifold scattering transform is a deep feature extractor for data defined on a Riemannian manifold. It is one of the first examples of extending convolutional neural network-like operators to general manifolds. The initial work on this model focused primarily on its theoretical stability and invariance properties but did not provide methods for its numerical implementation except in the case of two-dimensional surfaces with predefined meshes. In this work, we present practical schemes, based on the theory of diffusion maps, for implementing the manifold scattering transform to datasets arising in naturalistic systems, such as single cell genetics, where the data is a high-dimensional point cloud modeled as lying on a low-dimensional manifold. We show that our methods are effective for signal classification and manifold classification tasks.
△ Less
Submitted 21 January, 2024; v1 submitted 20 June, 2022;
originally announced June 2022.
-
Diffusion Curvature for Estimating Local Curvature in High Dimensional Data
Authors:
Dhananjay Bhaskar,
Kincaid MacDonald,
Oluwadamilola Fasina,
Dawson Thomas,
Bastian Rieck,
Ian Adelstein,
Smita Krishnaswamy
Abstract:
We introduce a new intrinsic measure of local curvature on point-cloud data called diffusion curvature. Our measure uses the framework of diffusion maps, including the data diffusion operator, to structure point cloud data and define local curvature based on the laziness of a random walk starting at a point or region of the data. We show that this laziness directly relates to volume comparison res…
▽ More
We introduce a new intrinsic measure of local curvature on point-cloud data called diffusion curvature. Our measure uses the framework of diffusion maps, including the data diffusion operator, to structure point cloud data and define local curvature based on the laziness of a random walk starting at a point or region of the data. We show that this laziness directly relates to volume comparison results from Riemannian geometry. We then extend this scalar curvature notion to an entire quadratic form using neural network estimations based on the diffusion map of point-cloud data. We show applications of both estimations on toy data, single-cell data, and on estimating local Hessian matrices of neural network loss landscapes.
△ Less
Submitted 8 June, 2022;
originally announced June 2022.
-
Time-inhomogeneous diffusion geometry and topology
Authors:
Guillaume Huguet,
Alexander Tong,
Bastian Rieck,
Jessie Huang,
Manik Kuchroo,
Matthew Hirn,
Guy Wolf,
Smita Krishnaswamy
Abstract:
Diffusion condensation is a dynamic process that yields a sequence of multiscale data representations that aim to encode meaningful abstractions. It has proven effective for manifold learning, denoising, clustering, and visualization of high-dimensional data. Diffusion condensation is constructed as a time-inhomogeneous process where each step first computes and then applies a diffusion operator t…
▽ More
Diffusion condensation is a dynamic process that yields a sequence of multiscale data representations that aim to encode meaningful abstractions. It has proven effective for manifold learning, denoising, clustering, and visualization of high-dimensional data. Diffusion condensation is constructed as a time-inhomogeneous process where each step first computes and then applies a diffusion operator to the data. We theoretically analyze the convergence and evolution of this process from geometric, spectral, and topological perspectives. From a geometric perspective, we obtain convergence bounds based on the smallest transition probability and the radius of the data, whereas from a spectral perspective, our bounds are based on the eigenspectrum of the diffusion kernel. Our spectral results are of particular interest since most of the literature on data diffusion is focused on homogeneous processes. From a topological perspective, we show diffusion condensation generalizes centroid-based hierarchical clustering. We use this perspective to obtain a bound based on the number of data points, independent of their location. To understand the evolution of the data geometry beyond convergence, we use topological data analysis. We show that the condensation process itself defines an intrinsic condensation homology. We use this intrinsic topology as well as the ambient persistent homology of the condensation process to study how the data changes over diffusion time. We demonstrate both types of topological information in well-understood toy examples. Our work gives theoretical insights into the convergence of diffusion condensation, and shows that it provides a link between topological and geometric data analysis.
△ Less
Submitted 5 January, 2023; v1 submitted 28 March, 2022;
originally announced March 2022.
-
ReLSO: A Transformer-based Model for Latent Space Optimization and Generation of Proteins
Authors:
Egbert Castro,
Abhinav Godavarthi,
Julian Rubinfien,
Kevin B. Givechian,
Dhananjay Bhaskar,
Smita Krishnaswamy
Abstract:
The development of powerful natural language models have increased the ability to learn meaningful representations of protein sequences. In addition, advances in high-throughput mutagenesis, directed evolution, and next-generation sequencing have allowed for the accumulation of large amounts of labeled fitness data. Leveraging these two trends, we introduce Regularized Latent Space Optimization (R…
▽ More
The development of powerful natural language models have increased the ability to learn meaningful representations of protein sequences. In addition, advances in high-throughput mutagenesis, directed evolution, and next-generation sequencing have allowed for the accumulation of large amounts of labeled fitness data. Leveraging these two trends, we introduce Regularized Latent Space Optimization (ReLSO), a deep transformer-based autoencoder which features a highly structured latent space that is trained to jointly generate sequences as well as predict fitness. Through regularized prediction heads, ReLSO introduces a powerful protein sequence encoder and novel approach for efficient fitness landscape traversal. Using ReLSO, we explicitly model the sequence-function landscape of large labeled datasets and generate new molecules by optimizing within the latent space using gradient-based methods. We evaluate this approach on several publicly-available protein datasets, including variant sets of anti-ranibizumab and GFP. We observe a greater sequence optimization efficiency (increase in fitness per optimization step) by ReLSO compared to other approaches, where ReLSO more robustly generates high-fitness sequences. Furthermore, the attention-based relationships learned by the jointly-trained ReLSO models provides a potential avenue towards sequence-level fitness attribution information.
△ Less
Submitted 31 May, 2022; v1 submitted 24 January, 2022;
originally announced January 2022.
-
H$_2$ Optimal Model Order Reduction over a Finite Time Interval
Authors:
Kasturi Das,
Srinivasan Krishnaswamy,
Somanath Majhi
Abstract:
For a time-limited version of the H$_2$ norm defined over a fixed time interval, we obtain a closed form expression of the gradients. After that, we use the gradients to propose a time-limited model order reduction method. The method involves obtaining a reduced model which minimizes the time-limited H$_2$ norm, formulated as a nonlinear optimization problem. The optimization problem is solved usi…
▽ More
For a time-limited version of the H$_2$ norm defined over a fixed time interval, we obtain a closed form expression of the gradients. After that, we use the gradients to propose a time-limited model order reduction method. The method involves obtaining a reduced model which minimizes the time-limited H$_2$ norm, formulated as a nonlinear optimization problem. The optimization problem is solved using standard optimization software.
△ Less
Submitted 3 January, 2022;
originally announced January 2022.
-
Learning shared neural manifolds from multi-subject FMRI data
Authors:
Jessie Huang,
Erica L. Busch,
Tom Wallenstein,
Michal Gerasimiuk,
Andrew Benz,
Guillaume Lajoie,
Guy Wolf,
Nicholas B. Turk-Browne,
Smita Krishnaswamy
Abstract:
Functional magnetic resonance imaging (fMRI) is a notoriously noisy measurement of brain activity because of the large variations between individuals, signals marred by environmental differences during collection, and spatiotemporal averaging required by the measurement resolution. In addition, the data is extremely high dimensional, with the space of the activity typically having much lower intri…
▽ More
Functional magnetic resonance imaging (fMRI) is a notoriously noisy measurement of brain activity because of the large variations between individuals, signals marred by environmental differences during collection, and spatiotemporal averaging required by the measurement resolution. In addition, the data is extremely high dimensional, with the space of the activity typically having much lower intrinsic dimension. In order to understand the connection between stimuli of interest and brain activity, and analyze differences and commonalities between subjects, it becomes important to learn a meaningful embedding of the data that denoises, and reveals its intrinsic structure. Specifically, we assume that while noise varies significantly between individuals, true responses to stimuli will share common, low-dimensional features between subjects which are jointly discoverable. Similar approaches have been exploited previously but they have mainly used linear methods such as PCA and shared response modeling (SRM). In contrast, we propose a neural network called MRMD-AE (manifold-regularized multiple decoder, autoencoder), that learns a common embedding from multiple subjects in an experiment while retaining the ability to decode to individual raw fMRI signals. We show that our learned common space represents an extensible manifold (where new points not seen during training can be mapped), improves the classification accuracy of stimulus features of unseen timepoints, as well as improves cross-subject translation of fMRI signals. We believe this framework can be used for many downstream applications such as guided brain-computer interface (BCI) training in the future.
△ Less
Submitted 22 December, 2021;
originally announced January 2022.
-
MURAL: An Unsupervised Random Forest-Based Embedding for Electronic Health Record Data
Authors:
Michal Gerasimiuk,
Dennis Shung,
Alexander Tong,
Adrian Stanley,
Michael Schultz,
Jeffrey Ngu,
Loren Laine,
Guy Wolf,
Smita Krishnaswamy
Abstract:
A major challenge in embedding or visualizing clinical patient data is the heterogeneity of variable types including continuous lab values, categorical diagnostic codes, as well as missing or incomplete data. In particular, in EHR data, some variables are {\em missing not at random (MNAR)} but deliberately not collected and thus are a source of information. For example, lab tests may be deemed nec…
▽ More
A major challenge in embedding or visualizing clinical patient data is the heterogeneity of variable types including continuous lab values, categorical diagnostic codes, as well as missing or incomplete data. In particular, in EHR data, some variables are {\em missing not at random (MNAR)} but deliberately not collected and thus are a source of information. For example, lab tests may be deemed necessary for some patients on the basis of suspected diagnosis, but not for others. Here we present the MURAL forest -- an unsupervised random forest for representing data with disparate variable types (e.g., categorical, continuous, MNAR). MURAL forests consist of a set of decision trees where node-splitting variables are chosen at random, such that the marginal entropy of all other variables is minimized by the split. This allows us to also split on MNAR variables and discrete variables in a way that is consistent with the continuous variables. The end goal is to learn the MURAL embedding of patients using average tree distances between those patients. These distances can be fed to nonlinear dimensionality reduction method like PHATE to derive visualizable embeddings. While such methods are ubiquitous in continuous-valued datasets (like single cell RNA-sequencing) they have not been used extensively in mixed variable data. We showcase the use of our method on one artificial and two clinical datasets. We show that using our approach, we can visualize and classify data more accurately than competing approaches. Finally, we show that MURAL can also be used to compare cohorts of patients via the recently proposed tree-sliced Wasserstein distances.
△ Less
Submitted 19 November, 2021;
originally announced November 2021.
-
Molecular Graph Generation via Geometric Scattering
Authors:
Dhananjay Bhaskar,
Jackson D. Grady,
Michael A. Perlmutter,
Smita Krishnaswamy
Abstract:
Graph neural networks (GNNs) have been used extensively for addressing problems in drug design and discovery. Both ligand and target molecules are represented as graphs with node and edge features encoding information about atomic elements and bonds respectively. Although existing deep learning models perform remarkably well at predicting physicochemical properties and binding affinities, the gene…
▽ More
Graph neural networks (GNNs) have been used extensively for addressing problems in drug design and discovery. Both ligand and target molecules are represented as graphs with node and edge features encoding information about atomic elements and bonds respectively. Although existing deep learning models perform remarkably well at predicting physicochemical properties and binding affinities, the generation of new molecules with optimized properties remains challenging. Inherently, most GNNs perform poorly in whole-graph representation due to the limitations of the message-passing paradigm. Furthermore, step-by-step graph generation frameworks that use reinforcement learning or other sequential processing can be slow and result in a high proportion of invalid molecules with substantial post-processing needed in order to satisfy the principles of stoichiometry. To address these issues, we propose a representation-first approach to molecular graph generation. We guide the latent representation of an autoencoder by capturing graph structure information with the geometric scattering transform and apply penalties that structure the representation also by molecular properties. We show that this highly structured latent space can be directly used for molecular graph generation by the use of a GAN. We demonstrate that our architecture learns meaningful representations of drug datasets and provides a platform for goal-directed drug synthesis.
△ Less
Submitted 12 October, 2021;
originally announced October 2021.
-
Near Optimal Interpolation based Time-Limited Model Order Reduction
Authors:
Kasturi Das,
Srinivasan Krishnaswamy,
Somanath Majhi
Abstract:
This paper presents an interpolatory framework for time-limited $H_2$ optimal model order reduction named Limited Time Iterative Rational Krylov Algorithm (LT-IRKA). The algorithm yields high fidelity reduced order models over limited time intervals of the form, $\begin{bmatrix}0 & τ\end{bmatrix}$ with $τ< \infty$ for linear time invariant (LTI) systems. Using the time limited $H_2$ norm, we deriv…
▽ More
This paper presents an interpolatory framework for time-limited $H_2$ optimal model order reduction named Limited Time Iterative Rational Krylov Algorithm (LT-IRKA). The algorithm yields high fidelity reduced order models over limited time intervals of the form, $\begin{bmatrix}0 & τ\end{bmatrix}$ with $τ< \infty$ for linear time invariant (LTI) systems. Using the time limited $H_2$ norm, we derive interpolation based $H_{2,τ}$ optimality conditions. The LT-IRKA yields a near optimal $H_2(τ)$ reduced order system. The nearness to the exact $H_2(τ)$ optimal reduced system is quantized in terms of the errors in the interpolation based $H_2(τ)$ optimality conditions. We demonstrate with numerical examples how the proposed algorithm nearly satisfies the time-limited optimality conditions and also how it performs with respect to the Time-Limited Two sided Iteration Algorithm (TL-TSIA), the Time-Limited Balanced Truncation (TL-BT), the Iterative Rational Krylov Algorithm (IRKA) and the Time-Limited Pseudo Optimal Rational Krylov (TL-PORK) Algorithm over a finite time interval.
△ Less
Submitted 8 October, 2021;
originally announced October 2021.
-
Embedding Signals on Knowledge Graphs with Unbalanced Diffusion Earth Mover's Distance
Authors:
Alexander Tong,
Guillaume Huguet,
Dennis Shung,
Amine Natik,
Manik Kuchroo,
Guillaume Lajoie,
Guy Wolf,
Smita Krishnaswamy
Abstract:
In modern relational machine learning it is common to encounter large graphs that arise via interactions or similarities between observations in many domains. Further, in many cases the target entities for analysis are actually signals on such graphs. We propose to compare and organize such datasets of graph signals by using an earth mover's distance (EMD) with a geodesic cost over the underlying…
▽ More
In modern relational machine learning it is common to encounter large graphs that arise via interactions or similarities between observations in many domains. Further, in many cases the target entities for analysis are actually signals on such graphs. We propose to compare and organize such datasets of graph signals by using an earth mover's distance (EMD) with a geodesic cost over the underlying graph. Typically, EMD is computed by optimizing over the cost of transporting one probability distribution to another over an underlying metric space. However, this is inefficient when computing the EMD between many signals. Here, we propose an unbalanced graph EMD that efficiently embeds the unbalanced EMD on an underlying graph into an $L^1$ space, whose metric we call unbalanced diffusion earth mover's distance (UDEMD). Next, we show how this gives distances between graph signals that are robust to noise. Finally, we apply this to organizing patients based on clinical notes, embedding cells modeled as signals on a gene graph, and organizing genes modeled as signals over a large cell graph. In each case, we show that UDEMD-based embeddings find accurate distances that are highly efficient compared to other methods.
△ Less
Submitted 28 March, 2022; v1 submitted 26 July, 2021;
originally announced July 2021.
-
Immersive Operation of a Semi-Autonomous Aerial Platform for Detecting and Mapping Radiation
Authors:
P. Dayani,
N. Orr,
A. Thomopoulos,
V. Saran,
S. Krishnaswamy,
E. Zhang,
N. Hu,
D. McPherson,
J. Menke,
A. Yang,
K. Vetter
Abstract:
Recent advancements in radiation detection and computer vision have enabled small unmanned aerial systems (sUASs) to produce 3D nuclear radiation maps in real-time. Currently these state-of-the-art systems still require two operators: one to pilot the sUAS and another operator to monitor the detected radiation. In this work we present a system that integrates real-time 3D radiation visualization w…
▽ More
Recent advancements in radiation detection and computer vision have enabled small unmanned aerial systems (sUASs) to produce 3D nuclear radiation maps in real-time. Currently these state-of-the-art systems still require two operators: one to pilot the sUAS and another operator to monitor the detected radiation. In this work we present a system that integrates real-time 3D radiation visualization with semi-autonomous sUAS control. Our Virtual Reality interface enables a single operator to define trajectories using waypoints to abstract complex flight control and utilize the semi-autonomous maneuvering capabilities of the sUAS. The interface also displays a fused radiation visualization and environment map, thereby enabling simultaneous remote operation and radiation monitoring by a single operator. This serves as the basis for development of a single system that deploys and autonomously controls fleets of sUASs.
△ Less
Submitted 18 March, 2021;
originally announced March 2021.
-
Diffusion Earth Mover's Distance and Distribution Embeddings
Authors:
Alexander Tong,
Guillaume Huguet,
Amine Natik,
Kincaid MacDonald,
Manik Kuchroo,
Ronald Coifman,
Guy Wolf,
Smita Krishnaswamy
Abstract:
We propose a new fast method of measuring distances between large numbers of related high dimensional datasets called the Diffusion Earth Mover's Distance (EMD). We model the datasets as distributions supported on common data graph that is derived from the affinity matrix computed on the combined data. In such cases where the graph is a discretization of an underlying Riemannian closed manifold, w…
▽ More
We propose a new fast method of measuring distances between large numbers of related high dimensional datasets called the Diffusion Earth Mover's Distance (EMD). We model the datasets as distributions supported on common data graph that is derived from the affinity matrix computed on the combined data. In such cases where the graph is a discretization of an underlying Riemannian closed manifold, we prove that Diffusion EMD is topologically equivalent to the standard EMD with a geodesic ground distance. Diffusion EMD can be computed in $\tilde{O}(n)$ time and is more accurate than similarly fast algorithms such as tree-based EMDs. We also show Diffusion EMD is fully differentiable, making it amenable to future uses in gradient-descent frameworks such as deep neural networks. Finally, we demonstrate an application of Diffusion EMD to single cell data collected from 210 COVID-19 patient samples at Yale New Haven Hospital. Here, Diffusion EMD can derive distances between patients on the manifold of cells at least two orders of magnitude faster than equally accurate methods. This distance matrix between patients can be embedded into a higher level patient manifold which uncovers structure and heterogeneity in patients. More generally, Diffusion EMD is applicable to all datasets that are massively collected in parallel in many medical and biological systems.
△ Less
Submitted 27 July, 2021; v1 submitted 25 February, 2021;
originally announced February 2021.
-
Multimodal Data Visualization and Denoising with Integrated Diffusion
Authors:
Manik Kuchroo,
Abhinav Godavarthi,
Alexander Tong,
Guy Wolf,
Smita Krishnaswamy
Abstract:
We propose a method called integrated diffusion for combining multimodal datasets, or data gathered via several different measurements on the same system, to create a joint data diffusion operator. As real world data suffers from both local and global noise, we introduce mechanisms to optimally calculate a diffusion operator that reflects the combined information from both modalities. We show the…
▽ More
We propose a method called integrated diffusion for combining multimodal datasets, or data gathered via several different measurements on the same system, to create a joint data diffusion operator. As real world data suffers from both local and global noise, we introduce mechanisms to optimally calculate a diffusion operator that reflects the combined information from both modalities. We show the utility of this joint operator in data denoising, visualization and clustering, performing better than other methods to integrate and analyze multimodal data. We apply our method to multi-omic data generated from blood cells, measuring both gene expression and chromatin accessibility. Our approach better visualizes the geometry of the joint data, captures known cross-modality associations and identifies known cellular populations. More generally, integrated diffusion is broadly applicable to multimodal datasets generated in many medical and biological systems.
△ Less
Submitted 3 March, 2022; v1 submitted 12 February, 2021;
originally announced February 2021.
-
Exploring the Geometry and Topology of Neural Network Loss Landscapes
Authors:
Stefan Horoi,
Jessie Huang,
Bastian Rieck,
Guillaume Lajoie,
Guy Wolf,
Smita Krishnaswamy
Abstract:
Recent work has established clear links between the generalization performance of trained neural networks and the geometry of their loss landscape near the local minima to which they converge. This suggests that qualitative and quantitative examination of the loss landscape geometry could yield insights about neural network generalization performance during training. To this end, researchers have…
▽ More
Recent work has established clear links between the generalization performance of trained neural networks and the geometry of their loss landscape near the local minima to which they converge. This suggests that qualitative and quantitative examination of the loss landscape geometry could yield insights about neural network generalization performance during training. To this end, researchers have proposed visualizing the loss landscape through the use of simple dimensionality reduction techniques. However, such visualization methods have been limited by their linear nature and only capture features in one or two dimensions, thus restricting sampling of the loss landscape to lines or planes. Here, we expand and improve upon these in three ways. First, we present a novel "jump and retrain" procedure for sampling relevant portions of the loss landscape. We show that the resulting sampled data holds more meaningful information about the network's ability to generalize. Next, we show that non-linear dimensionality reduction of the jump and retrain trajectories via PHATE, a trajectory and manifold-preserving method, allows us to visualize differences between networks that are generalizing well vs poorly. Finally, we combine PHATE trajectories with a computational homology characterization to quantify trajectory differences.
△ Less
Submitted 26 January, 2022; v1 submitted 31 January, 2021;
originally announced February 2021.
-
Data-Driven Learning of Geometric Scattering Networks
Authors:
Alexander Tong,
Frederik Wenkel,
Kincaid MacDonald,
Smita Krishnaswamy,
Guy Wolf
Abstract:
We propose a new graph neural network (GNN) module, based on relaxations of recently proposed geometric scattering transforms, which consist of a cascade of graph wavelet filters. Our learnable geometric scattering (LEGS) module enables adaptive tuning of the wavelets to encourage band-pass features to emerge in learned representations. The incorporation of our LEGS-module in GNNs enables the lear…
▽ More
We propose a new graph neural network (GNN) module, based on relaxations of recently proposed geometric scattering transforms, which consist of a cascade of graph wavelet filters. Our learnable geometric scattering (LEGS) module enables adaptive tuning of the wavelets to encourage band-pass features to emerge in learned representations. The incorporation of our LEGS-module in GNNs enables the learning of longer-range graph relations compared to many popular GNNs, which often rely on encoding graph structure via smoothness or similarity between neighbors. Further, its wavelet priors result in simplified architectures with significantly fewer learned parameters compared to competing GNNs. We demonstrate the predictive performance of LEGS-based networks on graph classification benchmarks, as well as the descriptive quality of their learned features in biochemical graph data exploration tasks.
△ Less
Submitted 28 March, 2022; v1 submitted 5 October, 2020;
originally announced October 2020.
-
On Passivity, Feedback Passivity, And Feedback Passivity Over Erasure Network: A Piecewise Affine Approximation Approach
Authors:
Abhijit Mazumdar,
Srinivasan Krishnaswamy,
Somanath Majhi
Abstract:
In this paper, we deal with the problem of passivity and feedback passification of smooth discrete-time nonlinear systems by considering their piecewise affine approximations. Sufficient conditions are derived for passivity and feedback passivity. These results are then extended to systems that operate over Gilbert-Elliott type communication channels. As a special case, results for feedback passiv…
▽ More
In this paper, we deal with the problem of passivity and feedback passification of smooth discrete-time nonlinear systems by considering their piecewise affine approximations. Sufficient conditions are derived for passivity and feedback passivity. These results are then extended to systems that operate over Gilbert-Elliott type communication channels. As a special case, results for feedback passivity of piecewise affine systems over a lossy channel are also derived.
△ Less
Submitted 6 August, 2020;
originally announced August 2020.
-
Image-to-image Mapping with Many Domains by Sparse Attribute Transfer
Authors:
Matthew Amodio,
Rim Assouel,
Victor Schmidt,
Tristan Sylvain,
Smita Krishnaswamy,
Yoshua Bengio
Abstract:
Unsupervised image-to-image translation consists of learning a pair of mappings between two domains without known pairwise correspondences between points. The current convention is to approach this task with cycle-consistent GANs: using a discriminator to encourage the generator to change the image to match the target domain, while training the generator to be inverted with another mapping. While…
▽ More
Unsupervised image-to-image translation consists of learning a pair of mappings between two domains without known pairwise correspondences between points. The current convention is to approach this task with cycle-consistent GANs: using a discriminator to encourage the generator to change the image to match the target domain, while training the generator to be inverted with another mapping. While ending up with paired inverse functions may be a good end result, enforcing this restriction at all times during training can be a hindrance to effective modeling. We propose an alternate approach that directly restricts the generator to performing a simple sparse transformation in a latent layer, motivated by recent work from cognitive neuroscience suggesting an architectural prior on representations corresponding to consciousness. Our biologically motivated approach leads to representations more amenable to transformation by disentangling high-level abstract concepts in the latent space. We demonstrate that image-to-image domain translation with many different domains can be learned more effectively with our architecturally constrained, simple transformation than with previous unconstrained architectures that rely on a cycle-consistency loss.
△ Less
Submitted 23 June, 2020;
originally announced June 2020.
-
Uncovering the Topology of Time-Varying fMRI Data using Cubical Persistence
Authors:
Bastian Rieck,
Tristan Yates,
Christian Bock,
Karsten Borgwardt,
Guy Wolf,
Nicholas Turk-Browne,
Smita Krishnaswamy
Abstract:
Functional magnetic resonance imaging (fMRI) is a crucial technology for gaining insights into cognitive processes in humans. Data amassed from fMRI measurements result in volumetric data sets that vary over time. However, analysing such data presents a challenge due to the large degree of noise and person-to-person variation in how information is represented in the brain. To address this challeng…
▽ More
Functional magnetic resonance imaging (fMRI) is a crucial technology for gaining insights into cognitive processes in humans. Data amassed from fMRI measurements result in volumetric data sets that vary over time. However, analysing such data presents a challenge due to the large degree of noise and person-to-person variation in how information is represented in the brain. To address this challenge, we present a novel topological approach that encodes each time point in an fMRI data set as a persistence diagram of topological features, i.e. high-dimensional voids present in the data. This representation naturally does not rely on voxel-by-voxel correspondence and is robust to noise. We show that these time-varying persistence diagrams can be clustered to find meaningful groupings between participants, and that they are also useful in studying within-subject brain state trajectories of subjects performing a particular task. Here, we apply both clustering and trajectory analysis techniques to a group of participants watching the movie 'Partly Cloudy'. We observe significant differences in both brain state trajectories and overall topological activity between adults and children watching the same movie.
△ Less
Submitted 22 October, 2020; v1 submitted 14 June, 2020;
originally announced June 2020.
-
Uncovering the Folding Landscape of RNA Secondary Structure with Deep Graph Embeddings
Authors:
Egbert Castro,
Andrew Benz,
Alexander Tong,
Guy Wolf,
Smita Krishnaswamy
Abstract:
Biomolecular graph analysis has recently gained much attention in the emerging field of geometric deep learning. Here we focus on organizing biomolecular graphs in ways that expose meaningful relations and variations between them. We propose a geometric scattering autoencoder (GSAE) network for learning such graph embeddings. Our embedding network first extracts rich graph features using the recen…
▽ More
Biomolecular graph analysis has recently gained much attention in the emerging field of geometric deep learning. Here we focus on organizing biomolecular graphs in ways that expose meaningful relations and variations between them. We propose a geometric scattering autoencoder (GSAE) network for learning such graph embeddings. Our embedding network first extracts rich graph features using the recently proposed geometric scattering transform. Then, it leverages a semi-supervised variational autoencoder to extract a low-dimensional embedding that retains the information in these features that enable prediction of molecular properties as well as characterize graphs. We show that GSAE organizes RNA graphs both by structure and energy, accurately reflecting bistable RNA structures. Also, the model is generative and can sample new folding trajectories.
△ Less
Submitted 28 March, 2022; v1 submitted 11 June, 2020;
originally announced June 2020.
-
The application of $σ$-LFSR in Key-Dependent Feedback Configuration for Word-Oriented Stream Ciphers
Authors:
Subrata Nandi,
Srinivasan Krishnaswamy,
Behrouz Zolfaghari,
Pinaki Mitra
Abstract:
In this paper, we propose and evaluate a method for generating key-dependent feedback configurations (KDFC) for $σ$-LFSRs. $σ$-LFSRs with such configurations can be applied to any stream cipher that uses a word-based LFSR. Here, a configuration generation algorithm uses the secret key(K) and the initialization vector (IV) to generate a feedback configuration. We have mathematically analysed the fe…
▽ More
In this paper, we propose and evaluate a method for generating key-dependent feedback configurations (KDFC) for $σ$-LFSRs. $σ$-LFSRs with such configurations can be applied to any stream cipher that uses a word-based LFSR. Here, a configuration generation algorithm uses the secret key(K) and the initialization vector (IV) to generate a feedback configuration. We have mathematically analysed the feedback configurations generated by this method. As a test case, we have applied this method on SNOW 2.0 and have studied its impact on resistance to various attacks. Further, we have also tested the generated keystream for randomness and have briefly described its implementation and the challenges involved in the same.
△ Less
Submitted 2 March, 2021; v1 submitted 20 March, 2020;
originally announced March 2020.
-
TrajectoryNet: A Dynamic Optimal Transport Network for Modeling Cellular Dynamics
Authors:
Alexander Tong,
Jessie Huang,
Guy Wolf,
David van Dijk,
Smita Krishnaswamy
Abstract:
It is increasingly common to encounter data from dynamic processes captured by static cross-sectional measurements over time, particularly in biomedical settings. Recent attempts to model individual trajectories from this data use optimal transport to create pairwise matchings between time points. However, these methods cannot model continuous dynamics and non-linear paths that entities can take i…
▽ More
It is increasingly common to encounter data from dynamic processes captured by static cross-sectional measurements over time, particularly in biomedical settings. Recent attempts to model individual trajectories from this data use optimal transport to create pairwise matchings between time points. However, these methods cannot model continuous dynamics and non-linear paths that entities can take in these systems. To address this issue, we establish a link between continuous normalizing flows and dynamic optimal transport, that allows us to model the expected paths of points over time. Continuous normalizing flows are generally under constrained, as they are allowed to take an arbitrary path from the source to the target distribution. We present TrajectoryNet, which controls the continuous paths taken between distributions to produce dynamic optimal transport. We show how this is particularly applicable for studying cellular dynamics in data from single-cell RNA sequencing (scRNA-seq) technologies, and that TrajectoryNet improves upon recently proposed static optimal transport-based models that can be used for interpolating cellular distributions.
△ Less
Submitted 26 July, 2020; v1 submitted 9 February, 2020;
originally announced February 2020.