-
Unleashing the Power of LLMs as Multi-Modal Encoders for Text and Graph-Structured Data
Authors:
Jiacheng Lin,
Kun Qian,
Haoyu Han,
Nurendra Choudhary,
Tianxin Wei,
Zhongruo Wang,
Sahika Genc,
Edward W Huang,
Sheng Wang,
Karthik Subbian,
Danai Koutra,
Jimeng Sun
Abstract:
Graph-structured information offers rich contextual information that can enhance language models by providing structured relationships and hierarchies, leading to more expressive embeddings for various applications such as retrieval, question answering, and classification. However, existing methods for integrating graph and text embeddings, often based on Multi-layer Perceptrons (MLPs) or shallow…
▽ More
Graph-structured information offers rich contextual information that can enhance language models by providing structured relationships and hierarchies, leading to more expressive embeddings for various applications such as retrieval, question answering, and classification. However, existing methods for integrating graph and text embeddings, often based on Multi-layer Perceptrons (MLPs) or shallow transformers, are limited in their ability to fully exploit the heterogeneous nature of these modalities. To overcome this, we propose Janus, a simple yet effective framework that leverages Large Language Models (LLMs) to jointly encode text and graph data. Specifically, Janus employs an MLP adapter to project graph embeddings into the same space as text embeddings, allowing the LLM to process both modalities jointly. Unlike prior work, we also introduce contrastive learning to align the graph and text spaces more effectively, thereby improving the quality of learned joint embeddings. Empirical results across six datasets spanning three tasks, knowledge graph-contextualized question answering, graph-text pair classification, and retrieval, demonstrate that Janus consistently outperforms existing baselines, achieving significant improvements across multiple datasets, with gains of up to 11.4% in QA tasks. These results highlight Janus's effectiveness in integrating graph and text data. Ablation studies further validate the effectiveness of our method.
△ Less
Submitted 14 October, 2024;
originally announced October 2024.
-
PSA: Private Set Alignment for Secure and Collaborative Analytics on Large-Scale Data
Authors:
Jiabo Wang,
Elmo Xuyun Huang,
Pu Duan,
Huaxiong Wang,
Kwok-Yan Lam
Abstract:
Enforcement of privacy regulation is essential for collaborative data analytics. In this work, we address a scenario in which two companies expect to securely join their datasets with respect to their common customers to maximize data insights. Apart from the necessary protection of raw data, it becomes more challenging to protect the identities and attributes of common customers, as it requires p…
▽ More
Enforcement of privacy regulation is essential for collaborative data analytics. In this work, we address a scenario in which two companies expect to securely join their datasets with respect to their common customers to maximize data insights. Apart from the necessary protection of raw data, it becomes more challenging to protect the identities and attributes of common customers, as it requires participants to align their records associated with common customers without knowing who they are. We proposed a solution, dubbed PSA, for this scenario, which is effectively applicable to real-world use cases, such as evaluating advertising conversion using data from both publishers and merchants. The contributions of this work are threefold: 1. We defined the notion of PSA with two levels of privacy protection and proposed novel PSA protocols based on the modified oblivious switching network, which leverages efficient symmetric key operations and offline precomputation to save online run time. 2. We implemented and benchmarked the proposed protocols in different network conditions by joining two datasets, each at the scale of one million records, in 35.5 sec on a single thread with a network bandwidth of 500 Mbps, resulting in an X100 improvement over the existing Homomorphic based protocols. 3. We give new proof for an algorithm of quasi-linear complexity that constructs an oblivious switching network to achieve a target permutation distinct from the existing one in the literature.
△ Less
Submitted 7 October, 2024;
originally announced October 2024.
-
Charge Susceptibility and Kubo Response in Hatsugai-Kohmoto-related Models
Authors:
Yuhao Ma,
Jinchao Zhao,
Edwin W. Huang,
Dhruv Kush,
Barry Bradlyn,
Philip W. Phillips
Abstract:
We study in depth the charge susceptibility for the band Hatsugai-Kohmoto (HK) and orbital (OHK) models. As either of these models describes a Mott insulator, the charge susceptibility takes on the form of a modified Lindhard function with lower and upper Hubbard bands, thereby giving rise to a multi-pole structure. The particle-hole continuum consists of hot spots along the $ω$ vs $q$ axis arisin…
▽ More
We study in depth the charge susceptibility for the band Hatsugai-Kohmoto (HK) and orbital (OHK) models. As either of these models describes a Mott insulator, the charge susceptibility takes on the form of a modified Lindhard function with lower and upper Hubbard bands, thereby giving rise to a multi-pole structure. The particle-hole continuum consists of hot spots along the $ω$ vs $q$ axis arising from inter-band transitions. Such transitions, which are strongly suppressed in non-interacting systems, are obtained here because of the non-rigidity of the Hubbard bands. This modified Lindhard function gives rise to a plasmon dispersion that is inversely dependent on the momentum, resulting in an additional contribution to the conventional f-sum rule. This extra contribution originates from a long-range diamagnetic contribution to the current. This results in a non-commutativity of the long-wavelength ($q\rightarrow 0$) and thermodynamic ($L\rightarrow\infty$) limits. When the correct limits are taken, we find that the Kubo response computed with either open or periodic boundary conditions yields identical results that are consistent with the continuity equation contrary to recent claims. We also show that the long wavelength pathology of the current noted previously also plagues the Anderson impurity model interpretation of dynamical mean-field theory (DMFT). Coupled with our previous work\cite{mai20231} which showed that HK is the correct $d=\infty$ limit of the Hubbard model, we arrive at the conclusion that single-orbital HK=DMFT.
△ Less
Submitted 27 September, 2024; v1 submitted 11 September, 2024;
originally announced September 2024.
-
Covering a Graph with Dense Subgraph Families, via Triangle-Rich Sets
Authors:
Sabyasachi Basu,
Daniel Paul-Pena,
Kun Qian,
C. Seshadhri,
Edward W Huang,
Karthik Subbian
Abstract:
Graphs are a fundamental data structure used to represent relationships in domains as diverse as the social sciences, bioinformatics, cybersecurity, the Internet, and more. One of the central observations in network science is that real-world graphs are globally sparse, yet contains numerous "pockets" of high edge density. A fundamental task in graph mining is to discover these dense subgraphs. Mo…
▽ More
Graphs are a fundamental data structure used to represent relationships in domains as diverse as the social sciences, bioinformatics, cybersecurity, the Internet, and more. One of the central observations in network science is that real-world graphs are globally sparse, yet contains numerous "pockets" of high edge density. A fundamental task in graph mining is to discover these dense subgraphs. Most common formulations of the problem involve finding a single (or a few) "optimally" dense subsets. But in most real applications, one does not care for the optimality. Instead, we want to find a large collection of dense subsets that covers a significant fraction of the input graph.
We give a mathematical formulation of this problem, using a new definition of regularly triangle-rich (RTR) families. These families capture the notion of dense subgraphs that contain many triangles and have degrees comparable to the subgraph size. We design a provable algorithm, RTRExtractor, that can discover RTR families that approximately cover any RTR set. The algorithm is efficient and is inspired by recent results that use triangle counts for community testing and clustering.
We show that RTRExtractor has excellent behavior on a large variety of real-world datasets. It is able to process graphs with hundreds of millions of edges within minutes. Across many datasets, RTRExtractor achieves high coverage using high edge density datasets. For example, the output covers a quarter of the vertices with subgraphs of edge density more than (say) $0.5$, for datasets with 10M+ edges. We show an example of how the output of RTRExtractor correlates with meaningful sets of similar vertices in a citation network, demonstrating the utility of RTRExtractor for unsupervised graph discovery tasks.
△ Less
Submitted 23 July, 2024;
originally announced July 2024.
-
All Against Some: Efficient Integration of Large Language Models for Message Passing in Graph Neural Networks
Authors:
Ajay Jaiswal,
Nurendra Choudhary,
Ravinarayana Adkathimar,
Muthu P. Alagappan,
Gaurush Hiranandani,
Ying Ding,
Zhangyang Wang,
Edward W Huang,
Karthik Subbian
Abstract:
Graph Neural Networks (GNNs) have attracted immense attention in the past decade due to their numerous real-world applications built around graph-structured data. On the other hand, Large Language Models (LLMs) with extensive pretrained knowledge and powerful semantic comprehension abilities have recently shown a remarkable ability to benefit applications using vision and text data. In this paper,…
▽ More
Graph Neural Networks (GNNs) have attracted immense attention in the past decade due to their numerous real-world applications built around graph-structured data. On the other hand, Large Language Models (LLMs) with extensive pretrained knowledge and powerful semantic comprehension abilities have recently shown a remarkable ability to benefit applications using vision and text data. In this paper, we investigate how LLMs can be leveraged in a computationally efficient fashion to benefit rich graph-structured data, a modality relatively unexplored in LLM literature. Prior works in this area exploit LLMs to augment every node features in an ad-hoc fashion (not scalable for large graphs), use natural language to describe the complex structural information of graphs, or perform computationally expensive finetuning of LLMs in conjunction with GNNs. We propose E-LLaGNN (Efficient LLMs augmented GNNs), a framework with an on-demand LLM service that enriches message passing procedure of graph learning by enhancing a limited fraction of nodes from the graph. More specifically, E-LLaGNN relies on sampling high-quality neighborhoods using LLMs, followed by on-demand neighborhood feature enhancement using diverse prompts from our prompt catalog, and finally information aggregation using message passing from conventional GNN architectures. We explore several heuristics-based active node selection strategies to limit the computational and memory footprint of LLMs when handling millions of nodes. Through extensive experiments & ablation on popular graph benchmarks of varying scales (Cora, PubMed, ArXiv, & Products), we illustrate the effectiveness of our E-LLaGNN framework and reveal many interesting capabilities such as improved gradient flow in deep GNNs, LLM-free inference ability etc.
△ Less
Submitted 20 July, 2024;
originally announced July 2024.
-
Improving Robustness and Clinical Applicability of Automatic Respiratory Sound Classification Using Deep Learning-Based Audio Enhancement: Algorithm Development and Validation Study
Authors:
Jing-Tong Tzeng,
Jeng-Lin Li,
Huan-Yu Chen,
Chun-Hsiang Huang,
Chi-Hsin Chen,
Cheng-Yi Fan,
Edward Pei-Chuan Huang,
Chi-Chun Lee
Abstract:
Deep learning techniques have shown promising results in the automatic classification of respiratory sounds. However, accurately distinguishing these sounds in real-world noisy conditions poses challenges for clinical deployment. Additionally, predicting signals with only background noise could undermine user trust in the system. This paper aims to investigate the feasibility and effectiveness of…
▽ More
Deep learning techniques have shown promising results in the automatic classification of respiratory sounds. However, accurately distinguishing these sounds in real-world noisy conditions poses challenges for clinical deployment. Additionally, predicting signals with only background noise could undermine user trust in the system. This paper aims to investigate the feasibility and effectiveness of incorporating a deep learning-based audio enhancement preprocessing step into automatic respiratory sound classification systems to improve robustness and clinical applicability. Multiple experiments were conducted using different audio enhancement model structures and classification models. The classification performance was compared to the baseline method of noise injection data augmentation. Experiments were performed on two datasets: the ICBHI respiratory sound dataset, which includes 5.5 hours of recordings, and the Formosa Archive of Breath Sounds (FABS) dataset, comprising 14.6 hours of recordings. Additionally, a physician validation study was conducted by 7 senior physicians to assess the clinical utility of the system.The integration of the audio enhancement pipeline resulted in a 21.88% increase in the ICBHI classification score on the ICBHI dataset and a 4.10% improvement on the FABS dataset in multi-class noisy scenarios. Quantitative analysis from the physician validation study revealed improvements in efficiency, diagnostic confidence, and trust during model-assisted diagnosis, with workflows integrating enhanced audio leading to an 11.61% increase in diagnostic sensitivity and facilitating high-confidence diagnoses. Incorporating an audio enhancement algorithm significantly enhances the robustness and clinical utility of automatic respiratory sound classification systems, improving performance in noisy environments and fostering greater trust among medical professionals.
△ Less
Submitted 7 October, 2024; v1 submitted 18 July, 2024;
originally announced July 2024.
-
Real-time Digital RF Emulation -- I: The Direct Path Computational Model
Authors:
Coleman DeLude,
Joe Driscoll,
Mandovi Mukherjee,
Nael Rahman,
Uday Kamal,
Xiangyu Mao,
Sharjeel Khan,
Hariharan Sivaraman,
Eric Huang,
Jeffrey McHarg,
Madhavan Swaminathan,
Santosh Pande,
Saibal Mukhopadhyay,
Justin Romberg
Abstract:
In this paper we consider the problem of developing a computational model for emulating an RF channel. The motivation for this is that an accurate and scalable emulator has the potential to minimize the need for field testing, which is expensive, slow, and difficult to replicate. Traditionally, emulators are built using a tapped delay line model where long filters modeling the physical interaction…
▽ More
In this paper we consider the problem of developing a computational model for emulating an RF channel. The motivation for this is that an accurate and scalable emulator has the potential to minimize the need for field testing, which is expensive, slow, and difficult to replicate. Traditionally, emulators are built using a tapped delay line model where long filters modeling the physical interactions of objects are implemented directly. For an emulation scenario consisting of $M$ objects all interacting with one another, the tapped delay line model's computational requirements scale as $O(M^3)$ per sample: there are $O(M^2)$ channels, each with $O(M)$ complexity. In this paper, we develop a new ``direct path" model that, while remaining physically faithful, allows us to carefully factor the emulator operations, resulting in an $O(M^2)$ per sample scaling of the computational requirements. The impact of this is drastic, a $200$ object scenario sees about a $100\times$ reduction in the number of per sample computations. Furthermore, the direct path model gives us a natural way to distribute the computations for an emulation: each object is mapped to a computational node, and these nodes are networked in a fully connected communication graph.
Alongside a discussion of the model and the physical phenomena it emulates, we show how to efficiently parameterize antenna responses and scattering profiles within this direct path framework. To verify the model and demonstrate its viability in hardware, we provide several numerical experiments produced using a cycle level C++ simulator of a hardware implementation of the model.
△ Less
Submitted 12 June, 2024;
originally announced June 2024.
-
Machine learning aided parameter analysis in Perovskite X-ray Detector
Authors:
Bobo Zhang,
Endai Huang,
Xinyi Du,
Xiaokang Ma,
Lu Zhang,
Jiaxue You,
Alex K. Y. Jen,
Shengzhong,
Liu
Abstract:
Many factors in perovskite X-ray detectors, such as crystal lattice and carrier dynamics, determine the final device performance (e.g., sensitivity and detection limit). However, the relationship between these factors remains unknown due to the complexity of the material. In this study, we employ machine learning to reveal the relationship between 15 intrinsic properties of halide perovskite mater…
▽ More
Many factors in perovskite X-ray detectors, such as crystal lattice and carrier dynamics, determine the final device performance (e.g., sensitivity and detection limit). However, the relationship between these factors remains unknown due to the complexity of the material. In this study, we employ machine learning to reveal the relationship between 15 intrinsic properties of halide perovskite materials and their device performance. We construct a database of X-ray detectors for the training of machine learning. The results show that the band gap is mainly influenced by the atomic number of the B-site metal, and the lattice length parameter b has the greatest impact on the carrier mobility-lifetime product (μτ). An X-ray detector (m-F-PEA)2PbI4 were generated in the experiment and it further verified the accuracy of our ML models. We suggest further study on random forest regression for X-ray detector applications.
△ Less
Submitted 7 May, 2024;
originally announced May 2024.
-
MathNet: A Data-Centric Approach for Printed Mathematical Expression Recognition
Authors:
Felix M. Schmitt-Koopmann,
Elaine M. Huang,
Hans-Peter Hutter,
Thilo Stadelmann,
Alireza Darvishy
Abstract:
Printed mathematical expression recognition (MER) models are usually trained and tested using LaTeX-generated mathematical expressions (MEs) as input and the LaTeX source code as ground truth. As the same ME can be generated by various different LaTeX source codes, this leads to unwanted variations in the ground truth data that bias test performance results and hinder efficient learning. In additi…
▽ More
Printed mathematical expression recognition (MER) models are usually trained and tested using LaTeX-generated mathematical expressions (MEs) as input and the LaTeX source code as ground truth. As the same ME can be generated by various different LaTeX source codes, this leads to unwanted variations in the ground truth data that bias test performance results and hinder efficient learning. In addition, the use of only one font to generate the MEs heavily limits the generalization of the reported results to realistic scenarios. We propose a data-centric approach to overcome this problem, and present convincing experimental results: Our main contribution is an enhanced LaTeX normalization to map any LaTeX ME to a canonical form. Based on this process, we developed an improved version of the benchmark dataset im2latex-100k, featuring 30 fonts instead of one. Second, we introduce the real-world dataset realFormula, with MEs extracted from papers. Third, we developed a MER model, MathNet, based on a convolutional vision transformer, with superior results on all four test sets (im2latex-100k, im2latexv2, realFormula, and InftyMDB-1), outperforming the previous state of the art by up to 88.3%.
△ Less
Submitted 21 April, 2024;
originally announced April 2024.
-
Enhanced pair-density-wave vertices in a bilayer Hubbard model at half-filling
Authors:
Fangze Liu,
Xu-Xin Huang,
Edwin W. Huang,
Brian Moritz,
Thomas P. Devereaux
Abstract:
Motivated by the pair-density-wave (PDW) state found in the one-dimensional Kondo-Heisenberg chain, we report on a determinant quantum Monte Carlo DQMC study of pair-fields for a two-dimensional half-filled Hubbard layer coupled to an itinerant, non-interacting layer with one electron per site. In a specific range of interlayer hopping, the pairing vertex associated with PDW order becomes more att…
▽ More
Motivated by the pair-density-wave (PDW) state found in the one-dimensional Kondo-Heisenberg chain, we report on a determinant quantum Monte Carlo DQMC study of pair-fields for a two-dimensional half-filled Hubbard layer coupled to an itinerant, non-interacting layer with one electron per site. In a specific range of interlayer hopping, the pairing vertex associated with PDW order becomes more attractive than that for uniform d-wave pairing, although both remain subdominant to the leading antiferromagnetic correlations at half-filling. Our result sheds light on where one potentially may find a PDW state in such a model.
△ Less
Submitted 9 October, 2024; v1 submitted 1 April, 2024;
originally announced April 2024.
-
Protocols and Trade-Offs of Quantum State Purification
Authors:
Hongshun Yao,
Yu-Ao Chen,
Erdong Huang,
Kaichu Chen,
Honghao Fu,
Xin Wang
Abstract:
Quantum state purification is crucial in quantum communication and computation, aiming to recover a purified state from multiple copies of an unknown noisy state. This work introduces a general state purification framework designed to achieve the highest fidelity with a specified probability and characterize the associated trade-offs. For i.i.d. quantum states under depolarizing noise, our framewo…
▽ More
Quantum state purification is crucial in quantum communication and computation, aiming to recover a purified state from multiple copies of an unknown noisy state. This work introduces a general state purification framework designed to achieve the highest fidelity with a specified probability and characterize the associated trade-offs. For i.i.d. quantum states under depolarizing noise, our framework can replicate the purification protocol proposed by [Barenco et al., SIAM Journal on Computing, 26(5), 1997] and further provide exact formulas for the purification fidelity and probability with explicit trade-offs. We prove the protocols' optimality for two copies of noisy states with any dimension and confirm its optimality for higher numbers of copies and dimensions through numerical analysis. Our methodological approach paves the way for proving the protocol's optimality in more general scenarios and leads to optimal protocols for other noise models. Furthermore, we present a systematic implementation method via block encoding and parameterized quantum circuits, providing explicit circuits for purifying three-copy and four-copy states under depolarizing noise. Finally, we estimate the sample complexity and generalize the protocol to a recursive form, demonstrating its practicality for quantum computers with limited memory.
△ Less
Submitted 26 September, 2024; v1 submitted 1 April, 2024;
originally announced April 2024.
-
An Interpretable Ensemble of Graph and Language Models for Improving Search Relevance in E-Commerce
Authors:
Nurendra Choudhary,
Edward W Huang,
Karthik Subbian,
Chandan K. Reddy
Abstract:
The problem of search relevance in the E-commerce domain is a challenging one since it involves understanding the intent of a user's short nuanced query and matching it with the appropriate products in the catalog. This problem has traditionally been addressed using language models (LMs) and graph neural networks (GNNs) to capture semantic and inter-product behavior signals, respectively. However,…
▽ More
The problem of search relevance in the E-commerce domain is a challenging one since it involves understanding the intent of a user's short nuanced query and matching it with the appropriate products in the catalog. This problem has traditionally been addressed using language models (LMs) and graph neural networks (GNNs) to capture semantic and inter-product behavior signals, respectively. However, the rapid development of new architectures has created a gap between research and the practical adoption of these techniques. Evaluating the generalizability of these models for deployment requires extensive experimentation on complex, real-world datasets, which can be non-trivial and expensive. Furthermore, such models often operate on latent space representations that are incomprehensible to humans, making it difficult to evaluate and compare the effectiveness of different models. This lack of interpretability hinders the development and adoption of new techniques in the field. To bridge this gap, we propose Plug and Play Graph LAnguage Model (PP-GLAM), an explainable ensemble of plug and play models. Our approach uses a modular framework with uniform data processing pipelines. It employs additive explanation metrics to independently decide whether to include (i) language model candidates, (ii) GNN model candidates, and (iii) inter-product behavioral signals. For the task of search relevance, we show that PP-GLAM outperforms several state-of-the-art baselines as well as a proprietary model on real-world multilingual, multi-regional e-commerce datasets. To promote better model comprehensibility and adoption, we also provide an analysis of the explainability and computational complexity of our model. We also provide the public codebase and provide a deployment strategy for practical implementation.
△ Less
Submitted 1 March, 2024;
originally announced March 2024.
-
Triad: A Framework Leveraging a Multi-Role LLM-based Agent to Solve Knowledge Base Question Answering
Authors:
Chang Zong,
Yuchen Yan,
Weiming Lu,
Jian Shao,
Eliot Huang,
Heng Chang,
Yueting Zhuang
Abstract:
Recent progress with LLM-based agents has shown promising results across various tasks. However, their use in answering questions from knowledge bases remains largely unexplored. Implementing a KBQA system using traditional methods is challenging due to the shortage of task-specific training data and the complexity of creating task-focused model structures. In this paper, we present Triad, a unifi…
▽ More
Recent progress with LLM-based agents has shown promising results across various tasks. However, their use in answering questions from knowledge bases remains largely unexplored. Implementing a KBQA system using traditional methods is challenging due to the shortage of task-specific training data and the complexity of creating task-focused model structures. In this paper, we present Triad, a unified framework that utilizes an LLM-based agent with three roles for KBQA tasks. The agent is assigned three roles to tackle different KBQA subtasks: agent as a generalist for mastering various subtasks, as a decision maker for the selection of candidates, and as an advisor for answering questions with knowledge. Our KBQA framework is executed in four phases, involving the collaboration of the agent's multiple roles. We evaluated the performance of our framework using three benchmark datasets, and the results show that our framework outperforms state-of-the-art systems on the LC-QuAD and YAGO-QA benchmarks, yielding F1 scores of 11.8% and 20.7%, respectively.
△ Less
Submitted 28 September, 2024; v1 submitted 22 February, 2024;
originally announced February 2024.
-
The emergence of antiferromagnetic correlations and Kondo-like features in a two-band model for infinite-layer nickelates
Authors:
Fangze Liu,
Cheng Peng,
Edwin W. Huang,
Brian Moritz,
Chunjing Jia,
Thomas P. Devereaux
Abstract:
We report a determinant quantum Monte Carlo study of a two-band model, inspired by infinite-layer nickelates, focusing on the influence of interlayer hybridization between $3d_{x^2-y^2}$ orbitals derived from Ni (or Ni and O) in one layer and rare-earth ($R$) 5d orbitals in the other layer, hereafter the NI and $R$ layers, respectively. For a filling with one electron shared between the two layers…
▽ More
We report a determinant quantum Monte Carlo study of a two-band model, inspired by infinite-layer nickelates, focusing on the influence of interlayer hybridization between $3d_{x^2-y^2}$ orbitals derived from Ni (or Ni and O) in one layer and rare-earth ($R$) 5d orbitals in the other layer, hereafter the NI and $R$ layers, respectively. For a filling with one electron shared between the two layers on average, interlayer hybridization leads to "self-doped" holes in the Ni layer and the absence of antiferromagnetic ordering, but rather the appearance of spin-density and charge-density stripe-like states. As the interlayer hybridization increases, both the Ni and $R$ layers develop antiferromagnetic correlations, even though either layer individually remains away from half-filling. For hybridization within an intermediate range, roughly comparable to the intralayer nearest-neighbor hopping $t_{\text{Ni}}$, the model develops signatures of Kondo-like physics.
△ Less
Submitted 30 April, 2024; v1 submitted 24 January, 2024;
originally announced January 2024.
-
Graph Coarsening via Convolution Matching for Scalable Graph Neural Network Training
Authors:
Charles Dickens,
Eddie Huang,
Aishwarya Reganti,
Jiong Zhu,
Karthik Subbian,
Danai Koutra
Abstract:
Graph summarization as a preprocessing step is an effective and complementary technique for scalable graph neural network (GNN) training. In this work, we propose the Coarsening Via Convolution Matching (CONVMATCH) algorithm and a highly scalable variant, A-CONVMATCH, for creating summarized graphs that preserve the output of graph convolution. We evaluate CONVMATCH on six real-world link predicti…
▽ More
Graph summarization as a preprocessing step is an effective and complementary technique for scalable graph neural network (GNN) training. In this work, we propose the Coarsening Via Convolution Matching (CONVMATCH) algorithm and a highly scalable variant, A-CONVMATCH, for creating summarized graphs that preserve the output of graph convolution. We evaluate CONVMATCH on six real-world link prediction and node classification graph datasets, and show it is efficient and preserves prediction performance while significantly reducing the graph size. Notably, CONVMATCH achieves up to 95% of the prediction performance of GNNs on node classification while trained on graphs summarized down to 1% the size of the original graph. Furthermore, on link prediction tasks, CONVMATCH consistently outperforms all baselines, achieving up to a 2x improvement.
△ Less
Submitted 24 December, 2023;
originally announced December 2023.
-
Minimum Decomposition on Maxmin Trees
Authors:
Emmy Huang,
Ray Tang
Abstract:
Maxmin trees are trees that consist of nodes that are either local minimums or maximums. Such trees were first studied by Postnikov. Later Dugan, Glennon, Gunnells, and Steingrimsson introduced the concept of weight to these trees and proved a bijection between maximum weight maxmin trees and permutations, defining weights for permutations. In addition, the q-Eulerian polynomial $E_n(x, q)$ is def…
▽ More
Maxmin trees are trees that consist of nodes that are either local minimums or maximums. Such trees were first studied by Postnikov. Later Dugan, Glennon, Gunnells, and Steingrimsson introduced the concept of weight to these trees and proved a bijection between maximum weight maxmin trees and permutations, defining weights for permutations. In addition, the q-Eulerian polynomial $E_n(x, q)$ is defined which relates descents and weights of permutations. This polynomial was later proven to exhibit a stabilization phenomenon by Agrawal et al. Extracting the formal power series $W_d(t)$ from the stabilization of these coefficients, $W_d(t)$ was conjectured to partially correspond to A256193. In our paper, we introduce a process called minimum decomposition to help us better understand maxmin trees. Using minimum decomposition, we present a new way to calculate the weight of different maxmin trees and prove the bijection between the coefficients of $W_d(t)$ and A256193.
△ Less
Submitted 24 October, 2023; v1 submitted 22 October, 2023;
originally announced October 2023.
-
ForeSeer: Product Aspect Forecasting Using Temporal Graph Embedding
Authors:
Zixuan Liu,
Gaurush Hiranandani,
Kun Qian,
Eddie W. Huang,
Yi Xu,
Belinda Zeng,
Karthik Subbian,
Sheng Wang
Abstract:
Developing text mining approaches to mine aspects from customer reviews has been well-studied due to its importance in understanding customer needs and product attributes. In contrast, it remains unclear how to predict the future emerging aspects of a new product that currently has little review information. This task, which we named product aspect forecasting, is critical for recommending new pro…
▽ More
Developing text mining approaches to mine aspects from customer reviews has been well-studied due to its importance in understanding customer needs and product attributes. In contrast, it remains unclear how to predict the future emerging aspects of a new product that currently has little review information. This task, which we named product aspect forecasting, is critical for recommending new products, but also challenging because of the missing reviews. Here, we propose ForeSeer, a novel textual mining and product embedding approach progressively trained on temporal product graphs for this novel product aspect forecasting task. ForeSeer transfers reviews from similar products on a large product graph and exploits these reviews to predict aspects that might emerge in future reviews. A key novelty of our method is to jointly provide review, product, and aspect embeddings that are both time-sensitive and less affected by extremely imbalanced aspect frequencies. We evaluated ForeSeer on a real-world product review system containing 11,536,382 reviews and 11,000 products over 3 years. We observe that ForeSeer substantially outperformed existing approaches with at least 49.1\% AUPRC improvement under the real setting where aspect associations are not given. ForeSeer further improves future link prediction on the product graph and the review aspect association prediction. Collectively, Foreseer offers a novel framework for review forecasting by effectively integrating review text, product network, and temporal information, opening up new avenues for online shopping recommendation and e-commerce applications.
△ Less
Submitted 7 October, 2023;
originally announced October 2023.
-
Charge-Density-Wave State in Extremely Overdoped Cuprates Driven by Phonons
Authors:
Jiarui Liu,
Shaozhi Li,
Edwin Huang,
Yao Wang
Abstract:
Recent resonant x-ray scattering (RXS) experiments revealed a novel charge order in extremely overdoped La$_{2-x}$Sr$_x$CuO$_4$ (LSCO) [Phys. Rev. Lett. 131,116002]. The observed charge order appears around the $(π/3,0)$ wavevector, distinct from the well-known stripe fluctuations near 1/8 doping, and persists from cryogenic temperatures to room temperature. To investigate the origin of this charg…
▽ More
Recent resonant x-ray scattering (RXS) experiments revealed a novel charge order in extremely overdoped La$_{2-x}$Sr$_x$CuO$_4$ (LSCO) [Phys. Rev. Lett. 131,116002]. The observed charge order appears around the $(π/3,0)$ wavevector, distinct from the well-known stripe fluctuations near 1/8 doping, and persists from cryogenic temperatures to room temperature. To investigate the origin of this charge order in the overdoped regime, we use determinant quantum Monte Carlo (DQMC) simulations to examine correlated models with various interactions. We demonstrate that this distinctive CDW originates from remnant correlations in extremely overdoped cuprates, with its specific pattern shaped by interactions beyond the Hubbard model, particularly electron-phonon couplings. The persistence of the $(π/3,0)$ wavevector across different doping levels indicates the presence of nonlocal couplings. Our study reveals the significant role of phonons in cuprates, which assist correlated electrons in the formation of unconventional phases.
△ Less
Submitted 1 October, 2024; v1 submitted 25 September, 2023;
originally announced September 2023.
-
Particle-hole asymmetric ferromagnetism and spin textures in the triangular Hubbard-Hofstadter model
Authors:
Jixun K. Ding,
Luhang Yang,
Wen O. Wang,
Ziyan Zhu,
Cheng Peng,
Peizhi Mai,
Edwin W. Huang,
Brian Moritz,
Philip W. Phillips,
Benjamin E. Feldman,
Thomas P. Devereaux
Abstract:
In a lattice model subject to a perpendicular magnetic field, when the lattice constant is comparable to the magnetic length, one enters the "Hofstadter regime," where continuum Landau levels become fractal magnetic Bloch bands. Strong mixing between bands alters the nature of the resulting quantum phases compared to the continuum limit; lattice potential, magnetic field, and Coulomb interaction m…
▽ More
In a lattice model subject to a perpendicular magnetic field, when the lattice constant is comparable to the magnetic length, one enters the "Hofstadter regime," where continuum Landau levels become fractal magnetic Bloch bands. Strong mixing between bands alters the nature of the resulting quantum phases compared to the continuum limit; lattice potential, magnetic field, and Coulomb interaction must be treated on equal footing. Using determinant quantum Monte Carlo (DQMC) and density matrix renormalization group (DMRG) techniques, we study this regime numerically in the context of the Hubbard-Hofstadter model on a triangular lattice. In the field-filling phase diagram, we find a broad wedge-shaped region of ferromagnetic ground states for filling factor $ν\leq 1$, bounded below by filling factor $ν= 1$ and bounded above by half-filling the lowest Hofstadter subband. We observe signatures of SU(2) quantum Hall ferromagnetism at filling factors $ν=1$ and $ν=3$. The phases near $ν=1$ are particle-hole asymmetric, and we observe a rapid decrease in ground state spin polarization consistent with the formation of skyrmions only on the electron doped side. At large fields, above the ferromagnetic wedge, we observe a low-spin metallic region with spin correlations peaked at small momenta. We argue that the phenomenology of this region likely results from exchange interaction mixing fractal Hofstadter subbands. The phase diagram derived beyond the continuum limit points to a rich landscape to explore interaction effects in magnetic Bloch bands.
△ Less
Submitted 1 March, 2024; v1 submitted 14 September, 2023;
originally announced September 2023.
-
Video Task Decathlon: Unifying Image and Video Tasks in Autonomous Driving
Authors:
Thomas E. Huang,
Yifan Liu,
Luc Van Gool,
Fisher Yu
Abstract:
Performing multiple heterogeneous visual tasks in dynamic scenes is a hallmark of human perception capability. Despite remarkable progress in image and video recognition via representation learning, current research still focuses on designing specialized networks for singular, homogeneous, or simple combination of tasks. We instead explore the construction of a unified model for major image and vi…
▽ More
Performing multiple heterogeneous visual tasks in dynamic scenes is a hallmark of human perception capability. Despite remarkable progress in image and video recognition via representation learning, current research still focuses on designing specialized networks for singular, homogeneous, or simple combination of tasks. We instead explore the construction of a unified model for major image and video recognition tasks in autonomous driving with diverse input and output structures. To enable such an investigation, we design a new challenge, Video Task Decathlon (VTD), which includes ten representative image and video tasks spanning classification, segmentation, localization, and association of objects and pixels. On VTD, we develop our unified network, VTDNet, that uses a single structure and a single set of weights for all ten tasks. VTDNet groups similar tasks and employs task interaction stages to exchange information within and between task groups. Given the impracticality of labeling all tasks on all frames, and the performance degradation associated with joint training of many tasks, we design a Curriculum training, Pseudo-labeling, and Fine-tuning (CPF) scheme to successfully train VTDNet on all tasks and mitigate performance loss. Armed with CPF, VTDNet significantly outperforms its single-task counterparts on most tasks with only 20% overall computations. VTD is a promising new direction for exploring the unification of perception tasks in autonomous driving.
△ Less
Submitted 26 November, 2023; v1 submitted 8 September, 2023;
originally announced September 2023.
-
Testing Meson Portal Dark Sector Solutions to the MiniBooNE Anomaly at CCM
Authors:
A. A. Aguilar-Arevalo,
S. Biedron,
J. Boissevain,
M. Borrego,
L. Bugel,
M. Chavez-Estrada,
J. M. Conrad,
R. L. Cooper,
A. Diaz,
J. R. Distel,
J. C. D'Olivo,
E. Dunton,
B. Dutta,
D. Fields,
J. R. Gochanour,
M. Gold,
E. Guardincerri,
E. C. Huang,
N. Kamp,
D. Kim,
K. Knickerbocker,
W. C. Louis,
J. T. M. Lyles,
R. Mahapatra,
S. Maludze
, et al. (20 additional authors not shown)
Abstract:
A solution to the MiniBooNE excess invoking rare three-body decays of the charged pions and kaons to new states in the MeV mass scale was recently proposed as a dark-sector explanation. This class of solution illuminates the fact that, while the charged pions were focused in the target-mode run, their decay products were isotropically suppressed in the beam-dump-mode run in which no excess was obs…
▽ More
A solution to the MiniBooNE excess invoking rare three-body decays of the charged pions and kaons to new states in the MeV mass scale was recently proposed as a dark-sector explanation. This class of solution illuminates the fact that, while the charged pions were focused in the target-mode run, their decay products were isotropically suppressed in the beam-dump-mode run in which no excess was observed. This suggests a new physics solution correlated to the mesonic sector. We investigate an extended set of phenomenological models that can explain the MiniBooNE excess as a dark sector solution, utilizing long-lived particles that might be produced in the three-body decays of the charged mesons and the two-body anomalous decays of the neutral mesons. Over a broad set of interactions with the long-lived particles, we show that these scenarios can be compatible with constraints from LSND, KARMEN, and MicroBooNE, and evaluate the sensitivity of the ongoing and future data taken by the Coherent CAPTAIN Mills experiment (CCM) to a potential discovery in this parameter space.
△ Less
Submitted 6 August, 2024; v1 submitted 5 September, 2023;
originally announced September 2023.
-
Experimental Safe Extremum Seeking for Accelerators
Authors:
Alan Williams,
Alexander Scheinker,
En-Chuan Huang,
Charles Taylor,
Miroslav Krstic
Abstract:
We demonstrate the recent designs of Safe Extremum Seeking (Safe ES) on the 1 kilometer-long charged particle accelerator at the Los Alamos Neutron Science Center (LANSCE). Safe ES is a modification of ES which, in addition to minimizing an analytically unknown cost, also employs a safety filter based on an analytically unknown control barrier function (CBF) safety metric. Accelerator tuning is ne…
▽ More
We demonstrate the recent designs of Safe Extremum Seeking (Safe ES) on the 1 kilometer-long charged particle accelerator at the Los Alamos Neutron Science Center (LANSCE). Safe ES is a modification of ES which, in addition to minimizing an analytically unknown cost, also employs a safety filter based on an analytically unknown control barrier function (CBF) safety metric. Accelerator tuning is necessitated by the accelerators being large, with many drifting parameters due to thermal effects and degradation. At the same time, safe operation (the maintenance of state constraints) is crucial, as damage brings astronomical costs, both financially and in operation downtime. Our measured (but analytically unknown) safety metric is the beam current. We perform multivariable Safe ES on three accelerator applications, in which we adapt 4, 6, and 3 magnet strength parameters, respectively. Two of the three applications are for validated simulation models of beamlines at LANSCE: the first for the Proton Radiography (pRad) beamline of 800 MeV protons for spot size tuning; the second on a high performance code, HPSim, for tuning the low energy beam transport (LEBT) region of of 750 keV protons. The third is an experimental tuning of the steering magnets in the LEBT at LANSCE.
△ Less
Submitted 29 August, 2023;
originally announced August 2023.
-
Equitable Restless Multi-Armed Bandits: A General Framework Inspired By Digital Health
Authors:
Jackson A. Killian,
Manish Jain,
Yugang Jia,
Jonathan Amar,
Erich Huang,
Milind Tambe
Abstract:
Restless multi-armed bandits (RMABs) are a popular framework for algorithmic decision making in sequential settings with limited resources. RMABs are increasingly being used for sensitive decisions such as in public health, treatment scheduling, anti-poaching, and -- the motivation for this work -- digital health. For such high stakes settings, decisions must both improve outcomes and prevent disp…
▽ More
Restless multi-armed bandits (RMABs) are a popular framework for algorithmic decision making in sequential settings with limited resources. RMABs are increasingly being used for sensitive decisions such as in public health, treatment scheduling, anti-poaching, and -- the motivation for this work -- digital health. For such high stakes settings, decisions must both improve outcomes and prevent disparities between groups (e.g., ensure health equity). We study equitable objectives for RMABs (ERMABs) for the first time. We consider two equity-aligned objectives from the fairness literature, minimax reward and max Nash welfare. We develop efficient algorithms for solving each -- a water filling algorithm for the former, and a greedy algorithm with theoretically motivated nuance to balance disparate group sizes for the latter. Finally, we demonstrate across three simulation domains, including a new digital health model, that our approaches can be multiple times more equitable than the current state of the art without drastic sacrifices to utility. Our findings underscore our work's urgency as RMABs permeate into systems that impact human and wildlife outcomes. Code is available at https://github.com/google-research/socialgood/tree/equitable-rmab
△ Less
Submitted 17 August, 2023;
originally announced August 2023.
-
Communication-Free Distributed GNN Training with Vertex Cut
Authors:
Kaidi Cao,
Rui Deng,
Shirley Wu,
Edward W Huang,
Karthik Subbian,
Jure Leskovec
Abstract:
Training Graph Neural Networks (GNNs) on real-world graphs consisting of billions of nodes and edges is quite challenging, primarily due to the substantial memory needed to store the graph and its intermediate node and edge features, and there is a pressing need to speed up the training process. A common approach to achieve speed up is to divide the graph into many smaller subgraphs, which are the…
▽ More
Training Graph Neural Networks (GNNs) on real-world graphs consisting of billions of nodes and edges is quite challenging, primarily due to the substantial memory needed to store the graph and its intermediate node and edge features, and there is a pressing need to speed up the training process. A common approach to achieve speed up is to divide the graph into many smaller subgraphs, which are then distributed across multiple GPUs in one or more machines and processed in parallel. However, existing distributed methods require frequent and substantial cross-GPU communication, leading to significant time overhead and progressively diminishing scalability. Here, we introduce CoFree-GNN, a novel distributed GNN training framework that significantly speeds up the training process by implementing communication-free training. The framework utilizes a Vertex Cut partitioning, i.e., rather than partitioning the graph by cutting the edges between partitions, the Vertex Cut partitions the edges and duplicates the node information to preserve the graph structure. Furthermore, the framework maintains high model accuracy by incorporating a reweighting mechanism to handle a distorted graph distribution that arises from the duplicated nodes. We also propose a modified DropEdge technique to further speed up the training process. Using an extensive set of experiments on real-world networks, we demonstrate that CoFree-GNN speeds up the GNN training process by up to 10 times over the existing state-of-the-art GNN training approaches.
△ Less
Submitted 6 August, 2023;
originally announced August 2023.
-
MVA2023 Small Object Detection Challenge for Spotting Birds: Dataset, Methods, and Results
Authors:
Yuki Kondo,
Norimichi Ukita,
Takayuki Yamaguchi,
Hao-Yu Hou,
Mu-Yi Shen,
Chia-Chi Hsu,
En-Ming Huang,
Yu-Chen Huang,
Yu-Cheng Xia,
Chien-Yao Wang,
Chun-Yi Lee,
Da Huo,
Marc A. Kastner,
Tingwei Liu,
Yasutomo Kawanishi,
Takatsugu Hirayama,
Takahiro Komamizu,
Ichiro Ide,
Yosuke Shinya,
Xinyao Liu,
Guang Liang,
Syusuke Yasui
Abstract:
Small Object Detection (SOD) is an important machine vision topic because (i) a variety of real-world applications require object detection for distant objects and (ii) SOD is a challenging task due to the noisy, blurred, and less-informative image appearances of small objects. This paper proposes a new SOD dataset consisting of 39,070 images including 137,121 bird instances, which is called the S…
▽ More
Small Object Detection (SOD) is an important machine vision topic because (i) a variety of real-world applications require object detection for distant objects and (ii) SOD is a challenging task due to the noisy, blurred, and less-informative image appearances of small objects. This paper proposes a new SOD dataset consisting of 39,070 images including 137,121 bird instances, which is called the Small Object Detection for Spotting Birds (SOD4SB) dataset. The detail of the challenge with the SOD4SB dataset is introduced in this paper. In total, 223 participants joined this challenge. This paper briefly introduces the award-winning methods. The dataset, the baseline code, and the website for evaluation on the public testset are publicly available.
△ Less
Submitted 18 July, 2023;
originally announced July 2023.
-
On $χ-$slice pretzel links
Authors:
Sophia Fanelle,
Evan Huang,
Ben Huenemann,
Weizhe Shen,
Jonathan Simone,
Hannah Turner
Abstract:
A link is called $χ-$slice if it bounds a smooth properly embedded surface in the 4-ball with no closed components and Euler characteristic 1. If a link has a single component, then it is $χ-$slice if and only if it is slice. One motivation for studying such links is that the double cover of the 3-sphere branched along a nonzero determinant $χ-$slice link is a rational homology 3-sphere that bound…
▽ More
A link is called $χ-$slice if it bounds a smooth properly embedded surface in the 4-ball with no closed components and Euler characteristic 1. If a link has a single component, then it is $χ-$slice if and only if it is slice. One motivation for studying such links is that the double cover of the 3-sphere branched along a nonzero determinant $χ-$slice link is a rational homology 3-sphere that bounds a rational homology 4-ball. This article aims to generalize known results about the sliceness of pretzel knots to the $χ-$sliceness of pretzel links. In particular, we completely classify positive and negative pretzel links that are $χ-$slice, and obtain partial classifications of 3-stranded and 4-stranded pretzel links that are $χ-$slice. As a consequence, we obtain infinite families of Seifert fiber spaces that bound rational homology 4-balls.
△ Less
Submitted 2 June, 2023;
originally announced June 2023.
-
ElectrodeNet -- A Deep Learning Based Sound Coding Strategy for Cochlear Implants
Authors:
Enoch Hsin-Ho Huang,
Rong Chao,
Yu Tsao,
Chao-Min Wu
Abstract:
ElectrodeNet, a deep learning based sound coding strategy for the cochlear implant (CI), is proposed to emulate the advanced combination encoder (ACE) strategy by replacing the conventional envelope detection using various artificial neural networks. The extended ElectrodeNet-CS strategy further incorporates the channel selection (CS). Network models of deep neural network (DNN), convolutional neu…
▽ More
ElectrodeNet, a deep learning based sound coding strategy for the cochlear implant (CI), is proposed to emulate the advanced combination encoder (ACE) strategy by replacing the conventional envelope detection using various artificial neural networks. The extended ElectrodeNet-CS strategy further incorporates the channel selection (CS). Network models of deep neural network (DNN), convolutional neural network (CNN), and long short-term memory (LSTM) were trained using the Fast Fourier Transformed bins and channel envelopes obtained from the processing of clean speech by the ACE strategy. Objective speech understanding using short-time objective intelligibility (STOI) and normalized covariance metric (NCM) was estimated for ElectrodeNet using CI simulations. Sentence recognition tests for vocoded Mandarin speech were conducted with normal-hearing listeners. DNN, CNN, and LSTM based ElectrodeNets exhibited strong correlations to ACE in objective and subjective scores using mean squared error (MSE), linear correlation coefficient (LCC) and Spearman's rank correlation coefficient (SRCC). The ElectrodeNet-CS strategy was capable of producing N-of-M compatible electrode patterns using a modified DNN network to embed maxima selection, and to perform in similar or even slightly higher average in STOI and sentence recognition compared to ACE. The methods and findings demonstrated the feasibility and potential of using deep learning in CI coding strategy.
△ Less
Submitted 26 May, 2023;
originally announced May 2023.
-
Simplifying Distributed Neural Network Training on Massive Graphs: Randomized Partitions Improve Model Aggregation
Authors:
Jiong Zhu,
Aishwarya Reganti,
Edward Huang,
Charles Dickens,
Nikhil Rao,
Karthik Subbian,
Danai Koutra
Abstract:
Distributed training of GNNs enables learning on massive graphs (e.g., social and e-commerce networks) that exceed the storage and computational capacity of a single machine. To reach performance comparable to centralized training, distributed frameworks focus on maximally recovering cross-instance node dependencies with either communication across instances or periodic fallback to centralized tra…
▽ More
Distributed training of GNNs enables learning on massive graphs (e.g., social and e-commerce networks) that exceed the storage and computational capacity of a single machine. To reach performance comparable to centralized training, distributed frameworks focus on maximally recovering cross-instance node dependencies with either communication across instances or periodic fallback to centralized training, which create overhead and limit the framework scalability. In this work, we present a simplified framework for distributed GNN training that does not rely on the aforementioned costly operations, and has improved scalability, convergence speed and performance over the state-of-the-art approaches. Specifically, our framework (1) assembles independent trainers, each of which asynchronously learns a local model on locally-available parts of the training graph, and (2) only conducts periodic (time-based) model aggregation to synchronize the local models. Backed by our theoretical analysis, instead of maximizing the recovery of cross-instance node dependencies -- which has been considered the key behind closing the performance gap between model aggregation and centralized training -- , our framework leverages randomized assignment of nodes or super-nodes (i.e., collections of original nodes) to partition the training graph such that it improves data uniformity and minimizes the discrepancy of gradient and loss function across instances. In our experiments on social and e-commerce networks with up to 1.3 billion edges, our proposed RandomTMA and SuperTMA approaches -- despite using less training data -- achieve state-of-the-art performance and 2.31x speedup compared to the fastest baseline, and show better robustness to trainer failures.
△ Less
Submitted 16 May, 2023;
originally announced May 2023.
-
You Only Transfer What You Share: Intersection-Induced Graph Transfer Learning for Link Prediction
Authors:
Wenqing Zheng,
Edward W Huang,
Nikhil Rao,
Zhangyang Wang,
Karthik Subbian
Abstract:
Link prediction is central to many real-world applications, but its performance may be hampered when the graph of interest is sparse. To alleviate issues caused by sparsity, we investigate a previously overlooked phenomenon: in many cases, a densely connected, complementary graph can be found for the original graph. The denser graph may share nodes with the original graph, which offers a natural b…
▽ More
Link prediction is central to many real-world applications, but its performance may be hampered when the graph of interest is sparse. To alleviate issues caused by sparsity, we investigate a previously overlooked phenomenon: in many cases, a densely connected, complementary graph can be found for the original graph. The denser graph may share nodes with the original graph, which offers a natural bridge for transferring selective, meaningful knowledge. We identify this setting as Graph Intersection-induced Transfer Learning (GITL), which is motivated by practical applications in e-commerce or academic co-authorship predictions. We develop a framework to effectively leverage the structural prior in this setting. We first create an intersection subgraph using the shared nodes between the two graphs, then transfer knowledge from the source-enriched intersection subgraph to the full target graph. In the second step, we consider two approaches: a modified label propagation, and a multi-layer perceptron (MLP) model in a teacher-student regime. Experimental results on proprietary e-commerce datasets and open-source citation graphs show that the proposed workflow outperforms existing transfer learning baselines that do not explicitly utilize the intersection structure.
△ Less
Submitted 18 June, 2023; v1 submitted 27 February, 2023;
originally announced February 2023.
-
Quantitative assessment of the universal thermopower in the Hubbard model
Authors:
Wen O. Wang,
Jixun K. Ding,
Edwin W. Huang,
Brian Moritz,
Thomas P. Devereaux
Abstract:
As primarily an electronic observable, the room-temperature thermopower $S$ in cuprates provides possibilities for a quantitative assessment of the Hubbard model. Using determinant quantum Monte Carlo, we demonstrate agreement between Hubbard model calculations and experimentally measured room-temperature $S$ across multiple cuprate families, both qualitatively in terms of the doping dependence an…
▽ More
As primarily an electronic observable, the room-temperature thermopower $S$ in cuprates provides possibilities for a quantitative assessment of the Hubbard model. Using determinant quantum Monte Carlo, we demonstrate agreement between Hubbard model calculations and experimentally measured room-temperature $S$ across multiple cuprate families, both qualitatively in terms of the doping dependence and quantitatively in terms of magnitude. We observe an upturn in $S$ with decreasing temperatures, which possesses a slope comparable to that observed experimentally in cuprates. From our calculations, the doping at which $S$ changes sign occurs in close proximity to a vanishing temperature dependence of the chemical potential at fixed density. Our results emphasize the importance of interaction effects in the systematic assessment of the thermopower $S$ in cuprates.
△ Less
Submitted 3 November, 2023; v1 submitted 25 February, 2023;
originally announced February 2023.
-
Lightweight Facial Attractiveness Prediction Using Dual Label Distribution
Authors:
Shu Liu,
Enquan Huang,
Ziyu Zhou,
Yan Xu,
Xiaoyan Kui,
Tao Lei,
Hongying Meng
Abstract:
Facial attractiveness prediction (FAP) aims to assess facial attractiveness automatically based on human aesthetic perception. Previous methods using deep convolutional neural networks have improved the performance, but their large-scale models have led to a deficiency in flexibility. In addition, most methods fail to take full advantage of the dataset. In this paper, we present a novel end-to-end…
▽ More
Facial attractiveness prediction (FAP) aims to assess facial attractiveness automatically based on human aesthetic perception. Previous methods using deep convolutional neural networks have improved the performance, but their large-scale models have led to a deficiency in flexibility. In addition, most methods fail to take full advantage of the dataset. In this paper, we present a novel end-to-end FAP approach that integrates dual label distribution and lightweight design. The manual ratings, attractiveness score, and standard deviation are aggregated explicitly to construct a dual-label distribution to make the best use of the dataset, including the attractiveness distribution and the rating distribution. Such distributions, as well as the attractiveness score, are optimized under a joint learning framework based on the label distribution learning (LDL) paradigm. The data processing is simplified to a minimum for a lightweight design, and MobileNetV2 is selected as our backbone. Extensive experiments are conducted on two benchmark datasets, where our approach achieves promising results and succeeds in balancing performance and efficiency. Ablation studies demonstrate that our delicately designed learning modules are indispensable and correlated. Additionally, the visualization indicates that our approach can perceive facial attractiveness and capture attractive facial regions to facilitate semantic predictions. The code is available at https://github.com/enquan/2D_FAP.
△ Less
Submitted 24 April, 2024; v1 submitted 3 December, 2022;
originally announced December 2022.
-
Search Behavior Prediction: A Hypergraph Perspective
Authors:
Yan Han,
Edward W Huang,
Wenqing Zheng,
Nikhil Rao,
Zhangyang Wang,
Karthik Subbian
Abstract:
Although the bipartite shopping graphs are straightforward to model search behavior, they suffer from two challenges: 1) The majority of items are sporadically searched and hence have noisy/sparse query associations, leading to a \textit{long-tail} distribution. 2) Infrequent queries are more likely to link to popular items, leading to another hurdle known as \textit{disassortative mixing}. To add…
▽ More
Although the bipartite shopping graphs are straightforward to model search behavior, they suffer from two challenges: 1) The majority of items are sporadically searched and hence have noisy/sparse query associations, leading to a \textit{long-tail} distribution. 2) Infrequent queries are more likely to link to popular items, leading to another hurdle known as \textit{disassortative mixing}. To address these two challenges, we go beyond the bipartite graph to take a hypergraph perspective, introducing a new paradigm that leverages \underline{auxiliary} information from anonymized customer engagement sessions to assist the \underline{main task} of query-item link prediction. This auxiliary information is available at web scale in the form of search logs. We treat all items appearing in the same customer session as a single hyperedge. The hypothesis is that items in a customer session are unified by a common shopping interest. With these hyperedges, we augment the original bipartite graph into a new \textit{hypergraph}. We develop a \textit{\textbf{D}ual-\textbf{C}hannel \textbf{A}ttention-Based \textbf{H}ypergraph Neural Network} (\textbf{DCAH}), which synergizes information from two potentially noisy sources (original query-item edges and item-item hyperedges). In this way, items on the tail are better connected due to the extra hyperedges, thereby enhancing their link prediction performance. We further integrate DCAH with self-supervised graph pre-training and/or DropEdge training, both of which effectively alleviate disassortative mixing. Extensive experiments on three proprietary E-Commerce datasets show that DCAH yields significant improvements of up to \textbf{24.6\% in mean reciprocal rank (MRR)} and \textbf{48.3\% in recall} compared to GNN-based baselines. Our source code is available at \url{https://github.com/amazon-science/dual-channel-hypergraph-neural-network}.
△ Less
Submitted 28 November, 2022; v1 submitted 23 November, 2022;
originally announced November 2022.
-
Tailoring three-dimensional topological codes for biased noise
Authors:
Eric Huang,
Arthur Pesah,
Christopher T. Chubb,
Michael Vasmer,
Arpit Dua
Abstract:
Tailored topological stabilizer codes in two dimensions have been shown to exhibit high storage threshold error rates and improved subthreshold performance under biased Pauli noise. Three-dimensional (3D) topological codes can allow for several advantages including a transversal implementation of non-Clifford logical gates, single-shot decoding strategies, parallelized decoding in the case of frac…
▽ More
Tailored topological stabilizer codes in two dimensions have been shown to exhibit high storage threshold error rates and improved subthreshold performance under biased Pauli noise. Three-dimensional (3D) topological codes can allow for several advantages including a transversal implementation of non-Clifford logical gates, single-shot decoding strategies, parallelized decoding in the case of fracton codes as well as construction of fractal lattice codes. Motivated by this, we tailor 3D topological codes for enhanced storage performance under biased Pauli noise. We present Clifford deformations of various 3D topological codes, such that they exhibit a threshold error rate of $50\%$ under infinitely biased Pauli noise. Our examples include the 3D surface code on the cubic lattice, the 3D surface code on a checkerboard lattice that lends itself to a subsystem code with a single-shot decoder, the 3D color code, as well as fracton models such as the X-cube model, the Sierpinski model and the Haah code. We use the belief propagation with ordered statistics decoder (BP-OSD) to study threshold error rates at finite bias. We also present a rotated layout for the 3D surface code, which uses roughly half the number of physical qubits for the same code distance under appropriate boundary conditions. Imposing coprime periodic dimensions on this rotated layout leads to logical operators of weight $O(n)$ at infinite bias and a corresponding $\exp[-O(n)]$ subthreshold scaling of the logical failure rate, where $n$ is the number of physical qubits in the code. Even though this scaling is unstable due to the existence of logical representations with $O(1)$ low-rate Pauli errors, the number of such representations scales only polynomially for the Clifford-deformed code, leading to an enhanced effective distance.
△ Less
Submitted 3 November, 2022;
originally announced November 2022.
-
TuneUp: A Simple Improved Training Strategy for Graph Neural Networks
Authors:
Weihua Hu,
Kaidi Cao,
Kexin Huang,
Edward W Huang,
Karthik Subbian,
Kenji Kawaguchi,
Jure Leskovec
Abstract:
Despite recent advances in Graph Neural Networks (GNNs), their training strategies remain largely under-explored. The conventional training strategy learns over all nodes in the original graph(s) equally, which can be sub-optimal as certain nodes are often more difficult to learn than others. Here we present TuneUp, a simple curriculum-based training strategy for improving the predictive performan…
▽ More
Despite recent advances in Graph Neural Networks (GNNs), their training strategies remain largely under-explored. The conventional training strategy learns over all nodes in the original graph(s) equally, which can be sub-optimal as certain nodes are often more difficult to learn than others. Here we present TuneUp, a simple curriculum-based training strategy for improving the predictive performance of GNNs. TuneUp trains a GNN in two stages. In the first stage, TuneUp applies conventional training to obtain a strong base GNN. The base GNN tends to perform well on head nodes (nodes with large degrees) but less so on tail nodes (nodes with small degrees). Therefore, the second stage of TuneUp focuses on improving prediction on the difficult tail nodes by further training the base GNN on synthetically generated tail node data. We theoretically analyze TuneUp and show it provably improves generalization performance on tail nodes. TuneUp is simple to implement and applicable to a broad range of GNN architectures and prediction tasks. Extensive evaluation of TuneUp on five diverse GNN architectures, three types of prediction tasks, and both transductive and inductive settings shows that TuneUp significantly improves the performance of the base GNN on tail nodes, while often even improving the performance on head nodes. Altogether, TuneUp produces up to 57.6% and 92.2% relative predictive performance improvement in the transductive and the challenging inductive settings, respectively.
△ Less
Submitted 26 August, 2023; v1 submitted 26 October, 2022;
originally announced October 2022.
-
Composite Learning for Robust and Effective Dense Predictions
Authors:
Menelaos Kanakis,
Thomas E. Huang,
David Bruggemann,
Fisher Yu,
Luc Van Gool
Abstract:
Multi-task learning promises better model generalization on a target task by jointly optimizing it with an auxiliary task. However, the current practice requires additional labeling efforts for the auxiliary task, while not guaranteeing better model performance. In this paper, we find that jointly training a dense prediction (target) task with a self-supervised (auxiliary) task can consistently im…
▽ More
Multi-task learning promises better model generalization on a target task by jointly optimizing it with an auxiliary task. However, the current practice requires additional labeling efforts for the auxiliary task, while not guaranteeing better model performance. In this paper, we find that jointly training a dense prediction (target) task with a self-supervised (auxiliary) task can consistently improve the performance of the target task, while eliminating the need for labeling auxiliary tasks. We refer to this joint training as Composite Learning (CompL). Experiments of CompL on monocular depth estimation, semantic segmentation, and boundary detection show consistent performance improvements in fully and partially labeled datasets. Further analysis on depth estimation reveals that joint training with self-supervision outperforms most labeled auxiliary tasks. We also find that CompL can improve model robustness when the models are evaluated in new domains. These results demonstrate the benefits of self-supervision as an auxiliary task, and establish the design of novel task-specific self-supervised methods as a new axis of investigation for future multi-task learning research.
△ Less
Submitted 13 October, 2022;
originally announced October 2022.
-
QDTrack: Quasi-Dense Similarity Learning for Appearance-Only Multiple Object Tracking
Authors:
Tobias Fischer,
Thomas E. Huang,
Jiangmiao Pang,
Linlu Qiu,
Haofeng Chen,
Trevor Darrell,
Fisher Yu
Abstract:
Similarity learning has been recognized as a crucial step for object tracking. However, existing multiple object tracking methods only use sparse ground truth matching as the training objective, while ignoring the majority of the informative regions in images. In this paper, we present Quasi-Dense Similarity Learning, which densely samples hundreds of object regions on a pair of images for contras…
▽ More
Similarity learning has been recognized as a crucial step for object tracking. However, existing multiple object tracking methods only use sparse ground truth matching as the training objective, while ignoring the majority of the informative regions in images. In this paper, we present Quasi-Dense Similarity Learning, which densely samples hundreds of object regions on a pair of images for contrastive learning. We combine this similarity learning with multiple existing object detectors to build Quasi-Dense Tracking (QDTrack), which does not require displacement regression or motion priors. We find that the resulting distinctive feature space admits a simple nearest neighbor search at inference time for object association. In addition, we show that our similarity learning scheme is not limited to video data, but can learn effective instance similarity even from static input, enabling a competitive tracking performance without training on videos or using tracking supervision. We conduct extensive experiments on a wide variety of popular MOT benchmarks. We find that, despite its simplicity, QDTrack rivals the performance of state-of-the-art tracking methods on all benchmarks and sets a new state-of-the-art on the large-scale BDD100K MOT benchmark, while introducing negligible computational overhead to the detector.
△ Less
Submitted 27 September, 2023; v1 submitted 12 October, 2022;
originally announced October 2022.
-
Exact solution for finite center-of-mass momentum Cooper pairing
Authors:
Chandan Setty,
Jinchao Zhao,
Laura Fanfarillo,
Edwin W. Huang,
Peter J. Hirschfeld,
Philip W. Phillips,
Kun Yang
Abstract:
Pair density waves (PDWs) are superconducting states formed by ``Cooper pairs" of electrons containing a non-zero center-of-mass momentum. They are characterized by a spatially modulated order parameter and may occur in a variety of emerging quantum materials such as cuprates, transition metal dichalcogenides (TMDs) and Kagome metals. Despite extensive theoretical and numerical studies seeking PDW…
▽ More
Pair density waves (PDWs) are superconducting states formed by ``Cooper pairs" of electrons containing a non-zero center-of-mass momentum. They are characterized by a spatially modulated order parameter and may occur in a variety of emerging quantum materials such as cuprates, transition metal dichalcogenides (TMDs) and Kagome metals. Despite extensive theoretical and numerical studies seeking PDWs in a variety of lattices and interacting settings, there is currently no generic and robust mechanism that favors a modulated solution of the superconducting order parameter in the presence of time reversal symmetry. Here, we study the problem of two electrons subject to an anisotropic ($d$-wave) attractive potential. We solve the two-body Schrodinger wave equation exactly to determine the pair binding energy as a function of the center-of-mass momentum. We find that a modulated (finite momentum) pair is favored over a homogeneous (zero momentum) solution above a critical interaction. Using this insight from the exact two-body solution, we construct a BCS-like variational many-body wave function and calculate the free energy and superconducting gap as a function of the center-of-mass momentum. A zero temperature analysis of the energy shows that the conclusions of the two-body problem are robust in the many-body limit. Our results lay the theoretical and microscopic foundation for the existence of PDWs.
△ Less
Submitted 21 September, 2022;
originally announced September 2022.
-
A Data-Centric Methodology and Task Typology for Time-Stamped Event Sequences
Authors:
Yasara Peiris,
Clara-Maria Barth,
Elaine M. Huang,
Jürgen Bernard
Abstract:
Task abstractions and taxonomic structures for tasks are useful for designers of interactive data analysis approaches, serving as design targets and evaluation criteria alike. For individual data types, dataset-specific taxonomic structures capture unique data characteristics, while being generalizable across application domains. The creation of dataset-centric but domain-agnostic taxonomic struct…
▽ More
Task abstractions and taxonomic structures for tasks are useful for designers of interactive data analysis approaches, serving as design targets and evaluation criteria alike. For individual data types, dataset-specific taxonomic structures capture unique data characteristics, while being generalizable across application domains. The creation of dataset-centric but domain-agnostic taxonomic structures is difficult, especially if best practices for a focused data type are still missing, observing experts is not feasible, and means for reflection and generalization are scarce. We discovered this need for methodological support when working with time-stamped event sequences, a datatype that has not yet been fully systematically studied in visualization research. To address this shortcoming, we present a methodology that enables researchers to abstract tasks and build dataset-centric taxonomic structures in five phases (data collection, coding, task categorization, task synthesis, and action-target(criterion) crosscut). We validate the methodology by applying it to time-stamped event sequences and present a task typology that uses triples as a novel language of description for tasks: (1) action, (2) data target, and (3) data criterion. We further evaluate the descriptive power of the typology with a real-world case on cybersecurity.
△ Less
Submitted 21 September, 2022;
originally announced September 2022.
-
The Wiedemann-Franz law in doped Mott insulators without quasiparticles
Authors:
Wen O. Wang,
Jixun K. Ding,
Yoni Schattner,
Edwin W. Huang,
Brian Moritz,
Thomas P. Devereaux
Abstract:
Many metallic quantum materials display anomalous transport phenomena that defy a Fermi liquid description. Here, we use numerical methods to calculate thermal and charge transport in the doped Hubbard model and observe a cross-over separating high- and low-temperature behaviors. Distinct from the behavior at high temperatures, the Lorenz number $L$ becomes weakly doping dependent and less sensiti…
▽ More
Many metallic quantum materials display anomalous transport phenomena that defy a Fermi liquid description. Here, we use numerical methods to calculate thermal and charge transport in the doped Hubbard model and observe a cross-over separating high- and low-temperature behaviors. Distinct from the behavior at high temperatures, the Lorenz number $L$ becomes weakly doping dependent and less sensitive to parameters at low temperatures. At the lowest numerically accessible temperatures, $L$ roughly approaches the Wiedemann-Franz constant $L_0$, even in a doped Mott insulator that lacks well-defined quasiparticles. Decomposing the energy current operator indicates a compensation between kinetic and potential contributions, which may help to clarify the interpretation of transport experiments beyond Boltzmann theory in strongly correlated metals.
△ Less
Submitted 1 December, 2023; v1 submitted 19 August, 2022;
originally announced August 2022.
-
Tracking Every Thing in the Wild
Authors:
Siyuan Li,
Martin Danelljan,
Henghui Ding,
Thomas E. Huang,
Fisher Yu
Abstract:
Current multi-category Multiple Object Tracking (MOT) metrics use class labels to group tracking results for per-class evaluation. Similarly, MOT methods typically only associate objects with the same class predictions. These two prevalent strategies in MOT implicitly assume that the classification performance is near-perfect. However, this is far from the case in recent large-scale MOT datasets,…
▽ More
Current multi-category Multiple Object Tracking (MOT) metrics use class labels to group tracking results for per-class evaluation. Similarly, MOT methods typically only associate objects with the same class predictions. These two prevalent strategies in MOT implicitly assume that the classification performance is near-perfect. However, this is far from the case in recent large-scale MOT datasets, which contain large numbers of classes with many rare or semantically similar categories. Therefore, the resulting inaccurate classification leads to sub-optimal tracking and inadequate benchmarking of trackers. We address these issues by disentangling classification from tracking. We introduce a new metric, Track Every Thing Accuracy (TETA), breaking tracking measurement into three sub-factors: localization, association, and classification, allowing comprehensive benchmarking of tracking performance even under inaccurate classification. TETA also deals with the challenging incomplete annotation problem in large-scale tracking datasets. We further introduce a Track Every Thing tracker (TETer), that performs association using Class Exemplar Matching (CEM). Our experiments show that TETA evaluates trackers more comprehensively, and TETer achieves significant improvements on the challenging large-scale datasets BDD100K and TAO compared to the state-of-the-art.
△ Less
Submitted 26 July, 2022;
originally announced July 2022.
-
A 'one-size-fits-most' walking recognition method for smartphones, smartwatches, and wearable accelerometers
Authors:
Marcin Straczkiewicz,
Emily J. Huang,
Jukka-Pekka Onnela
Abstract:
The ubiquity of personal digital devices offers unprecedented opportunities to study human behavior. Current state-of-the-art methods quantify physical activity using 'activity counts,' a measure which overlooks specific types of physical activities. We proposed a walking recognition method for sub-second tri-axial accelerometer data, in which activity classification is based on the inherent featu…
▽ More
The ubiquity of personal digital devices offers unprecedented opportunities to study human behavior. Current state-of-the-art methods quantify physical activity using 'activity counts,' a measure which overlooks specific types of physical activities. We proposed a walking recognition method for sub-second tri-axial accelerometer data, in which activity classification is based on the inherent features of walking: intensity, periodicity, and duration. We validated our method against 20 publicly available, annotated datasets on walking activity data collected at various body locations (thigh, waist, chest, arm, wrist). We demonstrated that our method can estimate walking periods with high sensitivity and specificity: average sensitivity ranged between 0.92 and 0.97 across various body locations, and average specificity for common daily activities was typically above 0.95. We also assessed the method's algorithmic fairness to demographic and anthropometric variables and measurement contexts (body location, environment). Finally, we have released our method as open-source software in MATLAB and Python.
△ Less
Submitted 15 July, 2022;
originally announced July 2022.
-
Occlusion-Resistant Instance Segmentation of Piglets in Farrowing Pens Using Center Clustering Network
Authors:
Endai Huang,
Axiu Mao,
Junhui Hou,
Yongjian Wu,
Weitao Xu,
Maria Camila Ceballos,
Thomas D. Parsons,
Kai Liu
Abstract:
Computer vision enables the development of new approaches to monitor the behavior, health, and welfare of animals. Instance segmentation is a high-precision method in computer vision for detecting individual animals of interest. This method can be used for in-depth analysis of animals, such as examining their subtle interactive behaviors, from videos and images. However, existing deep-learning-bas…
▽ More
Computer vision enables the development of new approaches to monitor the behavior, health, and welfare of animals. Instance segmentation is a high-precision method in computer vision for detecting individual animals of interest. This method can be used for in-depth analysis of animals, such as examining their subtle interactive behaviors, from videos and images. However, existing deep-learning-based instance segmentation methods have been mostly developed based on public datasets, which largely omit heavy occlusion problems; therefore, these methods have limitations in real-world applications involving object occlusions, such as farrowing pen systems used on pig farms in which the farrowing crates often impede the sow and piglets. In this paper, we adapt a Center Clustering Network originally designed for counting to achieve instance segmentation, dubbed as CClusnet-Inseg. Specifically, CClusnet-Inseg uses each pixel to predict object centers and trace these centers to form masks based on clustering results, which consists of a network for segmentation and center offset vector map, Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm, Centers-to-Mask (C2M), and Remain-Centers-to-Mask (RC2M) algorithms. In all, 4,600 images were extracted from six videos collected from three closed and three half-open farrowing crates to train and validate our method. CClusnet-Inseg achieves a mean average precision (mAP) of 84.1 and outperforms all other methods compared in this study. We conduct comprehensive ablation studies to demonstrate the advantages and effectiveness of core modules of our method. In addition, we apply CClusnet-Inseg to multi-object tracking for animal monitoring, and the predicted object center that is a conjunct output could serve as an occlusion-resistant representation of the location of an object.
△ Less
Submitted 17 February, 2023; v1 submitted 4 June, 2022;
originally announced June 2022.
-
Interaction-driven Spontaneous Ferromagnetic Insulating States with Odd Chern Numbers
Authors:
Peizhi Mai,
Edwin W. Huang,
Jiachen Yu,
Benjamin E. Feldman,
Philip W. Phillips
Abstract:
Motivated by recent experimental work on moiré systems in a strong magnetic field, we compute the compressibility as well as the spin correlations and Hofstadter spectrum of spinful electrons on a honeycomb lattice with Hubbard interactions using the determinantal quantum Monte Carlo method. While the interactions in general preserve quantum and anomalous Hall states, emergent features arise corre…
▽ More
Motivated by recent experimental work on moiré systems in a strong magnetic field, we compute the compressibility as well as the spin correlations and Hofstadter spectrum of spinful electrons on a honeycomb lattice with Hubbard interactions using the determinantal quantum Monte Carlo method. While the interactions in general preserve quantum and anomalous Hall states, emergent features arise corresponding to an antiferromagnetic insulator at half-filling and other incompressible states following the Chern sequence $\pm (2N+1)$. These odd integer Chern states exhibit strong ferromagnetic correlations and arise spontaneously without any external mechanism for breaking the spin-rotation symmetry. Analogs of these magnetic states should be observable in general interacting quantum Hall systems. In addition, the interacting Hofstadter spectrum is qualitatively similar to the experimental data at intermediate values of the on-site interaction.
△ Less
Submitted 20 February, 2023; v1 submitted 17 May, 2022;
originally announced May 2022.
-
LANSCE-PSR Short-Pulse Upgrade for Improved Dark Sector Particle Searches with the Coherent Captain Mills Experiment
Authors:
R. G. Van de Water,
S. G. Biedron,
E. -C. Huang,
A. J. Hurd,
W. C. Louis,
S. V. Milton,
N. A. Moody,
P. deNiverville,
C. E. Taylor,
R. T. Thornton,
M. Fazio,
S. I. Sosa,
T. J. Schaub,
J. W. Lewellen
Abstract:
Proton beam dumps are prolific sources of charged and neutral pions, enabling a powerful technique to search for dark matter, axions, sterile neutrinos, tests of short baseline anomalies, and precision measurements of coherent nucleus scattering neutrinos (CEvNS). The Lujan neutron elastic scattering center at the Los Alamos Neutron Science Center (LANSCE) consists of an 800-MeV, short-pulse, 100-…
▽ More
Proton beam dumps are prolific sources of charged and neutral pions, enabling a powerful technique to search for dark matter, axions, sterile neutrinos, tests of short baseline anomalies, and precision measurements of coherent nucleus scattering neutrinos (CEvNS). The Lujan neutron elastic scattering center at the Los Alamos Neutron Science Center (LANSCE) consists of an 800-MeV, short-pulse, 100-kW proton and spallation neutron source where such searches are ongoing with the Coherent CAPTAIN Mills (CCM) 10-ton, liquid argon detector. The employment of fast timing coincidence of the beam with the detector is used to identify signals and reject background. The current beam time width is 300 ns with an intensity of $3.1 \times 10^{13}$ protons per pulse at 20 Hz. With upgrades to the Proton Storage Ring (PSR), the beam time width may be compressed to 30 ns with minimal intensity loss, allowing an increase in the signal to background (S/B) of more than 100 and an increase in the sensitivity for dark matter and sterile neutrino searches of an order of magnitude. This can be achieved with PSR accelerator upgrades on a time scale of a few years and at a modest cost.
△ Less
Submitted 4 April, 2022;
originally announced April 2022.
-
Fluctuating intertwined stripes in the strange metal regime of the Hubbard model
Authors:
Edwin W. Huang,
Tianyi Liu,
Wen O. Wang,
Hong-Chen Jiang,
Peizhi Mai,
Thomas A. Maier,
Steven Johnston,
Brian Moritz,
Thomas P. Devereaux
Abstract:
Strongly correlated electron systems host a variety of poorly understood correlations in their high temperature normal state. Unlike ordered phases defined by order parameters, these normal state phases are often defined through unconventional properties such as strange metallic transport or spectroscopic pseudogaps. Characterizing the microscopic correlations in the normal state is necessary to e…
▽ More
Strongly correlated electron systems host a variety of poorly understood correlations in their high temperature normal state. Unlike ordered phases defined by order parameters, these normal state phases are often defined through unconventional properties such as strange metallic transport or spectroscopic pseudogaps. Characterizing the microscopic correlations in the normal state is necessary to elucidate mechanisms that lead to these properties and their connection to ground state orders. Here we establish the presence of intertwined charge and spin stripes in the strange metal normal state of the Hubbard model using determinant quantum Monte Carlo calculations. The charge and spin density waves constituting the stripes are fluctuating and short-ranged, yet they obey a mutual commensurability relation and remain microscopically interlocked, as evidenced through measurements of three-point spin-spin-hole correlation functions. Our findings demonstrate the ability of many-body numerical simulations to unravel the microscopic correlations that define quantum states of matter.
△ Less
Submitted 17 February, 2022;
originally announced February 2022.
-
Task-Agnostic Graph Explanations
Authors:
Yaochen Xie,
Sumeet Katariya,
Xianfeng Tang,
Edward Huang,
Nikhil Rao,
Karthik Subbian,
Shuiwang Ji
Abstract:
Graph Neural Networks (GNNs) have emerged as powerful tools to encode graph-structured data. Due to their broad applications, there is an increasing need to develop tools to explain how GNNs make decisions given graph-structured data. Existing learning-based GNN explanation approaches are task-specific in training and hence suffer from crucial drawbacks. Specifically, they are incapable of produci…
▽ More
Graph Neural Networks (GNNs) have emerged as powerful tools to encode graph-structured data. Due to their broad applications, there is an increasing need to develop tools to explain how GNNs make decisions given graph-structured data. Existing learning-based GNN explanation approaches are task-specific in training and hence suffer from crucial drawbacks. Specifically, they are incapable of producing explanations for a multitask prediction model with a single explainer. They are also unable to provide explanations in cases where the GNN is trained in a self-supervised manner, and the resulting representations are used in future downstream tasks. To address these limitations, we propose a Task-Agnostic GNN Explainer (TAGE) that is independent of downstream models and trained under self-supervision with no knowledge of downstream tasks. TAGE enables the explanation of GNN embedding models with unseen downstream tasks and allows efficient explanation of multitask models. Our extensive experiments show that TAGE can significantly speed up the explanation efficiency by using the same model to explain predictions for multiple downstream tasks while achieving explanation quality as good as or even better than current state-of-the-art GNN explanation approaches. Our code is pubicly available as part of the DIG library at https://github.com/divelab/DIG/tree/main/dig/xgraph/TAGE/.
△ Less
Submitted 23 September, 2022; v1 submitted 16 February, 2022;
originally announced February 2022.
-
Active Learning Over Multiple Domains in Natural Language Tasks
Authors:
Shayne Longpre,
Julia Reisler,
Edward Greg Huang,
Yi Lu,
Andrew Frank,
Nikhil Ramesh,
Chris DuBois
Abstract:
Studies of active learning traditionally assume the target and source data stem from a single domain. However, in realistic applications, practitioners often require active learning with multiple sources of out-of-distribution data, where it is unclear a priori which data sources will help or hurt the target domain. We survey a wide variety of techniques in active learning (AL), domain shift detec…
▽ More
Studies of active learning traditionally assume the target and source data stem from a single domain. However, in realistic applications, practitioners often require active learning with multiple sources of out-of-distribution data, where it is unclear a priori which data sources will help or hurt the target domain. We survey a wide variety of techniques in active learning (AL), domain shift detection (DS), and multi-domain sampling to examine this challenging setting for question answering and sentiment analysis. We ask (1) what family of methods are effective for this task? And, (2) what properties of selected examples and domains achieve strong results? Among 18 acquisition functions from 4 families of methods, we find H-Divergence methods, and particularly our proposed variant DAL-E, yield effective results, averaging 2-3% improvements over the random baseline. We also show the importance of a diverse allocation of domains, as well as room-for-improvement of existing methods on both domain and example selection. Our findings yield the first comprehensive analysis of both existing and novel methods for practitioners faced with multi-domain active learning for natural language tasks.
△ Less
Submitted 8 February, 2022; v1 submitted 1 February, 2022;
originally announced February 2022.
-
MiniBooNE and MicroBooNE Combined Fit to a 3+1 Sterile Neutrino Scenario
Authors:
A. A. Aguilar-Arevalo,
B. C. Brown,
J. M. Conrad,
R. Dharmapalan,
A. Diaz,
Z. Djurcic,
D. A. Finley,
R. Ford,
G. T. Garvey,
S. Gollapinni,
A. Hourlier,
E. -C. Huang,
N. W. Kamp,
G. Karagiorgi,
T. Katori,
T. Kobilarcik,
K. Lin,
W. C. Louis,
C. Mariani,
W. Marsh,
G. B. Mills,
J. Mirabal-Martinez,
C. D. Moore,
R. H. Nelson,
J. Nowak
, et al. (14 additional authors not shown)
Abstract:
This letter presents the results from the MiniBooNE experiment within a full "3+1" scenario where one sterile neutrino is introduced to the three-active-neutrino picture. In addition to electron-neutrino appearance at short-baselines, this scenario also allows for disappearance of the muon-neutrino and electron-neutrino fluxes in the Booster Neutrino Beam, which is shared by the MicroBooNE experim…
▽ More
This letter presents the results from the MiniBooNE experiment within a full "3+1" scenario where one sterile neutrino is introduced to the three-active-neutrino picture. In addition to electron-neutrino appearance at short-baselines, this scenario also allows for disappearance of the muon-neutrino and electron-neutrino fluxes in the Booster Neutrino Beam, which is shared by the MicroBooNE experiment. We present the 3+1 fit to the MiniBooNE electron-(anti)neutrino and muon-(anti)neutrino data alone, and in combination with MicroBooNE electron-neutrino data. The best-fit parameters of the combined fit with the exclusive CCQE analysis (inclusive analysis) are $Δm^2 = 0.29 eV^2 (0.33 eV^2)$, $|U_{e4}|^2 = 0.016 (0.500)$, $|U_{μ4}|^2 = 0.500 (0.500)$, and $\sin^2(2θ_{μe})=0.0316 (1.0)$. Comparing the no-oscillation scenario to the 3+1 model, the data prefer the 3+1 model with a $Δχ^2/\text{dof} = 24.7 / 3 (17.3 / 3)$, a $4.3σ(3.4σ)$ preference assuming the asymptotic approximation given by Wilks' theorem.
△ Less
Submitted 9 September, 2022; v1 submitted 5 January, 2022;
originally announced January 2022.
-
Prospects for detecting axionlike particles at the Coherent CAPTAIN-Mills experiment
Authors:
A. A. Aguilar-Arevalo,
D. S. M. Alves,
S. Biedron,
J. Boissevain,
M. Borrego,
L. Bugel,
M. Chavez-Estrada,
J. M. Conrad,
R. L. Cooper,
A. Diaz,
J. R. Distel,
J. C. D'Olivo,
E. Dunton,
B. Dutta,
D. Fields,
J. R. Gochanour,
M. Gold,
E. Guardincerri,
E. C. Huang,
N. Kamp,
D. Kim,
K. Knickerbocker,
W. C. Louis,
J. T. M. Lyles,
R. Mahapatra
, et al. (23 additional authors not shown)
Abstract:
We show results from the Coherent CAPTAIN Mills (CCM) 2019 engineering run which begin to constrain regions of parameter space for axion-like particles (ALPs) produced in electromagnetic particle showers in an 800 MeV proton beam dump, and further investigate the sensitivity of ongoing data-taking campaigns for the CCM200 upgraded detector. Based on beam-on background estimates from the engineerin…
▽ More
We show results from the Coherent CAPTAIN Mills (CCM) 2019 engineering run which begin to constrain regions of parameter space for axion-like particles (ALPs) produced in electromagnetic particle showers in an 800 MeV proton beam dump, and further investigate the sensitivity of ongoing data-taking campaigns for the CCM200 upgraded detector. Based on beam-on background estimates from the engineering run, we make realistic extrapolations for background reduction based on expected shielding improvements, reduced beam width, and analysis-based techniques for background rejection. We obtain reach projections for two classes of signatures; ALPs coupled primarily to photons can be produced in the tungsten target via the Primakoff process, and then produce a gamma-ray signal in the Liquid Argon (LAr) CCM detector either via inverse Primakoff scattering or decay to a photon pair. ALPs with significant electron couplings have several additional production mechanisms (Compton scattering, $e^+e^-$ annihilation, ALP-bremsstrahlung) and detection modes (inverse Compton scattering, external $e^+e^-$ pair conversion, and decay to $e^+e^-$). In some regions, the constraint is marginally better than both astrophysical and terrestrial constraints. With the beginning of a three year run, CCM will be more sensitive to this parameter space by up to an order of magnitude for both ALP-photon and ALP-electron couplings. The CCM experiment will also have sensitivity to well-motivated parameter space of QCD axion models. It is only a recent realization that accelerator-based large volume liquid argon detectors designed for low energy coherent neutrino and dark matter scattering searches are also ideal for probing ALPs in the unexplored $\sim$MeV mass scale.
△ Less
Submitted 26 May, 2023; v1 submitted 18 December, 2021;
originally announced December 2021.
-
Thermodynamics of an Exactly Solvable Model for Superconductivity in a Doped Mott Insulator
Authors:
Jinchao Zhao,
Luke Yeo,
Edwin Huang,
Philip W. Phillips
Abstract:
Computing superconducting properties starting from an exactly solvable model for a doped Mott insulator stands as a grand challenge. We have recently shown that this can be done starting from the Hatsugai-Kohmoto (HK) model which can be understood generally as the minimal model that breaks the non-local $\mathbb Z_2$ symmetry of a Fermi liquid, thereby constituting a new quartic fixed point for Mo…
▽ More
Computing superconducting properties starting from an exactly solvable model for a doped Mott insulator stands as a grand challenge. We have recently shown that this can be done starting from the Hatsugai-Kohmoto (HK) model which can be understood generally as the minimal model that breaks the non-local $\mathbb Z_2$ symmetry of a Fermi liquid, thereby constituting a new quartic fixed point for Mott physics [Phillips et al., Nature Physics 16, 1175 (2020); Huang et al., Nature Physics (2022)]. In the current work, we compute the thermodynamics, condensation energy, and electronic properties such as the NMR relaxation rate $1/T_1$ and ultrasonic attenuation rate. Key differences arise with the standard BCS analysis from a Fermi liquid: 1) the free energy exhibits a local minimum at $T_p$ where the pairing gap turns on discontinuously above a critical value of the repulsive HK interaction, thereby indicating a first-order transition, 2) a tri-critical point emerges, thereby demarcating the boundary between the standard second-order superconducting transition and the novel first-order regime, 3) Mottness changes the sign of the quartic coefficient in the Landau-Ginzburg free-energy fuctional relative to that in BCS, 4) as this obtains in the strongly interacting regime, it is Mott physics that underlies the generic first-order transition, 5) the condensation energy exceeds that in BCS theory suggesting that multiple Mott bands might be a way of enhancing superconducting, 6) the heat-capacity jump is non-universal and increases with the Mott scale, 7) Mottness destroys the Hebel-Slichter peak in NMR, and 8) Mottness enhances the fall-off of the ultrasonic attenuation at the pairing temperature $T_p$. As several of these properties are observed in the cuprates, our analysis here points a way forward in computing superconducting properties of strongly correlated electron matter.
△ Less
Submitted 25 April, 2022; v1 submitted 29 November, 2021;
originally announced November 2021.