Showing 1–50 of 356 results for author: Xie, T

From Tokens to Materials: Leveraging Language Models for Scientific Discovery

Authors: Yuwei Wan, Tong Xie, Nan Wu, Wenjie Zhang, Chunyu Kit, Bram Hoex

Abstract: Exploring the predictive capabilities of language models in material science is an ongoing interest. This study investigates the application of language model embeddings to enhance material property prediction in materials science. By evaluating various contextual embedding methods and pre-trained models, including Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-t… ▽ More Exploring the predictive capabilities of language models in material science is an ongoing interest. This study investigates the application of language model embeddings to enhance material property prediction in materials science. By evaluating various contextual embedding methods and pre-trained models, including Bidirectional Encoder Representations from Transformers (BERT) and Generative Pre-trained Transformers (GPT), we demonstrate that domain-specific models, particularly MatBERT significantly outperform general-purpose models in extracting implicit knowledge from compound names and material properties. Our findings reveal that information-dense embeddings from the third layer of MatBERT, combined with a context-averaging approach, offer the most effective method for capturing material-property relationships from the scientific literature. We also identify a crucial "tokenizer effect," highlighting the importance of specialized text processing techniques that preserve complete compound names while maintaining consistent token counts. These insights underscore the value of domain-specific training and tokenization in materials science applications and offer a promising pathway for accelerating the discovery and development of new materials through AI-driven approaches. △ Less

Submitted 21 October, 2024; originally announced October 2024.

Automated Proof Generation for Rust Code via Self-Evolution

Authors: Tianyu Chen, Shuai Lu, Shan Lu, Yeyun Gong, Chenyuan Yang, Xuheng Li, Md Rakib Hossain Misu, Hao Yu, Nan Duan, Peng Cheng, Fan Yang, Shuvendu K Lahiri, Tao Xie, Lidong Zhou

Abstract: Ensuring correctness is crucial for code generation. Formal verification offers a definitive assurance of correctness, but demands substantial human effort in proof construction and hence raises a pressing need for automation. The primary obstacle lies in the severe lack of data - there is much less proof than code for LLMs to train upon. In this paper, we introduce SAFE, a novel framework that ov… ▽ More Ensuring correctness is crucial for code generation. Formal verification offers a definitive assurance of correctness, but demands substantial human effort in proof construction and hence raises a pressing need for automation. The primary obstacle lies in the severe lack of data - there is much less proof than code for LLMs to train upon. In this paper, we introduce SAFE, a novel framework that overcomes the lack of human-written proof to enable automated proof generation of Rust code. SAFE establishes a self-evolving cycle where data synthesis and fine-tuning collaborate to enhance the model capability, leveraging the definitive power of a symbolic verifier in telling correct proof from incorrect ones. SAFE also re-purposes the large number of synthesized incorrect proofs to train the self-debugging capability of the fine-tuned models, empowering them to fix incorrect proofs based on the verifier's feedback. SAFE demonstrates superior efficiency and precision compared to GPT-4o. Through tens of thousands of synthesized proofs and the self-debugging mechanism, we improve the capability of open-source models, initially unacquainted with formal verification, to automatically write proof for Rust code. This advancement leads to a significant improvement in performance, achieving a 70.50% accuracy rate in a benchmark crafted by human experts, a significant leap over GPT-4o's performance of 24.46%. △ Less

Submitted 21 October, 2024; originally announced October 2024.

Diffusion-PINN Sampler

Authors: Zhekun Shi, Longlin Yu, Tianyu Xie, Cheng Zhang

Abstract: Recent success of diffusion models has inspired a surge of interest in developing sampling techniques using reverse diffusion processes. However, accurately estimating the drift term in the reverse stochastic differential equation (SDE) solely from the unnormalized target density poses significant challenges, hindering existing methods from achieving state-of-the-art performance. In this paper, we… ▽ More Recent success of diffusion models has inspired a surge of interest in developing sampling techniques using reverse diffusion processes. However, accurately estimating the drift term in the reverse stochastic differential equation (SDE) solely from the unnormalized target density poses significant challenges, hindering existing methods from achieving state-of-the-art performance. In this paper, we introduce the Diffusion-PINN Sampler (DPS), a novel diffusion-based sampling algorithm that estimates the drift term by solving the governing partial differential equation of the log-density of the underlying SDE marginals via physics-informed neural networks (PINN). We prove that the error of log-density approximation can be controlled by the PINN residual loss, enabling us to establish convergence guarantees of DPS. Experiments on a variety of sampling tasks demonstrate the effectiveness of our approach, particularly in accurately identifying mixing proportions when the target contains isolated components. △ Less

Submitted 20 October, 2024; originally announced October 2024.

Comments: 33 pages, 7 figures

EPIC: Efficient Position-Independent Context Caching for Serving Large Language Models

Authors: Junhao Hu, Wenrui Huang, Haoyi Wang, Weidong Wang, Tiancheng Hu, Qin Zhang, Hao Feng, Xusheng Chen, Yizhou Shan, Tao Xie

Abstract: Large Language Models (LLMs) are critical for a wide range of applications, but serving them efficiently becomes increasingly challenging as inputs become more complex. Context caching improves serving performance by exploiting inter-request dependency and reusing key-value (KV) cache across requests, thus improving time-to-first-token (TTFT). However, existing prefix-based context caching require… ▽ More Large Language Models (LLMs) are critical for a wide range of applications, but serving them efficiently becomes increasingly challenging as inputs become more complex. Context caching improves serving performance by exploiting inter-request dependency and reusing key-value (KV) cache across requests, thus improving time-to-first-token (TTFT). However, existing prefix-based context caching requires exact token prefix matches, limiting cache reuse in few-shot learning, multi-document QA, or retrieval-augmented generation, where prefixes may vary. In this paper, we present EPIC, an LLM serving system that introduces position-independent context caching (PIC), enabling modular KV cache reuse regardless of token chunk position (or prefix). EPIC features two key designs: AttnLink, which leverages static attention sparsity to minimize recomputation for accuracy recovery, and KVSplit, a customizable chunking method that preserves semantic coherence. Our experiments demonstrate that Epic delivers up to 8x improvements in TTFT and 7x throughput over existing systems, with negligible or no accuracy loss. By addressing the limitations of traditional caching approaches, Epic enables more scalable and efficient LLM inference. △ Less

Submitted 20 October, 2024; originally announced October 2024.

Continuous agent-based modeling of adult-child pairs based on a pseudo-energy: Relevance for public safety and egress efficiency

Authors: Chuan-Zhi Thomas Xie, Tie-Qiao Tang, Alexandre Nicolas

Abstract: Pushes, falls, stampedes, and crushes are safety hazards that emerge from the collective motion of crowds, but might be avoided by better design and guidance. While pedestrian dynamics are now getting better understood on the whole, complex heterogeneous flows involvinge.g. adult-child pairs, though widely found at e.g. crowded Chinese training schools, still defy the current understanding and cap… ▽ More Pushes, falls, stampedes, and crushes are safety hazards that emerge from the collective motion of crowds, but might be avoided by better design and guidance. While pedestrian dynamics are now getting better understood on the whole, complex heterogeneous flows involvinge.g. adult-child pairs, though widely found at e.g. crowded Chinese training schools, still defy the current understanding and capabilities of crowd simulation models. We substantially extend a recent agent-based model in which each agent's choice of motion results from the minimization ofa sum of intuitive contributions, in order to integrate adult-child pairs. This is achieved by adding a suitably defined pairing potential. The resulting model captures the relative positions of pair members in a quantitative fashion, as confirmed by small-scale controlled experiments, and alsosucceeds in describing collision avoidance between pairs. The model is used to simulate mixed adult-child flows at a T-junction and test the sensitivity to the design and pairing strategies. Simulation shows that making the post-confluence corridor wide enough is critical to avoidfriction in the flow, and that tight hand-holding is advisable for safer evacuations (whereas more loosely bound pairs get split at high density) and, more marginally, more efficient egresses in normal conditions. △ Less

Submitted 17 October, 2024; originally announced October 2024.

Journal ref: Safety Science, 2024, 177, pp.106576

ECGN: A Cluster-Aware Approach to Graph Neural Networks for Imbalanced Classification

Authors: Bishal Thapaliya, Anh Nguyen, Yao Lu, Tian Xie, Igor Grudetskyi, Fudong Lin, Antonios Valkanas, Jingyu Liu, Deepayan Chakraborty, Bilel Fehri

Abstract: Classifying nodes in a graph is a common problem. The ideal classifier must adapt to any imbalances in the class distribution. It must also use information in the clustering structure of real-world graphs. Existing Graph Neural Networks (GNNs) have not addressed both problems together. We propose the Enhanced Cluster-aware Graph Network (ECGN), a novel method that addresses these issues by integra… ▽ More Classifying nodes in a graph is a common problem. The ideal classifier must adapt to any imbalances in the class distribution. It must also use information in the clustering structure of real-world graphs. Existing Graph Neural Networks (GNNs) have not addressed both problems together. We propose the Enhanced Cluster-aware Graph Network (ECGN), a novel method that addresses these issues by integrating cluster-specific training with synthetic node generation. Unlike traditional GNNs that apply the same node update process for all nodes, ECGN learns different aggregations for different clusters. We also use the clusters to generate new minority-class nodes in a way that helps clarify the inter-class decision boundary. By combining cluster-aware embeddings with a global integration step, ECGN enhances the quality of the resulting node embeddings. Our method works with any underlying GNN and any cluster generation technique. Experimental results show that ECGN consistently outperforms its closest competitors by up to 11% on some widely studied benchmark datasets. △ Less

Submitted 15 October, 2024; originally announced October 2024.

Comments: 17 pages, 3 figures

Distributionally robust self-supervised learning for tabular data

Authors: Shantanu Ghosh, Tiankang Xie, Mikhail Kuznetsov

Abstract: Machine learning (ML) models trained using Empirical Risk Minimization (ERM) often exhibit systematic errors on specific subpopulations of tabular data, known as error slices. Learning robust representation in presence of error slices is challenging, especially in self-supervised settings during the feature reconstruction phase, due to high cardinality features and the complexity of constructing e… ▽ More Machine learning (ML) models trained using Empirical Risk Minimization (ERM) often exhibit systematic errors on specific subpopulations of tabular data, known as error slices. Learning robust representation in presence of error slices is challenging, especially in self-supervised settings during the feature reconstruction phase, due to high cardinality features and the complexity of constructing error sets. Traditional robust representation learning methods are largely focused on improving worst group performance in supervised setting in computer vision, leaving a gap in approaches tailored for tabular data. We address this gap by developing a framework to learn robust representation in tabular data during self-supervised pre-training. Our approach utilizes an encoder-decoder model trained with Masked Language Modeling (MLM) loss to learn robust latent representations. This paper applies the Just Train Twice (JTT) and Deep Feature Reweighting (DFR) methods during the pre-training phase for tabular data. These methods fine-tune the ERM pre-trained model by up-weighting error-prone samples or creating balanced datasets for specific categorical features. This results in specialized models for each feature, which are then used in an ensemble approach to enhance downstream classification performance. This methodology improves robustness across slices, thus enhancing overall generalization performance. Extensive experiments across various datasets demonstrate the efficacy of our approach. △ Less

Submitted 11 October, 2024; originally announced October 2024.

Comments: TRL Workshop@NeurIPS2024

Computing Generalized Eigenfunctions in Rigged Hilbert Spaces

Authors: Matthew J. Colbrook, Andrew Horning, Tianyiwa Xie

Abstract: We introduce a simple, general, and convergent scheme to compute generalized eigenfunctions of self-adjoint operators with continuous spectra on rigged Hilbert spaces. Our approach does not require prior knowledge about the eigenfunctions, such as asymptotics or other analytic properties. Instead, we carefully sample the range of the resolvent operator to construct smooth and accurate wave packet… ▽ More We introduce a simple, general, and convergent scheme to compute generalized eigenfunctions of self-adjoint operators with continuous spectra on rigged Hilbert spaces. Our approach does not require prior knowledge about the eigenfunctions, such as asymptotics or other analytic properties. Instead, we carefully sample the range of the resolvent operator to construct smooth and accurate wave packet approximations to generalized eigenfunctions. We prove high-order convergence in key topologies, including weak-star convergence for distributional eigenfunctions, uniform convergence on compact sets for locally smooth generalized eigenfunctions, and convergence in seminorms for separable Frechet spaces, covering the majority of physical scenarios. The method's performance is illustrated with applications to both differential and integral operators, culminating in the computation of spectral measures and generalized eigenfunctions for an operator associated with Poincare's internal waves problem. These computations corroborate experimental results and highlight the method's utility for a broad range of spectral problems in physics. △ Less

Submitted 10 October, 2024; originally announced October 2024.

Careful About What App Promotion Ads Recommend! Detecting and Explaining Malware Promotion via App Promotion Graph

Authors: Shang Ma, Chaoran Chen, Shao Yang, Shifu Hou, Toby Jia-Jun Li, Xusheng Xiao, Tao Xie, Yanfang Ye

Abstract: In Android apps, their developers frequently place app promotion ads, namely advertisements to promote other apps. Unfortunately, the inadequate vetting of ad content allows malicious developers to exploit app promotion ads as a new distribution channel for malware. To help detect malware distributed via app promotion ads, in this paper, we propose a novel approach, named ADGPE, that synergistical… ▽ More In Android apps, their developers frequently place app promotion ads, namely advertisements to promote other apps. Unfortunately, the inadequate vetting of ad content allows malicious developers to exploit app promotion ads as a new distribution channel for malware. To help detect malware distributed via app promotion ads, in this paper, we propose a novel approach, named ADGPE, that synergistically integrates app user interface (UI) exploration with graph learning to automatically collect app promotion ads, detect malware promoted by these ads, and explain the promotion mechanisms employed by the detected malware. Our evaluation on 18, 627 app promotion ads demonstrates the substantial risks in the app promotion ecosystem. △ Less

Submitted 9 October, 2024; originally announced October 2024.

Comments: NDSS Symposium 2025 Accepted Papers

MOOSE-Chem: Large Language Models for Rediscovering Unseen Chemistry Scientific Hypotheses

Authors: Zonglin Yang, Wanhao Liu, Ben Gao, Tong Xie, Yuqiang Li, Wanli Ouyang, Soujanya Poria, Erik Cambria, Dongzhan Zhou

Abstract: Scientific discovery contributes largely to human society's prosperity, and recent progress shows that LLMs could potentially catalyze this process. However, it is still unclear whether LLMs can discover novel and valid hypotheses in chemistry. In this work, we investigate this central research question: Can LLMs automatically discover novel and valid chemistry research hypotheses given only a che… ▽ More Scientific discovery contributes largely to human society's prosperity, and recent progress shows that LLMs could potentially catalyze this process. However, it is still unclear whether LLMs can discover novel and valid hypotheses in chemistry. In this work, we investigate this central research question: Can LLMs automatically discover novel and valid chemistry research hypotheses given only a chemistry research background (consisting of a research question and/or a background survey), without limitation on the domain of the research question? After extensive discussions with chemistry experts, we propose an assumption that a majority of chemistry hypotheses can be resulted from a research background and several inspirations. With this key insight, we break the central question into three smaller fundamental questions. In brief, they are: (1) given a background question, whether LLMs can retrieve good inspirations; (2) with background and inspirations, whether LLMs can lead to hypothesis; and (3) whether LLMs can identify good hypotheses to rank them higher. To investigate these questions, we construct a benchmark consisting of 51 chemistry papers published in Nature, Science, or a similar level in 2024 (all papers are only available online since 2024). Every paper is divided by chemistry PhD students into three components: background, inspirations, and hypothesis. The goal is to rediscover the hypothesis, given only the background and a large randomly selected chemistry literature corpus consisting the ground truth inspiration papers, with LLMs trained with data up to 2023. We also develop an LLM-based multi-agent framework that leverages the assumption, consisting of three stages reflecting the three smaller questions. The proposed method can rediscover many hypotheses with very high similarity with the ground truth ones, covering the main innovations. △ Less

Submitted 12 October, 2024; v1 submitted 9 October, 2024; originally announced October 2024.

Comments: Code and Benchmark are available at https://github.com/ZonglinY/MOOSE-Chem.git

LLaMA-Berry: Pairwise Optimization for O1-like Olympiad-Level Mathematical Reasoning

Authors: Di Zhang, Jianbo Wu, Jingdi Lei, Tong Che, Jiatong Li, Tong Xie, Xiaoshui Huang, Shufei Zhang, Marco Pavone, Yuqiang Li, Wanli Ouyang, Dongzhan Zhou

Abstract: This paper presents an advanced mathematical problem-solving framework, LLaMA-Berry, for enhancing the mathematical reasoning ability of Large Language Models (LLMs). The framework combines Monte Carlo Tree Search (MCTS) with iterative Self-Refine to optimize the reasoning path and utilizes a pairwise reward model to evaluate different paths globally. By leveraging the self-critic and rewriting ca… ▽ More This paper presents an advanced mathematical problem-solving framework, LLaMA-Berry, for enhancing the mathematical reasoning ability of Large Language Models (LLMs). The framework combines Monte Carlo Tree Search (MCTS) with iterative Self-Refine to optimize the reasoning path and utilizes a pairwise reward model to evaluate different paths globally. By leveraging the self-critic and rewriting capabilities of LLMs, Self-Refine applied to MCTS (SR-MCTS) overcomes the inefficiencies and limitations of conventional step-wise and greedy search algorithms by fostering a more efficient exploration of solution spaces. Pairwise Preference Reward Model~(PPRM), inspired by Reinforcement Learning from Human Feedback (RLHF), is then used to model pairwise preferences between solutions, utilizing an Enhanced Borda Count (EBC) method to synthesize these preferences into a global ranking score to find better answers. This approach addresses the challenges of scoring variability and non-independent distributions in mathematical reasoning tasks. The framework has been tested on general and advanced benchmarks, showing superior performance in terms of search efficiency and problem-solving capability compared to existing methods like ToT and rStar, particularly in complex Olympiad-level benchmarks, including GPQA, AIME24 and AMC23. △ Less

Submitted 3 October, 2024; originally announced October 2024.

Structural-Entropy-Based Sample Selection for Efficient and Effective Learning

Authors: Tianchi Xie, Jiangning Zhu, Guozu Ma, Minzhi Lin, Wei Chen, Weikai Yang, Shixia Liu

Abstract: Sample selection improves the efficiency and effectiveness of machine learning models by providing informative and representative samples. Typically, samples can be modeled as a sample graph, where nodes are samples and edges represent their similarities. Most existing methods are based on local information, such as the training difficulty of samples, thereby overlooking global information, such a… ▽ More Sample selection improves the efficiency and effectiveness of machine learning models by providing informative and representative samples. Typically, samples can be modeled as a sample graph, where nodes are samples and edges represent their similarities. Most existing methods are based on local information, such as the training difficulty of samples, thereby overlooking global information, such as connectivity patterns. This oversight can result in suboptimal selection because global information is crucial for ensuring that the selected samples well represent the structural properties of the graph. To address this issue, we employ structural entropy to quantify global information and losslessly decompose it from the whole graph to individual nodes using the Shapley value. Based on the decomposition, we present $\textbf{S}$tructural-$\textbf{E}$ntropy-based sample $\textbf{S}$election ($\textbf{SES}$), a method that integrates both global and local information to select informative and representative samples. SES begins by constructing a $k$NN-graph among samples based on their similarities. It then measures sample importance by combining structural entropy (global metric) with training difficulty (local metric). Finally, SES applies importance-biased blue noise sampling to select a set of diverse and representative samples. Comprehensive experiments on three learning scenarios -- supervised learning, active learning, and continual learning -- clearly demonstrate the effectiveness of our method. △ Less

Submitted 5 October, 2024; v1 submitted 3 October, 2024; originally announced October 2024.

Comments: Submitted to ICLR 2025

PEDRO: Parameter-Efficient Fine-tuning with Prompt DEpenDent Representation MOdification

Authors: Tianfang Xie, Tianjing Li, Wei Zhu, Wei Han, Yi Zhao

Abstract: Due to their substantial sizes, large language models (LLMs) are typically deployed within a single-backbone multi-tenant framework. In this setup, a single instance of an LLM backbone must cater to multiple users or tasks through the application of various parameter-efficient fine-tuning (PEFT) models. Despite the availability of numerous effective PEFT techniques such as LoRA, there remains a ne… ▽ More Due to their substantial sizes, large language models (LLMs) are typically deployed within a single-backbone multi-tenant framework. In this setup, a single instance of an LLM backbone must cater to multiple users or tasks through the application of various parameter-efficient fine-tuning (PEFT) models. Despite the availability of numerous effective PEFT techniques such as LoRA, there remains a need for a PEFT approach that achieves both high efficiency during inference and competitive performance on downstream tasks. In this research, we introduce a new and straightforward PEFT methodology named \underline{P}rompt D\underline{E}pen\underline{D}ent \underline{R}epresentation M\underline{O}dification (PEDRO). The proposed method involves integrating a lightweight vector generator into each Transformer layer, which generates vectors contingent upon the input prompts. These vectors then modify the hidden representations created by the LLM through a dot product operation, thereby influencing the semantic output and generated content of the model. Extensive experimentation across a variety of tasks indicates that: (a) PEDRO surpasses recent PEFT benchmarks when using a similar number of tunable parameters. (b) Under the single-backbone multi-tenant deployment model, PEDRO exhibits superior efficiency compared to LoRA, indicating significant industrial potential. △ Less

Submitted 26 September, 2024; originally announced September 2024.

Comments: arXiv admin note: text overlap with arXiv:2405.18203

Skill-Adpative Imitation Learning for UI Test Reuse

Authors: Mengzhou Wu, Hao Wang, Jun Ren, Yuan Cao, Yuetong Li, Alex Jiang, Dezhi Ran, Yitao Hu, Wei Yang, Tao Xie

Abstract: To alleviate the substantial cost of manually crafting user interface (UI) test cases, UI test migration aims to automatically generate test cases for a target mobile application (app) by adapting those from a source app that shares similar functionalities. Traditionally, this process has been approached as a sequential UI-event-mapping problem, where events in the source app are mapped to those i… ▽ More To alleviate the substantial cost of manually crafting user interface (UI) test cases, UI test migration aims to automatically generate test cases for a target mobile application (app) by adapting those from a source app that shares similar functionalities. Traditionally, this process has been approached as a sequential UI-event-mapping problem, where events in the source app are mapped to those in the target one based on their textual descriptions. Prior research has extensively focused on enhancing the event-mapping accuracy of NLP models. Although the advent of large language models (LLMs) with impressive NLP capabilities suggests the potential for near-perfect event-mapping, our study demonstrates that even the highly accurate event-mapping of LLMs is insufficient to address the implementation discrepancies between the source and the target apps, reducing the overall effectiveness of LLM-driven solutions for UI test migration. To address this challenge, in this paper, we propose SAIL, a skill-adaptive imitation learning framework designed to enhance the effectiveness of UI test migration through two key designs. First, SAIL leverages the source test cases as demonstrations and employs a multi-level abstraction of test cases' underlying skills, so as to extract the testing information from source test cases as the knowledge base for the subsequent test generation on the target app. Second, SAIL selectively reuses a subset of the learned skills to guide the generation of test cases for the target app with its novel context- and history-aware skill adaptation. While SAIL can be instantiated with any imitation learning techniques, we utilize the in-context learning capabilities of LLMs to instantiate SAIL. Evaluations results show that SAIL substantially improves the effectiveness of UI test migration, with 149\% higher success rate than state-of-the-art approaches. △ Less

Submitted 20 September, 2024; originally announced September 2024.

CtRNet-X: Camera-to-Robot Pose Estimation in Real-world Conditions Using a Single Camera

Authors: Jingpei Lu, Zekai Liang, Tristin Xie, Florian Ritcher, Shan Lin, Sainan Liu, Michael C. Yip

Abstract: Camera-to-robot calibration is crucial for vision-based robot control and requires effort to make it accurate. Recent advancements in markerless pose estimation methods have eliminated the need for time-consuming physical setups for camera-to-robot calibration. While the existing markerless pose estimation methods have demonstrated impressive accuracy without the need for cumbersome setups, they r… ▽ More Camera-to-robot calibration is crucial for vision-based robot control and requires effort to make it accurate. Recent advancements in markerless pose estimation methods have eliminated the need for time-consuming physical setups for camera-to-robot calibration. While the existing markerless pose estimation methods have demonstrated impressive accuracy without the need for cumbersome setups, they rely on the assumption that all the robot joints are visible within the camera's field of view. However, in practice, robots usually move in and out of view, and some portion of the robot may stay out-of-frame during the whole manipulation task due to real-world constraints, leading to a lack of sufficient visual features and subsequent failure of these approaches. To address this challenge and enhance the applicability to vision-based robot control, we propose a novel framework capable of estimating the robot pose with partially visible robot manipulators. Our approach leverages the Vision-Language Models for fine-grained robot components detection, and integrates it into a keypoint-based pose estimation network, which enables more robust performance in varied operational conditions. The framework is evaluated on both public robot datasets and self-collected partial-view datasets to demonstrate our robustness and generalizability. As a result, this method is effective for robot pose estimation in a wider range of real-world manipulation scenarios. △ Less

Submitted 16 September, 2024; originally announced September 2024.

Comments: 7 pages, 5 figures, project website: https://sites.google.com/ucsd.edu/ctrnet-x

Improving Tree Probability Estimation with Stochastic Optimization and Variance Reduction

Authors: Tianyu Xie, Musu Yuan, Minghua Deng, Cheng Zhang

Abstract: Probability estimation of tree topologies is one of the fundamental tasks in phylogenetic inference. The recently proposed subsplit Bayesian networks (SBNs) provide a powerful probabilistic graphical model for tree topology probability estimation by properly leveraging the hierarchical structure of phylogenetic trees. However, the expectation maximization (EM) method currently used for learning SB… ▽ More Probability estimation of tree topologies is one of the fundamental tasks in phylogenetic inference. The recently proposed subsplit Bayesian networks (SBNs) provide a powerful probabilistic graphical model for tree topology probability estimation by properly leveraging the hierarchical structure of phylogenetic trees. However, the expectation maximization (EM) method currently used for learning SBN parameters does not scale up to large data sets. In this paper, we introduce several computationally efficient methods for training SBNs and show that variance reduction could be the key for better performance. Furthermore, we also introduce the variance reduction technique to improve the optimization of SBN parameters for variational Bayesian phylogenetic inference (VBPI). Extensive synthetic and real data experiments demonstrate that our methods outperform previous baseline methods on the tasks of tree topology probability estimation as well as Bayesian phylogenetic inference using SBNs. △ Less

Submitted 8 September, 2024; originally announced September 2024.

Comments: 23 pages, 6 figures, 7 tables

MusicMamba: A Dual-Feature Modeling Approach for Generating Chinese Traditional Music with Modal Precision

Authors: Jiatao Chen, Tianming Xie, Xing Tang, Jing Wang, Wenjing Dong, Bing Shi

Abstract: In recent years, deep learning has significantly advanced the MIDI domain, solidifying music generation as a key application of artificial intelligence. However, existing research primarily focuses on Western music and encounters challenges in generating melodies for Chinese traditional music, especially in capturing modal characteristics and emotional expression. To address these issues, we propo… ▽ More In recent years, deep learning has significantly advanced the MIDI domain, solidifying music generation as a key application of artificial intelligence. However, existing research primarily focuses on Western music and encounters challenges in generating melodies for Chinese traditional music, especially in capturing modal characteristics and emotional expression. To address these issues, we propose a new architecture, the Dual-Feature Modeling Module, which integrates the long-range dependency modeling of the Mamba Block with the global structure capturing capabilities of the Transformer Block. Additionally, we introduce the Bidirectional Mamba Fusion Layer, which integrates local details and global structures through bidirectional scanning, enhancing the modeling of complex sequences. Building on this architecture, we propose the REMI-M representation, which more accurately captures and generates modal information in melodies. To support this research, we developed FolkDB, a high-quality Chinese traditional music dataset encompassing various styles and totaling over 11 hours of music. Experimental results demonstrate that the proposed architecture excels in generating melodies with Chinese traditional music characteristics, offering a new and effective solution for music generation. △ Less

Submitted 4 September, 2024; originally announced September 2024.

PSE-Net: Channel Pruning for Convolutional Neural Networks with Parallel-subnets Estimator

Authors: Shiguang Wang, Tao Xie, Haijun Liu, Xingcheng Zhang, Jian Cheng

Abstract: Channel Pruning is one of the most widespread techniques used to compress deep neural networks while maintaining their performances. Currently, a typical pruning algorithm leverages neural architecture search to directly find networks with a configurable width, the key step of which is to identify representative subnet for various pruning ratios by training a supernet. However, current methods mai… ▽ More Channel Pruning is one of the most widespread techniques used to compress deep neural networks while maintaining their performances. Currently, a typical pruning algorithm leverages neural architecture search to directly find networks with a configurable width, the key step of which is to identify representative subnet for various pruning ratios by training a supernet. However, current methods mainly follow a serial training strategy to optimize supernet, which is very time-consuming. In this work, we introduce PSE-Net, a novel parallel-subnets estimator for efficient channel pruning. Specifically, we propose a parallel-subnets training algorithm that simulate the forward-backward pass of multiple subnets by droping extraneous features on batch dimension, thus various subnets could be trained in one round. Our proposed algorithm facilitates the efficiency of supernet training and equips the network with the ability to interpolate the accuracy of unsampled subnets, enabling PSE-Net to effectively evaluate and rank the subnets. Over the trained supernet, we develop a prior-distributed-based sampling algorithm to boost the performance of classical evolutionary search. Such algorithm utilizes the prior information of supernet training phase to assist in the search of optimal subnets while tackling the challenge of discovering samples that satisfy resource constraints due to the long-tail distribution of network configuration. Extensive experiments demonstrate PSE-Net outperforms previous state-of-the-art channel pruning methods on the ImageNet dataset while retaining superior supernet training efficiency. For example, under 300M FLOPs constraint, our pruned MobileNetV2 achieves 75.2% Top-1 accuracy on ImageNet dataset, exceeding the original MobileNetV2 by 2.6 units while only cost 30%/16% times than BCNet/AutoAlim. △ Less

Submitted 28 August, 2024; originally announced August 2024.

Comments: 10pages, Neural Networks

SWE-bench-java: A GitHub Issue Resolving Benchmark for Java

Authors: Daoguang Zan, Zhirong Huang, Ailun Yu, Shaoxin Lin, Yifan Shi, Wei Liu, Dong Chen, Zongshuai Qi, Hao Yu, Lei Yu, Dezhi Ran, Muhan Zeng, Bo Shen, Pan Bian, Guangtai Liang, Bei Guan, Pengjie Huang, Tao Xie, Yongji Wang, Qianxiang Wang

Abstract: GitHub issue resolving is a critical task in software engineering, recently gaining significant attention in both industry and academia. Within this task, SWE-bench has been released to evaluate issue resolving capabilities of large language models (LLMs), but has so far only focused on Python version. However, supporting more programming languages is also important, as there is a strong demand in… ▽ More GitHub issue resolving is a critical task in software engineering, recently gaining significant attention in both industry and academia. Within this task, SWE-bench has been released to evaluate issue resolving capabilities of large language models (LLMs), but has so far only focused on Python version. However, supporting more programming languages is also important, as there is a strong demand in industry. As a first step toward multilingual support, we have developed a Java version of SWE-bench, called SWE-bench-java. We have publicly released the dataset, along with the corresponding Docker-based evaluation environment and leaderboard, which will be continuously maintained and updated in the coming months. To verify the reliability of SWE-bench-java, we implement a classic method SWE-agent and test several powerful LLMs on it. As is well known, developing a high-quality multi-lingual benchmark is time-consuming and labor-intensive, so we welcome contributions through pull requests or collaboration to accelerate its iteration and refinement, paving the way for fully automated programming. △ Less

Submitted 26 August, 2024; originally announced August 2024.

Comments: This work is in progress

Joint PET-MRI Reconstruction with Diffusion Stochastic Differential Model

Authors: Taofeng Xie, Zhuoxu Cui, Congcong Liu, Chen Luo, Huayu Wang, Yuanzhi Zhang, Xuemei Wang, Yihang Zhou, Qiyu Jin, Guoqing Chen, Dong Liang, Haifeng Wang

Abstract: PET suffers from a low signal-to-noise ratio. Meanwhile, the k-space data acquisition process in MRI is time-consuming by PET-MRI systems. We aim to accelerate MRI and improve PET image quality. This paper proposed a novel joint reconstruction model by diffusion stochastic differential equations based on learning the joint probability distribution of PET and MRI. Compare the results underscore the… ▽ More PET suffers from a low signal-to-noise ratio. Meanwhile, the k-space data acquisition process in MRI is time-consuming by PET-MRI systems. We aim to accelerate MRI and improve PET image quality. This paper proposed a novel joint reconstruction model by diffusion stochastic differential equations based on learning the joint probability distribution of PET and MRI. Compare the results underscore the qualitative and quantitative improvements our model brings to PET and MRI reconstruction, surpassing the current state-of-the-art methodologies. Joint PET-MRI reconstruction is a challenge in the PET-MRI system. This studies focused on the relationship extends beyond edges. In this study, PET is generated from MRI by learning joint probability distribution as the relationship. △ Less

Submitted 7 August, 2024; originally announced August 2024.

Comments: Accepted as ISMRM 2024 Digital poster 6575. 04-09 May 2024 Singapore

Journal ref: ISMRM 2024 Digital poster 6575

Superconductivity and spin canting in spin-orbit proximitized rhombohedral trilayer graphene

Authors: Caitlin L. Patterson, Owen I. Sheekey, Trevor B. Arp, Ludwig F. W. Holleis, Jin Ming Koh, Youngjoon Choi, Tian Xie, Siyuan Xu, Evgeny Redekop, Grigory Babikyan, Haoxin Zhou, Xiang Cheng, Takashi Taniguchi, Kenji Watanabe, Chenhao Jin, Etienne Lantagne-Hurtubise, Jason Alicea, Andrea F. Young

Abstract: Graphene and transition metal dichalcogenide flat-band systems show similar phase diagrams, replete with magnetic and superconducting phases. An abiding question has been whether magnetic ordering competes with superconductivity or facilitates pairing. The advent of crystalline graphene superconductors enables a new generation of controlled experiments to probe the microscopic origin of supercondu… ▽ More Graphene and transition metal dichalcogenide flat-band systems show similar phase diagrams, replete with magnetic and superconducting phases. An abiding question has been whether magnetic ordering competes with superconductivity or facilitates pairing. The advent of crystalline graphene superconductors enables a new generation of controlled experiments to probe the microscopic origin of superconductivity. For example, recent studies of Bernal bilayer graphene show a dramatic increase in the observed domain and critical temperature $T_c$ of superconducting states in the presence of enhanced spin-orbit coupling; the mechanism for this enhancement, however, remains unclear. Here, we show that introducing spin-orbit coupling in rhombohedral trilayer graphene (RTG) via substrate proximity effect generates new superconducting pockets for both electron and hole doping, with maximal $T_c\approx$ 300mK three times larger than in RTG encapsulated by hexagonal boron nitride alone. Using local magnetometry and thermodynamic compressibility measurements, we show that superconductivity straddles an apparently continuous transition between a spin-canted state with a finite in-plane magnetic moment and a state with complete spin-valley locking. This transition is reproduced in our Hartree-Fock calculations, where it is driven by the competition between spin-orbit coupling and the carrier-density-tuned Hund's interaction. Our experiment suggests that the enhancement of superconductivity by spin-orbit coupling is driven not by a change in the ground state symmetry or degeneracy but rather by a quantitative change in the canting angle. These results align with a recently proposed mechanism for the enhancement of superconductivity in spin-orbit coupled rhombohedral multilayers, in which fluctuations in the spin-canting order contribute to the pairing interaction. △ Less

Submitted 19 August, 2024; originally announced August 2024.

Variational Bayesian Phylogenetic Inference with Semi-implicit Branch Length Distributions

Authors: Tianyu Xie, Frederick A. Matsen IV, Marc A. Suchard, Cheng Zhang

Abstract: Reconstructing the evolutionary history relating a collection of molecular sequences is the main subject of modern Bayesian phylogenetic inference. However, the commonly used Markov chain Monte Carlo methods can be inefficient due to the complicated space of phylogenetic trees, especially when the number of sequences is large. An alternative approach is variational Bayesian phylogenetic inference… ▽ More Reconstructing the evolutionary history relating a collection of molecular sequences is the main subject of modern Bayesian phylogenetic inference. However, the commonly used Markov chain Monte Carlo methods can be inefficient due to the complicated space of phylogenetic trees, especially when the number of sequences is large. An alternative approach is variational Bayesian phylogenetic inference (VBPI) which transforms the inference problem into an optimization problem. While effective, the default diagonal lognormal approximation for the branch lengths of the tree used in VBPI is often insufficient to capture the complexity of the exact posterior. In this work, we propose a more flexible family of branch length variational posteriors based on semi-implicit hierarchical distributions using graph neural networks. We show that this semi-implicit construction emits straightforward permutation equivariant distributions, and therefore can handle the non-Euclidean branch length space across different tree topologies with ease. To deal with the intractable marginal probability of semi-implicit variational distributions, we develop several alternative lower bounds for stochastic optimization. We demonstrate the effectiveness of our proposed method over baseline methods on benchmark data examples, in terms of both marginal likelihood estimation and branch length posterior approximation. △ Less

Submitted 9 August, 2024; originally announced August 2024.

Comments: 26 pages, 7 figures

First-principles study on spin density wave in La$_3$Ni$_2$O$_{7-δ}$

Authors: Xiao-Sheng Ni, Yuyang Ji, Lixin He, Tao Xie, Dao-Xin Yao, Meng Wang, Kun Cao

Abstract: The recent discovery of high-temperature superconductivity in high-pressurized La$_3$Ni$_2$O$_{7-δ}$ has garnered significant attention. Using density functional theory, we investigate the magnetic properties of La$_3$Ni$_2$O$_7$ at ambient pressure. Our calculations suggest that with $δ=0$, the double spin stripe phase is favored as the magnetic ground state. Oxygen vacancies may effectively turn… ▽ More The recent discovery of high-temperature superconductivity in high-pressurized La$_3$Ni$_2$O$_{7-δ}$ has garnered significant attention. Using density functional theory, we investigate the magnetic properties of La$_3$Ni$_2$O$_7$ at ambient pressure. Our calculations suggest that with $δ=0$, the double spin stripe phase is favored as the magnetic ground state. Oxygen vacancies may effectively turn nearest Ni spins into charge sites. Consequently, with moderate $δ$ values, our theoretical magnetic ground state exhibits characteristics of both double spin stripe and spin-charge stripe configurations, providing a natural explanation to reconcile the seemingly contradictory experimental findings that suggest both the configurations as candidates for the spin-density-wave phase. With higher $δ$ values, we anticipate the ground state to become a spin-glass-like noncollinear magnetic phase with only short-range order. The oxygen vacancies are expected to significantly impact the magnetic excitations and the transition temperatures $T_{SDW}$. We further offer a plausible explanation for the experimental observations that the measured $T_{SDW}$ appears insensitive to the variation of samples and the lack of direct evidence for long-range magnetic ordering. △ Less

Submitted 27 July, 2024; originally announced July 2024.

Wonderful Matrices: More Efficient and Effective Architecture for Language Modeling Tasks

Authors: Jingze Shi, Bingheng Wu, Ting Xie, Lu He

Abstract: Recent studies have shown that, relative position encoding performs well in selective state space model scanning algorithms, and the architecture that balances SSM and Attention enhances the efficiency and effectiveness of the algorithm, while the sparse activation of the mixture of experts reduces the training cost. We studied the effectiveness of using different position encodings in structured… ▽ More Recent studies have shown that, relative position encoding performs well in selective state space model scanning algorithms, and the architecture that balances SSM and Attention enhances the efficiency and effectiveness of the algorithm, while the sparse activation of the mixture of experts reduces the training cost. We studied the effectiveness of using different position encodings in structured state space dual algorithms, and the more effective SSD-Attn internal and external function mixing method, and designed a more efficient cross domain mixture of experts. We found that the same matrix is very wonderful in different algorithms, which allows us to establish a new hybrid sparse architecture: Cheems. Compared with other hybrid architectures, it is more efficient and more effective in language modeling tasks. △ Less

Submitted 12 October, 2024; v1 submitted 23 July, 2024; originally announced July 2024.

Regulated magnetic anisotropy and charge density wave in uniformly fabricated Janus CrTeSe monolayer

Authors: Jin-Hua Nie, Cong Wang, Mao-Peng Miao, Kang-Di Niu, Tao Xie, Ting-Fei Guo, Wen-Hao Zhang, Chao-Fei Liu, Rui-Jing Sun, Jian-Wang Zhou, Jun-Hao Lin, Wei Ji, Ying-Shuang Fu

Abstract: Two-dimensional materials with Janus structure host novel physical properties due to their inversional symmetry breaking. However, it remains elusive to synthesize Janus monolayer crystals with tailored long-range magnetic orders. Here, we have developed a general method to fabricate uniform Janus CrTeSe monolayers by selective selenization of preformed CrTe2 monolayers with molecular beam epitaxy… ▽ More Two-dimensional materials with Janus structure host novel physical properties due to their inversional symmetry breaking. However, it remains elusive to synthesize Janus monolayer crystals with tailored long-range magnetic orders. Here, we have developed a general method to fabricate uniform Janus CrTeSe monolayers by selective selenization of preformed CrTe2 monolayers with molecular beam epitaxy. The uniform Janus structure of CrTeSe with high crystal quality is confirmed by high-resolution scanning transmission electron microscopy. Spin-polarized scanning tunneling microscopy/spectroscopy measurements unveil that the Janus CrTeSe undergoes a charge density wave (CDW) transition and a robust antiferromagnetic order. The magnetic anisotropy of CrTeSe is drastically altered compared to monolayer CrTe2 by the breaking symmetries induced from the Janus structure and the CDW transition, as is substantiated with first principles calculations. Our research achieves the construction of large-area Janus structures, and artificially tailors the electronic and magnetic properties of Janus systems at the two-dimensional limit. △ Less

Submitted 23 July, 2024; originally announced July 2024.

Comments: 19 pages, 4 figures

L^2CL: Embarrassingly Simple Layer-to-Layer Contrastive Learning for Graph Collaborative Filtering

Authors: Xinzhou Jin, Jintang Li, Liang Chen, Chenyun Yu, Yuanzhen Xie, Tao Xie, Chengxiang Zhuo, Zang Li, Zibin Zheng

Abstract: Graph neural networks (GNNs) have recently emerged as an effective approach to model neighborhood signals in collaborative filtering. Towards this research line, graph contrastive learning (GCL) demonstrates robust capabilities to address the supervision label shortage issue through generating massive self-supervised signals. Despite its effectiveness, GCL for recommendation suffers seriously from… ▽ More Graph neural networks (GNNs) have recently emerged as an effective approach to model neighborhood signals in collaborative filtering. Towards this research line, graph contrastive learning (GCL) demonstrates robust capabilities to address the supervision label shortage issue through generating massive self-supervised signals. Despite its effectiveness, GCL for recommendation suffers seriously from two main challenges: i) GCL relies on graph augmentation to generate semantically different views for contrasting, which could potentially disrupt key information and introduce unwanted noise; ii) current works for GCL primarily focus on contrasting representations using sophisticated networks architecture (usually deep) to capture high-order interactions, which leads to increased computational complexity and suboptimal training efficiency. To this end, we propose L2CL, a principled Layer-to-Layer Contrastive Learning framework that contrasts representations from different layers. By aligning the semantic similarities between different layers, L2CL enables the learning of complex structural relationships and gets rid of the noise perturbation in stochastic data augmentation. Surprisingly, we find that L2CL, using only one-hop contrastive learning paradigm, is able to capture intrinsic semantic structures and improve the quality of node representation, leading to a simple yet effective architecture. We also provide theoretical guarantees for L2CL in minimizing task-irrelevant information. Extensive experiments on five real-world datasets demonstrate the superiority of our model over various state-of-the-art collaborative filtering methods. Our code is available at https://github.com/downeykking/L2CL. △ Less

Submitted 19 July, 2024; originally announced July 2024.

Isospin Pomeranchuk effect and finite temperature resistivity minimum in rhombohedral graphene

Authors: Ludwig Holleis, Tian Xie, Siyuan Xu, Haoxin Zhou, Caitlin L. Patterson, Archisman Panigrahi, Takashi Taniguchi, Kenji Watanabe, Leonid S. Levitov, Chenhao Jin, Erez Berg, Andrea F. Young

Abstract: Magnetism typically arises from the effect of exchange interactions on highly localized fermionic wave functions in f- and d-atomic orbitals. In rhombohedral graphene multilayers, in contrast, magnetism -- manifesting as spontaneous polarization into one or more spin and valley flavors[1-7]-originates from fully itinerant electrons near a Van Hove singularity. Here, we show that despite the absenc… ▽ More Magnetism typically arises from the effect of exchange interactions on highly localized fermionic wave functions in f- and d-atomic orbitals. In rhombohedral graphene multilayers, in contrast, magnetism -- manifesting as spontaneous polarization into one or more spin and valley flavors[1-7]-originates from fully itinerant electrons near a Van Hove singularity. Here, we show that despite the absence of localized electronic orbitals on the scale of the interparticle separation, rhombohedral graphene bi- and trilayers show thermodynamic signatures typically associated with disordered local magnetic moments. Specifically, while long range valley order vanishes at temperatures of a few Kelvin, thermodynamic characteristics of fluctuating moments, probed via both electronic compressibility[2-4] and exciton sensing measurements[8], survive to temperatures an order of magnitude higher. Measurements of the electronic entropy in this regime reveal a contribution of $ΔS \approx 1k_B$/charge carrier, onsetting at the Curie temperature, while first order phase transitions show an isospin "Pomeranchuk effect" in which the fluctuating moment phase is entropically favored over the nearby symmetric Fermi liquid[9, 10]. Our results imply that despite the itinerant nature of the electron wave functions, the spin- and valley polarization of individual electrons are decoupled, a phenomenon typically associated with localized moments, as happens, for example, in solid 3He[11]. Transport measurements, surprisingly, show a finite temperature resistance minimum within the fluctuating moment regime, which we attribute to the interplay of fluctuating magnetic moments and electron phonon scattering. Our results highlight the universality of soft isospin modes to two dimensional flat band systems and their possible relevance both to low-temperature ground states and high temperature transport anomalies. △ Less

Submitted 18 July, 2024; originally announced July 2024.

Correcting the Mythos of KL-Regularization: Direct Alignment without Overoptimization via Chi-Squared Preference Optimization

Authors: Audrey Huang, Wenhao Zhan, Tengyang Xie, Jason D. Lee, Wen Sun, Akshay Krishnamurthy, Dylan J. Foster

Abstract: Language model alignment methods, such as reinforcement learning from human feedback (RLHF), have led to impressive advances in language model capabilities, but existing techniques are limited by a widely observed phenomenon known as overoptimization, where the quality of the language model plateaus or degrades over the course of the alignment process. Overoptimization is often attributed to overf… ▽ More Language model alignment methods, such as reinforcement learning from human feedback (RLHF), have led to impressive advances in language model capabilities, but existing techniques are limited by a widely observed phenomenon known as overoptimization, where the quality of the language model plateaus or degrades over the course of the alignment process. Overoptimization is often attributed to overfitting to an inaccurate reward model, and while it can be mitigated through online data collection, this is infeasible in many settings. This raises a fundamental question: Do existing offline alignment algorithms make the most of the data they have, or can their sample-efficiency be improved further? We address this question with a new algorithm for offline alignment, $χ^2$-Preference Optimization ($χ$PO). $χ$PO is a one-line change to Direct Preference Optimization (DPO; Rafailov et al., 2023), which only involves modifying the logarithmic link function in the DPO objective. Despite this minimal change, $χ$PO implicitly implements the principle of pessimism in the face of uncertainty via regularization with the $χ^2$-divergence -- which quantifies uncertainty more effectively than KL-regularization -- and provably alleviates overoptimization, achieving sample-complexity guarantees based on single-policy concentrability -- the gold standard in offline reinforcement learning. $χ$PO's simplicity and strong guarantees make it the first practical and general-purpose offline alignment algorithm that is provably robust to overoptimization. △ Less

Submitted 19 July, 2024; v1 submitted 18 July, 2024; originally announced July 2024.

Helical Spin Dynamics in Commensurate Magnets: a Study on Brochantite, Cu$_4$SO$_4$(OH)$_6$

Authors: S. E. Nikitin, Tao Xie, A. Gazizulina, B. Ouladdiaf, J. A. Rodríguez Velamazán, I. ~F. ~Díaz-Ortega, H. Nojiri, L. M. Anovitz, A. M. dos Santos, O. Prokhnenko, A. Podlesnyak

Abstract: We report the direct observation of a commensurate-ordered antiferromagnetic (AFM) state but incommensurate helical spin dynamics in the natural mineral brochantite Cu$_4$SO$_4$(OH)$_6$ through neutron diffraction and neutron spectroscopy measurements. Inelastic neutron scattering measurements reveal magnon-like excitations with considerable dispersion along the c-axis and almost flat branches in… ▽ More We report the direct observation of a commensurate-ordered antiferromagnetic (AFM) state but incommensurate helical spin dynamics in the natural mineral brochantite Cu$_4$SO$_4$(OH)$_6$ through neutron diffraction and neutron spectroscopy measurements. Inelastic neutron scattering measurements reveal magnon-like excitations with considerable dispersion along the c-axis and almost flat branches in other principal directions, indicating the strong one-dimensional character of the magnetic correlations. We experimentally observe the effect of the uniform Dzyaloshinskii-Moriya (DM) interaction, which elevates the degeneracy of the spin-wave modes shifting them in opposite directions in reciprocal space. The system has a commensurate AFM ground state, stabilized by the anisotropic symmetric Heisenberg exchange interactions, and quasi-one-dimensional chiral spin dynamics due to the antisymmetric DM interaction. Employing linear spin-wave theory, we were able to construct an effective Heisenberg Hamiltonian. We quantify both the symmetric exchange parameters and the DM vector components in Cu$_4$SO$_4$(OH)$_6$ and determine the mechanism of the magnetic frustration. Our work provides detailed insights into the complex dynamics of the spin chain in the presence of uniform DM interaction. △ Less

Submitted 17 July, 2024; originally announced July 2024.

Comments: 9 pages, 5 figures

QVD: Post-training Quantization for Video Diffusion Models

Authors: Shilong Tian, Hong Chen, Chengtao Lv, Yu Liu, Jinyang Guo, Xianglong Liu, Shengxi Li, Hao Yang, Tao Xie

Abstract: Recently, video diffusion models (VDMs) have garnered significant attention due to their notable advancements in generating coherent and realistic video content. However, processing multiple frame features concurrently, coupled with the considerable model size, results in high latency and extensive memory consumption, hindering their broader application. Post-training quantization (PTQ) is an effe… ▽ More Recently, video diffusion models (VDMs) have garnered significant attention due to their notable advancements in generating coherent and realistic video content. However, processing multiple frame features concurrently, coupled with the considerable model size, results in high latency and extensive memory consumption, hindering their broader application. Post-training quantization (PTQ) is an effective technique to reduce memory footprint and improve computational efficiency. Unlike image diffusion, we observe that the temporal features, which are integrated into all frame features, exhibit pronounced skewness. Furthermore, we investigate significant inter-channel disparities and asymmetries in the activation of video diffusion models, resulting in low coverage of quantization levels by individual channels and increasing the challenge of quantization. To address these issues, we introduce the first PTQ strategy tailored for video diffusion models, dubbed QVD. Specifically, we propose the High Temporal Discriminability Quantization (HTDQ) method, designed for temporal features, which retains the high discriminability of quantized features, providing precise temporal guidance for all video frames. In addition, we present the Scattered Channel Range Integration (SCRI) method which aims to improve the coverage of quantization levels across individual channels. Experimental validations across various models, datasets, and bit-width settings demonstrate the effectiveness of our QVD in terms of diverse metrics. In particular, we achieve near-lossless performance degradation on W8A8, outperforming the current methods by 205.12 in FVD. △ Less

Submitted 17 July, 2024; v1 submitted 16 July, 2024; originally announced July 2024.

Comments: accepted by ACMMM2024

High-Resolution Spectroscopy of the Intermediate Impurity States near a Quantum Phase Transition

Authors: Yao Zhang, Tao Xie, Zhen-Yu Liu, Rui Wang, Wenhao Zhang, Chaofei Liu, Ying-Shuang Fu

Abstract: The intermediate behavior near a quantum phase transition is crucial for understanding the quantum criticality of various competing phases and their separate origins, yet remains unexplored for the multiple Yu-Shiba-Rusinov (YSR) states. Here, we investigated the detailed spectroscopic change of the exchange coupling-dependent YSR states near a quantum phase transition. The initially developed one… ▽ More The intermediate behavior near a quantum phase transition is crucial for understanding the quantum criticality of various competing phases and their separate origins, yet remains unexplored for the multiple Yu-Shiba-Rusinov (YSR) states. Here, we investigated the detailed spectroscopic change of the exchange coupling-dependent YSR states near a quantum phase transition. The initially developed one pair of YSR states, induced by the Fe vacancy in monolayer Fe(Te,Se) superconductor, are clearly resolved with high resolution showing an evolution into two pairs of YSR peaks yet with dichotomy in their spectral features. Interestingly, while the lower-energy YSR branch enters the quantum phase transition region, the higher-lying one remains rigidly away from the lower-energy counterpart with a constant energy difference. Spectral weight analysis of the higher-energy branch yields an exponential dependence on the exchange coupling, which can be well rationalized by taking the two pairs of YSR states as a result of field splitting by the magnetic anisotropy. Our results unveil the intermediate region of a quantum phase transition with a magnetic anisotropy-induced splitting of the YSR resonance, and highlight a prospect for developing functional electronics based on the flexibly controllable multiple quantum states. △ Less

Submitted 16 July, 2024; originally announced July 2024.

Comments: 8 pages, 3 figures

Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?

Authors: Ruisheng Cao, Fangyu Lei, Haoyuan Wu, Jixuan Chen, Yeqiao Fu, Hongcheng Gao, Xinzhuang Xiong, Hanchong Zhang, Yuchen Mao, Wenjing Hu, Tianbao Xie, Hongshen Xu, Danyang Zhang, Sida Wang, Ruoxi Sun, Pengcheng Yin, Caiming Xiong, Ansong Ni, Qian Liu, Victor Zhong, Lu Chen, Kai Yu, Tao Yu

Abstract: Data science and engineering workflows often span multiple stages, from warehousing to orchestration, using tools like BigQuery, dbt, and Airbyte. As vision language models (VLMs) advance in multimodal understanding and code generation, VLM-based agents could potentially automate these workflows by generating SQL queries, Python code, and GUI operations. This automation can improve the productivit… ▽ More Data science and engineering workflows often span multiple stages, from warehousing to orchestration, using tools like BigQuery, dbt, and Airbyte. As vision language models (VLMs) advance in multimodal understanding and code generation, VLM-based agents could potentially automate these workflows by generating SQL queries, Python code, and GUI operations. This automation can improve the productivity of experts while democratizing access to large-scale data analysis. In this paper, we introduce Spider2-V, the first multimodal agent benchmark focusing on professional data science and engineering workflows, featuring 494 real-world tasks in authentic computer environments and incorporating 20 enterprise-level professional applications. These tasks, derived from real-world use cases, evaluate the ability of a multimodal agent to perform data-related tasks by writing code and managing the GUI in enterprise data software systems. To balance realistic simulation with evaluation simplicity, we devote significant effort to developing automatic configurations for task setup and carefully crafting evaluation metrics for each task. Furthermore, we supplement multimodal agents with comprehensive documents of these enterprise data software systems. Our empirical evaluation reveals that existing state-of-the-art LLM/VLM-based agents do not reliably automate full data workflows (14.0% success). Even with step-by-step guidance, these agents still underperform in tasks that require fine-grained, knowledge-intensive GUI actions (16.2%) and involve remote cloud-hosted workspaces (10.6%). We hope that Spider2-V paves the way for autonomous multimodal agents to transform the automation of data science and engineering workflow. Our code and data are available at https://spider2-v.github.io. △ Less

Submitted 15 July, 2024; originally announced July 2024.

Comments: 34 pages, 14 figures, 10 tables

Scalable microwave-to-optical transducers at single photon level with spins

Authors: Tian Xie, Rikuto Fukumori, Jiahui Li, Andrei Faraon

Abstract: Microwave-to-optical transduction of single photons will play an essential role in interconnecting future superconducting quantum devices, with applications in distributed quantum computing and secure communications. Various transducers that couple microwave and optical modes via an optical drive have been developed, utilizing nonlinear phenomena such as the Pockels effect and a combination of ele… ▽ More Microwave-to-optical transduction of single photons will play an essential role in interconnecting future superconducting quantum devices, with applications in distributed quantum computing and secure communications. Various transducers that couple microwave and optical modes via an optical drive have been developed, utilizing nonlinear phenomena such as the Pockels effect and a combination of electromechanical, piezoelectric, and optomechanical couplings. However, the limited strength of these nonlinearities, set by bulk material properties, requires the use of high quality factor resonators, often in conjunction with sophisticated nano-fabrication of suspended structures. Thus, an efficient and scalable transduction technology is still an outstanding goal. Rare-earth ion (REI) doped crystals provide high-quality atomic resonances that result in effective second-order nonlinearities stronger by many orders of magnitude compared to conventional materials. Here, we use ytterbium-171 ions doped in a YVO$_4$ crystal at 340 ppm with an effective resonant $χ^{(2)}$ nonlinearity of ~ 10$^7$ pm/V to implement an on-chip microwave-to-optical transducer. Without an engineered optical cavity, we achieve percent-level efficiencies with an added noise as low as 1.24(9) photons. To showcase scalability, we demonstrate the interference of photons originating from two simultaneously operated transducers, enabled by the inherent absolute frequencies of the atomic transitions. These results establish REI-based transducers as a highly competitive transduction platform, provide existing REI-based quantum technologies a native link to various leading quantum microwave platforms, and pave the way toward remote transducer-assisted entanglement of superconducting quantum machines. △ Less

Submitted 11 July, 2024; originally announced July 2024.

Comments: TX and RF contributed equally to this work

Foundation Model Engineering: Engineering Foundation Models Just as Engineering Software

Authors: Dezhi Ran, Mengzhou Wu, Wei Yang, Tao Xie

Abstract: By treating data and models as the source code, Foundation Models (FMs) become a new type of software. Mirroring the concept of software crisis, the increasing complexity of FMs making FM crisis a tangible concern in the coming decade, appealing for new theories and methodologies from the field of software engineering. In this paper, we outline our vision of introducing Foundation Model (FM) engin… ▽ More By treating data and models as the source code, Foundation Models (FMs) become a new type of software. Mirroring the concept of software crisis, the increasing complexity of FMs making FM crisis a tangible concern in the coming decade, appealing for new theories and methodologies from the field of software engineering. In this paper, we outline our vision of introducing Foundation Model (FM) engineering, a strategic response to the anticipated FM crisis with principled engineering methodologies. FM engineering aims to mitigate potential issues in FM development and application through the introduction of declarative, automated, and unified programming interfaces for both data and model management, reducing the complexities involved in working with FMs by providing a more structured and intuitive process for developers. Through the establishment of FM engineering, we aim to provide a robust, automated, and extensible framework that addresses the imminent challenges, and discovering new research opportunities for the software engineering field. △ Less

Submitted 11 July, 2024; originally announced July 2024.

Comments: Accepted by 2030 Software Engineering Workshop, co-located with FSE24; Invited to ACM TOSEM 2030 Roadmap for Software Engineering

MemServe: Context Caching for Disaggregated LLM Serving with Elastic Memory Pool

Authors: Cunchen Hu, Heyang Huang, Junhao Hu, Jiang Xu, Xusheng Chen, Tao Xie, Chenxi Wang, Sa Wang, Yungang Bao, Ninghui Sun, Yizhou Shan

Abstract: Large language model (LLM) serving has transformed from stateless to stateful systems, utilizing techniques like context caching and disaggregated inference. These optimizations extend the lifespan and domain of the KV cache, necessitating a new architectural approach. We present MemServe, a unified system that integrates both inter-request and intra-request optimizations. MemServe introduces MemP… ▽ More Large language model (LLM) serving has transformed from stateless to stateful systems, utilizing techniques like context caching and disaggregated inference. These optimizations extend the lifespan and domain of the KV cache, necessitating a new architectural approach. We present MemServe, a unified system that integrates both inter-request and intra-request optimizations. MemServe introduces MemPool, an elastic memory pool managing distributed memory and KV caches across serving instances. Using MemPool APIs, MemServe combines context caching with disaggregated inference for the first time, supported by a global scheduler that enhances cache reuse through a global prompt tree-based locality-aware policy. Tests show that MemServe significantly improves job completion time and time-to-first-time. △ Less

Submitted 26 June, 2024; v1 submitted 25 June, 2024; originally announced June 2024.

Retrieval Augmented Instruction Tuning for Open NER with Large Language Models

Authors: Tingyu Xie, Jian Zhang, Yan Zhang, Yuanyuan Liang, Qi Li, Hongwei Wang

Abstract: The strong capability of large language models (LLMs) has been applied to information extraction (IE) through either retrieval augmented prompting or instruction tuning (IT). However, the best way to incorporate information with LLMs for IE remains an open question. In this paper, we explore Retrieval Augmented Instruction Tuning (RA-IT) for IE, focusing on the task of open named entity recognitio… ▽ More The strong capability of large language models (LLMs) has been applied to information extraction (IE) through either retrieval augmented prompting or instruction tuning (IT). However, the best way to incorporate information with LLMs for IE remains an open question. In this paper, we explore Retrieval Augmented Instruction Tuning (RA-IT) for IE, focusing on the task of open named entity recognition (NER). Specifically, for each training sample, we retrieve semantically similar examples from the training dataset as the context and prepend them to the input of the original instruction. To evaluate our RA-IT approach more thoroughly, we construct a Chinese IT dataset for open NER and evaluate RA-IT in both English and Chinese scenarios. Experimental results verify the effectiveness of RA-IT across various data sizes and in both English and Chinese scenarios. We also conduct thorough studies to explore the impacts of various retrieval strategies in the proposed RA-IT framework. Code and data are available at: https://github.com/Emma1066/Retrieval-Augmented-IT-OpenNER △ Less

Submitted 25 June, 2024; originally announced June 2024.

Addressing Polarization and Unfairness in Performative Prediction

Authors: Kun Jin, Tian Xie, Yang Liu, Xueru Zhang

Abstract: When machine learning (ML) models are used in applications that involve humans (e.g., online recommendation, school admission, hiring, lending), the model itself may trigger changes in the distribution of targeted data it aims to predict. Performative prediction (PP) is a framework that explicitly considers such model-dependent distribution shifts when learning ML models. While significant efforts… ▽ More When machine learning (ML) models are used in applications that involve humans (e.g., online recommendation, school admission, hiring, lending), the model itself may trigger changes in the distribution of targeted data it aims to predict. Performative prediction (PP) is a framework that explicitly considers such model-dependent distribution shifts when learning ML models. While significant efforts have been devoted to finding performative stable (PS) solutions in PP for system robustness, their societal implications are less explored and it is unclear whether PS solutions are aligned with social norms such as fairness. In this paper, we set out to examine the fairness property of PS solutions in performative prediction. We first show that PS solutions can incur severe polarization effects and group-wise loss disparity. Although existing fairness mechanisms commonly used in literature can help mitigate unfairness, they may fail and disrupt the stability under model-dependent distribution shifts. We thus propose novel fairness intervention mechanisms that can simultaneously achieve both stability and fairness in PP settings. Both theoretical analysis and experiments are provided to validate the proposed method. △ Less

Submitted 24 June, 2024; originally announced June 2024.

OTCE: Hybrid SSM and Attention with Cross Domain Mixture of Experts to construct Observer-Thinker-Conceiver-Expresser

Authors: Jingze Shi, Ting Xie, Bingheng Wu, Chunjun Zheng, Kai Wang

Abstract: Recent research has shown that combining Mamba with Transformer architecture, which has selective state space and quadratic self-attention mechanism, outperforms using Mamba or Transformer architecture alone in language modeling tasks. The quadratic self-attention mechanism effectively alleviates the shortcomings of selective state space in handling long-term dependencies of any element in the seq… ▽ More Recent research has shown that combining Mamba with Transformer architecture, which has selective state space and quadratic self-attention mechanism, outperforms using Mamba or Transformer architecture alone in language modeling tasks. The quadratic self-attention mechanism effectively alleviates the shortcomings of selective state space in handling long-term dependencies of any element in the sequence. We propose a position information injection method that connects the selective state space model with the quadratic attention, and integrates these two architectures with hybrid experts with cross-sharing domains, so that we can enjoy the advantages of both. We design a new architecture with a more biomimetic idea: Observer-Thinker-Conceiver-Expresser (OTCE), which can compete with well-known medium-scale open-source language models on a small scale in language modeling tasks. △ Less

Submitted 19 July, 2024; v1 submitted 24 June, 2024; originally announced June 2024.

Competing excitation quenching and charge exchange in ultracold Li-Ba$^+$ collisions

Authors: Xiaodong Xing, Pascal Weckesser, Fabian Thielemann, Tibor Jónás, Romain Vexiau, Nadia Bouloufa-Maafa, Eliane Luc-Koenig, Kirk W. Madison, Andrea Orbán, Ting Xie, Tobias Schaetz, Olivier Dulieu

Abstract: Hybrid atom-ion systems are a rich and powerful platform for studying chemical reactions, as they feature both excellent control over the electronic state preparation and readout as well as a versatile tunability over the scattering energy, ranging from the few-partial wave regime to the quantum regime. In this work, we make use of these excellent control knobs, and present a joint experimental an… ▽ More Hybrid atom-ion systems are a rich and powerful platform for studying chemical reactions, as they feature both excellent control over the electronic state preparation and readout as well as a versatile tunability over the scattering energy, ranging from the few-partial wave regime to the quantum regime. In this work, we make use of these excellent control knobs, and present a joint experimental and theoretical study of the collisions of a single $^{138}$Ba$^+$ ion prepared in the $5d\,^2D_{3/2,5/2}$ metastable states with a ground state $^6$Li gas near quantum degeneracy. We show that in contrast to previously reported atom-ion mixtures, several non-radiative processes, including charge exchange, excitation exchange and quenching, compete with each other due to the inherent complexity of the ion-atom molecular structure. We present a full quantum model based on high-level electronic structure calculations involving spin-orbit couplings. Results are in excellent agreement with observations, highlighting the strong coupling between the internal angular momenta and the mechanical rotation of the colliding pair, which is relevant in any other hybrid system composed of an alkali-metal atom and an alkaline-earth ion. △ Less

Submitted 23 June, 2024; originally announced June 2024.

Comments: 17 pages, 15 figures, 4 tables

SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal Behaviors

Authors: Tinghao Xie, Xiangyu Qi, Yi Zeng, Yangsibo Huang, Udari Madhushani Sehwag, Kaixuan Huang, Luxi He, Boyi Wei, Dacheng Li, Ying Sheng, Ruoxi Jia, Bo Li, Kai Li, Danqi Chen, Peter Henderson, Prateek Mittal

Abstract: Evaluating aligned large language models' (LLMs) ability to recognize and reject unsafe user requests is crucial for safe, policy-compliant deployments. Existing evaluation efforts, however, face three limitations that we address with SORRY-Bench, our proposed benchmark. First, existing methods often use coarse-grained taxonomies of unsafe topics, and are over-representing some fine-grained topics… ▽ More Evaluating aligned large language models' (LLMs) ability to recognize and reject unsafe user requests is crucial for safe, policy-compliant deployments. Existing evaluation efforts, however, face three limitations that we address with SORRY-Bench, our proposed benchmark. First, existing methods often use coarse-grained taxonomies of unsafe topics, and are over-representing some fine-grained topics. For example, among the ten existing datasets that we evaluated, tests for refusals of self-harm instructions are over 3x less represented than tests for fraudulent activities. SORRY-Bench improves on this by using a fine-grained taxonomy of 45 potentially unsafe topics, and 450 class-balanced unsafe instructions, compiled through human-in-the-loop methods. Second, linguistic characteristics and formatting of prompts are often overlooked, like different languages, dialects, and more -- which are only implicitly considered in many evaluations. We supplement SORRY-Bench with 20 diverse linguistic augmentations to systematically examine these effects. Third, existing evaluations rely on large LLMs (e.g., GPT-4) for evaluation, which can be computationally expensive. We investigate design choices for creating a fast, accurate automated safety evaluator. By collecting 7K+ human annotations and conducting a meta-evaluation of diverse LLM-as-a-judge designs, we show that fine-tuned 7B LLMs can achieve accuracy comparable to GPT-4 scale LLMs, with lower computational cost. Putting these together, we evaluate over 40 proprietary and open-source LLMs on SORRY-Bench, analyzing their distinctive refusal behaviors. We hope our effort provides a building block for systematic evaluations of LLMs' safety refusal capabilities, in a balanced, granular, and efficient manner. △ Less

Submitted 20 June, 2024; originally announced June 2024.

Fantastic Copyrighted Beasts and How (Not) to Generate Them

Authors: Luxi He, Yangsibo Huang, Weijia Shi, Tinghao Xie, Haotian Liu, Yue Wang, Luke Zettlemoyer, Chiyuan Zhang, Danqi Chen, Peter Henderson

Abstract: Recent studies show that image and video generation models can be prompted to reproduce copyrighted content from their training data, raising serious legal concerns around copyright infringement. Copyrighted characters, in particular, pose a difficult challenge for image generation services, with at least one lawsuit already awarding damages based on the generation of these characters. Yet, little… ▽ More Recent studies show that image and video generation models can be prompted to reproduce copyrighted content from their training data, raising serious legal concerns around copyright infringement. Copyrighted characters, in particular, pose a difficult challenge for image generation services, with at least one lawsuit already awarding damages based on the generation of these characters. Yet, little research has empirically examined this issue. We conduct a systematic evaluation to fill this gap. First, we build CopyCat, an evaluation suite consisting of diverse copyrighted characters and a novel evaluation pipeline. Our evaluation considers both the detection of similarity to copyrighted characters and generated image's consistency with user input. Our evaluation systematically shows that both image and video generation models can still generate characters even if characters' names are not explicitly mentioned in the prompt, sometimes with only two generic keywords (e.g., prompting with "videogame, plumber" consistently generates Nintendo's Mario character). We then introduce techniques to semi-automatically identify such keywords or descriptions that trigger character generation. Using our evaluation suite, we study runtime mitigation strategies, including both existing methods and new strategies we propose. Our findings reveal that commonly employed strategies, such as prompt rewriting in the DALL-E system, are not sufficient as standalone guardrails. These strategies must be coupled with other approaches, like negative prompting, to effectively reduce the unintended generation of copyrighted characters. Our work provides empirical grounding to the discussion of copyright mitigation strategies and offers actionable insights for model deployers actively implementing them. △ Less

Submitted 20 June, 2024; originally announced June 2024.

Interpretable Preferences via Multi-Objective Reward Modeling and Mixture-of-Experts

Authors: Haoxiang Wang, Wei Xiong, Tengyang Xie, Han Zhao, Tong Zhang

Abstract: Reinforcement learning from human feedback (RLHF) has emerged as the primary method for aligning large language models (LLMs) with human preferences. The RLHF process typically starts by training a reward model (RM) using human preference data. Conventional RMs are trained on pairwise responses to the same user request, with relative ratings indicating which response humans prefer. The trained RM… ▽ More Reinforcement learning from human feedback (RLHF) has emerged as the primary method for aligning large language models (LLMs) with human preferences. The RLHF process typically starts by training a reward model (RM) using human preference data. Conventional RMs are trained on pairwise responses to the same user request, with relative ratings indicating which response humans prefer. The trained RM serves as a proxy for human preferences. However, due to the black-box nature of RMs, their outputs lack interpretability, as humans cannot intuitively understand why an RM thinks a response is good or not. As RMs act as human preference proxies, we believe they should be human-interpretable to ensure that their internal decision processes are consistent with human preferences and to prevent reward hacking in LLM alignment. To build RMs with interpretable preferences, we propose a two-stage approach: i) train an Absolute-Rating Multi-Objective Reward Model (ArmoRM) with multi-dimensional absolute-rating data, each dimension corresponding to a human-interpretable objective (e.g., honesty, verbosity, safety); ii) employ a Mixture-of-Experts (MoE) strategy with a gating network that automatically selects the most suitable reward objectives based on the context. We efficiently trained an ArmoRM with Llama-3 8B and a gating network consisting of a shallow MLP on top of the ArmoRM. Our trained model, ArmoRM-Llama3-8B, obtains state-of-the-art performance on RewardBench, a benchmark evaluating RMs for language modeling. Notably, the performance of our model surpasses the LLM-as-a-judge method with GPT-4 judges by a margin, and approaches the performance of the much larger Nemotron-4 340B reward model. △ Less

Submitted 18 June, 2024; originally announced June 2024.

Comments: Technical report v1. Code and model are released at https://github.com/RLHFlow/RLHF-Reward-Modeling/

Self-Play with Adversarial Critic: Provable and Scalable Offline Alignment for Language Models

Authors: Xiang Ji, Sanjeev Kulkarni, Mengdi Wang, Tengyang Xie

Abstract: This work studies the challenge of aligning large language models (LLMs) with offline preference data. We focus on alignment by Reinforcement Learning from Human Feedback (RLHF) in particular. While popular preference optimization methods exhibit good empirical performance in practice, they are not theoretically guaranteed to converge to the optimal policy and can provably fail when the data cover… ▽ More This work studies the challenge of aligning large language models (LLMs) with offline preference data. We focus on alignment by Reinforcement Learning from Human Feedback (RLHF) in particular. While popular preference optimization methods exhibit good empirical performance in practice, they are not theoretically guaranteed to converge to the optimal policy and can provably fail when the data coverage is sparse by classical offline reinforcement learning (RL) results. On the other hand, a recent line of work has focused on theoretically motivated preference optimization methods with provable guarantees, but these are not computationally efficient for large-scale applications like LLM alignment. To bridge this gap, we propose SPAC, a new offline preference optimization method with self-play, inspired by the on-average pessimism technique from the offline RL literature, to be the first provable and scalable approach to LLM alignment. We both provide theoretical analysis for its convergence under single-policy concentrability for the general function approximation setting and demonstrate its competitive empirical performance for LLM alignment on a 7B Mistral model with Open LLM Leaderboard evaluations. △ Less

Submitted 6 June, 2024; originally announced June 2024.

VideoPhy: Evaluating Physical Commonsense for Video Generation

Authors: Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai-Wei Chang, Aditya Grover

Abstract: Recent advances in internet-scale video data pretraining have led to the development of text-to-video generative models that can create high-quality videos across a broad range of visual concepts, synthesize realistic motions and render complex objects. Hence, these generative models have the potential to become general-purpose simulators of the physical world. However, it is unclear how far we ar… ▽ More Recent advances in internet-scale video data pretraining have led to the development of text-to-video generative models that can create high-quality videos across a broad range of visual concepts, synthesize realistic motions and render complex objects. Hence, these generative models have the potential to become general-purpose simulators of the physical world. However, it is unclear how far we are from this goal with the existing text-to-video generative models. To this end, we present VideoPhy, a benchmark designed to assess whether the generated videos follow physical commonsense for real-world activities (e.g. marbles will roll down when placed on a slanted surface). Specifically, we curate diverse prompts that involve interactions between various material types in the physical world (e.g., solid-solid, solid-fluid, fluid-fluid). We then generate videos conditioned on these captions from diverse state-of-the-art text-to-video generative models, including open models (e.g., CogVideoX) and closed models (e.g., Lumiere, Dream Machine). Our human evaluation reveals that the existing models severely lack the ability to generate videos adhering to the given text prompts, while also lack physical commonsense. Specifically, the best performing model, CogVideoX-5B, generates videos that adhere to the caption and physical laws for 39.6% of the instances. VideoPhy thus highlights that the video generative models are far from accurately simulating the physical world. Finally, we propose an auto-evaluator, VideoCon-Physics, to assess the performance reliably for the newly released models. △ Less

Submitted 3 October, 2024; v1 submitted 5 June, 2024; originally announced June 2024.

Comments: 43 pages, 29 figures, 12 tables. Added CogVideo and Dream Machine in v2

CodeR: Issue Resolving with Multi-Agent and Task Graphs

Authors: Dong Chen, Shaoxin Lin, Muhan Zeng, Daoguang Zan, Jian-Gang Wang, Anton Cheshkov, Jun Sun, Hao Yu, Guoliang Dong, Artem Aliev, Jie Wang, Xiao Cheng, Guangtai Liang, Yuchi Ma, Pan Bian, Tao Xie, Qianxiang Wang

Abstract: GitHub issue resolving recently has attracted significant attention from academia and industry. SWE-bench is proposed to measure the performance in resolving issues. In this paper, we propose CodeR, which adopts a multi-agent framework and pre-defined task graphs to Repair & Resolve reported bugs and add new features within code Repository. On SWE-bench lite, CodeR is able to solve 28.33% of issue… ▽ More GitHub issue resolving recently has attracted significant attention from academia and industry. SWE-bench is proposed to measure the performance in resolving issues. In this paper, we propose CodeR, which adopts a multi-agent framework and pre-defined task graphs to Repair & Resolve reported bugs and add new features within code Repository. On SWE-bench lite, CodeR is able to solve 28.33% of issues, when submitting only once for each issue. We examine the performance impact of each design of CodeR and offer insights to advance this research direction. △ Less

Submitted 10 June, 2024; v1 submitted 3 June, 2024; originally announced June 2024.

Comments: https://github.com/NL2Code/CodeR

Exploratory Preference Optimization: Harnessing Implicit Q*-Approximation for Sample-Efficient RLHF

Authors: Tengyang Xie, Dylan J. Foster, Akshay Krishnamurthy, Corby Rosset, Ahmed Awadallah, Alexander Rakhlin

Abstract: Reinforcement learning from human feedback (RLHF) has emerged as a central tool for language model alignment. We consider online exploration in RLHF, which exploits interactive access to human or AI feedback by deliberately encouraging the model to produce diverse, maximally informative responses. By allowing RLHF to confidently stray from the pre-trained model, online exploration offers the possi… ▽ More Reinforcement learning from human feedback (RLHF) has emerged as a central tool for language model alignment. We consider online exploration in RLHF, which exploits interactive access to human or AI feedback by deliberately encouraging the model to produce diverse, maximally informative responses. By allowing RLHF to confidently stray from the pre-trained model, online exploration offers the possibility of novel, potentially super-human capabilities, but its full potential as a paradigm for language model training has yet to be realized, owing to computational and statistical bottlenecks in directly adapting existing reinforcement learning techniques. We propose a new algorithm for online exploration in RLHF, Exploratory Preference Optimization (XPO), which is simple and practical -- a one-line change to (online) Direct Preference Optimization (DPO; Rafailov et al., 2023) -- yet enjoys the strongest known provable guarantees and promising empirical performance. XPO augments the DPO objective with a novel and principled exploration bonus, empowering the algorithm to explore outside the support of the initial model and human feedback data. In theory, we show that XPO is provably sample-efficient and converges to a near-optimal language model policy under natural exploration conditions, irrespective of whether the initial model has good coverage. Our analysis, which builds on the observation that DPO implicitly performs a form of $Q^{\star}$-approximation (or, Bellman error minimization), combines previously disparate techniques from language modeling and theoretical reinforcement learning in a serendipitous fashion through the perspective of KL-regularized Markov decision processes. Empirically, we find that XPO is more sample-efficient than non-exploratory DPO variants in a preliminary evaluation. △ Less

Submitted 31 May, 2024; originally announced May 2024.

AI Risk Management Should Incorporate Both Safety and Security

Authors: Xiangyu Qi, Yangsibo Huang, Yi Zeng, Edoardo Debenedetti, Jonas Geiping, Luxi He, Kaixuan Huang, Udari Madhushani, Vikash Sehwag, Weijia Shi, Boyi Wei, Tinghao Xie, Danqi Chen, Pin-Yu Chen, Jeffrey Ding, Ruoxi Jia, Jiaqi Ma, Arvind Narayanan, Weijie J Su, Mengdi Wang, Chaowei Xiao, Bo Li, Dawn Song, Peter Henderson, Prateek Mittal

Abstract: The exposure of security vulnerabilities in safety-aligned language models, e.g., susceptibility to adversarial attacks, has shed light on the intricate interplay between AI safety and AI security. Although the two disciplines now come together under the overarching goal of AI risk management, they have historically evolved separately, giving rise to differing perspectives. Therefore, in this pape… ▽ More The exposure of security vulnerabilities in safety-aligned language models, e.g., susceptibility to adversarial attacks, has shed light on the intricate interplay between AI safety and AI security. Although the two disciplines now come together under the overarching goal of AI risk management, they have historically evolved separately, giving rise to differing perspectives. Therefore, in this paper, we advocate that stakeholders in AI risk management should be aware of the nuances, synergies, and interplay between safety and security, and unambiguously take into account the perspectives of both disciplines in order to devise mostly effective and holistic risk mitigation approaches. Unfortunately, this vision is often obfuscated, as the definitions of the basic concepts of "safety" and "security" themselves are often inconsistent and lack consensus across communities. With AI risk management being increasingly cross-disciplinary, this issue is particularly salient. In light of this conceptual challenge, we introduce a unified reference framework to clarify the differences and interplay between AI safety and AI security, aiming to facilitate a shared understanding and effective collaboration across communities. △ Less

Submitted 29 May, 2024; originally announced May 2024.

Kernel Semi-Implicit Variational Inference

Authors: Ziheng Cheng, Longlin Yu, Tianyu Xie, Shiyue Zhang, Cheng Zhang

Abstract: Semi-implicit variational inference (SIVI) extends traditional variational families with semi-implicit distributions defined in a hierarchical manner. Due to the intractable densities of semi-implicit distributions, classical SIVI often resorts to surrogates of evidence lower bound (ELBO) that would introduce biases for training. A recent advancement in SIVI, named SIVI-SM, utilizes an alternative… ▽ More Semi-implicit variational inference (SIVI) extends traditional variational families with semi-implicit distributions defined in a hierarchical manner. Due to the intractable densities of semi-implicit distributions, classical SIVI often resorts to surrogates of evidence lower bound (ELBO) that would introduce biases for training. A recent advancement in SIVI, named SIVI-SM, utilizes an alternative score matching objective made tractable via a minimax formulation, albeit requiring an additional lower-level optimization. In this paper, we propose kernel SIVI (KSIVI), a variant of SIVI-SM that eliminates the need for lower-level optimization through kernel tricks. Specifically, we show that when optimizing over a reproducing kernel Hilbert space (RKHS), the lower-level problem has an explicit solution. This way, the upper-level objective becomes the kernel Stein discrepancy (KSD), which is readily computable for stochastic gradient descent due to the hierarchical structure of semi-implicit variational distributions. An upper bound for the variance of the Monte Carlo gradient estimators of the KSD objective is derived, which allows us to establish novel convergence guarantees of KSIVI. We demonstrate the effectiveness and efficiency of KSIVI on both synthetic distributions and a variety of real data Bayesian inference tasks. △ Less

Submitted 29 May, 2024; originally announced May 2024.

Comments: ICML 2024 camera ready

Atlas3D: Physically Constrained Self-Supporting Text-to-3D for Simulation and Fabrication

Authors: Yunuo Chen, Tianyi Xie, Zeshun Zong, Xuan Li, Feng Gao, Yin Yang, Ying Nian Wu, Chenfanfu Jiang

Abstract: Existing diffusion-based text-to-3D generation methods primarily focus on producing visually realistic shapes and appearances, often neglecting the physical constraints necessary for downstream tasks. Generated models frequently fail to maintain balance when placed in physics-based simulations or 3D printed. This balance is crucial for satisfying user design intentions in interactive gaming, embod… ▽ More Existing diffusion-based text-to-3D generation methods primarily focus on producing visually realistic shapes and appearances, often neglecting the physical constraints necessary for downstream tasks. Generated models frequently fail to maintain balance when placed in physics-based simulations or 3D printed. This balance is crucial for satisfying user design intentions in interactive gaming, embodied AI, and robotics, where stable models are needed for reliable interaction. Additionally, stable models ensure that 3D-printed objects, such as figurines for home decoration, can stand on their own without requiring additional supports. To fill this gap, we introduce Atlas3D, an automatic and easy-to-implement method that enhances existing Score Distillation Sampling (SDS)-based text-to-3D tools. Atlas3D ensures the generation of self-supporting 3D models that adhere to physical laws of stability under gravity, contact, and friction. Our approach combines a novel differentiable simulation-based loss function with physically inspired regularization, serving as either a refinement or a post-processing module for existing frameworks. We verify Atlas3D's efficacy through extensive generation tasks and validate the resulting 3D models in both simulated and real-world environments. △ Less

Submitted 28 May, 2024; originally announced May 2024.

Reflected Flow Matching

Authors: Tianyu Xie, Yu Zhu, Longlin Yu, Tong Yang, Ziheng Cheng, Shiyue Zhang, Xiangyu Zhang, Cheng Zhang

Abstract: Continuous normalizing flows (CNFs) learn an ordinary differential equation to transform prior samples into data. Flow matching (FM) has recently emerged as a simulation-free approach for training CNFs by regressing a velocity model towards the conditional velocity field. However, on constrained domains, the learned velocity model may lead to undesirable flows that result in highly unnatural sampl… ▽ More Continuous normalizing flows (CNFs) learn an ordinary differential equation to transform prior samples into data. Flow matching (FM) has recently emerged as a simulation-free approach for training CNFs by regressing a velocity model towards the conditional velocity field. However, on constrained domains, the learned velocity model may lead to undesirable flows that result in highly unnatural samples, e.g., oversaturated images, due to both flow matching error and simulation error. To address this, we add a boundary constraint term to CNFs, which leads to reflected CNFs that keep trajectories within the constrained domains. We propose reflected flow matching (RFM) to train the velocity model in reflected CNFs by matching the conditional velocity fields in a simulation-free manner, similar to the vanilla FM. Moreover, the analytical form of conditional velocity fields in RFM avoids potentially biased approximations, making it superior to existing score-based generative models on constrained domains. We demonstrate that RFM achieves comparable or better results on standard image benchmarks and produces high-quality class-conditioned samples under high guidance weight. △ Less

Submitted 26 May, 2024; originally announced May 2024.

Comments: ICML 2024 camera-ready