Showing 1–50 of 6,016 results for author: Li, W

ARTS: Semi-Analytical Regressor using Disentangled Skeletal Representations for Human Mesh Recovery from Videos

Authors: Tao Tang, Hong Liu, Yingxuan You, Ti Wang, Wenhao Li

Abstract: Although existing video-based 3D human mesh recovery methods have made significant progress, simultaneously estimating human pose and shape from low-resolution image features limits their performance. These image features lack sufficient spatial information about the human body and contain various noises (e.g., background, lighting, and clothing), which often results in inaccurate pose and inconsi… ▽ More Although existing video-based 3D human mesh recovery methods have made significant progress, simultaneously estimating human pose and shape from low-resolution image features limits their performance. These image features lack sufficient spatial information about the human body and contain various noises (e.g., background, lighting, and clothing), which often results in inaccurate pose and inconsistent motion. Inspired by the rapid advance in human pose estimation, we discover that compared to image features, skeletons inherently contain accurate human pose and motion. Therefore, we propose a novel semiAnalytical Regressor using disenTangled Skeletal representations for human mesh recovery from videos, called ARTS. Specifically, a skeleton estimation and disentanglement module is proposed to estimate the 3D skeletons from a video and decouple them into disentangled skeletal representations (i.e., joint position, bone length, and human motion). Then, to fully utilize these representations, we introduce a semi-analytical regressor to estimate the parameters of the human mesh model. The regressor consists of three modules: Temporal Inverse Kinematics (TIK), Bone-guided Shape Fitting (BSF), and Motion-Centric Refinement (MCR). TIK utilizes joint position to estimate initial pose parameters and BSF leverages bone length to regress bone-aligned shape parameters. Finally, MCR combines human motion representation with image features to refine the initial human model parameters. Extensive experiments demonstrate that our ARTS surpasses existing state-of-the-art video-based methods in both per-frame accuracy and temporal consistency on popular benchmarks: 3DPW, MPI-INF-3DHP, and Human3.6M. Code is available at https://github.com/TangTao-PKU/ARTS. △ Less

Submitted 20 October, 2024; originally announced October 2024.

Comments: Accepted by ACM MM 2024. Project page: https://github.com/TangTao-PKU/ARTS

Multimodal Learning for Embryo Viability Prediction in Clinical IVF

Authors: Junsik Kim, Zhiyi Shi, Davin Jeong, Johannes Knittel, Helen Y. Yang, Yonghyun Song, Wanhua Li, Yicong Li, Dalit Ben-Yosef, Daniel Needleman, Hanspeter Pfister

Abstract: In clinical In-Vitro Fertilization (IVF), identifying the most viable embryo for transfer is important to increasing the likelihood of a successful pregnancy. Traditionally, this process involves embryologists manually assessing embryos' static morphological features at specific intervals using light microscopy. This manual evaluation is not only time-intensive and costly, due to the need for expe… ▽ More In clinical In-Vitro Fertilization (IVF), identifying the most viable embryo for transfer is important to increasing the likelihood of a successful pregnancy. Traditionally, this process involves embryologists manually assessing embryos' static morphological features at specific intervals using light microscopy. This manual evaluation is not only time-intensive and costly, due to the need for expert analysis, but also inherently subjective, leading to variability in the selection process. To address these challenges, we develop a multimodal model that leverages both time-lapse video data and Electronic Health Records (EHRs) to predict embryo viability. One of the primary challenges of our research is to effectively combine time-lapse video and EHR data, owing to their inherent differences in modality. We comprehensively analyze our multimodal model with various modality inputs and integration approaches. Our approach will enable fast and automated embryo viability predictions in scale for clinical IVF. △ Less

Submitted 20 October, 2024; originally announced October 2024.

Comments: Accepted to MICCAI 2024

MMCS: A Multimodal Medical Diagnosis System Integrating Image Analysis and Knowledge-based Departmental Consultation

Authors: Yi Ren, HanZhi Zhang, Weibin Li, Diandong Liu, Tianyi Zhang, Jie He

Abstract: We present MMCS, a system capable of recognizing medical images and patient facial details, and providing professional medical diagnoses. The system consists of two core components: The first component is the analysis of medical images and videos. We trained a specialized multimodal medical model capable of interpreting medical images and accurately analyzing patients' facial emotions and facial p… ▽ More We present MMCS, a system capable of recognizing medical images and patient facial details, and providing professional medical diagnoses. The system consists of two core components: The first component is the analysis of medical images and videos. We trained a specialized multimodal medical model capable of interpreting medical images and accurately analyzing patients' facial emotions and facial paralysis conditions. The model achieved an accuracy of 72.59% on the FER2013 facial emotion recognition dataset, with a 91.1% accuracy in recognizing the happy emotion. In facial paralysis recognition, the model reached an accuracy of 92%, which is 30% higher than that of GPT-4o. Based on this model, we developed a parser for analyzing facial movement videos of patients with facial paralysis, achieving precise grading of the paralysis severity. In tests on 30 videos of facial paralysis patients, the system demonstrated a grading accuracy of 83.3%.The second component is the generation of professional medical responses. We employed a large language model, integrated with a medical knowledge base, to generate professional diagnoses based on the analysis of medical images or videos. The core innovation lies in our development of a department-specific knowledge base routing management mechanism, in which the large language model categorizes data by medical departments and, during the retrieval process, determines the appropriate knowledge base to query. This significantly improves retrieval accuracy in the RAG (retrieval-augmented generation) process. This mechanism led to an average increase of 4 percentage points in accuracy for various large language models on the MedQA dataset.Our code is open-sourced and available at: https://github.com/renllll/MMCS. △ Less

Submitted 20 October, 2024; originally announced October 2024.

HyQE: Ranking Contexts with Hypothetical Query Embeddings

Authors: Weichao Zhou, Jiaxin Zhang, Hilaf Hasson, Anu Singh, Wenchao Li

Abstract: In retrieval-augmented systems, context ranking techniques are commonly employed to reorder the retrieved contexts based on their relevance to a user query. A standard approach is to measure this relevance through the similarity between contexts and queries in the embedding space. However, such similarity often fails to capture the relevance. Alternatively, large language models (LLMs) have been u… ▽ More In retrieval-augmented systems, context ranking techniques are commonly employed to reorder the retrieved contexts based on their relevance to a user query. A standard approach is to measure this relevance through the similarity between contexts and queries in the embedding space. However, such similarity often fails to capture the relevance. Alternatively, large language models (LLMs) have been used for ranking contexts. However, they can encounter scalability issues when the number of candidate contexts grows and the context window sizes of the LLMs remain constrained. Additionally, these approaches require fine-tuning LLMs with domain-specific data. In this work, we introduce a scalable ranking framework that combines embedding similarity and LLM capabilities without requiring LLM fine-tuning. Our framework uses a pre-trained LLM to hypothesize the user query based on the retrieved contexts and ranks the context based on the similarity between the hypothesized queries and the user query. Our framework is efficient at inference time and is compatible with many other retrieval and ranking techniques. Experimental results show that our method improves the ranking performance across multiple benchmarks. The complete code and data are available at https://github.com/zwc662/hyqe △ Less

Submitted 19 October, 2024; originally announced October 2024.

Simulating and investigating various dynamic aspects of $\rm{H}_2\rm{O}$-related hydrogen bond model

Authors: Jiangchuan You, Ran Chen, Wanshun Li, Hui-hui Miao, Yuri Igorevich Ozhigov

Abstract: A simple $\rm{H}_2\rm{O}$-related hydrogen bond model, modified from the Jaynes-Cummings model, is proposed and its various dynamic aspects are investigated theoretically. In this model, the formation and breaking processes of hydrogen bond are accompanied by the creation and annihilation of the thermal phonon of the medium. A number of simplifying assumptions about the dynamics of the molecules i… ▽ More A simple $\rm{H}_2\rm{O}$-related hydrogen bond model, modified from the Jaynes-Cummings model, is proposed and its various dynamic aspects are investigated theoretically. In this model, the formation and breaking processes of hydrogen bond are accompanied by the creation and annihilation of the thermal phonon of the medium. A number of simplifying assumptions about the dynamics of the molecules involved are used. Rotating wave approximation is applied under consideration of the strong-coupling condition. Dissipative dynamics under the Markovian approximation is obtained through solving the quantum master equation - Lindbladian. The probabilities of reaction channels involving hydrogen bond depending on the parameters of the external environment, are obtained. Differences between unitary and dissipative evolutions are disciussed. Consideration is given to the effect of all kinds of potential interactions and dissipations on evolution. Consideration is also given to the reverse processes (inflows) of dissipations. The results show that the magnitude changes of the interactions and dissipations have slight effect on the formation of hydrogen bond, but the variation of the reverse processes of dissipations significantly affect the formation of hydrogen bond. According to the findings, the dynamics of $\rm{H}_2\rm{O}$-related hydrogen bond model can be controlled by selectively choosing system parameters. The results will be used as a basis to extend the research to more complex chemical and biological model in the future. △ Less

Submitted 19 October, 2024; originally announced October 2024.

Comments: 15 pages, 11 figures, 1 table

Nonconvex Robust Quaternion Matrix Completion for Imaging Processing

Authors: Baohua Huang, Jiakai Chen, Wen Li

Abstract: One of the tasks in color image processing and computer vision is to recover clean data from partial observations corrupted by noise. To this end, robust quaternion matrix completion (QMC) has recently attracted more attention and shown its effectiveness, whose convex relaxation is to minimize the quaternion nuclear norm plus the quaternion $L_1$-norm. However, there is still room to improve due t… ▽ More One of the tasks in color image processing and computer vision is to recover clean data from partial observations corrupted by noise. To this end, robust quaternion matrix completion (QMC) has recently attracted more attention and shown its effectiveness, whose convex relaxation is to minimize the quaternion nuclear norm plus the quaternion $L_1$-norm. However, there is still room to improve due to the convexity of the convex surrogates. This paper proposes a new nonconvex robust QMC model, in which the nonconvex MCP function and the quaternion $L_p$-norm are used to enhance the low-rankness and sparseness of the low-rank term and sparse term, respectively. An alternating direction method of multipliers (ADMM) algorithm is developed to solve the proposed model and its convergence is given. Moreover, a novel nonlocal-self-similarity-based nonconvex robust quaternion completion method is proposed to handle large-scale data. Numerical results on color images and videos indicate the advantages of the proposed method over some existing ones. △ Less

Submitted 19 October, 2024; originally announced October 2024.

Multistable Synaptic Plasticity induces Memory Effects and Cohabitation of Chimera and Bump States in Leaky Integrate-and-Fire Networks

Authors: Astero Provata, Yannis Almirantis, Wentian Li

Abstract: Chimera states and bump states are collective synchronization phenomena observed independently (at different parameter regions) in networks of coupled nonlinear oscillators. And while chimera states are characterized by coexistence of coherent and incoherent domains, bump states consist of active domains operating on a silent background. Multistable plasticity in the network connections originates… ▽ More Chimera states and bump states are collective synchronization phenomena observed independently (at different parameter regions) in networks of coupled nonlinear oscillators. And while chimera states are characterized by coexistence of coherent and incoherent domains, bump states consist of active domains operating on a silent background. Multistable plasticity in the network connections originates from brain dynamics and is based on the idea that neural cells may transmit inhibitory or excitatory signals depending on various factors, such as local connectivity, influence of neighboring cells etc. During the system/network integration, the link weights adapt and, in the case of multistability, they may organize in coexisting excitatory and/or inhibitory domains. Here, we explore the influence of bistable plasticity on collective synchronization states and we numerically demonstrate that the dynamics of the linking may give rise to co-existence of bump-like and chimera-like states simultaneously in the network. In the case of bump and chimera co-existence, confinement effects are developed: the different domains stay localized and do not travel around the network. Memory effects are also reported in the sense that the final spatial arrangement of the coupling strengths reflects some of the local properties of the initial link distribution. For the quantification of the system's spatial and temporal features, the global and local entropy functions are employed as measures of the network organization, while the average firing rates account for the network evolution and dynamics. △ Less

Submitted 18 October, 2024; originally announced October 2024.

Comments: 22 pages, 7 figures

The KnowWhereGraph Ontology

Authors: Cogan Shimizu, Shirly Stephe, Adrita Barua, Ling Cai, Antrea Christou, Kitty Currier, Abhilekha Dalal, Colby K. Fisher, Pascal Hitzler, Krzysztof Janowicz, Wenwen Li, Zilong Liu, Mohammad Saeid Mahdavinejad, Gengchen Mai, Dean Rehberger, Mark Schildhauer, Meilin Shi, Sanaz Saki Norouzi, Yuanyuan Tian, Sizhe Wang, Zhangyu Wang, Joseph Zalewski, Lu Zhou, Rui Zhu

Abstract: KnowWhereGraph is one of the largest fully publicly available geospatial knowledge graphs. It includes data from 30 layers on natural hazards (e.g., hurricanes, wildfires), climate variables (e.g., air temperature, precipitation), soil properties, crop and land-cover types, demographics, and human health, various place and region identifiers, among other themes. These have been leveraged through t… ▽ More KnowWhereGraph is one of the largest fully publicly available geospatial knowledge graphs. It includes data from 30 layers on natural hazards (e.g., hurricanes, wildfires), climate variables (e.g., air temperature, precipitation), soil properties, crop and land-cover types, demographics, and human health, various place and region identifiers, among other themes. These have been leveraged through the graph by a variety of applications to address challenges in food security and agricultural supply chains; sustainability related to soil conservation practices and farm labor; and delivery of emergency humanitarian aid following a disaster. In this paper, we introduce the ontology that acts as the schema for KnowWhereGraph. This broad overview provides insight into the requirements and design specifications for the graph and its schema, including the development methodology (modular ontology modeling) and the resources utilized to implement, materialize, and deploy KnowWhereGraph with its end-user interfaces and public query SPARQL endpoint. △ Less

Submitted 17 October, 2024; originally announced October 2024.

A spatiotemporal knowledge graph-based method for identifying individual activity locations from mobile phone data

Authors: Jian Li, Tian Gan, Weifeng Li, Yuhang Liu

Abstract: In recent years, mobile phone data has been widely used for human mobility analytics. Identifying individual activity locations is the fundamental step for mobile phone data processing. Current methods typically aggregate spatially adjacent location records over multiple days to identify activity locations. However, only considering spatial relationships while overlooking temporal ones may lead to… ▽ More In recent years, mobile phone data has been widely used for human mobility analytics. Identifying individual activity locations is the fundamental step for mobile phone data processing. Current methods typically aggregate spatially adjacent location records over multiple days to identify activity locations. However, only considering spatial relationships while overlooking temporal ones may lead to inaccurate activity location identification, and also affect activity pattern analysis. In this study, we propose a spatiotemporal knowledge graph-based (STKG) method for identifying activity locations from mobile phone data. An STKG is designed and constructed to describe individual mobility characteristics. The spatial and temporal relationships of individual stays are inferred and transformed into a spatiotemporal graph. The modularity-optimization community detection algorithm is applied to identify stays with dense spatiotemporal relationships, which are considering as activity locations. A case study in Shanghai was conducted to verify the performance of the proposed method. The results show that compared with two baseline methods, the STKG-based method can limit an additional 45% of activity locations with the longest daytime stay within a reasonable spatial range; In addition, the STKG-based method exhibit lower variance in the start and end times of activities across different days, performing approximately 10% to 20% better than the two baseline methods. Moreover, the STKG-based method effectively distinguishes between locations that are geographically close but exhibit different temporal patterns. These findings demonstrate the effectiveness of STKG-based method in enhancing both spatial precision and temporal consistency. △ Less

Submitted 16 October, 2024; originally announced October 2024.

Comments: 24 pages, 10 figures, 1 table

Arthur packets for metaplectic groups

Authors: Wen-Wei Li

Abstract: For metaplectic groups over a local field of characteristic zero, we define the Arthur packet attached to any Arthur parameter $ψ$ as a multi-set of unitary genuine irreducible representations, characterized by endoscopic character relations. Over number fields, we obtain a multiplicity formula for the genuine discrete $L^2$-automorphic spectrum in terms of global Arthur parameters and $ε$-factors… ▽ More For metaplectic groups over a local field of characteristic zero, we define the Arthur packet attached to any Arthur parameter $ψ$ as a multi-set of unitary genuine irreducible representations, characterized by endoscopic character relations. Over number fields, we obtain a multiplicity formula for the genuine discrete $L^2$-automorphic spectrum in terms of global Arthur parameters and $ε$-factors, by leveraging the trace formula for metaplectic groups. This confirms a conjecture of Gan, and extends earlier results of Gan-Ichino on the Shimura-Waldspurger correspondences, whereas their works play a critical role in our proof. Furthermore, all these are shown to be compatible with existing results in rank one (Waldspurger) and two (Gan-Ichino). △ Less

Submitted 17 October, 2024; originally announced October 2024.

Comments: 90 pages

MSC Class: 22E50 (Primary) 11F70; 11F72 (Secondary)

SDI-Paste: Synthetic Dynamic Instance Copy-Paste for Video Instance Segmentation

Authors: Sahir Shrestha, Weihao Li, Gao Zhu, Nick Barnes

Abstract: Data augmentation methods such as Copy-Paste have been studied as effective ways to expand training datasets while incurring minimal costs. While such methods have been extensively implemented for image level tasks, we found no scalable implementation of Copy-Paste built specifically for video tasks. In this paper, we leverage the recent growth in video fidelity of generative models to explore eff… ▽ More Data augmentation methods such as Copy-Paste have been studied as effective ways to expand training datasets while incurring minimal costs. While such methods have been extensively implemented for image level tasks, we found no scalable implementation of Copy-Paste built specifically for video tasks. In this paper, we leverage the recent growth in video fidelity of generative models to explore effective ways of incorporating synthetically generated objects into existing video datasets to artificially expand object instance pools. We first procure synthetic video sequences featuring objects that morph dynamically with time. Our carefully devised pipeline automatically segments then copy-pastes these dynamic instances across the frames of any target background video sequence. We name our video data augmentation pipeline Synthetic Dynamic Instance Copy-Paste, and test it on the complex task of Video Instance Segmentation which combines detection, segmentation and tracking of object instances across a video sequence. Extensive experiments on the popular Youtube-VIS 2021 dataset using two separate popular networks as baselines achieve strong gains of +2.9 AP (6.5%) and +2.1 AP (4.9%). We make our code and models publicly available. △ Less

Submitted 16 October, 2024; originally announced October 2024.

Observation of a rare beta decay of the charmed baryon with a Graph Neural Network

Authors: BESIII Collaboration, M. Ablikim, M. N. Achasov, P. Adlarson, O. Afedulidis, X. C. Ai, R. Aliberti, A. Amoroso, Q. An, Y. Bai, O. Bakina, I. Balossino, Y. Ban, H. -R. Bao, V. Batozskaya, K. Begzsuren, N. Berger, M. Berlowski, M. Bertani, D. Bettoni, F. Bianchi, E. Bianco, A. Bortone, I. Boyko, R. A. Briere , et al. (637 additional authors not shown)

Abstract: The study of beta decay of the charmed baryon provides unique insights into the fundamental mechanism of the strong and electro-weak interactions. The $Λ_c^+$, being the lightest charmed baryon, undergoes disintegration solely through the charm quark weak decay. Its beta decay provides an ideal laboratory for investigating non-perturbative effects in quantum chromodynamics and for constraining the… ▽ More The study of beta decay of the charmed baryon provides unique insights into the fundamental mechanism of the strong and electro-weak interactions. The $Λ_c^+$, being the lightest charmed baryon, undergoes disintegration solely through the charm quark weak decay. Its beta decay provides an ideal laboratory for investigating non-perturbative effects in quantum chromodynamics and for constraining the fundamental parameters of the Cabibbo-Kobayashi-Maskawa matrix in weak interaction theory. This article presents the first observation of the Cabibbo-suppressed $Λ_c^+$ beta decay into a neutron $Λ_c^+ \rightarrow n e^+ ν_{e}$, based on $4.5~\mathrm{fb}^{-1}$ of electron-positron annihilation data collected with the BESIII detector in the energy region above the $Λ^+_c\barΛ^-_c$ threshold. A novel machine learning technique, leveraging Graph Neural Networks, has been utilized to effectively separate signals from dominant backgrounds, particularly $Λ_c^+ \rightarrow Λe^+ ν_{e}$. This approach has yielded a statistical significance of more than $10σ$. The absolute branching fraction of $Λ_c^+ \rightarrow n e^+ ν_{e}$ is measured to be $(3.57\pm0.34_{\mathrm{stat}}\pm0.14_{\mathrm{syst}})\times 10^{-3}$. For the first time, the CKM matrix element $\left|V_{cd}\right|$ is extracted via a charmed baryon decay to be $0.208\pm0.011_{\rm exp.}\pm0.007_{\rm LQCD}\pm0.001_{τ_{Λ_c^+}}$. This study provides a new probe to further understand fundamental interactions in the charmed baryon sector, and demonstrates the power of modern machine learning techniques in enhancing experimental capability in high energy physics research. △ Less

Submitted 17 October, 2024; originally announced October 2024.

Comments: 28 pages, 6 figures

Observation of $χ_{c0}\toΣ^{+}\barΣ^{-}η$ and evidence for $χ_{c1,2}\toΣ^{+}\barΣ^{-}η$

Authors: BESIII Collaboration, M. Ablikim, M. N. Achasov, P. Adlarson, O. Afedulidis, X. C. Ai, R. Aliberti, A. Amoroso, Q. An, Y. Bai, O. Bakina, I. Balossino, Y. Ban, H. -R. Bao, V. Batozskaya, K. Begzsuren, N. Berger, M. Berlowski, M. Bertani, D. Bettoni, F. Bianchi, E. Bianco, A. Bortone, I. Boyko, R. A. Briere , et al. (634 additional authors not shown)

Abstract: Using $(27.12\pm 0.14)\times10^{8}$ $ψ(3686)$ events collected with the BESIII detector, the decay $χ_{c0}\toΣ^{+}\barΣ^{-}η$ is observed for the first time with a statistical significance of $7.0σ$, and evidence for $χ_{c1}\toΣ^{+}\barΣ^{-}η$ and $χ_{c2}\toΣ^{+}\barΣ^{-}η$ is found with statistical significances of $4.3σ$ and $4.6σ$, respectively. The branching fractions are determined to be… ▽ More Using $(27.12\pm 0.14)\times10^{8}$ $ψ(3686)$ events collected with the BESIII detector, the decay $χ_{c0}\toΣ^{+}\barΣ^{-}η$ is observed for the first time with a statistical significance of $7.0σ$, and evidence for $χ_{c1}\toΣ^{+}\barΣ^{-}η$ and $χ_{c2}\toΣ^{+}\barΣ^{-}η$ is found with statistical significances of $4.3σ$ and $4.6σ$, respectively. The branching fractions are determined to be $\mathcal{B}(χ_{c0}\toΣ^{+}\barΣ^{-}η)=({1.26 \pm 0.20 \pm 0.13}) \times 10^{-4}, ~\mathcal{B}(χ_{c1}\toΣ^{+}\barΣ^{-}η)=({5.10 \pm 1.21 \pm 0.67}) \times 10^{-5}$, and $\mathcal{B}(χ_{c2}\toΣ^{+}\barΣ^{-}η)=({5.46 \pm 1.18 \pm 0.50}) \times 10^{-5}$, where the first uncertainties are statistical, and the second ones are systematic. △ Less

Submitted 17 October, 2024; originally announced October 2024.

Observation of the Singly Cabibbo-Suppressed Decay $Λ_c^{+}\to pπ^0$

Authors: BESIII Collaboration, M. Ablikim, M. N. Achasov, P. Adlarson, O. Afedulidis, X. C. Ai, R. Aliberti, A. Amoroso, Q. An, Y. Bai, O. Bakina, I. Balossino, Y. Ban, H. -R. Bao, V. Batozskaya, K. Begzsuren, N. Berger, M. Berlowski, M. Bertani, D. Bettoni, F. Bianchi, E. Bianco, A. Bortone, I. Boyko, R. A. Briere , et al. (638 additional authors not shown)

Abstract: Utilizing 4.5${~\rm{fb}}^{-1}$ of $e^+e^-$ annihilation data collected with the BESIII detector at the BEPCII collider at center-of-mass energies between 4.600 and 4.699 GeV, the first observation of the singly Cabibbo-suppressed decay $Λ_c^{+}\to pπ^0$ is presented, with a statistical significance of $5.4σ$. The ratio of the branching fractions of $Λ_c^{+}\to pπ^0$ and $Λ_c^{+}\to pη$ is measured… ▽ More Utilizing 4.5${~\rm{fb}}^{-1}$ of $e^+e^-$ annihilation data collected with the BESIII detector at the BEPCII collider at center-of-mass energies between 4.600 and 4.699 GeV, the first observation of the singly Cabibbo-suppressed decay $Λ_c^{+}\to pπ^0$ is presented, with a statistical significance of $5.4σ$. The ratio of the branching fractions of $Λ_c^{+}\to pπ^0$ and $Λ_c^{+}\to pη$ is measured as $\mathcal{B}(Λ_c^{+}\to pπ^0)/\mathcal{B}(Λ_c^{+}\to pη)=(0.120\pm0.026_{\rm stat.}\pm0.007_{\rm syst.})$. This result resolves the longstanding discrepancy between earlier experimental searches, providing both a decisive conclusion and valuable input for QCD-inspired theoretical models. A sophisticated deep learning approach using a Transformer-based architecture is employed to distinguish the signal from the prevalent hadronic backgrounds, complemented by thorough validation and systematic uncertainty quantification. △ Less

Submitted 17 October, 2024; originally announced October 2024.

Comments: 9 pages, 4 figures

Cerberus: Efficient Inference with Adaptive Parallel Decoding and Sequential Knowledge Enhancement

Authors: Yuxuan Liu, Wenyuan Li, Laizhong Cui, Hailiang Yang

Abstract: Large language models (LLMs) often face a bottleneck in inference speed due to their reliance on auto-regressive decoding. Recently, parallel decoding has shown significant promise in enhancing inference efficiency. However, we have identified two key issues with existing parallel decoding frameworks: (1) decoding heads fail to balance prediction accuracy and the parallelism of execution, and (2)… ▽ More Large language models (LLMs) often face a bottleneck in inference speed due to their reliance on auto-regressive decoding. Recently, parallel decoding has shown significant promise in enhancing inference efficiency. However, we have identified two key issues with existing parallel decoding frameworks: (1) decoding heads fail to balance prediction accuracy and the parallelism of execution, and (2) parallel decoding is not a universal solution, as it can bring unnecessary overheads at some challenging decoding steps. To address these issues, we propose Cerberus, an adaptive parallel decoding framework introduces the gating mechanism to enable the LLMs to adaptively choose appropriate decoding approaches at each decoding step, along with introducing a new paradigm of decoding heads that introduce the sequential knowledge while maintaining execution parallelism. The experiment results demonstrate that the Cerberus can achieve up to 2.12x speed up compared to auto-regressive decoding, and outperforms one of the leading parallel decoding frameworks, Medusa, with a 10% - 30% increase in acceleration and superior generation quality. △ Less

Submitted 17 October, 2024; originally announced October 2024.

On the Boltzmann equation with soft potentials: Existence, uniqueness and smoothing effect of mild solutions

Authors: Ling-Bing He, Jie Ji, Wei-Xi Li

Abstract: We consider the spatially inhomogeneous Boltzmann equation without angular cutoff for soft potentials. For any given initial datum such that the mass, energy and entropy densities are bounded and the mass is away from vacuum, we establish the local-in-time existence and uniqueness of mild solutions, and further provide the first result on sharp smoothing effect in analytic space or Gevrey space fo… ▽ More We consider the spatially inhomogeneous Boltzmann equation without angular cutoff for soft potentials. For any given initial datum such that the mass, energy and entropy densities are bounded and the mass is away from vacuum, we establish the local-in-time existence and uniqueness of mild solutions, and further provide the first result on sharp smoothing effect in analytic space or Gevrey space for soft potentials. △ Less

Submitted 17 October, 2024; originally announced October 2024.

Comments: 60pages,0 figure

MSC Class: 35B65; 35Q20

ACCEPT: Adaptive Codebook for Composite and Efficient Prompt Tuning

Authors: Yu-Chen Lin, Wei-Hua Li, Jun-Cheng Chen, Chu-Song Chen

Abstract: Prompt Tuning has been a popular Parameter-Efficient Fine-Tuning method attributed to its remarkable performance with few updated parameters on various large-scale pretrained Language Models (PLMs). Traditionally, each prompt has been considered indivisible and updated independently, leading the parameters increase proportionally as prompt length grows. To address this issue, we propose Adaptive C… ▽ More Prompt Tuning has been a popular Parameter-Efficient Fine-Tuning method attributed to its remarkable performance with few updated parameters on various large-scale pretrained Language Models (PLMs). Traditionally, each prompt has been considered indivisible and updated independently, leading the parameters increase proportionally as prompt length grows. To address this issue, we propose Adaptive Codebook for Composite and Efficient Prompt Tuning (ACCEPT). In our method, we refer to the concept of product quantization (PQ), allowing all soft prompts to share a set of learnable codebook vectors in each subspace, with each prompt differentiated by a set of adaptive weights. We achieve the superior performance on 17 diverse natural language tasks including natural language understanding (NLU) and question answering (QA) tasks by tuning only 0.3% of parameters of the PLMs. Our approach also excels in few-shot and large model settings, highlighting its significant potential. △ Less

Submitted 17 October, 2024; v1 submitted 10 October, 2024; originally announced October 2024.

Comments: EMNLP Findings 2024

Search for $e^{+}e^{-} \to φχ_{c0}$ and $φη_{c2}(1D)$ at center-of-mass energies from 4.47 to 4.95 GeV

Authors: BESIII Collaboration, M. Ablikim, M. N. Achasov, P. Adlarson, O. Afedulidis, X. C. Ai, R. Aliberti, A. Amoroso, Q. An, Y. Bai, O. Bakina, I. Balossino, Y. Ban, H. -R. Bao, V. Batozskaya, K. Begzsuren, N. Berger, M. Berlowski, M. Bertani, D. Bettoni, F. Bianchi, E. Bianco, A. Bortone, I. Boyko, R. A. Briere , et al. (644 additional authors not shown)

Abstract: Utilizing a data set of $6.7$ fb$^{-1}$ from electron-positron collisions recorded by the BESIII detector at the BEPCII storage ring, a search is conducted for the processes $e^{+}e^{-} \to φχ_{c0}$ and $φη_{c2}(1D)$ across center-of-mass energies from 4.47 to 4.95 GeV. In the absence of any significant signals, upper limits are set. These include limits on the Born cross sections for… ▽ More Utilizing a data set of $6.7$ fb$^{-1}$ from electron-positron collisions recorded by the BESIII detector at the BEPCII storage ring, a search is conducted for the processes $e^{+}e^{-} \to φχ_{c0}$ and $φη_{c2}(1D)$ across center-of-mass energies from 4.47 to 4.95 GeV. In the absence of any significant signals, upper limits are set. These include limits on the Born cross sections for $e^{+}e^{-} \to φχ_{c0}$, as well as the product of the Born cross section for $e^{+}e^{-} \to φη_{c2}(1D)$ and a sum of five branching fractions. Furthermore, the product of the electronic width of $Y(4660)$ and the branching fraction of the $Y(4660) \to φχ_{c0}$, denoted as $Γ^{Y(4660)}_{e^{+}e^{-}} \mathcal{B}_{Y(4660) \to φχ_{c0}}$, is determined to be $< 0.40$ eV at the 90\% confidence level. △ Less

Submitted 16 October, 2024; originally announced October 2024.

Comments: 14 pages, 6 figures

Highly anisotropic Drude-weight-reduction and enhanced linear-dichroism in van der Waals Weyl semimetal Td-MoTe2 with coherent interlayer electronic transport

Authors: Bo Su, Weikang Wu, Jianzhou Zhao, Xiutong Deng, Wenhui Li, Shengyuan A. Yang, Youguo Shi, Qiang Li, Jianlin Luo, Genda Gu, Zhi-Guo Chen

Abstract: Weyl semimetal (WSM) states can be achieved by breaking spatial-inversion symmetry or time reversal symmetry. However, the anisotropy of the energy reduction contributing to the emergence of WSM states has seldom been investigated by experiments. A van der Waals metal MoTe2 exhibits a type-II WSM phase below the monoclinic-to-orthorhombic-phase-transition temperature Tc ~ 250 K. Here, we report a… ▽ More Weyl semimetal (WSM) states can be achieved by breaking spatial-inversion symmetry or time reversal symmetry. However, the anisotropy of the energy reduction contributing to the emergence of WSM states has seldom been investigated by experiments. A van der Waals metal MoTe2 exhibits a type-II WSM phase below the monoclinic-to-orthorhombic-phase-transition temperature Tc ~ 250 K. Here, we report a combined linearly-polarized optical-spectroscopy and electrical-transport study of MoTe2 at different temperatures. The Drude components in the a-axis, b-axis and c-axis optical conductivity spectra, together with the metallic out-of-plane and in-plane electrical resistivities, indicate the coherent inter-layer and in-plane charge transports. Moreover, the Drude weight in σ1a(ω), rather than the Drude weights in σ1b(ω) and σ1c(ω), decreases dramatically below Tc, which exhibits a highly anisotropic decrease in its Drude weight and thus suggests a strongly anisotropic reduction of the electronic kinetic energy in the WSM phase. Furthermore, below Tc, due to the in-plane anisotropic spectral-weight transfer from Drude component to high-energy region, the in-plane inter-band-absorption anisotropy increases remarkably around 770 meV, and has the largest value (~ 0.68) of normalized linear dichroism among the reported type-II WSMs. Our work sheds light on seeking new WSMs and developing novel photonic devices based on WSMs. △ Less

Submitted 16 October, 2024; originally announced October 2024.

Comments: Accepted by Laser & Photonics Reviews

The EMC Effect of Tritium and Helium-3 from the JLab MARATHON Experiment

Authors: D. Abrams, H. Albataineh, B. S. Aljawrneh, S. Alsalmi, D. Androic, K. Aniol, W. Armstrong, J. Arrington, H. Atac, T. Averett, C. Ayerbe Gayoso, X. Bai, J. Bane, S. Barcus, A. Beck, V. Bellini, H. Bhatt, D. Bhetuwal, D. Biswas, D. Blyth, W. Boeglin, D. Bulumulla, J. Butler, A. Camsonne, M. Carmignotto , et al. (109 additional authors not shown)

Abstract: Measurements of the EMC effect in the tritium and helium-3 mirror nuclei are reported. The data were obtained by the MARATHON Jefferson Lab experiment, which performed deep inelastic electron scattering from deuterium and the three-body nuclei, using a cryogenic gas target system and the High Resolution Spectrometers of the Hall A Facility of the Lab. The data cover the Bjorken $x$ range from 0.20… ▽ More Measurements of the EMC effect in the tritium and helium-3 mirror nuclei are reported. The data were obtained by the MARATHON Jefferson Lab experiment, which performed deep inelastic electron scattering from deuterium and the three-body nuclei, using a cryogenic gas target system and the High Resolution Spectrometers of the Hall A Facility of the Lab. The data cover the Bjorken $x$ range from 0.20 to 0.83, corresponding to a squared four-momentum transfer $Q^2$ range from 2.7 to $11.9\gevsq$, and to an invariant mass $W$ of the final hadronic state greater than 1.84 GeV/${\it c}^2$. The tritium EMC effect measurement is the first of its kind. The MARATHON experimental results are compared to results from previous measurements by DESY-HERMES and JLab-Hall C experiments, as well as with few-body theoretical predictions. △ Less

Submitted 15 October, 2024; originally announced October 2024.

Comments: arXiv admin note: text overlap with arXiv:2104.05850

WeatherDG: LLM-assisted Procedural Weather Generation for Domain-Generalized Semantic Segmentation

Authors: Chenghao Qian, Yuhu Guo, Yuhong Mo, Wenjing Li

Abstract: In this work, we propose a novel approach, namely WeatherDG, that can generate realistic, weather-diverse, and driving-screen images based on the cooperation of two foundation models, i.e, Stable Diffusion (SD) and Large Language Model (LLM). Specifically, we first fine-tune the SD with source data, aligning the content and layout of generated samples with real-world driving scenarios. Then, we pr… ▽ More In this work, we propose a novel approach, namely WeatherDG, that can generate realistic, weather-diverse, and driving-screen images based on the cooperation of two foundation models, i.e, Stable Diffusion (SD) and Large Language Model (LLM). Specifically, we first fine-tune the SD with source data, aligning the content and layout of generated samples with real-world driving scenarios. Then, we propose a procedural prompt generation method based on LLM, which can enrich scenario descriptions and help SD automatically generate more diverse, detailed images. In addition, we introduce a balanced generation strategy, which encourages the SD to generate high-quality objects of tailed classes under various weather conditions, such as riders and motorcycles. This segmentation-model-agnostic method can improve the generalization ability of existing models by additionally adapting them with the generated synthetic data. Experiments on three challenging datasets show that our method can significantly improve the segmentation performance of different state-of-the-art models on target domains. Notably, in the setting of ''Cityscapes to ACDC'', our method improves the baseline HRDA by 13.9% in mIoU. △ Less

Submitted 15 October, 2024; originally announced October 2024.

Extending 1089 attractor to any number of digits and any number of steps

Authors: Yannis Almirantis, Wentian Li

Abstract: The well-known 1089 trick reflects an amazing trait of digital reversal process and reminisces of a limiting attractor in dynamical systems even though it takes only two steps. It is natural to consider the situations when the number of digits is beyond three as in the original 1089 trick, as well as situations when the number of steps is beyond two. The first part has been mostly done by Webster… ▽ More The well-known 1089 trick reflects an amazing trait of digital reversal process and reminisces of a limiting attractor in dynamical systems even though it takes only two steps. It is natural to consider the situations when the number of digits is beyond three as in the original 1089 trick, as well as situations when the number of steps is beyond two. The first part has been mostly done by Webster which we will reproduce. After two steps, the resulting integers are called Papadakis-Webster integers (PWI), which is always divisible by 99, and the resulting quotients consist of only 0's and 1's, which we name Papadakis-Webster binary strings (PWBS). Not all binary strings could be PWBS, and we define the hairpin pairing rule to determine if a binary string is a PWBS. For the second part, we propose a two-option iteration system named iterative digital reversal (IDR) suitably interweaving additions and subtractions. The simplest limiting behavior of IDR is 2-cycles. The elements in an IDR 2-cycle are all composed of repetitions of the 10(9)$_L$89 (L>=0) motif, and are all PWIs. The lower 2-cycle elements after division of 99 belong to the subset of PWBS that are palindromic and consist of 0- and 1-blocks with a minimal length of two. IDR also has higher p-cycles (p=10,12,71) whose elements seem to contain at least one PWI. Another interesting finding about IDR is that it contains non-periodic and diverging trajectories, as the integer values grow to infinity. In these diverging trajectories, while the number of flanking digits around the middle point increases by the iteration, the middle part has an 8-cycle rhythm or signature which has been found in all diverging trajectories. Overall, the generalization of the original 1089 trick in both space and time leads to new patterns in integers and new phenomenology in dynamics. △ Less

Submitted 15 October, 2024; originally announced October 2024.

Comments: 1 figure

Stochastic diagonal estimation with adaptive parameter selection

Authors: Zongyuan Han, Wenhao Li, Shengxin Zhu

Abstract: In this paper, we investigate diagonal estimation for large or implicit matrices, aiming to develop a novel and efficient stochastic algorithm that incorporates adaptive parameter selection. We explore the influence of different eigenvalue distributions on diagonal estimation and analyze the necessity of introducing the projection method and adaptive parameter optimization into the stochastic diag… ▽ More In this paper, we investigate diagonal estimation for large or implicit matrices, aiming to develop a novel and efficient stochastic algorithm that incorporates adaptive parameter selection. We explore the influence of different eigenvalue distributions on diagonal estimation and analyze the necessity of introducing the projection method and adaptive parameter optimization into the stochastic diagonal estimator. Based on this analysis, we derive a lower bound on the number of random query vectors needed to satisfy a given probabilistic error bound, which forms the foundation of our adaptive stochastic diagonal estimation algorithm. Finally, numerical experiments demonstrate the effectiveness of the proposed estimator for various matrix types, showcasing its efficiency and stability compared to other existing stochastic diagonal estimation methods. △ Less

Submitted 15 October, 2024; originally announced October 2024.

Observation of $χ_{cJ}\to p \bar p K^0_S K^- π^+ + c.c.$

Authors: BESIII Collaboration, M. Ablikim, M. N. Achasov, P. Adlarson, O. Afedulidis, X. C. Ai, R. Aliberti, A. Amoroso, Y. Bai, O. Bakina, I. Balossino, Y. Ban, H. -R. Bao, V. Batozskaya, K. Begzsuren, N. Berger, M. Berlowski, M. Bertani, D. Bettoni, F. Bianchi, E. Bianco, A. Bortone, I. Boyko, R. A. Briere, A. Brueggemann , et al. (648 additional authors not shown)

Abstract: By analyzing $(27.12\pm0.14)\times10^8$ $ψ(3686)$ events collected with the BESIII detector operating at the BEPCII collider, the decays of $χ_{cJ} \to p \bar{p} K^0_S K^- π^+ +c.c.(J=0, 1, 2)$ are observed for the first time with statistical significances greater than $10σ$. The branching fractions of these decays are determined to be… ▽ More By analyzing $(27.12\pm0.14)\times10^8$ $ψ(3686)$ events collected with the BESIII detector operating at the BEPCII collider, the decays of $χ_{cJ} \to p \bar{p} K^0_S K^- π^+ +c.c.(J=0, 1, 2)$ are observed for the first time with statistical significances greater than $10σ$. The branching fractions of these decays are determined to be $\mathcal{B}(χ_{c0}\to p \bar p K^{0}_{S} K^- π^+ + c.c.)=(2.61\pm0.27\pm0.32)\times10^{-5},$ $\mathcal{B}(χ_{c1}\to p \bar p K^{0}_{S} K^- π^+ + c.c.)=(4.16\pm0.24\pm0.46)\times10^{-5},$ and $\mathcal{B}(χ_{c2}\to p \bar p K^{0}_{S} K^- π^+ + c.c.)=(5.63\pm0.28\pm0.46)\times10^{-5}$, respectively. The processes $χ_{c1,2} \to \bar{p} Λ(1520) K^0_S π^{+} + c.c.$ are also observed, with statistical significances of 5.7$σ$ and 7.0$σ$, respectively. Evidence for $χ_{c0} \to\bar{p} Λ(1520) K^0_S π^{+} + c.c.$ is found with statistical significances of 3.3$σ$ each. The corresponding branching fractions are determined to be $\mathcal{B}(χ_{c0}\to \bar{p} Λ(1520) K^0_S π^{+} + c.c.) =(1.61^{+0.68}_{-0.64}\pm0.23)\times10^{-5}$, $\mathcal{B}(χ_{c1}\to \bar{p} Λ(1520) K^0_S π^{+} + c.c.)=(4.06^{+0.80}_{-0.76}\pm0.52)\times10^{-5}$, and $\mathcal{B}(χ_{c2}\to \bar{p} Λ(1520) K^0_S π^{+} + c.c.)=(4.09^{+0.87}_{-0.84}\pm0.42)\times10^{-5}$. Here, the first uncertainties are statistical and the second ones are systematic. △ Less

Submitted 15 October, 2024; originally announced October 2024.

Comments: 12 pages, 5 figures

Process Reward Model with Q-Value Rankings

Authors: Wendi Li, Yixuan Li

Abstract: Process Reward Modeling (PRM) is critical for complex reasoning and decision-making tasks where the accuracy of intermediate steps significantly influences the overall outcome. Existing PRM approaches, primarily framed as classification problems, employ cross-entropy loss to independently evaluate each step's correctness. This method can lead to suboptimal reward distribution and does not adequate… ▽ More Process Reward Modeling (PRM) is critical for complex reasoning and decision-making tasks where the accuracy of intermediate steps significantly influences the overall outcome. Existing PRM approaches, primarily framed as classification problems, employ cross-entropy loss to independently evaluate each step's correctness. This method can lead to suboptimal reward distribution and does not adequately address the interdependencies among steps. To address these limitations, we introduce the Process Q-value Model (PQM), a novel framework that redefines PRM in the context of a Markov Decision Process. PQM optimizes Q-value rankings based on a novel comparative loss function, enhancing the model's ability to capture the intricate dynamics among sequential decisions. This approach provides a more granular and theoretically grounded methodology for process rewards. Our extensive empirical evaluations across various sampling policies, language model backbones, and multi-step reasoning benchmarks show that PQM outperforms classification-based PRMs. The effectiveness of the comparative loss function is highlighted in our comprehensive ablation studies, confirming PQM's practical efficacy and theoretical advantage. △ Less

Submitted 15 October, 2024; originally announced October 2024.

Tree of Attributes Prompt Learning for Vision-Language Models

Authors: Tong Ding, Wanhua Li, Zhongqi Miao, Hanspeter Pfister

Abstract: Prompt learning has proven effective in adapting vision language models for downstream tasks. However, existing methods usually append learnable prompt tokens solely with the category names to obtain textual features, which fails to fully leverage the rich context indicated in the category name. To address this issue, we propose the Tree of Attributes Prompt learning (TAP), which first instructs L… ▽ More Prompt learning has proven effective in adapting vision language models for downstream tasks. However, existing methods usually append learnable prompt tokens solely with the category names to obtain textual features, which fails to fully leverage the rich context indicated in the category name. To address this issue, we propose the Tree of Attributes Prompt learning (TAP), which first instructs LLMs to generate a tree of attributes with a "concept - attribute - description" structure for each category, and then learn the hierarchy with vision and text prompt tokens. Unlike existing methods that merely augment category names with a set of unstructured descriptions, our approach essentially distills structured knowledge graphs associated with class names from LLMs. Furthermore, our approach introduces text and vision prompts designed to explicitly learn the corresponding visual attributes, effectively serving as domain experts. Additionally, the general and diverse descriptions generated based on the class names may be wrong or absent in the specific given images. To address this misalignment, we further introduce a vision-conditional pooling module to extract instance-specific text features. Extensive experimental results demonstrate that our approach outperforms state-of-the-art methods on the zero-shot base-to-novel generalization, cross-dataset transfer, as well as few-shot classification across 11 diverse datasets. △ Less

Submitted 14 October, 2024; originally announced October 2024.

KNN Transformer with Pyramid Prompts for Few-Shot Learning

Authors: Wenhao Li, Qiangchang Wang, Peng Zhao, Yilong Yin

Abstract: Few-Shot Learning (FSL) aims to recognize new classes with limited labeled data. Recent studies have attempted to address the challenge of rare samples with textual prompts to modulate visual features. However, they usually struggle to capture complex semantic relationships between textual and visual features. Moreover, vanilla self-attention is heavily affected by useless information in images, s… ▽ More Few-Shot Learning (FSL) aims to recognize new classes with limited labeled data. Recent studies have attempted to address the challenge of rare samples with textual prompts to modulate visual features. However, they usually struggle to capture complex semantic relationships between textual and visual features. Moreover, vanilla self-attention is heavily affected by useless information in images, severely constraining the potential of semantic priors in FSL due to the confusion of numerous irrelevant tokens during interaction. To address these aforementioned issues, a K-NN Transformer with Pyramid Prompts (KTPP) is proposed to select discriminative information with K-NN Context Attention (KCA) and adaptively modulate visual features with Pyramid Cross-modal Prompts (PCP). First, for each token, the KCA only selects the K most relevant tokens to compute the self-attention matrix and incorporates the mean of all tokens as the context prompt to provide the global context in three cascaded stages. As a result, irrelevant tokens can be progressively suppressed. Secondly, pyramid prompts are introduced in the PCP to emphasize visual features via interactions between text-based class-aware prompts and multi-scale visual features. This allows the ViT to dynamically adjust the importance weights of visual features based on rich semantic information at different scales, making models robust to spatial variations. Finally, augmented visual features and class-aware prompts are interacted via the KCA to extract class-specific features. Consequently, our model further enhances noise-free visual representations via deep cross-modal interactions, extracting generalized visual representation in scenarios with few labeled samples. Extensive experiments on four benchmark datasets demonstrate the effectiveness of our method. △ Less

Submitted 14 October, 2024; originally announced October 2024.

Comments: 10 pages, 5 figures, accepted by ACM Multimedia 2024

ACM Class: I.4.9

Shear viscoelasticity in anisotropic holographic axion model

Authors: Lei Li, Wei-Jia Li, Xiao-Mei Kuang

Abstract: In this work, we investigate the shear viscoelasticity in a simple holographic axion model with broken translational symmetry and rotational symmetry in space via the perturbation computation. We find that, in the case of spontaneous symmetry breaking, the broken translations and anisotropy both enhance the shear elasticity of the system. While in all cases, they introduce a double suppression on… ▽ More In this work, we investigate the shear viscoelasticity in a simple holographic axion model with broken translational symmetry and rotational symmetry in space via the perturbation computation. We find that, in the case of spontaneous symmetry breaking, the broken translations and anisotropy both enhance the shear elasticity of the system. While in all cases, they introduce a double suppression on the shear viscosity, which is in contrast to the result from the study of the p-wave holographic superfluid where the shear viscosity is enhanced when the rotational symmetry is broken spontaneously. △ Less

Submitted 14 October, 2024; originally announced October 2024.

Hi-Mamba: Hierarchical Mamba for Efficient Image Super-Resolution

Authors: Junbo Qiao, Jincheng Liao, Wei Li, Yulun Zhang, Yong Guo, Yi Wen, Zhangxizi Qiu, Jiao Xie, Jie Hu, Shaohui Lin

Abstract: State Space Models (SSM), such as Mamba, have shown strong representation ability in modeling long-range dependency with linear complexity, achieving successful applications from high-level to low-level vision tasks. However, SSM's sequential nature necessitates multiple scans in different directions to compensate for the loss of spatial dependency when unfolding the image into a 1D sequence. This… ▽ More State Space Models (SSM), such as Mamba, have shown strong representation ability in modeling long-range dependency with linear complexity, achieving successful applications from high-level to low-level vision tasks. However, SSM's sequential nature necessitates multiple scans in different directions to compensate for the loss of spatial dependency when unfolding the image into a 1D sequence. This multi-direction scanning strategy significantly increases the computation overhead and is unbearable for high-resolution image processing. To address this problem, we propose a novel Hierarchical Mamba network, namely, Hi-Mamba, for image super-resolution (SR). Hi-Mamba consists of two key designs: (1) The Hierarchical Mamba Block (HMB) assembled by a Local SSM (L-SSM) and a Region SSM (R-SSM) both with the single-direction scanning, aggregates multi-scale representations to enhance the context modeling ability. (2) The Direction Alternation Hierarchical Mamba Group (DA-HMG) allocates the isomeric single-direction scanning into cascading HMBs to enrich the spatial relationship modeling. Extensive experiments demonstrate the superiority of Hi-Mamba across five benchmark datasets for efficient SR. For example, Hi-Mamba achieves a significant PSNR improvement of 0.29 dB on Manga109 for $\times3$ SR, compared to the strong lightweight MambaIR. △ Less

Submitted 14 October, 2024; originally announced October 2024.

Towards Defining an Efficient and Expandable File Format for AI-Generated Contents

Authors: Yixin Gao, Runsen Feng, Xin Li, Weiping Li, Zhibo Chen

Abstract: Recently, AI-generated content (AIGC) has gained significant traction due to its powerful creation capability. However, the storage and transmission of large amounts of high-quality AIGC images inevitably pose new challenges for recent file formats. To overcome this, we define a new file format for AIGC images, named AIGIF, enabling ultra-low bitrate coding of AIGC images. Unlike compressing AIGC… ▽ More Recently, AI-generated content (AIGC) has gained significant traction due to its powerful creation capability. However, the storage and transmission of large amounts of high-quality AIGC images inevitably pose new challenges for recent file formats. To overcome this, we define a new file format for AIGC images, named AIGIF, enabling ultra-low bitrate coding of AIGC images. Unlike compressing AIGC images intuitively with pixel-wise space as existing file formats, AIGIF instead compresses the generation syntax. This raises a crucial question: Which generation syntax elements, e.g., text prompt, device configuration, etc, are necessary for compression/transmission? To answer this question, we systematically investigate the effects of three essential factors: platform, generative model, and data configuration. We experimentally find that a well-designed composable bitstream structure incorporating the above three factors can achieve an impressive compression ratio of even up to 1/10,000 while still ensuring high fidelity. We also introduce an expandable syntax in AIGIF to support the extension of the most advanced generation models to be developed in the future. △ Less

Submitted 15 October, 2024; v1 submitted 13 October, 2024; originally announced October 2024.

LOKI: A Comprehensive Synthetic Data Detection Benchmark using Large Multimodal Models

Authors: Junyan Ye, Baichuan Zhou, Zilong Huang, Junan Zhang, Tianyi Bai, Hengrui Kang, Jun He, Honglin Lin, Zihao Wang, Tong Wu, Zhizheng Wu, Yiping Chen, Dahua Lin, Conghui He, Weijia Li

Abstract: With the rapid development of AI-generated content, the future internet may be inundated with synthetic data, making the discrimination of authentic and credible multimodal data increasingly challenging. Synthetic data detection has thus garnered widespread attention, and the performance of large multimodal models (LMMs) in this task has attracted significant interest. LMMs can provide natural lan… ▽ More With the rapid development of AI-generated content, the future internet may be inundated with synthetic data, making the discrimination of authentic and credible multimodal data increasingly challenging. Synthetic data detection has thus garnered widespread attention, and the performance of large multimodal models (LMMs) in this task has attracted significant interest. LMMs can provide natural language explanations for their authenticity judgments, enhancing the explainability of synthetic content detection. Simultaneously, the task of distinguishing between real and synthetic data effectively tests the perception, knowledge, and reasoning capabilities of LMMs. In response, we introduce LOKI, a novel benchmark designed to evaluate the ability of LMMs to detect synthetic data across multiple modalities. LOKI encompasses video, image, 3D, text, and audio modalities, comprising 18K carefully curated questions across 26 subcategories with clear difficulty levels. The benchmark includes coarse-grained judgment and multiple-choice questions, as well as fine-grained anomaly selection and explanation tasks, allowing for a comprehensive analysis of LMMs. We evaluated 22 open-source LMMs and 6 closed-source models on LOKI, highlighting their potential as synthetic data detectors and also revealing some limitations in the development of LMM capabilities. More information about LOKI can be found at https://opendatalab.github.io/LOKI/ △ Less

Submitted 13 October, 2024; originally announced October 2024.

Comments: 79 pages, 63 figures

Phonon-Mediated Nonlinear Optical Responses and Quantum Geometry

Authors: Jiaming Hu, Wenbin Li, Hua Wang, Kai Chang

Abstract: Unraveling the complexities of nonlinear optical (NLO) responses, particularly the intricate many-body interactions among photons, electrons, and phonons, remains a significant challenge in condensed matter physics. Here, we present a diagrammatic approach to explore NLO responses with electron-phonon coupling (EPC), focusing on the phonon-mediated nonlinear optical (Ph-NLO) responses up to the se… ▽ More Unraveling the complexities of nonlinear optical (NLO) responses, particularly the intricate many-body interactions among photons, electrons, and phonons, remains a significant challenge in condensed matter physics. Here, we present a diagrammatic approach to explore NLO responses with electron-phonon coupling (EPC), focusing on the phonon-mediated nonlinear optical (Ph-NLO) responses up to the second order in photon perturbation. We systematically analyze the shift and ballistic mechanisms responsible for phonon-mediated electron-photon interactions. By incorporating EPC effects, we elucidate phenomena such as phonon-mediated shift current (Ph-SC) and second-harmonic generation (Ph-SHG) in a comprehensive Ph-NLO framework. This approach enables below-gap resonant responses at terahertz photon frequencies, offering a promising mechanism for terahertz optical applications that surpasses the constraints of conventional pure-electronic NLO theories. Additionally, we explore the geometric and topological consequences of Ph-NLO responses by introducing the EPC Berry curvature, EPC quantum metric, and EPC shift vector. These concepts unveil a unique quantum geometric structure within the Hilbert space, parameterized by both the electronic wavevector and phonon-displacement, thereby extending the established pure-electronic quantum geometry. Using a general Rice-Mele model, we demonstrate the connection between Ph-NLO responses and EPC geometry, discussing the implications and predicting observable effects for future experimental validation. This framework also provides a foundation for advancing first-principles calculations aimed at the discovery and engineering of NLO materials. The insights gained from this study contribute to a more profound understanding of NLO responses and EPC quantum geometry. △ Less

Submitted 12 October, 2024; originally announced October 2024.

Comments: 24 pages, 16 figures

Exploring Behavior-Relevant and Disentangled Neural Dynamics with Generative Diffusion Models

Authors: Yule Wang, Chengrui Li, Weihan Li, Anqi Wu

Abstract: Understanding the neural basis of behavior is a fundamental goal in neuroscience. Current research in large-scale neuro-behavioral data analysis often relies on decoding models, which quantify behavioral information in neural data but lack details on behavior encoding. This raises an intriguing scientific question: ``how can we enable in-depth exploration of neural representations in behavioral ta… ▽ More Understanding the neural basis of behavior is a fundamental goal in neuroscience. Current research in large-scale neuro-behavioral data analysis often relies on decoding models, which quantify behavioral information in neural data but lack details on behavior encoding. This raises an intriguing scientific question: ``how can we enable in-depth exploration of neural representations in behavioral tasks, revealing interpretable neural dynamics associated with behaviors''. However, addressing this issue is challenging due to the varied behavioral encoding across different brain regions and mixed selectivity at the population level. To tackle this limitation, our approach, named ``BeNeDiff'', first identifies a fine-grained and disentangled neural subspace using a behavior-informed latent variable model. It then employs state-of-the-art generative diffusion models to synthesize behavior videos that interpret the neural dynamics of each latent factor. We validate the method on multi-session datasets containing widefield calcium imaging recordings across the dorsal cortex. Through guiding the diffusion model to activate individual latent factors, we verify that the neural dynamics of latent factors in the disentangled neural subspace provide interpretable quantifications of the behaviors of interest. At the same time, the neural subspace in BeNeDiff demonstrates high disentanglement and neural reconstruction quality. △ Less

Submitted 12 October, 2024; originally announced October 2024.

Can a large language model be a gaslighter?

Authors: Wei Li, Luyao Zhu, Yang Song, Ruixi Lin, Rui Mao, Yang You

Abstract: Large language models (LLMs) have gained human trust due to their capabilities and helpfulness. However, this in turn may allow LLMs to affect users' mindsets by manipulating language. It is termed as gaslighting, a psychological effect. In this work, we aim to investigate the vulnerability of LLMs under prompt-based and fine-tuning-based gaslighting attacks. Therefore, we propose a two-stage fram… ▽ More Large language models (LLMs) have gained human trust due to their capabilities and helpfulness. However, this in turn may allow LLMs to affect users' mindsets by manipulating language. It is termed as gaslighting, a psychological effect. In this work, we aim to investigate the vulnerability of LLMs under prompt-based and fine-tuning-based gaslighting attacks. Therefore, we propose a two-stage framework DeepCoG designed to: 1) elicit gaslighting plans from LLMs with the proposed DeepGaslighting prompting template, and 2) acquire gaslighting conversations from LLMs through our Chain-of-Gaslighting method. The gaslighting conversation dataset along with a corresponding safe dataset is applied to fine-tuning-based attacks on open-source LLMs and anti-gaslighting safety alignment on these LLMs. Experiments demonstrate that both prompt-based and fine-tuning-based attacks transform three open-source LLMs into gaslighters. In contrast, we advanced three safety alignment strategies to strengthen (by 12.05%) the safety guardrail of LLMs. Our safety alignment strategies have minimal impacts on the utility of LLMs. Empirical studies indicate that an LLM may be a potential gaslighter, even if it passed the harmfulness test on general dangerous queries. △ Less

Submitted 11 October, 2024; originally announced October 2024.

Comments: 10/26 (Main Body/Total), 8 figures

Gradient-adjusted underdamped Langevin dynamics for sampling

Authors: Xinzhe Zuo, Stanley Osher, Wuchen Li

Abstract: Sampling from a target distribution is a fundamental problem. Traditional Markov chain Monte Carlo (MCMC) algorithms, such as the unadjusted Langevin algorithm (ULA), derived from the overdamped Langevin dynamics, have been extensively studied. From an optimization perspective, the Kolmogorov forward equation of the overdamped Langevin dynamics can be treated as the gradient flow of the relative e… ▽ More Sampling from a target distribution is a fundamental problem. Traditional Markov chain Monte Carlo (MCMC) algorithms, such as the unadjusted Langevin algorithm (ULA), derived from the overdamped Langevin dynamics, have been extensively studied. From an optimization perspective, the Kolmogorov forward equation of the overdamped Langevin dynamics can be treated as the gradient flow of the relative entropy in the space of probability densities embedded with Wassrstein-2 metrics. Several efforts have also been devoted to including momentum-based methods, such as underdamped Langevin dynamics for faster convergence of sampling algorithms. Recent advances in optimizations have demonstrated the effectiveness of primal-dual damping and Hessian-driven damping dynamics for achieving faster convergence in solving optimization problems. Motivated by these developments, we introduce a class of stochastic differential equations (SDEs) called gradient-adjusted underdamped Langevin dynamics (GAUL), which add stochastic perturbations in primal-dual damping dynamics and Hessian-driven damping dynamics from optimization. We prove that GAUL admits the correct stationary distribution, whose marginal is the target distribution. The proposed method outperforms overdamped and underdamped Langevin dynamics regarding convergence speed in the total variation distance for Gaussian target distributions. Moreover, using the Euler-Maruyama discretization, we show that the mixing time towards a biased target distribution only depends on the square root of the condition number of the target covariance matrix. Numerical experiments for non-Gaussian target distributions, such as Bayesian regression problems and Bayesian neural networks, further illustrate the advantages of our approach. △ Less

Submitted 11 October, 2024; originally announced October 2024.

Observation of time-dependent $CP$ violation and measurement of the branching fraction of $B^0 \to J/ψπ^0$ decays

Authors: Belle II Collaboration, I. Adachi, L. Aggarwal, H. Ahmed, H. Aihara, N. Akopov, A. Aloisio, N. Althubiti, N. Anh Ky, D. M. Asner, H. Atmacan, V. Aushev, M. Aversano, R. Ayad, V. Babu, H. Bae, N. K. Baghel, S. Bahinipati, P. Bambade, Sw. Banerjee, S. Bansal, J. Baudot, A. Baur, A. Beaubien, F. Becherer , et al. (369 additional authors not shown)

Abstract: We present a measurement of the branching fraction and time-dependent charge-parity ($CP$) decay-rate asymmetries in $B^0 \to J/ψπ^0$ decays. The data sample was collected with the Belle~II detector at the SuperKEKB asymmetric $e^+e^-$ collider in 2019-2022 and contains $(387\pm 6)\times 10^6$ $B\overline{B}$ meson pairs from $Υ(4S)$ decays. We reconstruct $392\pm 24$ signal decays and fit the… ▽ More We present a measurement of the branching fraction and time-dependent charge-parity ($CP$) decay-rate asymmetries in $B^0 \to J/ψπ^0$ decays. The data sample was collected with the Belle~II detector at the SuperKEKB asymmetric $e^+e^-$ collider in 2019-2022 and contains $(387\pm 6)\times 10^6$ $B\overline{B}$ meson pairs from $Υ(4S)$ decays. We reconstruct $392\pm 24$ signal decays and fit the $CP$ parameters from the distribution of the proper-decay-time difference of the two $B$ mesons. We measure the branching fraction to be $B(B^0 \to J/ψπ^0)=(2.02 \pm 0.12 \pm 0.10)\times 10^{-5}$ and the direct and mixing-induced $CP$ asymmetries to be $C_{CP}=0.13 \pm 0.12 \pm 0.03$ and $S_{CP}=-0.88 \pm 0.17 \pm 0.03$, respectively, where the first uncertainties are statistical and the second are systematic. We observe mixing-induced $CP$ violation with a significance of $5.0$ standard deviations for the first time in this mode. △ Less

Submitted 11 October, 2024; originally announced October 2024.

Report number: Belle II preprint: 2024-018, KEK preprint: 2024-14

Observation of $D^+\toη^\primeμ^+ν_μ$ and First Study of $D^+\to η^\prime \ell^+ν_\ell$ Decay Dynamics

Authors: BESIII Collaboration, M. Ablikim, M. N. Achasov, P. Adlarson, O. Afedulidis, X. C. Ai, R. Aliberti, A. Amoroso, Q. An, Y. Bai, O. Bakina, I. Balossino, Y. Ban, H. -R. Bao, V. Batozskaya, K. Begzsuren, N. Berger, M. Berlowski, M. Bertani, D. Bettoni, F. Bianchi, E. Bianco, A. Bortone, I. Boyko, R. A. Briere , et al. (643 additional authors not shown)

Abstract: Using $20.3\,\rm fb^{-1}$ of $e^+e^-$ collision data collected at the center-of-mass energy 3.773\,GeV with the BESIII detector, we report the first observation of the semileptonic decay $D^+\to η^\prime μ^+ν_μ$ with significance of $8.6σ$ including systematic uncertainties, and an improved measurement of $D^+\to η^\prime e^+ν_e$. The branching fractions of $D^+\to η^\prime μ^+ν_μ$ and… ▽ More Using $20.3\,\rm fb^{-1}$ of $e^+e^-$ collision data collected at the center-of-mass energy 3.773\,GeV with the BESIII detector, we report the first observation of the semileptonic decay $D^+\to η^\prime μ^+ν_μ$ with significance of $8.6σ$ including systematic uncertainties, and an improved measurement of $D^+\to η^\prime e^+ν_e$. The branching fractions of $D^+\to η^\prime μ^+ν_μ$ and $D^+\to η^\prime e^+ν_e$ are determined to be $(1.92\pm0.28_{\rm stat}\pm 0.08_{\rm syst})\times 10^{-4}$ and $(1.79\pm0.19_{\rm stat}\pm 0.07_{\rm syst})\times 10^{-4}$, respectively. From an analysis of the $D^+\to η^\prime \ell^+ν_\ell$ decay dynamics, the product of the hadronic form factor $f_+^{η^{\prime}}(0)$ and the CKM matrix element $|V_{cd}|$ is measured for the first time, giving $f^{η^\prime}_+(0)|V_{cd}| = (5.92\pm0.56_{\rm stat}\pm0.13_{\rm syst})\times 10^{-2}$. No evidence for violation of $μ-e$ lepton-flavor universality is found in both the full range and several bins of $\ell^+ν_\ell$ four-momentum transfer. The $η-η^\prime$ mixing angle in the quark flavor basis is determined to be $φ_{\rm P} =(39.8\pm0.8_{\rm stat}\pm0.3_{\rm syst})^\circ$. △ Less

Submitted 11 October, 2024; originally announced October 2024.

Integrated adaptive coherent LiDAR for 4D bionic vision

Authors: Ruixuan Chen, Yichen Wu, Ke Zhang, Chuxin Liu, Yikun Chen, Wencan Li, Bitao Shen, Zhaoxi Chen, Hanke Feng, Zhangfeng Ge, Yan Zhou, Zihan Tao, Weihan Xu, Yimeng Wang, Pengfei Cai, Dong Pan, Haowen Shu, Linjie Zhou, Cheng Wang, Xingjun Wang

Abstract: Light detection and ranging (LiDAR) is a ubiquitous tool to provide precise spatial awareness in various perception environments. A bionic LiDAR that can mimic human-like vision by adaptively gazing at selected regions of interest within a broad field of view is crucial to achieve high-resolution imaging in an energy-saving and cost-effective manner. However, current LiDARs based on stacking fixed… ▽ More Light detection and ranging (LiDAR) is a ubiquitous tool to provide precise spatial awareness in various perception environments. A bionic LiDAR that can mimic human-like vision by adaptively gazing at selected regions of interest within a broad field of view is crucial to achieve high-resolution imaging in an energy-saving and cost-effective manner. However, current LiDARs based on stacking fixed-wavelength laser arrays and inertial scanning have not been able to achieve the desired dynamic focusing patterns and agile scalability simultaneously. Moreover, the ability to synchronously acquire multi-dimensional physical parameters, including distance, direction, Doppler, and color, through seamless fusion between multiple sensors, still remains elusive in LiDAR. Here, we overcome these limitations and demonstrate a bio-inspired frequency-modulated continuous wave (FMCW) LiDAR system with dynamic and scalable gazing capability. Our chip-scale LiDAR system is built using hybrid integrated photonic solutions, where a frequency-chirped external cavity laser provides broad spectral tunability, while on-chip electro-optic combs with elastic channel spacing allow customizable imaging granularity. Using the dynamic zoom-in capability and the coherent FMCW scheme, we achieve a state-of-the-art resolution of 0.012 degrees, providing up to 15 times the resolution of conventional 3D LiDAR sensors, with 115 equivalent scanning lines and 4D parallel imaging. We further demonstrate cooperative sensing between our adaptive coherent LiDAR and a camera to enable high-resolution color-enhanced machine vision. △ Less

Submitted 11 October, 2024; originally announced October 2024.

Neural Material Adaptor for Visual Grounding of Intrinsic Dynamics

Authors: Junyi Cao, Shanyan Guan, Yanhao Ge, Wei Li, Xiaokang Yang, Chao Ma

Abstract: While humans effortlessly discern intrinsic dynamics and adapt to new scenarios, modern AI systems often struggle. Current methods for visual grounding of dynamics either use pure neural-network-based simulators (black box), which may violate physical laws, or traditional physical simulators (white box), which rely on expert-defined equations that may not fully capture actual dynamics. We propose… ▽ More While humans effortlessly discern intrinsic dynamics and adapt to new scenarios, modern AI systems often struggle. Current methods for visual grounding of dynamics either use pure neural-network-based simulators (black box), which may violate physical laws, or traditional physical simulators (white box), which rely on expert-defined equations that may not fully capture actual dynamics. We propose the Neural Material Adaptor (NeuMA), which integrates existing physical laws with learned corrections, facilitating accurate learning of actual dynamics while maintaining the generalizability and interpretability of physical priors. Additionally, we propose Particle-GS, a particle-driven 3D Gaussian Splatting variant that bridges simulation and observed images, allowing back-propagate image gradients to optimize the simulator. Comprehensive experiments on various dynamics in terms of grounded particle accuracy, dynamic rendering quality, and generalization ability demonstrate that NeuMA can accurately capture intrinsic dynamics. △ Less

Submitted 10 October, 2024; originally announced October 2024.

Comments: NeurIPS 2024, the project page: https://xjay18.github.io/projects/neuma.html

HybridBooth: Hybrid Prompt Inversion for Efficient Subject-Driven Generation

Authors: Shanyan Guan, Yanhao Ge, Ying Tai, Jian Yang, Wei Li, Mingyu You

Abstract: Recent advancements in text-to-image diffusion models have shown remarkable creative capabilities with textual prompts, but generating personalized instances based on specific subjects, known as subject-driven generation, remains challenging. To tackle this issue, we present a new hybrid framework called HybridBooth, which merges the benefits of optimization-based and direct-regression methods. Hy… ▽ More Recent advancements in text-to-image diffusion models have shown remarkable creative capabilities with textual prompts, but generating personalized instances based on specific subjects, known as subject-driven generation, remains challenging. To tackle this issue, we present a new hybrid framework called HybridBooth, which merges the benefits of optimization-based and direct-regression methods. HybridBooth operates in two stages: the Word Embedding Probe, which generates a robust initial word embedding using a fine-tuned encoder, and the Word Embedding Refinement, which further adapts the encoder to specific subject images by optimizing key parameters. This approach allows for effective and fast inversion of visual concepts into textual embedding, even from a single image, while maintaining the model's generalization capabilities. △ Less

Submitted 10 October, 2024; originally announced October 2024.

Comments: ECCV 2024, the project page: https://sites.google.com/view/hybridbooth

Autonomous Driving in Unstructured Environments: How Far Have We Come?

Authors: Chen Min, Shubin Si, Xu Wang, Hanzhang Xue, Weizhong Jiang, Yang Liu, Juan Wang, Qingtian Zhu, Qi Zhu, Lun Luo, Fanjie Kong, Jinyu Miao, Xudong Cai, Shuai An, Wei Li, Jilin Mei, Tong Sun, Heng Zhai, Qifeng Liu, Fangzhou Zhao, Liang Chen, Shuai Wang, Erke Shang, Linzhi Shang, Kunlong Zhao , et al. (13 additional authors not shown)

Abstract: Research on autonomous driving in unstructured outdoor environments is less advanced than in structured urban settings due to challenges like environmental diversities and scene complexity. These environments-such as rural areas and rugged terrains-pose unique obstacles that are not common in structured urban areas. Despite these difficulties, autonomous driving in unstructured outdoor environment… ▽ More Research on autonomous driving in unstructured outdoor environments is less advanced than in structured urban settings due to challenges like environmental diversities and scene complexity. These environments-such as rural areas and rugged terrains-pose unique obstacles that are not common in structured urban areas. Despite these difficulties, autonomous driving in unstructured outdoor environments is crucial for applications in agriculture, mining, and military operations. Our survey reviews over 250 papers for autonomous driving in unstructured outdoor environments, covering offline mapping, pose estimation, environmental perception, path planning, end-to-end autonomous driving, datasets, and relevant challenges. We also discuss emerging trends and future research directions. This review aims to consolidate knowledge and encourage further research for autonomous driving in unstructured environments. To support ongoing work, we maintain an active repository with up-to-date literature and open-source projects at: https://github.com/chaytonmin/Survey-Autonomous-Driving-in-Unstructured-Environments. △ Less

Submitted 12 October, 2024; v1 submitted 10 October, 2024; originally announced October 2024.

Comments: Survey paper; 38 pages

Adversarial Robustness Overestimation and Instability in TRADES

Authors: Jonathan Weiping Li, Ren-Wei Liang, Cheng-Han Yeh, Cheng-Chang Tsai, Kuanchun Yu, Chun-Shien Lu, Shang-Tse Chen

Abstract: This paper examines the phenomenon of probabilistic robustness overestimation in TRADES, a prominent adversarial training method. Our study reveals that TRADES sometimes yields disproportionately high PGD validation accuracy compared to the AutoAttack testing accuracy in the multiclass classification task. This discrepancy highlights a significant overestimation of robustness for these instances,… ▽ More This paper examines the phenomenon of probabilistic robustness overestimation in TRADES, a prominent adversarial training method. Our study reveals that TRADES sometimes yields disproportionately high PGD validation accuracy compared to the AutoAttack testing accuracy in the multiclass classification task. This discrepancy highlights a significant overestimation of robustness for these instances, potentially linked to gradient masking. We further analyze the parameters contributing to unstable models that lead to overestimation. Our findings indicate that smaller batch sizes, lower beta values (which control the weight of the robust loss term in TRADES), larger learning rates, and higher class complexity (e.g., CIFAR-100 versus CIFAR-10) are associated with an increased likelihood of robustness overestimation. By examining metrics such as the First-Order Stationary Condition (FOSC), inner-maximization, and gradient information, we identify the underlying cause of this phenomenon as gradient masking and provide insights into it. Furthermore, our experiments show that certain unstable training instances may return to a state without robust overestimation, inspiring our attempts at a solution. In addition to adjusting parameter settings to reduce instability or retraining when overestimation occurs, we recommend incorporating Gaussian noise in inputs when the FOSC score exceed the threshold. This method aims to mitigate robustness overestimation of TRADES and other similar methods at its source, ensuring more reliable representation of adversarial robustness during evaluation. △ Less

Submitted 10 October, 2024; originally announced October 2024.

Precision Measurement of the Branching Fraction of $D^{+}\to μ^{+}ν_μ$

Authors: BESIII Collaboration, M. Ablikim, M. N. Achasov, P. Adlarson, O. Afedulidis, X. C. Ai, R. Aliberti, A. Amoroso, Q. An, Y. Bai, O. Bakina, I. Balossino, Y. Ban, H. -R. Bao, V. Batozskaya, K. Begzsuren, N. Berger, M. Berlowski, M. Bertani, D. Bettoni, F. Bianchi, E. Bianco, A. Bortone, I. Boyko, R. A. Briere , et al. (643 additional authors not shown)

Abstract: Using $20.3~\mathrm{fb}^{-1}$ of $e^+e^-$ collision data collected at a center-of-mass energy of $E_{\rm cm}=3.773$ GeV with the BESIII detector operating at the BEPCII collider, we determine the branching fraction of the leptonic decay $D^+\toμ^+ν_μ$ to be $(3.981\pm0.079_{\rm stat}\pm0.040_{\rm syst})\times10^{-4}$. Interpreting our measurement with knowledge of the Fermi coupling constant… ▽ More Using $20.3~\mathrm{fb}^{-1}$ of $e^+e^-$ collision data collected at a center-of-mass energy of $E_{\rm cm}=3.773$ GeV with the BESIII detector operating at the BEPCII collider, we determine the branching fraction of the leptonic decay $D^+\toμ^+ν_μ$ to be $(3.981\pm0.079_{\rm stat}\pm0.040_{\rm syst})\times10^{-4}$. Interpreting our measurement with knowledge of the Fermi coupling constant $G_F$, the masses of the $D^+$ and $μ^+$ as well as the lifetime of the $D^+$, we determine $f_{D^+}|V_{cd}|=(47.53\pm0.48_{\rm stat}\pm0.24_{\rm syst}\pm0.12_{\rm input})~\mathrm{MeV}$. This result is a factor of 2.3 more precise than the previous best measurement. Using the value of the magnitude of the Cabibbo-Kobayashi-Maskawa matrix element $|V_{cd}|$ given by the global standard model fit, we obtain the $D^+$ decay constant $f_{D^+}=(211.5\pm2.3_{\rm stat}\pm1.1_{\rm syst}\pm0.8_{\rm input})$ MeV. Alternatively, using the value of $f_{D^+}$ from a precise lattice quantum chromodynamics calculation, we extract $|V_{cd}|=0.2242\pm0.0023_{\rm stat}\pm0.0011_{\rm syst}\pm0.0009_{\rm input}$. △ Less

Submitted 10 October, 2024; originally announced October 2024.

Comments: 9 pages, 2 figures

SWIFT: On-the-Fly Self-Speculative Decoding for LLM Inference Acceleration

Authors: Heming Xia, Yongqi Li, Jun Zhang, Cunxiao Du, Wenjie Li

Abstract: Speculative decoding (SD) has emerged as a widely used paradigm to accelerate the inference of large language models (LLMs) without compromising generation quality. It works by first employing a compact model to draft multiple tokens efficiently and then using the target LLM to verify them in parallel. While this technique has achieved notable speedups, most existing approaches necessitate either… ▽ More Speculative decoding (SD) has emerged as a widely used paradigm to accelerate the inference of large language models (LLMs) without compromising generation quality. It works by first employing a compact model to draft multiple tokens efficiently and then using the target LLM to verify them in parallel. While this technique has achieved notable speedups, most existing approaches necessitate either additional parameters or extensive training to construct effective draft models, thereby restricting their applicability across different LLMs and tasks. To address this limitation, we explore a novel plug-and-play SD solution with layer-skipping, which skips intermediate layers of the target LLM as the compact draft model. Our analysis reveals that LLMs exhibit great potential for self-acceleration through layer sparsity and the task-specific nature of this sparsity. Building on these insights, we introduce SWIFT, an on-the-fly self-speculative decoding algorithm that adaptively selects intermediate layers of LLMs to skip during inference. SWIFT does not require auxiliary models or additional training, making it a plug-and-play solution for accelerating LLM inference across diverse input data streams. Our extensive experiments across a wide range of models and downstream tasks demonstrate that SWIFT can achieve over a 1.3x-1.6x speedup while preserving the original distribution of the generated text. △ Less

Submitted 9 October, 2024; originally announced October 2024.

The global phase diagram of the cluster-XY spin chain with dissipation

Authors: Wei-Lin Li, Ying-Ao Chen, Zheng-Xin Guo, Xue-Jia Yu, Zhi Li

Abstract: We study the ground-state phase diagram of a non-Hermitian cluster-XY spin chain in the language of free fermions. By calculating the second derivative of ground-state energy density and various types of order parameters, we establish the global ground-state phase diagram of the model, exhibiting rich quantum phases and corresponding phase transitions. Specially, the results reveal that the non-He… ▽ More We study the ground-state phase diagram of a non-Hermitian cluster-XY spin chain in the language of free fermions. By calculating the second derivative of ground-state energy density and various types of order parameters, we establish the global ground-state phase diagram of the model, exhibiting rich quantum phases and corresponding phase transitions. Specially, the results reveal that the non-Hermitian cluster-XY model contains five different phases and two critical regions, i.e., ferromagnetic (FM), antiferromagnetic (AFM), symmetry-protected topological (SPT), paramagnetic (PM), Luttinger liquid-like phase, as well as critical region I and II. The order parameters and critical behaviors are investigated and the correctness of the theory is confirmed. △ Less

Submitted 9 October, 2024; originally announced October 2024.

Optical and near-infrared photometry of 94 type II supernovae from the Carnegie Supernova Project

Authors: J. P. Anderson, C. Contreras, M. D. Stritzinger, M. Hamuy, M. M. Phillips, N. B. Suntzeff, N. Morrell, S. Gonzalez-Gaitan, C. P. Gutierrez, C. R. Burns, E. Y. Hsiao, J. Anais, C. Ashall, C. Baltay, E. Baron, M. Bersten, L. Busta, S. Castellon, T. de Jaeger, D. DePoy, A. V. Filippenko, G. Folatelli, F. Forster, L. Galbany, C. Gall , et al. (21 additional authors not shown)

Abstract: Type II supernovae (SNeII) mark the endpoint in the lives of hydrogen-rich massive stars. Their large explosion energies and luminosities allow us to measure distances, metallicities, and star formation rates into the distant Universe. To fully exploit their use in answering different astrophysical problems, high-quality low-redshift data sets are required. Such samples are vital to understand the… ▽ More Type II supernovae (SNeII) mark the endpoint in the lives of hydrogen-rich massive stars. Their large explosion energies and luminosities allow us to measure distances, metallicities, and star formation rates into the distant Universe. To fully exploit their use in answering different astrophysical problems, high-quality low-redshift data sets are required. Such samples are vital to understand the physics of SNeII, but also to serve as calibrators for distinct - and often lower-quality - samples. We present uBgVri optical and YJH near-infrared (NIR) photometry for 94 low-redshift SNeII observed by the Carnegie Supernova Project (CSP). A total of 9817 optical and 1872 NIR photometric data points are released, leading to a sample of high-quality SNII light curves during the first ~150 days post explosion on a well-calibrated photometric system. The sample is presented and its properties are analysed and discussed through comparison to literature events. We also focus on individual SNeII as examples of classically defined subtypes and outlier objects. Making a cut in the plateau decline rate of our sample (s2), a new subsample of fast-declining SNeII is presented. The sample has a median redshift of 0.015, with the nearest event at 0.001 and the most distant at 0.07. At optical wavelengths (V), the sample has a median cadence of 4.7 days over the course of a median coverage of 80 days. In the NIR (J), the median cadence is 7.2 days over the course of 59 days. The fast-declining subsample is more luminous than the full sample and shows shorter plateau phases. Of the non-standard SNeII highlighted, SN2009A particularly stands out with a steeply declining then rising light curve, together with what appears to be two superimposed P-Cygni profiles of H-alpha in its spectra. We outline the significant utility of these data, and finally provide an outlook of future SNII science. △ Less

Submitted 9 October, 2024; originally announced October 2024.

Comments: Accepted for publication in A&A. Photometric data will be uploaded to the CDS and the CSP website, and can also be requested from the first author

Enhancing Multimodal LLM for Detailed and Accurate Video Captioning using Multi-Round Preference Optimization

Authors: Changli Tang, Yixuan Li, Yudong Yang, Jimin Zhuang, Guangzhi Sun, Wei Li, Zujun Ma, Chao Zhang

Abstract: Videos contain a wealth of information, and generating detailed and accurate descriptions in natural language is a key aspect of video understanding. In this paper, we present video-SALMONN 2, an advanced audio-visual large language model (LLM) with low-rank adaptation (LoRA) designed for enhanced video (with paired audio) captioning through directed preference optimization (DPO). We propose new m… ▽ More Videos contain a wealth of information, and generating detailed and accurate descriptions in natural language is a key aspect of video understanding. In this paper, we present video-SALMONN 2, an advanced audio-visual large language model (LLM) with low-rank adaptation (LoRA) designed for enhanced video (with paired audio) captioning through directed preference optimization (DPO). We propose new metrics to evaluate the completeness and accuracy of video descriptions, which are optimized using DPO. To further improve training, we introduce a novel multi-round DPO (mrDPO) approach, which involves periodically updating the DPO reference model, merging and re-initializing the LoRA module as a proxy for parameter updates after each training round (1,000 steps), and incorporating guidance from ground-truth video captions to stabilize the process. To address potential catastrophic forgetting of non-captioning abilities due to mrDPO, we propose rebirth tuning, which finetunes the pre-DPO LLM by using the captions generated by the mrDPO-trained model as supervised labels. Experiments show that mrDPO significantly enhances video-SALMONN 2's captioning accuracy, reducing global and local error rates by 40\% and 20\%, respectively, while decreasing the repetition rate by 35\%. The final video-SALMONN 2 model, with just 7 billion parameters, surpasses leading models such as GPT-4o and Gemini-1.5-Pro in video captioning tasks, while maintaining competitive performance to the state-of-the-art on widely used video question-answering benchmark among models of similar size. Upon acceptance, we will release the code, model checkpoints, and training and test data. Demos are available at \href{https://video-salmonn-2.github.io}{https://video-salmonn-2.github.io}. △ Less

Submitted 10 October, 2024; v1 submitted 9 October, 2024; originally announced October 2024.

Large Language Models as Code Executors: An Exploratory Study

Authors: Chenyang Lyu, Lecheng Yan, Rui Xing, Wenxi Li, Younes Samih, Tianbo Ji, Longyue Wang

Abstract: The capabilities of Large Language Models (LLMs) have significantly evolved, extending from natural language processing to complex tasks like code understanding and generation. We expand the scope of LLMs' capabilities to a broader context, using LLMs to execute code snippets to obtain the output. This paper pioneers the exploration of LLMs as code executors, where code snippets are directly fed t… ▽ More The capabilities of Large Language Models (LLMs) have significantly evolved, extending from natural language processing to complex tasks like code understanding and generation. We expand the scope of LLMs' capabilities to a broader context, using LLMs to execute code snippets to obtain the output. This paper pioneers the exploration of LLMs as code executors, where code snippets are directly fed to the models for execution, and outputs are returned. We are the first to comprehensively examine this feasibility across various LLMs, including OpenAI's o1, GPT-4o, GPT-3.5, DeepSeek, and Qwen-Coder. Notably, the o1 model achieved over 90% accuracy in code execution, while others demonstrated lower accuracy levels. Furthermore, we introduce an Iterative Instruction Prompting (IIP) technique that processes code snippets line by line, enhancing the accuracy of weaker models by an average of 7.22% (with the highest improvement of 18.96%) and an absolute average improvement of 3.86% against CoT prompting (with the highest improvement of 19.46%). Our study not only highlights the transformative potential of LLMs in coding but also lays the groundwork for future advancements in automated programming and the completion of complex tasks. △ Less

Submitted 10 October, 2024; v1 submitted 9 October, 2024; originally announced October 2024.

Subtle Errors Matter: Preference Learning via Error-injected Self-editing

Authors: Kaishuai Xu, Tiezheng Yu, Wenjun Hou, Yi Cheng, Chak Tou Leong, Liangyou Li, Xin Jiang, Lifeng Shang, Qun Liu, Wenjie Li

Abstract: Large Language Models (LLMs) have exhibited strong mathematical reasoning and computational prowess, tackling tasks ranging from basic arithmetic to advanced competition-level problems. However, frequently occurring subtle errors, such as miscalculations or incorrect substitutions, limit the models' full mathematical potential. Existing studies to improve mathematical ability typically involve dis… ▽ More Large Language Models (LLMs) have exhibited strong mathematical reasoning and computational prowess, tackling tasks ranging from basic arithmetic to advanced competition-level problems. However, frequently occurring subtle errors, such as miscalculations or incorrect substitutions, limit the models' full mathematical potential. Existing studies to improve mathematical ability typically involve distilling reasoning skills from stronger LLMs or applying preference learning to step-wise response pairs. Although these methods leverage samples of varying granularity to mitigate reasoning errors, they overlook the frequently occurring subtle errors. A major reason is that sampled preference pairs involve differences unrelated to the errors, which may distract the model from focusing on subtle errors. In this work, we propose a novel preference learning framework called eRror-Injected Self-Editing (RISE), which injects predefined subtle errors into partial tokens of correct solutions to construct hard pairs for error mitigation. In detail, RISE uses the model itself to edit a small number of tokens in the solution, injecting designed subtle errors. Then, pairs composed of self-edited solutions and their corresponding correct ones, along with pairs of correct and incorrect solutions obtained through sampling, are used together for subtle error-aware DPO training. Compared with other preference learning methods, RISE further refines the training objective to focus on predefined errors and their tokens, without requiring fine-grained sampling or preference annotation. Extensive experiments validate the effectiveness of RISE, with preference learning on Qwen2-7B-Instruct yielding notable improvements of 3.0% on GSM8K and 7.9% on MATH. △ Less

Submitted 9 October, 2024; originally announced October 2024.

Bots can Snoop: Uncovering and Mitigating Privacy Risks of Bots in Group Chats

Authors: Kai-Hsiang Chou, Yi-Min Lin, Yi-An Wang, Jonathan Weiping Li, Tiffany Hyun-Jin Kim, Hsu-Chun Hsiao

Abstract: New privacy concerns arise with chatbots on group messaging platforms. Chatbots may access information beyond their intended functionalities, such as messages unintended for chatbots or sender's identities. Chatbot operators may exploit such information to infer personal information and link users across groups, potentially leading to personal data breaches, pervasive tracking, and targeted advert… ▽ More New privacy concerns arise with chatbots on group messaging platforms. Chatbots may access information beyond their intended functionalities, such as messages unintended for chatbots or sender's identities. Chatbot operators may exploit such information to infer personal information and link users across groups, potentially leading to personal data breaches, pervasive tracking, and targeted advertising. Our analysis of conversation datasets shows that (1) chatbots often access far more messages than needed, and (2) when a user joins a new group with chatbots, there is a 3.4% chance that at least one of the chatbots can recognize and associate the user with their previous interactions in other groups. Although state-of-the-art group messaging protocols provide robust end-to-end security and some platforms have implemented policies to limit chatbot access, no platforms successfully combine these features. This paper introduces SnoopGuard, a secure group messaging protocol that ensures user privacy against chatbots while maintaining strong end-to-end security. Our method offers selective message access, preventing chatbots from accessing unrelated messages, and ensures sender anonymity within the group. SnoopGuard achieves $O(\log n + m)$ message-sending complexity for a group of $n$ users and $m$ chatbots, compared to $O(\log(n + m))$ in state-of-the-art protocols, with acceptable overhead for enhanced privacy. Our prototype implementation shows that sending a message in a group of 50 users and 10 chatbots takes about 30 milliseconds when integrated with Message Layer Security (MLS). △ Less

Submitted 9 October, 2024; originally announced October 2024.

Comments: 18 pages, 5 figures