-
The Impact of Scanner Domain Shift on Deep Learning Performance in Medical Imaging: an Experimental Study
Authors:
Brian Guo,
Darui Lu,
Gregory Szumel,
Rongze Gui,
Tingyu Wang,
Nicholas Konz,
Maciej A. Mazurowski
Abstract:
Purpose: Medical images acquired using different scanners and protocols can differ substantially in their appearance. This phenomenon, scanner domain shift, can result in a drop in the performance of deep neural networks which are trained on data acquired by one scanner and tested on another. This significant practical issue is well-acknowledged, however, no systematic study of the issue is availa…
▽ More
Purpose: Medical images acquired using different scanners and protocols can differ substantially in their appearance. This phenomenon, scanner domain shift, can result in a drop in the performance of deep neural networks which are trained on data acquired by one scanner and tested on another. This significant practical issue is well-acknowledged, however, no systematic study of the issue is available across different modalities and diagnostic tasks. Materials and Methods: In this paper, we present a broad experimental study evaluating the impact of scanner domain shift on convolutional neural network performance for different automated diagnostic tasks. We evaluate this phenomenon in common radiological modalities, including X-ray, CT, and MRI. Results: We find that network performance on data from a different scanner is almost always worse than on same-scanner data, and we quantify the degree of performance drop across different datasets. Notably, we find that this drop is most severe for MRI, moderate for X-ray, and quite small for CT, on average, which we attribute to the standardized nature of CT acquisition systems which is not present in MRI or X-ray. We also study how injecting varying amounts of target domain data into the training set, as well as adding noise to the training data, helps with generalization. Conclusion: Our results provide extensive experimental evidence and quantification of the extent of performance drop caused by scanner domain shift in deep learning across different modalities, with the goal of guiding the future development of robust deep learning models for medical image analysis.
△ Less
Submitted 2 October, 2024; v1 submitted 6 September, 2024;
originally announced September 2024.
-
Advancing Gamma-Ray Burst Identification through Transfer Learning with Convolutional Neural Networks
Authors:
Peng Zhang,
Bing Li,
Ren-zhou Gui,
Shao-lin Xiong,
Yu Wang,
Yan-qiu Zhang,
Chen-wei Wang,
Jia-cong Liu,
Wang-chen Xue,
Chao Zheng,
Zheng-hang Yu,
Wen-long Zhang
Abstract:
The Rapid and accurate identification of Gamma-Ray Bursts (GRBs) is crucial for unraveling their origins. However, current burst search algorithms frequently miss low-threshold signals or lack universality for observations. In this study, we propose a novel approach utilizing transfer learning experiment based on convolutional neural network (CNN) to establish a universal GRB identification method…
▽ More
The Rapid and accurate identification of Gamma-Ray Bursts (GRBs) is crucial for unraveling their origins. However, current burst search algorithms frequently miss low-threshold signals or lack universality for observations. In this study, we propose a novel approach utilizing transfer learning experiment based on convolutional neural network (CNN) to establish a universal GRB identification method, which validated successfully using GECAM-B data. By employing data augmentation techniques, we enhance the diversity and quantity of the GRB sample. We develop a 1D CNN model with a multi-scale feature cross fusion module (MSCFM) to extract features from samples and perform classification. The comparative results demonstrated significant performance improvements following pre-training and transferring on a large-scale dataset. Our optimal model achieved an impressive accuracy of 96.41% on the source dataset of GECAM-B, and identified three previously undiscovered GRBs by contrast with manual analysis of GECAM-B observations. These innovative transfer learning and data augmentation methods presented in this work hold promise for applications in multi-satellite exploration scenarios characterized by limited data sets and a scarcity of labeled samples in high-energy astronomy.
△ Less
Submitted 24 August, 2024;
originally announced August 2024.
-
CRAG -- Comprehensive RAG Benchmark
Authors:
Xiao Yang,
Kai Sun,
Hao Xin,
Yushi Sun,
Nikita Bhalla,
Xiangsen Chen,
Sajal Choudhary,
Rongze Daniel Gui,
Ziran Will Jiang,
Ziyu Jiang,
Lingkun Kong,
Brian Moran,
Jiaqi Wang,
Yifan Ethan Xu,
An Yan,
Chenyu Yang,
Eting Yuan,
Hanwen Zha,
Nan Tang,
Lei Chen,
Nicolas Scheffer,
Yue Liu,
Nirav Shah,
Rakesh Wanga,
Anuj Kumar
, et al. (2 additional authors not shown)
Abstract:
Retrieval-Augmented Generation (RAG) has recently emerged as a promising solution to alleviate Large Language Model (LLM)'s deficiency in lack of knowledge. Existing RAG datasets, however, do not adequately represent the diverse and dynamic nature of real-world Question Answering (QA) tasks. To bridge this gap, we introduce the Comprehensive RAG Benchmark (CRAG), a factual question answering bench…
▽ More
Retrieval-Augmented Generation (RAG) has recently emerged as a promising solution to alleviate Large Language Model (LLM)'s deficiency in lack of knowledge. Existing RAG datasets, however, do not adequately represent the diverse and dynamic nature of real-world Question Answering (QA) tasks. To bridge this gap, we introduce the Comprehensive RAG Benchmark (CRAG), a factual question answering benchmark of 4,409 question-answer pairs and mock APIs to simulate web and Knowledge Graph (KG) search. CRAG is designed to encapsulate a diverse array of questions across five domains and eight question categories, reflecting varied entity popularity from popular to long-tail, and temporal dynamisms ranging from years to seconds. Our evaluation on this benchmark highlights the gap to fully trustworthy QA. Whereas most advanced LLMs achieve <=34% accuracy on CRAG, adding RAG in a straightforward manner improves the accuracy only to 44%. State-of-the-art industry RAG solutions only answer 63% questions without any hallucination. CRAG also reveals much lower accuracy in answering questions regarding facts with higher dynamism, lower popularity, or higher complexity, suggesting future research directions. The CRAG benchmark laid the groundwork for a KDD Cup 2024 challenge, attracting thousands of participants and submissions within the first 50 days of the competition. We commit to maintaining CRAG to serve research communities in advancing RAG solutions and general QA solutions.
△ Less
Submitted 7 June, 2024;
originally announced June 2024.
-
Leveraging Diversity in Online Interactions
Authors:
Nardine Osman,
Bruno Rosell i Gui,
Carles Sierra
Abstract:
This paper addresses the issue of connecting people online to help them find support with their day-to-day problems. We make use of declarative norms for mediating online interactions, and we specifically focus on the issue of leveraging diversity when connecting people. We run pilots at different university sites, and the results show relative success in the diversity of the selected profiles, ba…
▽ More
This paper addresses the issue of connecting people online to help them find support with their day-to-day problems. We make use of declarative norms for mediating online interactions, and we specifically focus on the issue of leveraging diversity when connecting people. We run pilots at different university sites, and the results show relative success in the diversity of the selected profiles, backed by high user satisfaction.
△ Less
Submitted 20 September, 2023;
originally announced September 2023.
-
Application of Deep Learning Methods for Distinguishing Gamma-Ray Bursts from Fermi/GBM TTE Data
Authors:
Peng Zhang,
Bing Li,
RenZhou Gui,
Shaolin Xiong,
Ze-Cheng Zou,
Xianggao Wang,
Xiaobo Li,
Ce Cai,
Yi Zhao,
Yanqiu Zhang,
Wangchen Xue,
Chao Zheng,
Hongyu Zhao
Abstract:
To investigate GRBs in depth, it is crucial to develop an effective method for identifying GRBs accurately. Current criteria, e.g., onboard blind search, ground blind search, and target search, are limited by manually set thresholds and perhaps miss GRBs, especially for sub-threshold events. We propose a novel approach that utilizes convolutional neural networks (CNNs) to distinguish GRBs and non-…
▽ More
To investigate GRBs in depth, it is crucial to develop an effective method for identifying GRBs accurately. Current criteria, e.g., onboard blind search, ground blind search, and target search, are limited by manually set thresholds and perhaps miss GRBs, especially for sub-threshold events. We propose a novel approach that utilizes convolutional neural networks (CNNs) to distinguish GRBs and non-GRBs directly. We structured three CNN models, plain-CNN, ResNet, and ResNet-CBAM, and endeavored to exercise fusing strategy models. Count maps of NaI detectors onboard Fermi/GBM were employed as the input samples of datasets and models were implemented to evaluate their performance on different time scale data. The ResNet-CBAM model trained on 64 ms dataset achieves high accuracy overall, which includes residual and attention mechanism modules. The visualization methods of Grad-CAM and t-SNE explicitly displayed that the optimal model focuses on the key features of GRBs precisely. The model was applied to analyze one-year data, accurately identifying approximately 98% of GRBs listed in the Fermi burst catalog, 8 out of 9 sub-threshold GRBs, and 5 GRBs triggered by other satellites, which demonstrated the deep learning methods could effectively distinguish GRBs from observational data. Besides, thousands of unknown candidates were retrieved and compared with the bursts of SGR J1935+2154 for instance, which exemplified the potential scientific value of these candidates indeed. Detailed studies on integrating our model into real-time analysis pipelines thus may improve their accuracy of inspection, and provide valuable guidance for rapid follow-up observations of multi-band telescopes.
△ Less
Submitted 11 March, 2024; v1 submitted 1 March, 2023;
originally announced March 2023.
-
Categorical Difference and Related Brain Regions of the Attentional Blink Effect
Authors:
Renzhou Gui,
Xiaohong Ji
Abstract:
Attentional blink (AB) is a biological effect, showing that for 200 to 500ms after paying attention to one visual target, it is difficult to notice another target that appears next, and attentional blink magnitude (ABM) is a indicating parameter to measure the degree of this effect. Researchers have shown that different categories of images can access the consciousness of human mind differently, a…
▽ More
Attentional blink (AB) is a biological effect, showing that for 200 to 500ms after paying attention to one visual target, it is difficult to notice another target that appears next, and attentional blink magnitude (ABM) is a indicating parameter to measure the degree of this effect. Researchers have shown that different categories of images can access the consciousness of human mind differently, and produce different ranges of ABM values. So in this paper, we compare two different types of images, categorized as animal and object, by predicting ABM values directly from image features extracted from convolutional neural network (CNN), and indirectly from functional magnetic resonance imaging (fMRI) data. First, for two sets of images, we separately extract their average features from layers of Alexnet, a classic model of CNN, then input the features into a trained linear regression model to predict ABM values, and we find higher-level instead of lower-level image features determine the categorical difference in AB effect, and mid-level image features predict ABM values more correctly than low-level and high-level image features. Then we employ fMRI data from different brain regions collected when the subjects viewed 50 test images to predict ABM values, and conclude that brain regions covering relatively broader areas, like LVC, HVC and VC, perform better than other smaller brain regions, which means AB effect is more related to synthetic impact of several visual brain regions than only one particular visual regions.
△ Less
Submitted 3 November, 2021;
originally announced November 2021.
-
Transit-Gym: A Simulation and Evaluation Engine for Analysis of Bus Transit Systems
Authors:
Ruixiao Sun,
Rongze Gui,
Himanshu Neema,
Yuche Chen,
Juliette Ugirumurera,
Joseph Severino,
Philip Pugliese,
Aron Laszka,
Abhishek Dubey
Abstract:
Public-transit systems face a number of operational challenges: (a) changing ridership patterns requiring optimization of fixed line services, (b) optimizing vehicle-to-trip assignments to reduce maintenance and operation codes, and (c) ensuring equitable and fair coverage to areas with low ridership. Optimizing these objectives presents a hard computational problem due to the size and complexity…
▽ More
Public-transit systems face a number of operational challenges: (a) changing ridership patterns requiring optimization of fixed line services, (b) optimizing vehicle-to-trip assignments to reduce maintenance and operation codes, and (c) ensuring equitable and fair coverage to areas with low ridership. Optimizing these objectives presents a hard computational problem due to the size and complexity of the decision space. State-of-the-art methods formulate these problems as variants of the vehicle routing problem and use data-driven heuristics for optimizing the procedures. However, the evaluation and training of these algorithms require large datasets that provide realistic coverage of various operational uncertainties. This paper presents a dynamic simulation platform, called Transit-Gym, that can bridge this gap by providing the ability to simulate scenarios, focusing on variation of demand models, variations of route networks, and variations of vehicle-to-trip assignments. The central contribution of this work is a domain-specific language and associated experimentation tool-chain and infrastructure to enable subject-matter experts to intuitively specify, simulate, and analyze large-scale transit scenarios and their parametric variations. Of particular significance is an integrated microscopic energy consumption model that also helps to analyze the energy cost of various transit decisions made by the transportation agency of a city.
△ Less
Submitted 30 June, 2021;
originally announced July 2021.
-
Outage Performance Analysis of Widely Linear Receivers in Uplink Multi-user MIMO Systems
Authors:
Ronghua Gui,
Naveen Mysore Balasubramanya,
Lutz Lampe
Abstract:
This paper considers the application of widely linear (WL) receivers in an uplink multi-user system using real-valued modulation schemes, where the cellular base station (BS) with multiple antennas provides connectivity for randomly deployed single-antenna users. The targeted use case is massive machine type communication (mMTC) with grant-free access in the uplink, where the network is required t…
▽ More
This paper considers the application of widely linear (WL) receivers in an uplink multi-user system using real-valued modulation schemes, where the cellular base station (BS) with multiple antennas provides connectivity for randomly deployed single-antenna users. The targeted use case is massive machine type communication (mMTC) with grant-free access in the uplink, where the network is required to host a large number of low data rate devices transmitting in an uncoordinated fashion. Four types of WL receivers are investigated, namely the WL zero-forcing (ZF) and the WL minimum meansquared error (MMSE) receivers, along with their enhanced versions employing successive interference cancellation (SIC) with channel-dependent ordering, i.e., the WL-ZF-SIC and WL-MMSE-SIC receivers. The outage performances of these receivers are analytically characterized in the high signal-to-noise ratio (SNR) regime and compared to those of conventional linear (CL) receivers using complex-valued modulation schemes. For the non-SIC receivers, we show that, when compared to the CL counterparts, the WL receivers yield a higher diversity gain when decoding the same number of users and have the same diversity gain but a decreased coding gain when the number of users is nearly doubled. The outage performance analysis of WL-SIC receivers is facilitated by the marginal distribution of ordered eigenvalues of a real-valued Wishart matrix. It is shown that the SIC operation with channel-dependent ordering brings no additional diversity gain to the WL receivers but instead increases the coding gain. Moreover, the coding gain of WL-SIC receivers grows as the number of users increases and even exceeds that of CL-SIC receivers under suitable conditions.
△ Less
Submitted 27 June, 2021;
originally announced June 2021.
-
Ultra-broad near-infrared photoluminescence from crystalline (K-crypt)2Bi2 containing [Bi2]2- dimers
Authors:
Hong-Tao Sun,
Tetsu Yonezawa,
Miriam M. Gillett-Kunnath,
Yoshio Sakka,
Naoto Shirahata,
Sa Chu Rong Gui,
Minoru Fujii,
Slavi C. Sevov
Abstract:
For the first time, we report that a single crystal of (K-crypt)2Bi2 containing [Bi2]2+ displays ultra-broad near-infrared photoluminescence (PL) peaking at around 1190 nm and having a full width at the half maximum of 212 nm, stemming from the inherent electronic transitions of [Bi2]2+.The results not only add to the number of charged Bi species with luminescence, but also deepen the understandin…
▽ More
For the first time, we report that a single crystal of (K-crypt)2Bi2 containing [Bi2]2+ displays ultra-broad near-infrared photoluminescence (PL) peaking at around 1190 nm and having a full width at the half maximum of 212 nm, stemming from the inherent electronic transitions of [Bi2]2+.The results not only add to the number of charged Bi species with luminescence, but also deepen the understanding of Bi-related near-infrared emission behavior and lead to the reconsideration of the fundamentally important issue of Bi-related PL mechanisms in some material systems such as bulk glasses, fibers, and conventional optical crystals.
△ Less
Submitted 12 September, 2012; v1 submitted 28 May, 2012;
originally announced May 2012.