Showing 1–50 of 283 results for author: Ren, B

PointOBB-v2: Towards Simpler, Faster, and Stronger Single Point Supervised Oriented Object Detection

Authors: Botao Ren, Xue Yang, Yi Yu, Junwei Luo, Zhidong Deng

Abstract: Single point supervised oriented object detection has gained attention and made initial progress within the community. Diverse from those approaches relying on one-shot samples or powerful pretrained models (e.g. SAM), PointOBB has shown promise due to its prior-free feature. In this paper, we propose PointOBB-v2, a simpler, faster, and stronger method to generate pseudo rotated boxes from points… ▽ More Single point supervised oriented object detection has gained attention and made initial progress within the community. Diverse from those approaches relying on one-shot samples or powerful pretrained models (e.g. SAM), PointOBB has shown promise due to its prior-free feature. In this paper, we propose PointOBB-v2, a simpler, faster, and stronger method to generate pseudo rotated boxes from points without relying on any other prior. Specifically, we first generate a Class Probability Map (CPM) by training the network with non-uniform positive and negative sampling. We show that the CPM is able to learn the approximate object regions and their contours. Then, Principal Component Analysis (PCA) is applied to accurately estimate the orientation and the boundary of objects. By further incorporating a separation mechanism, we resolve the confusion caused by the overlapping on the CPM, enabling its operation in high-density scenarios. Extensive comparisons demonstrate that our method achieves a training speed 15.58x faster and an accuracy improvement of 11.60%/25.15%/21.19% on the DOTA-v1.0/v1.5/v2.0 datasets compared to the previous state-of-the-art, PointOBB. This significantly advances the cutting edge of single point supervised oriented detection in the modular track. △ Less

Submitted 10 October, 2024; originally announced October 2024.

Comments: 13 pages, 4 figures, 5 tables

GrabDAE: An Innovative Framework for Unsupervised Domain Adaptation Utilizing Grab-Mask and Denoise Auto-Encoder

Authors: Junzhou Chen, Xuan Wen, Ronghui Zhang, Bingtao Ren, Di Wu, Zhigang Xu, Danwei Wang

Abstract: Unsupervised Domain Adaptation (UDA) aims to adapt a model trained on a labeled source domain to an unlabeled target domain by addressing the domain shift. Existing Unsupervised Domain Adaptation (UDA) methods often fall short in fully leveraging contextual information from the target domain, leading to suboptimal decision boundary separation during source and target domain alignment. To address t… ▽ More Unsupervised Domain Adaptation (UDA) aims to adapt a model trained on a labeled source domain to an unlabeled target domain by addressing the domain shift. Existing Unsupervised Domain Adaptation (UDA) methods often fall short in fully leveraging contextual information from the target domain, leading to suboptimal decision boundary separation during source and target domain alignment. To address this, we introduce GrabDAE, an innovative UDA framework designed to tackle domain shift in visual classification tasks. GrabDAE incorporates two key innovations: the Grab-Mask module, which blurs background information in target domain images, enabling the model to focus on essential, domain-relevant features through contrastive learning; and the Denoising Auto-Encoder (DAE), which enhances feature alignment by reconstructing features and filtering noise, ensuring a more robust adaptation to the target domain. These components empower GrabDAE to effectively handle unlabeled target domain data, significantly improving both classification accuracy and robustness. Extensive experiments on benchmark datasets, including VisDA-2017, Office-Home, and Office31, demonstrate that GrabDAE consistently surpasses state-of-the-art UDA methods, setting new performance benchmarks. By tackling UDA's critical challenges with its novel feature masking and denoising approach, GrabDAE offers both significant theoretical and practical advancements in domain adaptation. △ Less

Submitted 10 October, 2024; originally announced October 2024.

FCDM: Sparse-view Sinogram Inpainting with Frequency Domain Convolution Enhanced Diffusion Models

Authors: Jiaze E, Srutarshi Banerjee, Tekin Bicer, Guannan Wang, Bin Ren

Abstract: Reducing the radiation dose in computed tomography (CT) is crucial, but it often results in sparse-view CT, where the number of available projections is significantly reduced. This reduction in projection data makes it challenging to accurately reconstruct high-quality CT images. In this condition, a sinogram, which is a collection of these projections, becomes incomplete. Sinogram inpainting then… ▽ More Reducing the radiation dose in computed tomography (CT) is crucial, but it often results in sparse-view CT, where the number of available projections is significantly reduced. This reduction in projection data makes it challenging to accurately reconstruct high-quality CT images. In this condition, a sinogram, which is a collection of these projections, becomes incomplete. Sinogram inpainting then becomes essential because it enables accurate image reconstruction with limited projections. Existing models performing well on conventional RGB images for inpainting mostly fail in the case of sinograms. Further, these models usually do not make full use of unique properties, e.g., frequency features and absorption characteristics in the sinogram, and cannot handle large-area masks and complex real-world projections well. To address these limitations, we propose a novel model called the Frequency Convolution Diffusion Model (FCDM). It employs frequency domain convolutions to extract frequency information from various angles and capture the intricate relationships between these angles, which is essential for high-quality CT reconstruction. We also design a specific loss function based on the unique properties of a sinogram to maintain the consistency in physical properties, which allows the model to learn more effectively even in larger mask areas. We compare FCDM using both simulations and real data with nine inpainting models examples, among which two are designed for sinogram and seven for RGB. The results indicate that our model significantly improves the quality of the inpainted sinograms in terms of both visually and quantitatively, with an SSIM of more than 0.95 and PSNR of more than 30, achieving up to a 33% improvement in SSIM and a 29% improvement in PSNR compared to the baseline. △ Less

Submitted 26 August, 2024; originally announced September 2024.

PVAFN: Point-Voxel Attention Fusion Network with Multi-Pooling Enhancing for 3D Object Detection

Authors: Yidi Li, Jiahao Wen, Bin Ren, Wenhao Li, Zhenhuan Xu, Hao Guo, Hong Liu, Nicu Sebe

Abstract: The integration of point and voxel representations is becoming more common in LiDAR-based 3D object detection. However, this combination often struggles with capturing semantic information effectively. Moreover, relying solely on point features within regions of interest can lead to information loss and limitations in local feature representation. To tackle these challenges, we propose a novel two… ▽ More The integration of point and voxel representations is becoming more common in LiDAR-based 3D object detection. However, this combination often struggles with capturing semantic information effectively. Moreover, relying solely on point features within regions of interest can lead to information loss and limitations in local feature representation. To tackle these challenges, we propose a novel two-stage 3D object detector, called Point-Voxel Attention Fusion Network (PVAFN). PVAFN leverages an attention mechanism to improve multi-modal feature fusion during the feature extraction phase. In the refinement stage, it utilizes a multi-pooling strategy to integrate both multi-scale and region-specific information effectively. The point-voxel attention mechanism adaptively combines point cloud and voxel-based Bird's-Eye-View (BEV) features, resulting in richer object representations that help to reduce false detections. Additionally, a multi-pooling enhancement module is introduced to boost the model's perception capabilities. This module employs cluster pooling and pyramid pooling techniques to efficiently capture key geometric details and fine-grained shape structures, thereby enhancing the integration of local and global features. Extensive experiments on the KITTI and Waymo datasets demonstrate that the proposed PVAFN achieves competitive performance. The code and models will be available. △ Less

Submitted 26 August, 2024; originally announced August 2024.

Comments: 3D Object Detection

Global-Local Distillation Network-Based Audio-Visual Speaker Tracking with Incomplete Modalities

Authors: Yidi Li, Yihan Li, Yixin Guo, Bin Ren, Zhenhuan Xu, Hao Guo, Hong Liu, Nicu Sebe

Abstract: In speaker tracking research, integrating and complementing multi-modal data is a crucial strategy for improving the accuracy and robustness of tracking systems. However, tracking with incomplete modalities remains a challenging issue due to noisy observations caused by occlusion, acoustic noise, and sensor failures. Especially when there is missing data in multiple modalities, the performance of… ▽ More In speaker tracking research, integrating and complementing multi-modal data is a crucial strategy for improving the accuracy and robustness of tracking systems. However, tracking with incomplete modalities remains a challenging issue due to noisy observations caused by occlusion, acoustic noise, and sensor failures. Especially when there is missing data in multiple modalities, the performance of existing multi-modal fusion methods tends to decrease. To this end, we propose a Global-Local Distillation-based Tracker (GLDTracker) for robust audio-visual speaker tracking. GLDTracker is driven by a teacher-student distillation model, enabling the flexible fusion of incomplete information from each modality. The teacher network processes global signals captured by camera and microphone arrays, and the student network handles local information subject to visual occlusion and missing audio channels. By transferring knowledge from teacher to student, the student network can better adapt to complex dynamic scenes with incomplete observations. In the student network, a global feature reconstruction module based on the generative adversarial network is constructed to reconstruct global features from feature embedding with missing local information. Furthermore, a multi-modal multi-level fusion attention is introduced to integrate the incomplete feature and the reconstructed feature, leveraging the complementarity and consistency of audio-visual and global-local features. Experimental results on the AV16.3 dataset demonstrate that the proposed GLDTracker outperforms existing state-of-the-art audio-visual trackers and achieves leading performance on both standard and incomplete modalities datasets, highlighting its superiority and robustness in complex conditions. The code and models will be available. △ Less

Submitted 26 August, 2024; originally announced August 2024.

Comments: Audio-Visual Speaker Tracking with Incomplete Modalities

Reconstruction-based Multi-Normal Prototypes Learning for Weakly Supervised Anomaly Detection

Authors: Zhijin Dong, Hongzhi Liu, Boyuan Ren, Weimin Xiong, Zhonghai Wu

Abstract: Anomaly detection is a crucial task in various domains. Most of the existing methods assume the normal sample data clusters around a single central prototype while the real data may consist of multiple categories or subgroups. In addition, existing methods always assume all unlabeled data are normal while they inevitably contain some anomalous samples. To address these issues, we propose a reconst… ▽ More Anomaly detection is a crucial task in various domains. Most of the existing methods assume the normal sample data clusters around a single central prototype while the real data may consist of multiple categories or subgroups. In addition, existing methods always assume all unlabeled data are normal while they inevitably contain some anomalous samples. To address these issues, we propose a reconstruction-based multi-normal prototypes learning framework that leverages limited labeled anomalies in conjunction with abundant unlabeled data for anomaly detection. Specifically, we assume the normal sample data may satisfy multi-modal distribution, and utilize deep embedding clustering and contrastive learning to learn multiple normal prototypes to represent it. Additionally, we estimate the likelihood of each unlabeled sample being normal based on the multi-normal prototypes, guiding the training process to mitigate the impact of contaminated anomalies in the unlabeled data. Extensive experiments on various datasets demonstrate the superior performance of our method compared to state-of-the-art techniques. △ Less

Submitted 23 August, 2024; originally announced August 2024.

ShapeSplat: A Large-scale Dataset of Gaussian Splats and Their Self-Supervised Pretraining

Authors: Qi Ma, Yue Li, Bin Ren, Nicu Sebe, Ender Konukoglu, Theo Gevers, Luc Van Gool, Danda Pani Paudel

Abstract: 3D Gaussian Splatting (3DGS) has become the de facto method of 3D representation in many vision tasks. This calls for the 3D understanding directly in this representation space. To facilitate the research in this direction, we first build a large-scale dataset of 3DGS using the commonly used ShapeNet and ModelNet datasets. Our dataset ShapeSplat consists of 65K objects from 87 unique categories, w… ▽ More 3D Gaussian Splatting (3DGS) has become the de facto method of 3D representation in many vision tasks. This calls for the 3D understanding directly in this representation space. To facilitate the research in this direction, we first build a large-scale dataset of 3DGS using the commonly used ShapeNet and ModelNet datasets. Our dataset ShapeSplat consists of 65K objects from 87 unique categories, whose labels are in accordance with the respective datasets. The creation of this dataset utilized the compute equivalent of 2 GPU years on a TITAN XP GPU. We utilize our dataset for unsupervised pretraining and supervised finetuning for classification and segmentation tasks. To this end, we introduce \textbf{\textit{Gaussian-MAE}}, which highlights the unique benefits of representation learning from Gaussian parameters. Through exhaustive experiments, we provide several valuable insights. In particular, we show that (1) the distribution of the optimized GS centroids significantly differs from the uniformly sampled point cloud (used for initialization) counterpart; (2) this change in distribution results in degradation in classification but improvement in segmentation tasks when using only the centroids; (3) to leverage additional Gaussian parameters, we propose Gaussian feature grouping in a normalized feature space, along with splats pooling layer, offering a tailored solution to effectively group and embed similar Gaussians, which leads to notable improvement in finetuning tasks. △ Less

Submitted 20 August, 2024; originally announced August 2024.

Deepest limits on scattered light emission from the Epsilon Eridani inner debris disk with HST/STIS

Authors: Sai Krishanth P. M., Ewan S. Douglas, Ramya M. Anche, Justin Hom, Kerri L. Cahoy, John H. Debes, Hannah Jang-Condell, Isabel Rebollido, Bin B. Ren, Christopher C. Stark, Robert Thompson, Yinzi Xin

Abstract: Epsilon Eridani ($ε$ Eri) is one of the first debris disk systems detected by the Infrared Astronomical Satellite (IRAS). However, the system has thus far eluded detection in scattered light with no components having been directly imaged. Its similarity to a relatively young Solar System combined with its proximity makes it an excellent candidate to further our understanding of planetary system ev… ▽ More Epsilon Eridani ($ε$ Eri) is one of the first debris disk systems detected by the Infrared Astronomical Satellite (IRAS). However, the system has thus far eluded detection in scattered light with no components having been directly imaged. Its similarity to a relatively young Solar System combined with its proximity makes it an excellent candidate to further our understanding of planetary system evolution. We present a set of coronagraphic images taken using the Space Telescope Imaging Spectrograph (STIS) coronagraph on the Hubble space telescope at a small inner working angle to detect a predicted warm inner debris disk inside 1". We used three different post-processing approaches; Non-negative Matrix Factorization (NMF), Karhunen-Lo`eve Image Processing (KLIP), and Classical reference differential imaging (RDI), to best optimize reference star subtraction, and find that NMF performed the best overall while KLIP produced the absolute best contrast inside 1". We present limits on scattered light from warm dust, with constraints on surface brightness at 6 mJy/as$^2$ at our inner working angle of 0.6". We also place a constraint of 0.5 mJy/as$^2$ outside 1", which gives us an upper limit on the brightness for outer disks and substellar companions. Finally, we calculated an upper limit on the dust albedo at $ω<$ 0.487. △ Less

Submitted 14 August, 2024; v1 submitted 13 August, 2024; originally announced August 2024.

Comments: 13+2 pages, 7+2 figures; Accepted for publication in the Astronomical Journal

Journal ref: The Astronomical Journal, 168:169 (10pp), 2024 October

A Survey of Protoplanetary Disks Using the Keck/NIRC2 Vortex Coronagraph

Authors: Nicole L. Wallack, Jean-Baptiste Ruffio, Garreth Ruane, Bin B. Ren, Jerry W. Xuan, Marion Villenave, Dimitri Mawet, Karl Stapelfeldt, Jason J. Wang, Michael C. Liu, Olivier Absil, Carlos Alvarez, Jaehan Bae, Charlotte Bond, Michael Bottom, Benjamin Calvin, Élodie Choquet, Valentin Christiaens, Therese Cook, Bruno Femenía Castellá, Carlos Gomez Gonzalez, Greta Guidi, Elsa Huby, Joel Kastner, Heather A. Knutson , et al. (12 additional authors not shown)

Abstract: Recent Atacama Large Millimeter/submillimeter Array (ALMA) observations of protoplanetary disks in the millimeter continuum have shown a variety of radial gaps, cavities, and spiral features. These substructures may be signposts for ongoing planet formation, and therefore these systems are promising targets for direct imaging planet searches in the near-infrared. To this end, we present results fr… ▽ More Recent Atacama Large Millimeter/submillimeter Array (ALMA) observations of protoplanetary disks in the millimeter continuum have shown a variety of radial gaps, cavities, and spiral features. These substructures may be signposts for ongoing planet formation, and therefore these systems are promising targets for direct imaging planet searches in the near-infrared. To this end, we present results from a deep imaging survey in the $L'$-band (3.8 $μ$m) with the Keck/NIRC2 vortex coronagraph to search for young planets in 43 disks with resolved features in the millimeter continuum or evidence for gaps/central cavities from their spectral energy distributions. Although we do not detect any new point sources, using the vortex coronagraph allows for high sensitivity to faint sources at small angular separations (down to ${\sim}$0$^{\prime\prime}$.1), allowing us to place strong upper limits on the masses of potential gas giant planets. We compare our mass sensitivities to the masses of planets derived using ALMA observations, and while we are sensitive to $\sim$1 M$_{Jup}$ planets in the gaps in some of our systems, we are generally not sensitive to planets of the masses expected from the ALMA observations. In addition to placing upper limits on the masses of gas giant planets that could be interacting with the dust in the disks to form the observed millimeter substructures, we are also able to map the micron-sized dust as seen in scattered light for 8 of these systems. Our large sample of systems also allows us to investigate limits on planetary accretion rates and disk viscosities. △ Less

Submitted 7 August, 2024; originally announced August 2024.

Comments: 23 pages, 14 figures, 3 tables, accepted for publication in AJ

Masked Angle-Aware Autoencoder for Remote Sensing Images

Authors: Zhihao Li, Biao Hou, Siteng Ma, Zitong Wu, Xianpeng Guo, Bo Ren, Licheng Jiao

Abstract: To overcome the inherent domain gap between remote sensing (RS) images and natural images, some self-supervised representation learning methods have made promising progress. However, they have overlooked the diverse angles present in RS objects. This paper proposes the Masked Angle-Aware Autoencoder (MA3E) to perceive and learn angles during pre-training. We design a \textit{scaling center crop} o… ▽ More To overcome the inherent domain gap between remote sensing (RS) images and natural images, some self-supervised representation learning methods have made promising progress. However, they have overlooked the diverse angles present in RS objects. This paper proposes the Masked Angle-Aware Autoencoder (MA3E) to perceive and learn angles during pre-training. We design a \textit{scaling center crop} operation to create the rotated crop with random orientation on each original image, introducing the explicit angle variation. MA3E inputs this composite image while reconstruct the original image, aiming to effectively learn rotation-invariant representations by restoring the angle variation introduced on the rotated crop. To avoid biases caused by directly reconstructing the rotated crop, we propose an Optimal Transport (OT) loss that automatically assigns similar original image patches to each rotated crop patch for reconstruction. MA3E demonstrates more competitive performance than existing pre-training methods on seven different RS image datasets in three downstream tasks. △ Less

Submitted 4 August, 2024; originally announced August 2024.

Comments: This paper has been accepted by ECCV 2024

Any Image Restoration with Efficient Automatic Degradation Adaptation

Authors: Bin Ren, Eduard Zamfir, Yawei Li, Zongwei Wu, Danda Pani Paudel, Radu Timofte, Nicu Sebe, Luc Van Gool

Abstract: With the emergence of mobile devices, there is a growing demand for an efficient model to restore any degraded image for better perceptual quality. However, existing models often require specific learning modules tailored for each degradation, resulting in complex architectures and high computation costs. Different from previous work, in this paper, we propose a unified manner to achieve joint emb… ▽ More With the emergence of mobile devices, there is a growing demand for an efficient model to restore any degraded image for better perceptual quality. However, existing models often require specific learning modules tailored for each degradation, resulting in complex architectures and high computation costs. Different from previous work, in this paper, we propose a unified manner to achieve joint embedding by leveraging the inherent similarities across various degradations for efficient and comprehensive restoration. Specifically, we first dig into the sub-latent space of each input to analyze the key components and reweight their contributions in a gated manner. The intrinsic awareness is further integrated with contextualized attention in an X-shaped scheme, maximizing local-global intertwining. Extensive comparison on benchmarking all-in-one restoration setting validates our efficiency and effectiveness, i.e., our network sets new SOTA records while reducing model complexity by approximately -82% in trainable parameters and -85\% in FLOPs. Our code will be made publicly available at:https://github.com/Amazingren/AnyIR. △ Less

Submitted 18 July, 2024; originally announced July 2024.

Comments: Efficient Any Image Restoration

Bringing Masked Autoencoders Explicit Contrastive Properties for Point Cloud Self-Supervised Learning

Authors: Bin Ren, Guofeng Mei, Danda Pani Paudel, Weijie Wang, Yawei Li, Mengyuan Liu, Rita Cucchiara, Luc Van Gool, Nicu Sebe

Abstract: Contrastive learning (CL) for Vision Transformers (ViTs) in image domains has achieved performance comparable to CL for traditional convolutional backbones. However, in 3D point cloud pretraining with ViTs, masked autoencoder (MAE) modeling remains dominant. This raises the question: Can we take the best of both worlds? To answer this question, we first empirically validate that integrating MAE-ba… ▽ More Contrastive learning (CL) for Vision Transformers (ViTs) in image domains has achieved performance comparable to CL for traditional convolutional backbones. However, in 3D point cloud pretraining with ViTs, masked autoencoder (MAE) modeling remains dominant. This raises the question: Can we take the best of both worlds? To answer this question, we first empirically validate that integrating MAE-based point cloud pre-training with the standard contrastive learning paradigm, even with meticulous design, can lead to a decrease in performance. To address this limitation, we reintroduce CL into the MAE-based point cloud pre-training paradigm by leveraging the inherent contrastive properties of MAE. Specifically, rather than relying on extensive data augmentation as commonly used in the image domain, we randomly mask the input tokens twice to generate contrastive input pairs. Subsequently, a weight-sharing encoder and two identically structured decoders are utilized to perform masked token reconstruction. Additionally, we propose that for an input token masked by both masks simultaneously, the reconstructed features should be as similar as possible. This naturally establishes an explicit contrastive constraint within the generative MAE-based pre-training paradigm, resulting in our proposed method, Point-CMAE. Consequently, Point-CMAE effectively enhances the representation quality and transfer performance compared to its MAE counterpart. Experimental evaluations across various downstream applications, including classification, part segmentation, and few-shot learning, demonstrate the efficacy of our framework in surpassing state-of-the-art techniques under standard ViTs and single-modal settings. The source code and trained models are available at: https://github.com/Amazingren/Point-CMAE. △ Less

Submitted 8 July, 2024; originally announced July 2024.

Comments: Bringing Masked Autoencoders Explicit Contrastive Properties for Point Cloud Self-Supervised Learning

High-order WENO finite-difference methods for hyperbolic nonconservative systems of Partial Differential Equations

Authors: B. Ren, C. Parés

Abstract: This work aims to extend the well-known high-order WENO finite-difference methods for systems of conservation laws to nonconservative hyperbolic systems. The main difficulty of these systems both from the theoretical and the numerical points of view comes from the fact that the definition of weak solution is not unique: according to the theory developed by Dal Maso, LeFloch, and Murat in 1995, it… ▽ More This work aims to extend the well-known high-order WENO finite-difference methods for systems of conservation laws to nonconservative hyperbolic systems. The main difficulty of these systems both from the theoretical and the numerical points of view comes from the fact that the definition of weak solution is not unique: according to the theory developed by Dal Maso, LeFloch, and Murat in 1995, it depends on the choice of a family of paths. A general strategy is proposed here in which WENO operators are not only used to reconstruct fluxes but also the nonconservative products of the system. Moreover, if a Roe linearization is available, the nonconservative products can be computed through matrix-vector operations instead of path-integrals. The methods are extended to problems with source terms and two different strategies are introduced to obtain well-balanced schemes. These numerical schemes will be then applied to the two-layer shallow water equations in one- and two- dimensions to obtain high-order methods that preserve water-at-rest steady states. △ Less

Submitted 5 July, 2024; originally announced July 2024.

GRB 221009A/SN 2022xiw: A Supernova Obscured by a Gamma-Ray Burst Afterglow?

Authors: De-Feng Kong, Xiang-Gao Wang, WeiKang Zheng, Hou-Jun Lü, L. P. Xin, Da-Bin Lin, Jia-Xin Cao, Ming-Xuan Lu, B. Ren, Edgar P. Vidal, J. Y. Wei, En-Wei Liang, Alexei V. Filippenko

Abstract: We present optical photometry for the afterglow of GRB 221009A, in some respects the most extraordinary gamma-ray burst (GRB) ever observed. Good quality in the R-band light curve is obtained, covering 0.32-19.57 days since the Fermi-GBM trigger. We find that a weak bump emerges fromthe declining afterglow at $t \approx 11$ days; a supernova (SN) may be responsible. We use a smooth broken power-la… ▽ More We present optical photometry for the afterglow of GRB 221009A, in some respects the most extraordinary gamma-ray burst (GRB) ever observed. Good quality in the R-band light curve is obtained, covering 0.32-19.57 days since the Fermi-GBM trigger. We find that a weak bump emerges fromthe declining afterglow at $t \approx 11$ days; a supernova (SN) may be responsible. We use a smooth broken power-law and $^{56}\mathrm{Ni}$ model to fit the light curve. The best-fitting results reveal that the SN ejected a total mass of $M_\mathrm{ej} = 3.70 M_\odot$, a $^{56}\mathrm{Ni}$ mass of $M_\mathrm{Ni} = 0.23 M_\odot$, and a kinetic energy of $E_\mathrm{SN,K} = 2.35 \times 10^{52} \mathrm{erg}$. We also compare GRB 221009A with other GRB-SN events based on a GRB-associated SN sample, and find that only SN 2003lw and SN 2011kl can be obviously revealed in the afterglow of GRB 221009A by setting these objects at its distance. This suggests that a supernova (SN 2022xiw) is possibly obscured by the brighter afterglow emission from GRB 221009A. △ Less

Submitted 30 June, 2024; originally announced July 2024.

Lighting Every Darkness with 3DGS: Fast Training and Real-Time Rendering for HDR View Synthesis

Authors: Xin Jin, Pengyi Jiao, Zheng-Peng Duan, Xingchao Yang, Chun-Le Guo, Bo Ren, Chongyi Li

Abstract: Volumetric rendering based methods, like NeRF, excel in HDR view synthesis from RAWimages, especially for nighttime scenes. While, they suffer from long training times and cannot perform real-time rendering due to dense sampling requirements. The advent of 3D Gaussian Splatting (3DGS) enables real-time rendering and faster training. However, implementing RAW image-based view synthesis directly usi… ▽ More Volumetric rendering based methods, like NeRF, excel in HDR view synthesis from RAWimages, especially for nighttime scenes. While, they suffer from long training times and cannot perform real-time rendering due to dense sampling requirements. The advent of 3D Gaussian Splatting (3DGS) enables real-time rendering and faster training. However, implementing RAW image-based view synthesis directly using 3DGS is challenging due to its inherent drawbacks: 1) in nighttime scenes, extremely low SNR leads to poor structure-from-motion (SfM) estimation in distant views; 2) the limited representation capacity of spherical harmonics (SH) function is unsuitable for RAW linear color space; and 3) inaccurate scene structure hampers downstream tasks such as refocusing. To address these issues, we propose LE3D (Lighting Every darkness with 3DGS). Our method proposes Cone Scatter Initialization to enrich the estimation of SfM, and replaces SH with a Color MLP to represent the RAW linear color space. Additionally, we introduce depth distortion and near-far regularizations to improve the accuracy of scene structure for downstream tasks. These designs enable LE3D to perform real-time novel view synthesis, HDR rendering, refocusing, and tone-mapping changes. Compared to previous volumetric rendering based methods, LE3D reduces training time to 1% and improves rendering speed by up to 4,000 times for 2K resolution images in terms of FPS. Code and viewer can be found in https://github.com/Srameo/LE3D . △ Less

Submitted 10 June, 2024; originally announced June 2024.

Sharing Key Semantics in Transformer Makes Efficient Image Restoration

Authors: Bin Ren, Yawei Li, Jingyun Liang, Rakesh Ranjan, Mengyuan Liu, Rita Cucchiara, Luc Van Gool, Ming-Hsuan Yang, Nicu Sebe

Abstract: Image Restoration (IR), a classic low-level vision task, has witnessed significant advancements through deep models that effectively model global information. Notably, the Vision Transformers (ViTs) emergence has further propelled these advancements. When computing, the self-attention mechanism, a cornerstone of ViTs, tends to encompass all global cues, even those from semantically unrelated objec… ▽ More Image Restoration (IR), a classic low-level vision task, has witnessed significant advancements through deep models that effectively model global information. Notably, the Vision Transformers (ViTs) emergence has further propelled these advancements. When computing, the self-attention mechanism, a cornerstone of ViTs, tends to encompass all global cues, even those from semantically unrelated objects or regions. This inclusivity introduces computational inefficiencies, particularly noticeable with high input resolution, as it requires processing irrelevant information, thereby impeding efficiency. Additionally, for IR, it is commonly noted that small segments of a degraded image, particularly those closely aligned semantically, provide particularly relevant information to aid in the restoration process, as they contribute essential contextual cues crucial for accurate reconstruction. To address these challenges, we propose boosting IR's performance by sharing the key semantics via Transformer for IR (i.e., SemanIR) in this paper. Specifically, SemanIR initially constructs a sparse yet comprehensive key-semantic dictionary within each transformer stage by establishing essential semantic connections for every degraded patch. Subsequently, this dictionary is shared across all subsequent transformer blocks within the same stage. This strategy optimizes attention calculation within each block by focusing exclusively on semantically related components stored in the key-semantic dictionary. As a result, attention calculation achieves linear computational complexity within each window. Extensive experiments across 6 IR tasks confirm the proposed SemanIR's state-of-the-art performance, quantitatively and qualitatively showcasing advancements. △ Less

Submitted 30 May, 2024; originally announced May 2024.

Comments: 9 pages

JUNO Sensitivity to Invisible Decay Modes of Neutrons

Authors: JUNO Collaboration, Angel Abusleme, Thomas Adam, Kai Adamowicz, Shakeel Ahmad, Rizwan Ahmed, Sebastiano Aiello, Fengpeng An, Qi An, Giuseppe Andronico, Nikolay Anfimov, Vito Antonelli, Tatiana Antoshkina, João Pedro Athayde Marcondes de André, Didier Auguste, Weidong Bai, Nikita Balashov, Wander Baldini, Andrea Barresi, Davide Basilico, Eric Baussan, Marco Bellato, Marco Beretta, Antonio Bergnoli, Daniel Bick , et al. (635 additional authors not shown)

Abstract: We explore the bound neutrons decay into invisible particles (e.g., $n\rightarrow 3 ν$ or $nn \rightarrow 2 ν$) in the JUNO liquid scintillator detector. The invisible decay includes two decay modes: $ n \rightarrow { inv} $ and $ nn \rightarrow { inv} $. The invisible decays of $s$-shell neutrons in $^{12}{\rm C}$ will leave a highly excited residual nucleus. Subsequently, some de-excitation mode… ▽ More We explore the bound neutrons decay into invisible particles (e.g., $n\rightarrow 3 ν$ or $nn \rightarrow 2 ν$) in the JUNO liquid scintillator detector. The invisible decay includes two decay modes: $ n \rightarrow { inv} $ and $ nn \rightarrow { inv} $. The invisible decays of $s$-shell neutrons in $^{12}{\rm C}$ will leave a highly excited residual nucleus. Subsequently, some de-excitation modes of the excited residual nuclei can produce a time- and space-correlated triple coincidence signal in the JUNO detector. Based on a full Monte Carlo simulation informed with the latest available data, we estimate all backgrounds, including inverse beta decay events of the reactor antineutrino $\barν_e$, natural radioactivity, cosmogenic isotopes and neutral current interactions of atmospheric neutrinos. Pulse shape discrimination and multivariate analysis techniques are employed to further suppress backgrounds. With two years of exposure, JUNO is expected to give an order of magnitude improvement compared to the current best limits. After 10 years of data taking, the JUNO expected sensitivities at a 90% confidence level are $τ/B( n \rightarrow { inv} ) > 5.0 \times 10^{31} \, {\rm yr}$ and $τ/B( nn \rightarrow { inv} ) > 1.4 \times 10^{32} \, {\rm yr}$. △ Less

Submitted 27 May, 2024; originally announced May 2024.

Comments: 28 pages, 7 figures, 4 tables

QML-IB: Quantized Collaborative Intelligence between Multiple Devices and the Mobile Network

Authors: Jingchen Peng, Boxiang Ren, Lu Yang, Chenghui Peng, Panpan Niu, Hao Wu

Abstract: The integration of artificial intelligence (AI) and mobile networks is regarded as one of the most important scenarios for 6G. In 6G, a major objective is to realize the efficient transmission of task-relevant data. Then a key problem arises, how to design collaborative AI models for the device side and the network side, so that the transmitted data between the device and the network is efficient… ▽ More The integration of artificial intelligence (AI) and mobile networks is regarded as one of the most important scenarios for 6G. In 6G, a major objective is to realize the efficient transmission of task-relevant data. Then a key problem arises, how to design collaborative AI models for the device side and the network side, so that the transmitted data between the device and the network is efficient enough, which means the transmission overhead is low but the AI task result is accurate. In this paper, we propose the multi-link information bottleneck (ML-IB) scheme for such collaborative models design. We formulate our problem based on a novel performance metric, which can evaluate both task accuracy and transmission overhead. Then we introduce a quantizer that is adjustable in the quantization bit depth, amplitudes, and breakpoints. Given the infeasibility of calculating our proposed metric on high-dimensional data, we establish a variational upper bound for this metric. However, due to the incorporation of quantization, the closed form of the variational upper bound remains uncomputable. Hence, we employ the Log-Sum Inequality to derive an approximation and provide a theoretical guarantee. Based on this, we devise the quantized multi-link information bottleneck (QML-IB) algorithm for collaborative AI models generation. Finally, numerical experiments demonstrate the superior performance of our QML-IB algorithm compared to the state-of-the-art algorithm. △ Less

Submitted 30 April, 2024; originally announced April 2024.

SmartMem: Layout Transformation Elimination and Adaptation for Efficient DNN Execution on Mobile

Authors: Wei Niu, Md Musfiqur Rahman Sanim, Zhihao Shu, Jiexiong Guan, Xipeng Shen, Miao Yin, Gagan Agrawal, Bin Ren

Abstract: This work is motivated by recent developments in Deep Neural Networks, particularly the Transformer architectures underlying applications such as ChatGPT, and the need for performing inference on mobile devices. Focusing on emerging transformers (specifically the ones with computationally efficient Swin-like architectures) and large models (e.g., Stable Diffusion and LLMs) based on transformers, w… ▽ More This work is motivated by recent developments in Deep Neural Networks, particularly the Transformer architectures underlying applications such as ChatGPT, and the need for performing inference on mobile devices. Focusing on emerging transformers (specifically the ones with computationally efficient Swin-like architectures) and large models (e.g., Stable Diffusion and LLMs) based on transformers, we observe that layout transformations between the computational operators cause a significant slowdown in these applications. This paper presents SmartMem, a comprehensive framework for eliminating most layout transformations, with the idea that multiple operators can use the same tensor layout through careful choice of layout and implementation of operations. Our approach is based on classifying the operators into four groups, and considering combinations of producer-consumer edges between the operators. We develop a set of methods for searching such layouts. Another component of our work is developing efficient memory layouts for 2.5 dimensional memory commonly seen in mobile devices. Our experimental results show that SmartMem outperforms 5 state-of-the-art DNN execution frameworks on mobile devices across 18 varied neural networks, including CNNs, Transformers with both local and global attention, as well as LLMs. In particular, compared to DNNFusion, SmartMem achieves an average speedup of 2.8$\times$, and outperforms TVM and MNN with speedups of 6.9$\times$ and 7.9$\times$, respectively, on average. △ Less

Submitted 21 April, 2024; originally announced April 2024.

PDS 70 unveiled by star-hopping: total intensity, polarimetry and mm-imaging modeled in concert

Authors: Z. Wahhaj, M. Benisty, C. Ginski, C. Swastik, S. Arora, R. G. van Holstein, R. J. De Rosa, B. Yang, J. Bae, B. Ren

Abstract: Context. Most ground-based planet search direct imaging campaigns use angular differential imaging, which distorts the signal from extended sources like protoplanetary disks. In the case PDS 70, a young system with two planets found within the cavity of a protoplanetary disk, obtaining a reliable image of both planets and disk is essential to understanding planet-disk interactions. Aims. Our goals… ▽ More Context. Most ground-based planet search direct imaging campaigns use angular differential imaging, which distorts the signal from extended sources like protoplanetary disks. In the case PDS 70, a young system with two planets found within the cavity of a protoplanetary disk, obtaining a reliable image of both planets and disk is essential to understanding planet-disk interactions. Aims. Our goals are to reveal the true intensity of the planets and disk without self-subtraction effects for the first time, search for new giant planets beyond separations of 0.1" and to study the morphology of the disk shaped by two massive planets. Methods. We present YJHK-band imaging, polarimetry, and spatially resolved spectroscopy of PDS 70 using near-simultaneous reference star differential imaging, also known as star-hopping. We created a radiative transfer model of the system to match the near-infrared imaging and polarimetric data, along with sub-millimeter imaging from ALMA. Furthermore, we extracted the spectra of the planets and the disk and compared them. Results. We find that the disk is quite flared with a scale height of ~15% at the outer edge of the disk at ~90 au, similar to some disks in the literature. The gap inside of ~50 au is estimated to have ~1% of the dust density of the outer disk. The Northeast outer disk arc seen in previous observations is likely the outer lip of the flared disk. Abundance ratios of grains estimated by the modeling indicate a shallow grain-size index > -2.7, instead of the canonical -3.5. There is both vertical and radial segregation of grains. Planet c is well separated from the disk and has a spectrum similar to planet b, clearly redder than the disk spectra. Planet c is possibly associated with the sudden flaring of the disk starting at ~50 au. No new planets > 5 Mj were found. △ Less

Submitted 17 April, 2024; originally announced April 2024.

Comments: Accepted to A&A on April 11, 2024. 20 pages, 19 figures

Journal ref: A&A 687, A257 (2024)

The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Report

Authors: Bin Ren, Yawei Li, Nancy Mehta, Radu Timofte, Hongyuan Yu, Cheng Wan, Yuxin Hong, Bingnan Han, Zhuoyuan Wu, Yajun Zou, Yuqing Liu, Jizhe Li, Keji He, Chao Fan, Heng Zhang, Xiaolin Zhang, Xuanwu Yin, Kunlong Zuo, Bohao Liao, Peizhe Xia, Long Peng, Zhibo Du, Xin Di, Wangkai Li, Yang Wang , et al. (109 additional authors not shown)

Abstract: This paper provides a comprehensive review of the NTIRE 2024 challenge, focusing on efficient single-image super-resolution (ESR) solutions and their outcomes. The task of this challenge is to super-resolve an input image with a magnification factor of x4 based on pairs of low and corresponding high-resolution images. The primary objective is to develop networks that optimize various aspects such… ▽ More This paper provides a comprehensive review of the NTIRE 2024 challenge, focusing on efficient single-image super-resolution (ESR) solutions and their outcomes. The task of this challenge is to super-resolve an input image with a magnification factor of x4 based on pairs of low and corresponding high-resolution images. The primary objective is to develop networks that optimize various aspects such as runtime, parameters, and FLOPs, while still maintaining a peak signal-to-noise ratio (PSNR) of approximately 26.90 dB on the DIV2K_LSDIR_valid dataset and 26.99 dB on the DIV2K_LSDIR_test dataset. In addition, this challenge has 4 tracks including the main track (overall performance), sub-track 1 (runtime), sub-track 2 (FLOPs), and sub-track 3 (parameters). In the main track, all three metrics (ie runtime, FLOPs, and parameter count) were considered. The ranking of the main track is calculated based on a weighted sum-up of the scores of all other sub-tracks. In sub-track 1, the practical runtime performance of the submissions was evaluated, and the corresponding score was used to determine the ranking. In sub-track 2, the number of FLOPs was considered. The score calculated based on the corresponding FLOPs was used to determine the ranking. In sub-track 3, the number of parameters was considered. The score calculated based on the corresponding parameters was used to determine the ranking. RLFN is set as the baseline for efficiency measurement. The challenge had 262 registered participants, and 34 teams made valid submissions. They gauge the state-of-the-art in efficient single-image super-resolution. To facilitate the reproducibility of the challenge and enable other researchers to build upon these findings, the code and the pre-trained model of validated solutions are made publicly available at https://github.com/Amazingren/NTIRE2024_ESR/. △ Less

Submitted 25 June, 2024; v1 submitted 16 April, 2024; originally announced April 2024.

Comments: The report paper of NTIRE2024 Efficient Super-resolution, accepted by CVPRW2024

Socially Pertinent Robots in Gerontological Healthcare

Authors: Xavier Alameda-Pineda, Angus Addlesee, Daniel Hernández García, Chris Reinke, Soraya Arias, Federica Arrigoni, Alex Auternaud, Lauriane Blavette, Cigdem Beyan, Luis Gomez Camara, Ohad Cohen, Alessandro Conti, Sébastien Dacunha, Christian Dondrup, Yoav Ellinson, Francesco Ferro, Sharon Gannot, Florian Gras, Nancie Gunson, Radu Horaud, Moreno D'Incà, Imad Kimouche, Séverin Lemaignan, Oliver Lemon, Cyril Liotard , et al. (19 additional authors not shown)

Abstract: Despite the many recent achievements in developing and deploying social robotics, there are still many underexplored environments and applications for which systematic evaluation of such systems by end-users is necessary. While several robotic platforms have been used in gerontological healthcare, the question of whether or not a social interactive robot with multi-modal conversational capabilitie… ▽ More Despite the many recent achievements in developing and deploying social robotics, there are still many underexplored environments and applications for which systematic evaluation of such systems by end-users is necessary. While several robotic platforms have been used in gerontological healthcare, the question of whether or not a social interactive robot with multi-modal conversational capabilities will be useful and accepted in real-life facilities is yet to be answered. This paper is an attempt to partially answer this question, via two waves of experiments with patients and companions in a day-care gerontological facility in Paris with a full-sized humanoid robot endowed with social and conversational interaction capabilities. The software architecture, developed during the H2020 SPRING project, together with the experimental protocol, allowed us to evaluate the acceptability (AES) and usability (SUS) with more than 60 end-users. Overall, the users are receptive to this technology, especially when the robot perception and action skills are robust to environmental clutter and flexible to handle a plethora of different interactions. △ Less

Submitted 11 April, 2024; originally announced April 2024.

Improving Detection in Aerial Images by Capturing Inter-Object Relationships

Authors: Botao Ren, Botian Xu, Yifan Pu, Jingyi Wang, Zhidong Deng

Abstract: In many image domains, the spatial distribution of objects in a scene exhibits meaningful patterns governed by their semantic relationships. In most modern detection pipelines, however, the detection proposals are processed independently, overlooking the underlying relationships between objects. In this work, we introduce a transformer-based approach to capture these inter-object relationships to… ▽ More In many image domains, the spatial distribution of objects in a scene exhibits meaningful patterns governed by their semantic relationships. In most modern detection pipelines, however, the detection proposals are processed independently, overlooking the underlying relationships between objects. In this work, we introduce a transformer-based approach to capture these inter-object relationships to refine classification and regression outcomes for detected objects. Building on two-stage detectors, we tokenize the region of interest (RoI) proposals to be processed by a transformer encoder. Specific spatial and geometric relations are incorporated into the attention weights and adaptively modulated and regularized. Experimental results demonstrate that the proposed method achieves consistent performance improvement on three benchmarks including DOTA-v1.0, DOTA-v1.5, and HRSC 2016, especially ranking first on both DOTA-v1.5 and HRSC 2016. Specifically, our new method has an increase of 1.59 mAP on DOTA-v1.0, 4.88 mAP on DOTA-v1.5, and 2.1 mAP on HRSC 2016, respectively, compared to the baselines. △ Less

Submitted 5 April, 2024; originally announced April 2024.

The First High-Contrast Images of Near High-Mass X-Ray Binaries with Keck/NIRC2

Authors: M. Prasow-Émond, J. Hlavacek-Larrondo, K. Fogarty, É. Artigau, D. Mawet, P. Gandhi, J. F. Steiner, J. Rameau, D. Lafrenière, A. C. Fabian, D. J. Walton, R. Doyon, B. B. Ren

Abstract: Although the study of X-ray binaries has led to major breakthroughs in high-energy astrophysics, their circumbinary environment at scales of $\sim$100--10,000 astronomical units has not been thoroughly investigated. In this paper, we undertake a novel and exploratory study by employing direct and high-contrast imaging techniques on a sample of X-ray binaries, using adaptive optics and the vortex c… ▽ More Although the study of X-ray binaries has led to major breakthroughs in high-energy astrophysics, their circumbinary environment at scales of $\sim$100--10,000 astronomical units has not been thoroughly investigated. In this paper, we undertake a novel and exploratory study by employing direct and high-contrast imaging techniques on a sample of X-ray binaries, using adaptive optics and the vortex coronagraph on Keck/NIRC2. High-contrast imaging opens up the possibility to search for exoplanets, brown dwarfs, circumbinary companion stars, and protoplanetary disks in these extreme systems. Here, we present the first near-infrared high-contrast images of 13 high-mass X-ray binaries located within $\sim$2--3 kpc. The key results of this campaign involve the discovery of several candidate circumbinary companions ranging from sub-stellar (brown dwarf) to stellar masses. By conducting an analysis based on galactic population models, we discriminate sources that are likely background/foreground stars and isolate those that have a high probability ($\gtrsim 60 - 99\%$) of being gravitationally bound to the X-ray binary. This publication seeks to establish a preliminary catalog for future analyses of proper motion and subsequent observations. With our preliminary results, we calculate the first estimate of the companion frequency and the multiplicity frequency for X-ray binaries: $\approx$0.6 and 1.8 $\pm$ 0.9 respectively, considering only the sources that are most likely bound to the X-ray binary. In addition to extending our comprehension of how brown dwarfs and stars can form and survive in such extreme systems, our study opens a new window to our understanding of the formation of X-ray binaries. △ Less

Submitted 23 March, 2024; originally announced March 2024.

Comments: 26 pages, 6 figures, accepted for publication in ApJ

Effect of particle oxidation, size and material on deformation, bonding and deposition during cold spray: a peridynamic investigation

Authors: Baihua Ren, Jun Song

Abstract: Cold spray (CS) has emerged as an important additive manufacturing technology over the past decade. This study investigates the effect of oxide layers on the CS process, focusing on the deformation behavior of copper (Cu) and iron (Fe) particles upon collision with a matching substrate. Using a peridynamics-based approach, we examine the effects of oxide thickness, particle size, and particle/subs… ▽ More Cold spray (CS) has emerged as an important additive manufacturing technology over the past decade. This study investigates the effect of oxide layers on the CS process, focusing on the deformation behavior of copper (Cu) and iron (Fe) particles upon collision with a matching substrate. Using a peridynamics-based approach, we examine the effects of oxide thickness, particle size, and particle/substrate material on material deformation and oxide fracture processes. Our results show that thicker oxide films restrict particle deformation, delay oxide discontinuities and material jetting, and increase the critical velocity required for metal to metal contact. Larger particles, despite uniform deformation across sizes, require lower velocities to initiate jetting and oxide separation because of their higher kinetic energy, leading to metallurgical bonding at lower velocities. Soft to soft impacts induce oxide film cracking at lower velocities, resulting in larger interface areas and more oxide-free contact zones, thereby reducing the critical velocity. Furthermore, the volume of residual oxide has a power-law relationship with the particle size, indicating that the oxide-cleaning ability of the particles affects the critical velocity. This study highlights the importance of oxide deformation and fracture during CS processes and provides valuable insights into the breakage and removal of oxides and subsequent metallic bond formation. These findings offer beneficial new knowledge for the rational design and optimization of CS processes. △ Less

Submitted 14 August, 2024; v1 submitted 2 March, 2024; originally announced March 2024.

SoD$^2$: Statically Optimizing Dynamic Deep Neural Network

Authors: Wei Niu, Gagan Agrawal, Bin Ren

Abstract: Though many compilation and runtime systems have been developed for DNNs in recent years, the focus has largely been on static DNNs. Dynamic DNNs, where tensor shapes and sizes and even the set of operators used are dependent upon the input and/or execution, are becoming common. This paper presents SoD$^2$, a comprehensive framework for optimizing Dynamic DNNs. The basis of our approach is a class… ▽ More Though many compilation and runtime systems have been developed for DNNs in recent years, the focus has largely been on static DNNs. Dynamic DNNs, where tensor shapes and sizes and even the set of operators used are dependent upon the input and/or execution, are becoming common. This paper presents SoD$^2$, a comprehensive framework for optimizing Dynamic DNNs. The basis of our approach is a classification of common operators that form DNNs, and the use of this classification towards a Rank and Dimension Propagation (RDP) method. This framework statically determines the shapes of operators as known constants, symbolic constants, or operations on these. Next, using RDP we enable a series of optimizations, like fused code generation, execution (order) planning, and even runtime memory allocation plan generation. By evaluating the framework on 10 emerging Dynamic DNNs and comparing it against several existing systems, we demonstrate both reductions in execution latency and memory requirements, with RDP-enabled key optimizations responsible for much of the gains. Our evaluation results show that SoD$^2$ runs up to $3.9\times$ faster than these systems while saving up to $88\%$ peak memory consumption. △ Less

Submitted 29 February, 2024; originally announced March 2024.

Multi-band reflectance and shadowing of RX J1604.3-2130 protoplanetary disk in scattered light

Authors: Huisheng Zhong, Bin B. Ren, Bo Ma, Chen Xie, Jie Ma, Nicole L. Wallack, Dimitri Mawet, Garreth Ruane

Abstract: Context.Spatially-resoved cicrumstellar disk spectrum and composition can provide valuable insights into the bulk composition of forming planets, as well as the mineralogical signatures that emerge during and after planet formation. Aims. We aim to systemically extract the RX~J1604.3-213010 (J1604 hereafter) protoplanetary disk in high-contrast imaging observations, and obtain its multi-band refle… ▽ More Context.Spatially-resoved cicrumstellar disk spectrum and composition can provide valuable insights into the bulk composition of forming planets, as well as the mineralogical signatures that emerge during and after planet formation. Aims. We aim to systemically extract the RX~J1604.3-213010 (J1604 hereafter) protoplanetary disk in high-contrast imaging observations, and obtain its multi-band reflectance in visible to near-infrared wavelengths. Methods. We obtained coronagraphic observations of J1604 from the Keck Observatory's NIRC2 instrument, and archival data from the Very Large Telescope's SPHERE instrument. Using archival images to remove star light and speckles, we recovered the J1604 disk and obtained its surface brightness using forward modeling. Together with polarization data, we obtained the relative reflectance of the disk in $R$, $J$, $H$ ($H2$ and $H3$), $K$ ($K1$ and $K2$), and $L'$ bands spanning two years. Results. Relative to the J1604 star, the resolved disk has a reflectance of ${\sim}10^{-1}$~arcsec$^{-2}$ in $R$ through $H$ bands and ${\sim}10^{-2}$~arcsec$^{-2}$ in $K$ and $L'$ bands, showing a blue color. Together with other systems, we summarized the multi-band reflectance for 9 systems. We also identified varying disk geometry structure, and a shadow that vanished between June and August in 2015. Conclusions. Motivated by broad-band observations, the deployment of cutting-edge technologies could yield higher-resolution reflection spectra, thereby informing the dust composition of disks in scattered light in the future. With multi-epoch observations, variable shadows have the potential to deepen insights into the dynamic characteristics of inner disk regions. △ Less

Submitted 26 February, 2024; originally announced February 2024.

Comments: 13 pages, 6 figures

A uniformly ergodic Gibbs sampler for Bayesian survival analysis

Authors: Benny Ren, Jeffrey Morris, Ian Barnett

Abstract: Finite sample inference for Cox models is an important problem in many settings, such as clinical trials. Bayesian procedures provide a means for finite sample inference and incorporation of prior information if MCMC algorithms and posteriors are well behaved. On the other hand, estimation procedures should also retain inferential properties in high dimensional settings. In addition, estimation pr… ▽ More Finite sample inference for Cox models is an important problem in many settings, such as clinical trials. Bayesian procedures provide a means for finite sample inference and incorporation of prior information if MCMC algorithms and posteriors are well behaved. On the other hand, estimation procedures should also retain inferential properties in high dimensional settings. In addition, estimation procedures should be able to incorporate constraints and multilevel modeling such as cure models and frailty models in a straightforward manner. In order to tackle these modeling challenges, we propose a uniformly ergodic Gibbs sampler for a broad class of convex set constrained multilevel Cox models. We develop two key strategies. First, we exploit a connection between Cox models and negative binomial processes through the Poisson process to reduce Bayesian computation to iterative Gaussian sampling. Next, we appeal to sufficient dimension reduction to address the difficult computation of nonparametric baseline hazards, allowing for the collapse of the Markov transition operator within the Gibbs sampler based on sufficient statistics. We demonstrate our approach using open source data and simulations. △ Less

Submitted 22 February, 2024; originally announced February 2024.

3C 273 Host Galaxy with Hubble Space Telescope Coronagraphy

Authors: Bin B. Ren, Kevin Fogarty, John H. Debes, Eileen T. Meyer, Youbin Mo, Dimitri Mawet, Marshall D. Perrin, Patrick M. Ogle, Johannes Sahlmann

Abstract: The close-in regions of bright quasars' host galaxies have been difficult to image due to the overwhelming light from the quasars. With coronagraphic observations in visible light using the Space Telescope Imaging Spectrograph (STIS) on the Hubble Space Telescope, we removed 3C 273 quasar light using color-matching reference stars. The observations revealed the host galaxy from 60" to 0.2" with ne… ▽ More The close-in regions of bright quasars' host galaxies have been difficult to image due to the overwhelming light from the quasars. With coronagraphic observations in visible light using the Space Telescope Imaging Spectrograph (STIS) on the Hubble Space Telescope, we removed 3C 273 quasar light using color-matching reference stars. The observations revealed the host galaxy from 60" to 0.2" with nearly full angular coverage. Isophote modeling revealed a new core jet, a core blob, and multiple smaller-scale blobs within 2.5". The blobs could potentially be satellite galaxies or infalling materials towards the central quasar. Using archival STIS data, we constrained the apparent motion of its large scale jets over a 22 yr timeline. By resolving the 3C 273 host galaxy with STIS, our study validates the coronagraph usage on extragalactic sources in obtaining new insights into the central ~kpc regions of quasar hosts. △ Less

Submitted 14 February, 2024; originally announced February 2024.

Comments: 13 pages, 11 figures, 2 tables, A&A Letters accepted

Key-Graph Transformer for Image Restoration

Authors: Bin Ren, Yawei Li, Jingyun Liang, Rakesh Ranjan, Mengyuan Liu, Rita Cucchiara, Luc Van Gool, Nicu Sebe

Abstract: While it is crucial to capture global information for effective image restoration (IR), integrating such cues into transformer-based methods becomes computationally expensive, especially with high input resolution. Furthermore, the self-attention mechanism in transformers is prone to considering unnecessary global cues from unrelated objects or regions, introducing computational inefficiencies. In… ▽ More While it is crucial to capture global information for effective image restoration (IR), integrating such cues into transformer-based methods becomes computationally expensive, especially with high input resolution. Furthermore, the self-attention mechanism in transformers is prone to considering unnecessary global cues from unrelated objects or regions, introducing computational inefficiencies. In response to these challenges, we introduce the Key-Graph Transformer (KGT) in this paper. Specifically, KGT views patch features as graph nodes. The proposed Key-Graph Constructor efficiently forms a sparse yet representative Key-Graph by selectively connecting essential nodes instead of all the nodes. Then the proposed Key-Graph Attention is conducted under the guidance of the Key-Graph only among selected nodes with linear computational complexity within each window. Extensive experiments across 6 IR tasks confirm the proposed KGT's state-of-the-art performance, showcasing advancements both quantitatively and qualitatively. △ Less

Submitted 4 February, 2024; originally announced February 2024.

Comments: 9 pages, 6 figures

Uncertainty-Aware Testing-Time Optimization for 3D Human Pose Estimation

Authors: Ti Wang, Mengyuan Liu, Hong Liu, Bin Ren, Yingxuan You, Wenhao Li, Nicu Sebe, Xia Li

Abstract: Although data-driven methods have achieved success in 3D human pose estimation, they often suffer from domain gaps and exhibit limited generalization. In contrast, optimization-based methods excel in fine-tuning for specific cases but are generally inferior to data-driven methods in overall performance. We observe that previous optimization-based methods commonly rely on projection constraint, whi… ▽ More Although data-driven methods have achieved success in 3D human pose estimation, they often suffer from domain gaps and exhibit limited generalization. In contrast, optimization-based methods excel in fine-tuning for specific cases but are generally inferior to data-driven methods in overall performance. We observe that previous optimization-based methods commonly rely on projection constraint, which only ensures alignment in 2D space, potentially leading to the overfitting problem. To address this, we propose an Uncertainty-Aware testing-time Optimization (UAO) framework, which keeps the prior information of pre-trained model and alleviates the overfitting problem using the uncertainty of joints. Specifically, during the training phase, we design an effective 2D-to-3D network for estimating the corresponding 3D pose while quantifying the uncertainty of each 3D joint. For optimization during testing, the proposed optimization framework freezes the pre-trained model and optimizes only a latent state. Projection loss is then employed to ensure the generated poses are well aligned in 2D space for high-quality optimization. Furthermore, we utilize the uncertainty of each joint to determine how much each joint is allowed for optimization. The effectiveness and superiority of the proposed framework are validated through extensive experiments on two challenging datasets: Human3.6M and MPI-INF-3DHP. Notably, our approach outperforms the previous best result by a large margin of 4.5% on Human3.6M. Our source code will be open-sourced. △ Less

Submitted 3 February, 2024; originally announced February 2024.

Mitigating Prior Shape Bias in Point Clouds via Differentiable Center Learning

Authors: Zhe Li, Ziyang Zhang, Jinglin Zhao, Zheng Wang, Bocheng Ren, Debin Liu, Laurence T. Yang

Abstract: Masked autoencoding and generative pretraining have achieved remarkable success in computer vision and natural language processing, and more recently, they have been extended to the point cloud domain. Nevertheless, existing point cloud models suffer from the issue of information leakage due to the pre-sampling of center points, which leads to trivial proxy tasks for the models. These approaches p… ▽ More Masked autoencoding and generative pretraining have achieved remarkable success in computer vision and natural language processing, and more recently, they have been extended to the point cloud domain. Nevertheless, existing point cloud models suffer from the issue of information leakage due to the pre-sampling of center points, which leads to trivial proxy tasks for the models. These approaches primarily focus on local feature reconstruction, limiting their ability to capture global patterns within point clouds. In this paper, we argue that the reduced difficulty of pretext tasks hampers the model's capacity to learn expressive representations. To address these limitations, we introduce a novel solution called the Differentiable Center Sampling Network (DCS-Net). It tackles the information leakage problem by incorporating both global feature reconstruction and local feature reconstruction as non-trivial proxy tasks, enabling simultaneous learning of both the global and local patterns within point cloud. Experimental results demonstrate that our method enhances the expressive capacity of existing point cloud models and effectively addresses the issue of information leakage. △ Less

Submitted 11 October, 2024; v1 submitted 3 February, 2024; originally announced February 2024.

MLIP: Enhancing Medical Visual Representation with Divergence Encoder and Knowledge-guided Contrastive Learning

Authors: Zhe Li, Laurence T. Yang, Bocheng Ren, Xin Nie, Zhangyang Gao, Cheng Tan, Stan Z. Li

Abstract: The scarcity of annotated data has sparked significant interest in unsupervised pre-training methods that leverage medical reports as auxiliary signals for medical visual representation learning. However, existing research overlooks the multi-granularity nature of medical visual representation and lacks suitable contrastive learning techniques to improve the models' generalizability across differe… ▽ More The scarcity of annotated data has sparked significant interest in unsupervised pre-training methods that leverage medical reports as auxiliary signals for medical visual representation learning. However, existing research overlooks the multi-granularity nature of medical visual representation and lacks suitable contrastive learning techniques to improve the models' generalizability across different granularities, leading to the underutilization of image-text information. To address this, we propose MLIP, a novel framework leveraging domain-specific medical knowledge as guiding signals to integrate language information into the visual domain through image-text contrastive learning. Our model includes global contrastive learning with our designed divergence encoder, local token-knowledge-patch alignment contrastive learning, and knowledge-guided category-level contrastive learning with expert knowledge. Experimental evaluations reveal the efficacy of our model in enhancing transfer performance for tasks such as image classification, object detection, and semantic segmentation. Notably, MLIP surpasses state-of-the-art methods even with limited annotated data, highlighting the potential of multimodal pre-training in advancing medical representation learning. △ Less

Submitted 3 February, 2024; originally announced February 2024.

A Uniform Analysis of Debris Disks with the Gemini Planet Imager II: Constraints on Dust Density Distribution Using Empirically-Informed Scattering Phase Functions

Authors: Justin Hom, Jennifer Patience, Christine H. Chen, Gaspard Duchêne, Johan Mazoyer, Maxwell A. Millar-Blanchaer, Thomas M. Esposito, Paul Kalas, Katie A. Crotts, Eileen C. Gonzales, Ludmilla Kolokolova, Briley L. Lewis, Brenda C. Matthews, Malena Rice, Alycia J. Weinberger, David J. Wilner, Schuyler G. Wolff, Sebastián Bruzzone, Elodie Choquet, John Debes, Robert J. De Rosa, Jessica Donaldson, Zachary Draper, Michael P. Fitzgerald, Dean C. Hines , et al. (18 additional authors not shown)

Abstract: Spatially-resolved images of debris disks are necessary to determine disk morphological properties and the scattering phase function (SPF) which quantifies the brightness of scattered light as a function of phase angle. Current high-contrast imaging instruments have successfully resolved several dozens of debris disks around other stars, but few studies have investigated trends in the scattered-li… ▽ More Spatially-resolved images of debris disks are necessary to determine disk morphological properties and the scattering phase function (SPF) which quantifies the brightness of scattered light as a function of phase angle. Current high-contrast imaging instruments have successfully resolved several dozens of debris disks around other stars, but few studies have investigated trends in the scattered-light, resolved population of debris disks in a uniform and consistent manner. We have combined Karhunen-Loeve Image Projection (KLIP) with radiative-transfer disk forward modeling in order to obtain the highest quality image reductions and constrain disk morphological properties of eight debris disks imaged by the Gemini Planet Imager at H-band with a consistent and uniformly-applied approach. In describing the scattering properties of our models, we assume a common SPF informed from solar system dust scattering measurements and apply it to all systems. We identify a diverse range of dust density properties among the sample, including critical radius, radial width, and vertical width. We also identify radially narrow and vertically extended disks that may have resulted from substellar companion perturbations, along with a tentative positive trend in disk eccentricity with relative disk width. We also find that using a common SPF can achieve reasonable model fits for disks that are axisymmetric and asymmetric when fitting models to each side of the disk independently, suggesting that scattering behavior from debris disks may be similar to Solar System dust. △ Less

Submitted 31 January, 2024; originally announced February 2024.

Comments: 23+5 pages, 12+6 figures, 15 pages of Online Supplemental Material included; Accepted for publication in MNRAS

Equivariant Index Theorem on $\mathbb{R}^n$ in the Context of Continuous Fields of $C^*$-algebras

Authors: Baiying Ren, Hang Wang, Zijing Wang

Abstract: We prove an equivariant index theorem on the Euclidean space using a continuous field of $C^*$-algebras. This generalizes the work of Elliott, Natsume and Nest, which is a special case of the algebraic index theorem by Nest-Tsygan. Using our formula, the equivariant index of the Bott-Dirac operator on $\mathbb{R}^{2n}$ can be explicitly calculated. We prove an equivariant index theorem on the Euclidean space using a continuous field of $C^*$-algebras. This generalizes the work of Elliott, Natsume and Nest, which is a special case of the algebraic index theorem by Nest-Tsygan. Using our formula, the equivariant index of the Bott-Dirac operator on $\mathbb{R}^{2n}$ can be explicitly calculated. △ Less

Submitted 15 January, 2024; originally announced January 2024.

Revisiting Recommendation Loss Functions through Contrastive Learning (Technical Report)

Authors: Dong Li, Ruoming Jin, Bin Ren

Abstract: Inspired by the success of contrastive learning, we systematically examine recommendation losses, including listwise (softmax), pairwise (BPR), and pointwise (MSE and CCL) losses. In this endeavor, we introduce InfoNCE+, an optimized generalization of InfoNCE with balance coefficients, and highlight its performance advantages, particularly when aligned with our new decoupled contrastive loss, MINE… ▽ More Inspired by the success of contrastive learning, we systematically examine recommendation losses, including listwise (softmax), pairwise (BPR), and pointwise (MSE and CCL) losses. In this endeavor, we introduce InfoNCE+, an optimized generalization of InfoNCE with balance coefficients, and highlight its performance advantages, particularly when aligned with our new decoupled contrastive loss, MINE+. We also leverage debiased InfoNCE to debias pointwise recommendation loss (CCL) as Debiased CCL. Interestingly, our analysis reveals that linear models like iALS and EASE are inherently debiased. Empirical results demonstrates the effectiveness of MINE+ and Debiased-CCL. △ Less

Submitted 13 December, 2023; originally announced December 2023.

Comments: This manuscript was initially submitted for review in August 2023

Multi-source domain adaptation for regression

Authors: Yujie Wu, Giovanni Parmigiani, Boyu Ren

Abstract: Multi-source domain adaptation (DA) aims at leveraging information from more than one source domain to make predictions in a target domain, where different domains may have different data distributions. Most existing methods for multi-source DA focus on classification problems while there is only limited investigation in the regression settings. In this paper, we fill in this gap through a two-ste… ▽ More Multi-source domain adaptation (DA) aims at leveraging information from more than one source domain to make predictions in a target domain, where different domains may have different data distributions. Most existing methods for multi-source DA focus on classification problems while there is only limited investigation in the regression settings. In this paper, we fill in this gap through a two-step procedure. First, we extend a flexible single-source DA algorithm for classification through outcome-coarsening to enable its application to regression problems. We then augment our single-source DA algorithm for regression with ensemble learning to achieve multi-source DA. We consider three learning paradigms in the ensemble algorithm, which combines linearly the target-adapted learners trained with each source domain: (i) a multi-source stacking algorithm to obtain the ensemble weights; (ii) a similarity-based weighting where the weights reflect the quality of DA of each target-adapted learner; and (iii) a combination of the stacking and similarity weights. We illustrate the performance of our algorithms with simulations and a data application where the goal is to predict High-density lipoprotein (HDL) cholesterol levels using gut microbiome. We observe a consistent improvement in prediction performance of our multi-source DA algorithm over the routinely used methods in all these scenarios. △ Less

Submitted 8 December, 2023; originally announced December 2023.

The JWST Early Release Science Program for Direct Observations of Exoplanetary Systems V: Do Self-Consistent Atmospheric Models Represent JWST Spectra? A Showcase With VHS 1256 b

Authors: Simon Petrus, Niall Whiteford, Polychronis Patapis, Beth A. Biller, Andrew Skemer, Sasha Hinkley, Genaro Suárez, Anna Lueber, Paulina Palma-Bifani, Jordan M. Stone, Johanna M. Vos, Caroline V. Morley, Pascal Tremblin, Benjamin Charnay, Christiane Helling, Brittany E. Miles, Aarynn L. Carter, Jason J. Wang, Markus Janson, Eileen C. Gonzales, Ben Sutlieff, Kielan K. W. Hoch, Mickaël Bonnefoy, Gaël Chauvin, Olivier Absil , et al. (97 additional authors not shown)

Abstract: The unprecedented medium-resolution (R~1500-3500) near- and mid-infrared (1-18um) spectrum provided by JWST for the young (140+/-20Myr) low-mass (12-20MJup) L-T transition (L7) companion VHS1256b gives access to a catalogue of molecular absorptions. In this study, we present a comprehensive analysis of this dataset utilizing a forward modelling approach, applying our Bayesian framework, ForMoSA. W… ▽ More The unprecedented medium-resolution (R~1500-3500) near- and mid-infrared (1-18um) spectrum provided by JWST for the young (140+/-20Myr) low-mass (12-20MJup) L-T transition (L7) companion VHS1256b gives access to a catalogue of molecular absorptions. In this study, we present a comprehensive analysis of this dataset utilizing a forward modelling approach, applying our Bayesian framework, ForMoSA. We explore five distinct atmospheric models to assess their performance in estimating key atmospheric parameters: Teff, log(g), [M/H], C/O, gamma, fsed, and R. Our findings reveal that each parameter's estimate is significantly influenced by factors such as the wavelength range considered and the model chosen for the fit. This is attributed to systematic errors in the models and their challenges in accurately replicating the complex atmospheric structure of VHS1256b, notably the complexity of its clouds and dust distribution. To propagate the impact of these systematic uncertainties on our atmospheric property estimates, we introduce innovative fitting methodologies based on independent fits performed on different spectral windows. We finally derived a Teff consistent with the spectral type of the target, considering its young age, which is confirmed by our estimate of log(g). Despite the exceptional data quality, attaining robust estimates for chemical abundances [M/H] and C/O, often employed as indicators of formation history, remains challenging. Nevertheless, the pioneering case of JWST's data for VHS1256b has paved the way for future acquisitions of substellar spectra that will be systematically analyzed to directly compare the properties of these objects and correct the systematics in the models. △ Less

Submitted 31 January, 2024; v1 submitted 6 December, 2023; originally announced December 2023.

Comments: 32 pages, 16 figures, 6 tables, 2 appendices

Zero-Shot Point Cloud Registration

Authors: Weijie Wang, Guofeng Mei, Bin Ren, Xiaoshui Huang, Fabio Poiesi, Luc Van Gool, Nicu Sebe, Bruno Lepri

Abstract: Learning-based point cloud registration approaches have significantly outperformed their traditional counterparts. However, they typically require extensive training on specific datasets. In this paper, we propose , the first zero-shot point cloud registration approach that eliminates the need for training on point cloud datasets. The cornerstone of ZeroReg is the novel transfer of image features… ▽ More Learning-based point cloud registration approaches have significantly outperformed their traditional counterparts. However, they typically require extensive training on specific datasets. In this paper, we propose , the first zero-shot point cloud registration approach that eliminates the need for training on point cloud datasets. The cornerstone of ZeroReg is the novel transfer of image features from keypoints to the point cloud, enriched by aggregating information from 3D geometric neighborhoods. Specifically, we extract keypoints and features from 2D image pairs using a frozen pretrained 2D backbone. These features are then projected in 3D, and patches are constructed by searching for neighboring points. We integrate the geometric and visual features of each point using our novel parameter-free geometric decoder. Subsequently, the task of determining correspondences between point clouds is formulated as an optimal transport problem. Extensive evaluations of ZeroReg demonstrate its competitive performance against both traditional and learning-based methods. On benchmarks such as 3DMatch, 3DLoMatch, and ScanNet, ZeroReg achieves impressive Recall Ratios (RR) of over 84%, 46%, and 75%, respectively. △ Less

Submitted 8 December, 2023; v1 submitted 5 December, 2023; originally announced December 2023.

Feedback RoI Features Improve Aerial Object Detection

Authors: Botao Ren, Botian Xu, Tengyu Liu, Jingyi Wang, Zhidong Deng

Abstract: Neuroscience studies have shown that the human visual system utilizes high-level feedback information to guide lower-level perception, enabling adaptation to signals of different characteristics. In light of this, we propose Feedback multi-Level feature Extractor (Flex) to incorporate a similar mechanism for object detection. Flex refines feature selection based on image-wise and instance-level fe… ▽ More Neuroscience studies have shown that the human visual system utilizes high-level feedback information to guide lower-level perception, enabling adaptation to signals of different characteristics. In light of this, we propose Feedback multi-Level feature Extractor (Flex) to incorporate a similar mechanism for object detection. Flex refines feature selection based on image-wise and instance-level feedback information in response to image quality variation and classification uncertainty. Experimental results show that Flex offers consistent improvement to a range of existing SOTA methods on the challenging aerial object detection datasets including DOTA-v1.0, DOTA-v1.5, and HRSC2016. Although the design originates in aerial image detection, further experiments on MS COCO also reveal our module's efficacy in general detection models. Quantitative and qualitative analyses indicate that the improvements are closely related to image qualities, which match our motivation. △ Less

Submitted 28 November, 2023; originally announced November 2023.

A Uniform Analysis of Debris Disks with the Gemini Planet Imager I: An Empirical Search for Perturbations from Planetary Companions in Polarized Light Images

Authors: Katie A. Crotts, Brenda C. Matthews, Gaspard Duchêne, Thomas M. Esposito, Ruobing Dong, Justin Hom, Rebecca Oppenheimer, Malena Rice, Schuyler G. Wolff, Christine H. Chen, Clarissa R. Do Ó, Paul Kalas, Briley L. Lewis, Alycia J. Weinberger, David J. Wilner, Mark Ammons, Pauline Arriaga, Robert J. De Rosa, John H. Debes, Michael P. Fitzgerald, Eileen C. Gonzales, Dean C. Hines, Sasha Hinkley, A. Meredith Hughes, Ludmilla Kolokolova , et al. (15 additional authors not shown)

Abstract: The Gemini Planet Imager (GPI) has excelled in imaging debris disks in the near-infrared. The GPI Exoplanet Survey (GPIES) imaged twenty-four debris disks in polarized $H$-band light, while other programs observed half of these disks in polarized $J$- and/or $K1$-bands. Using these data, we present a uniform analysis of the morphology of each disk to find asymmetries suggestive of perturbations, p… ▽ More The Gemini Planet Imager (GPI) has excelled in imaging debris disks in the near-infrared. The GPI Exoplanet Survey (GPIES) imaged twenty-four debris disks in polarized $H$-band light, while other programs observed half of these disks in polarized $J$- and/or $K1$-bands. Using these data, we present a uniform analysis of the morphology of each disk to find asymmetries suggestive of perturbations, particularly those due to planet-disk interactions. The multi-wavelength surface brightness, the disk color and geometry permit identification of any asymmetries such as warps or disk offsets from the central star. We find that nineteen of the disks in this sample exhibit asymmetries in surface brightness, disk color, disk geometry, or a combination of the three, suggesting that for this sample, perturbations, as seen in scattered light, are common. The relationship between these perturbations and potential planets in the system are discussed. We also explore correlations among stellar temperatures, ages, disk properties, and observed perturbations. We find significant trends between the vertical aspect ratio and the stellar temperature, disk radial extent, and the dust grain size distribution power-law, $q$. We also confirm a trend between the disk color and stellar effective temperature, where the disk becomes increasingly red/neutral with increasing temperature. Such results have important implications on the evolution of debris disk systems around stars of various spectral types. △ Less

Submitted 24 November, 2023; originally announced November 2023.

Comments: 46 pages, 20 figures, 6 tables, accepted for publication in ApJ

A Companion in V1247 Ori Supported by Spiral Arm Pattern Motion

Authors: Bin B. Ren, Chen Xie, Myriam Benisty, Ruobing Dong, Jaehan Bae, Tomas Stolker, Rob G. van Holstein, John H. Debes, Antonio Garufi, Christian Ginski, Stefan Kraus

Abstract: While there have been nearly two dozen of spiral arms detected from planet-forming disks in near-infrared scattered light, none of their substellar drivers have been confirmed. By observing spiral systems in at least two epochs spanning multiple years, and measuring the motion of the spirals, we can distinguish the cause of the spirals, and locate the orbits of the driving planets if they trigger… ▽ More While there have been nearly two dozen of spiral arms detected from planet-forming disks in near-infrared scattered light, none of their substellar drivers have been confirmed. By observing spiral systems in at least two epochs spanning multiple years, and measuring the motion of the spirals, we can distinguish the cause of the spirals, and locate the orbits of the driving planets if they trigger the spirals. Upon a recent validation of this approach using the co-motion between a stellar companion and a spiral, we obtained a second epoch observation for the spiral system in the disk of V1247 Ori in the $H$-band polarized scattered light using VLT/SPHERE/IRDIS. Combining our observations with archival IRDIS data, we established a $4.8$ yr timeline to constrain the V1247 Ori spiral motion. We obtained a pattern speed of $0.40^{\circ} \pm 0.09^{\circ}$ yr$^{-1}$ for the north-east spiral. This corresponds to an orbital period of $900\pm200$ yr, and thus the semi-major axis of the hidden planetary driver is $118\pm19$ au for a 2.0 $\pm$ 0.1 M$_\odot$ central star. The location agrees with the gap in ALMA dust continuum observations, providing joint support for the existence of a companion driving the scattered-light spirals while carving a millimeter gap. With an angular separation of 0.29" $\pm$ 0.05", this hidden companion is an ideal target for JWST imaging. △ Less

Submitted 7 December, 2023; v1 submitted 23 October, 2023; originally announced October 2023.

Comments: Accepted for publication in Astronomy and Astrophysics; 6 pages, 5 figures

The JWST Early Release Science Program for Direct Observations of Exoplanetary Systems III: Aperture Masking Interferometric Observations of the star HIP 65426

Authors: Shrishmoy Ray, Steph Sallum, Sasha Hinkley, Anand Sivamarakrishnan, Rachel Cooper, Jens Kammerer, Alexandra Z. Greebaum, Deepashri Thatte, Cecilia Lazzoni, Andrei Tokovinin, Matthew de Furio, Samuel Factor, Michael Meyer, Jordan M. Stone, Aarynn Carter, Beth Biller, Andrew Skemer, Genaro Suarez, Jarron M. Leisenring, Marshall D. Perrin, Adam L. Kraus, Olivier Absil, William O. Balmer, Mickael Bonnefoy, Marta L. Bryan , et al. (98 additional authors not shown)

Abstract: We present aperture masking interferometry (AMI) observations of the star HIP 65426 at $3.8\,\rm{μm}$ as a part of the JWST Direct Imaging Early Release Science (ERS) program obtained using the Near Infrared Imager and Slitless Spectrograph (NIRISS) instrument. This mode provides access to very small inner working angles (even separations slightly below the Michelson limit of $0.5λ/D$ for an inter… ▽ More We present aperture masking interferometry (AMI) observations of the star HIP 65426 at $3.8\,\rm{μm}$ as a part of the JWST Direct Imaging Early Release Science (ERS) program obtained using the Near Infrared Imager and Slitless Spectrograph (NIRISS) instrument. This mode provides access to very small inner working angles (even separations slightly below the Michelson limit of $0.5λ/D$ for an interferometer), which are inaccessible with the classical inner working angles of the JWST coronagraphs. When combined with JWST's unprecedented infrared sensitivity, this mode has the potential to probe a new portion of parameter space across a wide array of astronomical observations. Using this mode, we are able to achieve a $5σ$ contrast of $Δm{\sim}7.62{\pm}0.13$ mag relative to the host star at separations ${\gtrsim}0.07{"}$, and the contrast deteriorates steeply at separations ${\lesssim}0.07{"}$. However, we detect no additional companions interior to the known companion HIP 65426 b (at separation ${\sim}0.82{"}$ or, $87^{+108}_{-31}\,\rm{au}$). Our observations thus rule out companions more massive than $10{-}12\,\rm{M_{Jup}}$ at separations ${\sim}10{-}20\,\rm{au}$ from HIP 65426, a region out of reach of ground or space-based coronagraphic imaging. These observations confirm that the AMI mode on JWST is sensitive to planetary mass companions at close-in separations (${\gtrsim}0.07{"}$), even for thousands of more distant stars at $\sim$100 pc, in addition to the stars in the nearby young moving groups as stated in previous works. This result will allow the planning and successful execution of future observations to probe the inner regions of nearby stellar systems, opening an essentially unexplored parameter space. △ Less

Submitted 14 October, 2024; v1 submitted 17 October, 2023; originally announced October 2023.

Comments: 20 pages, 12 figures, submitted to ApJL

The JWST Early Release Science Program for Direct Observations of Exoplanetary Systems IV: NIRISS Aperture Masking Interferometry Performance and Lessons Learned

Authors: Steph Sallum, Shrishmoy Ray, Jens Kammerer, Anand Sivaramakrishnan, Rachel Cooper, Alexandra Z. Greebaum, Deepashri Thatte, Matthew de Furio, Samuel Factor, Michael Meyer, Jordan M. Stone, Aarynn Carter, Beth Biller, Sasha Hinkley, Andrew Skemer, Genaro Suarez, Jarron M. Leisenring, Marshall D. Perrin, Adam L. Kraus, Olivier Absil, William O. Balmer, Mickael Bonnefoy, Marta L. Bryan, Sarah K. Betti, Anthony Boccaletti , et al. (98 additional authors not shown)

Abstract: We present a performance analysis for the aperture masking interferometry (AMI) mode on board the James Webb Space Telescope Near Infrared Imager and Slitless Spectrograph (JWST/NIRISS). Thanks to self-calibrating observables, AMI accesses inner working angles down to and even within the classical diffraction limit. The scientific potential of this mode has recently been demonstrated by the Early… ▽ More We present a performance analysis for the aperture masking interferometry (AMI) mode on board the James Webb Space Telescope Near Infrared Imager and Slitless Spectrograph (JWST/NIRISS). Thanks to self-calibrating observables, AMI accesses inner working angles down to and even within the classical diffraction limit. The scientific potential of this mode has recently been demonstrated by the Early Release Science (ERS) 1386 program with a deep search for close-in companions in the HIP 65426 exoplanetary system. As part of ERS 1386, we use the same data set to explore the random, static, and calibration errors of NIRISS AMI observables. We compare the observed noise properties and achievable contrast to theoretical predictions. We explore possible sources of calibration errors and show that differences in charge migration between the observations of HIP 65426 and point-spread function calibration stars can account for the achieved contrast curves. Lastly, we use self-calibration tests to demonstrate that with adequate calibration NIRISS F380M AMI can reach contrast levels of $\sim9-10$ mag at $\gtrsim λ/D$. These tests lead us to observation planning recommendations and strongly motivate future studies aimed at producing sophisticated calibration strategies taking these systematic effects into account. This will unlock the unprecedented capabilities of JWST/NIRISS AMI, with sensitivity to significantly colder, lower-mass exoplanets than lower-contrast ground-based AMI setups, at orbital separations inaccessible to JWST coronagraphy. △ Less

Submitted 11 March, 2024; v1 submitted 17 October, 2023; originally announced October 2023.

Comments: 20 pages, 12 figures, accepted to Astrophysical Journal Letters

Protoplanetary disks in $K_s$-band total intensity and polarized light

Authors: Bin B. Ren, Myriam Benisty, Christian Ginski, Ryo Tazaki, Nicole L. Wallack, Julien Milli, Antonio Garufi, Jaehan Bae, Stefano Facchini, François Ménard, Paola Pinilla, C. Swastik, Richard Teague, Zahed Wahhaj

Abstract: Diverse protoplanetary disk morphology can result from planet-disk interaction, suggesting planetary presence. To date, most scattered light imaging campaigns have probed polarized light, which is only a fraction of the total light and not very sensitive to planets. To observe and characterize protoplanetary disk systems in the near-infrared in both polarized and total intensity light, we carried… ▽ More Diverse protoplanetary disk morphology can result from planet-disk interaction, suggesting planetary presence. To date, most scattered light imaging campaigns have probed polarized light, which is only a fraction of the total light and not very sensitive to planets. To observe and characterize protoplanetary disk systems in the near-infrared in both polarized and total intensity light, we carried out an unprecedented study of scattering properties of disks, as well as of any planetary companions. Using SPHERE with star-hopping at the Very Large Telescope, we observed 29 disk hosts and their reference stars in $K_s$-band polarized light. We extracted disks in total intensity by adopting the data imputation concept with sequential non-negative matrix factorization (DI-sNMF). We obtained high-quality disk images in total intensity for 15 systems and in polarized light for 23. For well-recovered disks in polarized light and total intensity, we parameterized the polarization fraction phase functions using scaled beta distribution: the peak of polarization fraction tentatively correlates with the peak scattering angle, which could be reproduced using certain compact dust, yet more detailed modeling studies are needed. We investigated the empirical DI-sNMF detectability of disks using logistic regression: total intensity detectability of disks primarily depends on host star brightness. For disks with SPHERE data in $Y$-/$J$-/$H$-band, we summarized their polarized color at ~90 deg scattering angle: most of disks are blue in polarized $J-K_s$ color, and they are relatively redder as stellar luminosity increases, indicating larger scatterers. High-quality disk imagery in both total intensity and polarized light thus allows for disk characterization in polarization fraction, and reduces the confusion between disk and planetary signals. △ Less

Submitted 12 October, 2023; originally announced October 2023.

Comments: 25 pages, 16 figures, 3 tables, A&A accepted. Data files in FITS format will be publicly available

Journal ref: A&A 680, A114 (2023)

Real-time Monitoring for the Next Core-Collapse Supernova in JUNO

Authors: Angel Abusleme, Thomas Adam, Shakeel Ahmad, Rizwan Ahmed, Sebastiano Aiello, Muhammad Akram, Abid Aleem, Fengpeng An, Qi An, Giuseppe Andronico, Nikolay Anfimov, Vito Antonelli, Tatiana Antoshkina, Burin Asavapibhop, João Pedro Athayde Marcondes de André, Didier Auguste, Weidong Bai, Nikita Balashov, Wander Baldini, Andrea Barresi, Davide Basilico, Eric Baussan, Marco Bellato, Marco Beretta, Antonio Bergnoli , et al. (606 additional authors not shown)

Abstract: The core-collapse supernova (CCSN) is considered one of the most energetic astrophysical events in the universe. The early and prompt detection of neutrinos before (pre-SN) and during the supernova (SN) burst presents a unique opportunity for multi-messenger observations of CCSN events. In this study, we describe the monitoring concept and present the sensitivity of the system to pre-SN and SN neu… ▽ More The core-collapse supernova (CCSN) is considered one of the most energetic astrophysical events in the universe. The early and prompt detection of neutrinos before (pre-SN) and during the supernova (SN) burst presents a unique opportunity for multi-messenger observations of CCSN events. In this study, we describe the monitoring concept and present the sensitivity of the system to pre-SN and SN neutrinos at the Jiangmen Underground Neutrino Observatory (JUNO), a 20 kton liquid scintillator detector currently under construction in South China. The real-time monitoring system is designed to ensure both prompt alert speed and comprehensive coverage of progenitor stars. It incorporates prompt monitors on the electronic board as well as online monitors at the data acquisition stage. Assuming a false alert rate of 1 per year, this monitoring system exhibits sensitivity to pre-SN neutrinos up to a distance of approximately 1.6 (0.9) kiloparsecs and SN neutrinos up to about 370 (360) kiloparsecs for a progenitor mass of 30 solar masses, considering both normal and inverted mass ordering scenarios. The pointing ability of the CCSN is evaluated by analyzing the accumulated event anisotropy of inverse beta decay interactions from pre-SN or SN neutrinos. This, along with the early alert, can play a crucial role in facilitating follow-up multi-messenger observations of the next galactic or nearby extragalactic CCSN. △ Less

Submitted 4 December, 2023; v1 submitted 13 September, 2023; originally announced September 2023.

Comments: 24 pages, 9 figures, accepted for the publication at JCAP

Vortex Fiber Nulling for Exoplanet Observations: Implementation and First Light

Authors: Daniel Echeverri, Jerry Xuan, Nemanja Jovanovic, Garreth Ruane, Jacques-Robert Delorme, Dimitri Mawet, Bertrand Mennesson, Eugene Serabyn, J. Kent Wallace, Jason Wang, Jean-Baptiste Ruffio, Luke Finnerty, Yinzi Xin, Maxwell Millar-Blanchaer, Ashley Baker, Randall Bartos, Benjamin Calvin, Sylvain Cetre, Greg Doppmann, Michael P. Fitzgerald, Sofia Hillman, Katelyn Horstman, Chih-Chun Hsu, Joshua Liberman, Ronald Lopez , et al. (9 additional authors not shown)

Abstract: Vortex fiber nulling (VFN) is a single-aperture interferometric technique for detecting and characterizing exoplanets separated from their host star by less than a diffracted beam width. VFN uses a vortex mask and single mode fiber to selectively reject starlight while coupling off-axis planet light with a simple optical design that can be readily implemented on existing direct imaging instruments… ▽ More Vortex fiber nulling (VFN) is a single-aperture interferometric technique for detecting and characterizing exoplanets separated from their host star by less than a diffracted beam width. VFN uses a vortex mask and single mode fiber to selectively reject starlight while coupling off-axis planet light with a simple optical design that can be readily implemented on existing direct imaging instruments that can feed light to an optical fiber. With its axially symmetric coupling region peaking within the inner working angle of conventional coronagraphs, VFN is more efficient at detecting new companions at small separations than conventional direct imaging, thereby increasing the yield of on-going exoplanet search campaigns. We deployed a VFN mode operating in K band ($2.0{-}2.5~μ$m) on the Keck Planet Imager and Characterizer (KPIC) instrument at the Keck II Telescope. In this paper we present the instrument design of this first on-sky demonstration of VFN and the results from on-sky commissioning, including planet and star throughput measurements and predicted flux-ratio detection limits for close-in companions. The instrument performance is shown to be sufficient for detecting a companion $10^3$ times fainter than a $5^{\mathrm{th}}$ magnitude host star in 1 hour at a separation of 50 mas (1.1$λ/D$). This makes the instrument capable of efficiently detecting substellar companions around young stars. We also discuss several routes for improvement that will reduce the required integration time for a detection by a factor ${>}$3. △ Less

Submitted 12 September, 2023; originally announced September 2023.

Comments: 26 pages, 5 figures; Accepted to JATIS

Journal ref: Journal of Astronomical Telescopes, Instruments, and Systems, Vol. 9, Issue 3, 035002 (September 2023)

NMF-based GPU accelerated coronagraphy pipeline

Authors: Sai Krishanth P. M., Ewan S. Douglas, Justin Hom, Ramya M. Anche, John Debes, Isabel Rebollido, Bin B. Ren

Abstract: We present a generalized Non-negative factorization (NMF)-based data reduction pipeline for circumstellar disk and exoplanet detection. By using an adaptable pre-processing routine that applies algorithmic masks and corrections to improper data, we are able to easily offload the computationally-intensive NMF algorithm to a graphics processing unit (GPU), significantly increasing computational effi… ▽ More We present a generalized Non-negative factorization (NMF)-based data reduction pipeline for circumstellar disk and exoplanet detection. By using an adaptable pre-processing routine that applies algorithmic masks and corrections to improper data, we are able to easily offload the computationally-intensive NMF algorithm to a graphics processing unit (GPU), significantly increasing computational efficiency. NMF has been shown to better preserve disk structural features compared to other post-processing approaches and has demonstrated improvements in the analysis of archival data. The adaptive pre-processing routine of this pipeline, which automatically aligns and applies image corrections to the raw data, is shown to significantly improve chromatic halo suppression. Utilizing HST-STIS and JWST-MIRI coronagraphic datasets, we demonstrate a factor of five increase in real-time computational efficiency by using GPUs to perform NMF compared to using CPUs. Additionally, we demonstrate the usefulness of higher numbers of NMF components with SNR and contrast improvements, which necessitates the use of a more computationally efficient approach for data reduction. △ Less

Submitted 8 September, 2023; originally announced September 2023.

Journal ref: Proceedings Volume 12680, Techniques and Instrumentation for Detection of Exoplanets XI; 1268021 (2023)

Karhunen-Loève Data Imputation in High Contrast Imaging

Authors: Bin B. Ren

Abstract: Detection and characterization of extended structures is a crucial goal in high contrast imaging. However, these structures face challenges in data reduction, leading to over-subtraction from speckles and self-subtraction with most existing methods. Iterative post-processing methods offer promising results, but their integration into existing pipelines is hindered by selective algorithms, high com… ▽ More Detection and characterization of extended structures is a crucial goal in high contrast imaging. However, these structures face challenges in data reduction, leading to over-subtraction from speckles and self-subtraction with most existing methods. Iterative post-processing methods offer promising results, but their integration into existing pipelines is hindered by selective algorithms, high computational cost, and algorithmic regularization. To address this for reference differential imaging (RDI), here we propose the data imputation concept to Karhunen-Loève transform (DIKL) by modifying two steps in the standard Karhunen-Loève image projection (KLIP) method. Specifically, we partition an image to two matrices: an anchor matrix which focuses only on the speckles to obtain the DIKL coefficients, and a boat matrix which focuses on the regions of astrophysical interest for speckle removal using DIKL components. As an analytical approach, DIKL achieves high-quality results with significantly reduced computational cost (~3 orders of magnitude less than iterative methods). Being a derivative method of KLIP, DIKL is seamlessly integrable into high contrast imaging pipelines for RDI observations. △ Less

Submitted 31 August, 2023; originally announced August 2023.

Comments: 7 pages, 5 figures, A&A accepted

Journal ref: A&A 679, A18 (2023)

An inner warp discovered in the disk around HD 110058 using VLT/SPHERE and HST/STIS

Authors: S. Stasevic, J. Milli, J. Mazoyer, A. -M. Lagrange, M. Bonnefoy, V. Faramaz-Gorka, F. Ménard, A. Boccaletti, E. Choquet, L. Shuai, J. Olofsson, A. Chomez, B. Ren, P. Rubini, C. Desgrange, R. Gratton, G. Chauvin, A. Vigan, E. Matthews

Abstract: An edge-on debris disk was detected in 2015 around the young, nearby A0V star HD 110058. The disk showed features resembling those seen in the disk of beta Pictoris that could indicate the presence of a perturbing planetary-mass companion in the system. We investigated new and archival scattered light images of the disk in order to characterise its morphology and spectrum. In particular, we analys… ▽ More An edge-on debris disk was detected in 2015 around the young, nearby A0V star HD 110058. The disk showed features resembling those seen in the disk of beta Pictoris that could indicate the presence of a perturbing planetary-mass companion in the system. We investigated new and archival scattered light images of the disk in order to characterise its morphology and spectrum. In particular, we analysed the disk's warp to constrain the properties of possible planetary perturbers. Our work uses data from two VLT/SPHERE observations and archival data from HST/STIS. We measured the morphology of the disk by analysing vertical profiles along the length of the disk to extract the centroid spine position and vertical height. We extracted the surface brightness and reflectance spectrum of the disk. We detect the disk between 20 au (with SPHERE) and 150 au (with STIS), at a position angle of 159.6$^\circ\pm$0.6$^\circ$. Analysis of the spine shows an asymmetry between the two sides of the disk, with a 3.4$^\circ\pm$0.9$^\circ$ warp between ~20 au and 60 au. The disk is marginally vertically resolved in scattered light, with a vertical aspect ratio of 9.3$\pm$0.7% at 45 au. The extracted reflectance spectrum is featureless, flat between 0.95 micron and 1.1 micron, and red from 1.1 micron to 1.65 micron. The outer parts of the disk are also asymmetric with a tilt between the two sides compatible with a disk made of forward-scattering particles and seen not perfectly edge-on, suggesting an inclination of <84$^\circ$. The presence of an undetected planetary-mass companion on an inclined orbit with respect to the disk could explain the warp. The misalignment of the inner parts of the disk with respect to the outer disk suggests a warp that has not yet propagated to the outer parts of the disk, favouring the scenario of an inner perturber as the origin of the warp. △ Less

Submitted 10 August, 2023; originally announced August 2023.

Comments: 17 pages, 15 figures, 3 tables; accepted for publication in A&A

Journal ref: A&A 678, A8 (2023)