Showing 1–14 of 14 results for author: Intes, X

A 3D-Printed Table for Hybrid X-ray CT and Optical Imaging of a Live Mouse

Authors: Wenxuan Xue, Yuxuan Liang, Mengzhou Li, Shan Gao, Xavier R. Intes, Ge Wang

Abstract: Multimodal imaging has shown great potential in cancer research by concurrently providing anatomical, functional, and molecular information in live, intact animals. During preclinical imaging of small animals like mice, anesthesia is required to prevent movement and improve image quality. However, their high surface area-to-body weight ratio predisposes mice, particularly nude mice, to hypothermia… ▽ More Multimodal imaging has shown great potential in cancer research by concurrently providing anatomical, functional, and molecular information in live, intact animals. During preclinical imaging of small animals like mice, anesthesia is required to prevent movement and improve image quality. However, their high surface area-to-body weight ratio predisposes mice, particularly nude mice, to hypothermia under anesthesia. To address this, we developed a detachable mouse scanning table with heating function for hybrid x-ray and optical imaging modalities, without introducing metal artifacts. Specifically, we employed Polylactic Acid (PLA) 3D printing technology to fabricate a customized scanning table, compatible with both CT and optical imaging systems. This innovation enables seamless transportation of the table between different imaging setups, while its detachable design facilitates maintaining a clutter-free operational environment within the imaging systems. This is crucial for accommodating various projects within the same scanner. The table features positioned fixation points to secure mice, ensuring positional consistency across imaging modalities. Additionally, we integrated a carbon nanotube-based heating pad into the table to regulate the body temperature of mice during examinations, providing an ethical and effective temperature maintenance solution. Our evaluations confirmed the table's ability to maintain a 30g water bag at approximately 40$^\circ$C, effectively regulating mouse body temperature to an optimal 36$^\circ$C during preclinical imaging sessions. This scanning table serves as a useful tool in preclinical cancer research, offering a versatile tool that upholds animal welfare standards. △ Less

Submitted 9 October, 2024; originally announced October 2024.

Unlocking Real-Time Fluorescence Lifetime Imaging: Multi-Pixel Parallelism for FPGA-Accelerated Processing

Authors: Ismail Erbas, Aporva Amarnath, Vikas Pandey, Karthik Swaminathan, Naigang Wang, Xavier Intes

Abstract: Fluorescence lifetime imaging (FLI) is a widely used technique in the biomedical field for measuring the decay times of fluorescent molecules, providing insights into metabolic states, protein interactions, and ligand-receptor bindings. However, its broader application in fast biological processes, such as dynamic activity monitoring, and clinical use, such as in guided surgery, is limited by long… ▽ More Fluorescence lifetime imaging (FLI) is a widely used technique in the biomedical field for measuring the decay times of fluorescent molecules, providing insights into metabolic states, protein interactions, and ligand-receptor bindings. However, its broader application in fast biological processes, such as dynamic activity monitoring, and clinical use, such as in guided surgery, is limited by long data acquisition times and computationally demanding data processing. While deep learning has reduced post-processing times, time-resolved data acquisition remains a bottleneck for real-time applications. To address this, we propose a method to achieve real-time FLI using an FPGA-based hardware accelerator. Specifically, we implemented a GRU-based sequence-to-sequence (Seq2Seq) model on an FPGA board compatible with time-resolved cameras. The GRU model balances accurate processing with the resource constraints of FPGAs, which have limited DSP units and BRAM. The limited memory and computational resources on the FPGA require efficient scheduling of operations and memory allocation to deploy deep learning models for low-latency applications. We address these challenges by using STOMP, a queue-based discrete-event simulator that automates and optimizes task scheduling and memory management on hardware. By integrating a GRU-based Seq2Seq model and its compressed version, called Seq2SeqLite, generated through knowledge distillation, we were able to process multiple pixels in parallel, reducing latency compared to sequential processing. We explore various levels of parallelism to achieve an optimal balance between performance and resource utilization. Our results indicate that the proposed techniques achieved a 17.7x and 52.0x speedup over manual scheduling for the Seq2Seq model and the Seq2SeqLite model, respectively. △ Less

Submitted 9 October, 2024; originally announced October 2024.

Comments: 7 pages, 6 figures

Compressing Recurrent Neural Networks for FPGA-accelerated Implementation in Fluorescence Lifetime Imaging

Authors: Ismail Erbas, Vikas Pandey, Aporva Amarnath, Naigang Wang, Karthik Swaminathan, Stefan T. Radev, Xavier Intes

Abstract: Fluorescence lifetime imaging (FLI) is an important technique for studying cellular environments and molecular interactions, but its real-time application is limited by slow data acquisition, which requires capturing large time-resolved images and complex post-processing using iterative fitting algorithms. Deep learning (DL) models enable real-time inference, but can be computationally demanding d… ▽ More Fluorescence lifetime imaging (FLI) is an important technique for studying cellular environments and molecular interactions, but its real-time application is limited by slow data acquisition, which requires capturing large time-resolved images and complex post-processing using iterative fitting algorithms. Deep learning (DL) models enable real-time inference, but can be computationally demanding due to complex architectures and large matrix operations. This makes DL models ill-suited for direct implementation on field-programmable gate array (FPGA)-based camera hardware. Model compression is thus crucial for practical deployment for real-time inference generation. In this work, we focus on compressing recurrent neural networks (RNNs), which are well-suited for FLI time-series data processing, to enable deployment on resource-constrained FPGA boards. We perform an empirical evaluation of various compression techniques, including weight reduction, knowledge distillation (KD), post-training quantization (PTQ), and quantization-aware training (QAT), to reduce model size and computational load while preserving inference accuracy. Our compressed RNN model, Seq2SeqLite, achieves a balance between computational efficiency and prediction accuracy, particularly at 8-bit precision. By applying KD, the model parameter size was reduced by 98\% while retaining performance, making it suitable for concurrent real-time FLI analysis on FPGA during data capture. This work represents a big step towards integrating hardware-accelerated real-time FLI analysis for fast biological processes. △ Less

Submitted 1 October, 2024; originally announced October 2024.

Comments: 8 pages, 2 figures

Cognitive-Motor Integration in Assessing Bimanual Motor Skills

Authors: Erim Yanik, Xavier Intes, Suvranu De

Abstract: Accurate assessment of bimanual motor skills is essential across various professions, yet, traditional methods often rely on subjective assessments or focus solely on motor actions, overlooking the integral role of cognitive processes. This study introduces a novel approach by leveraging deep neural networks (DNNs) to analyze and integrate both cognitive decision-making and motor execution. We tes… ▽ More Accurate assessment of bimanual motor skills is essential across various professions, yet, traditional methods often rely on subjective assessments or focus solely on motor actions, overlooking the integral role of cognitive processes. This study introduces a novel approach by leveraging deep neural networks (DNNs) to analyze and integrate both cognitive decision-making and motor execution. We tested this methodology by assessing laparoscopic surgery skills within the Fundamentals of Laparoscopic Surgery program, which is a prerequisite for general surgery certification. Utilizing video capture of motor actions and non-invasive functional near-infrared spectroscopy (fNIRS) for measuring neural activations, our approach precisely classifies subjects by expertise level and predicts FLS behavioral performance scores, significantly surpassing traditional single-modality assessments. △ Less

Submitted 16 April, 2024; originally announced April 2024.

Comments: 12 pages, 3 figures, 2 tables

Roadmap on Deep Learning for Microscopy

Authors: Giovanni Volpe, Carolina Wählby, Lei Tian, Michael Hecht, Artur Yakimovich, Kristina Monakhova, Laura Waller, Ivo F. Sbalzarini, Christopher A. Metzler, Mingyang Xie, Kevin Zhang, Isaac C. D. Lenton, Halina Rubinsztein-Dunlop, Daniel Brunner, Bijie Bai, Aydogan Ozcan, Daniel Midtvedt, Hao Wang, Nataša Sladoje, Joakim Lindblad, Jason T. Smith, Marien Ochoa, Margarida Barroso, Xavier Intes, Tong Qiu , et al. (50 additional authors not shown)

Abstract: Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the… ▽ More Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap is written collectively by prominent researchers and encompasses selected aspects of how machine learning is applied to microscopy image data, with the aim of gaining scientific knowledge by improved image quality, automated detection, segmentation, classification and tracking of objects, and efficient merging of information from multiple imaging modalities. We aim to give the reader an overview of the key developments and an understanding of possibilities and limitations of machine learning for microscopy. It will be of interest to a wide cross-disciplinary audience in the physical sciences and life sciences. △ Less

Submitted 7 March, 2023; originally announced March 2023.

One-shot skill assessment in high-stakes domains with limited data via meta learning

Authors: Erim Yanik, Steven Schwaitzberg, Gene Yang, Xavier Intes, Jack Norfleet, Matthew Hackett, Suvranu De

Abstract: Deep Learning (DL) has achieved robust competency assessment in various high-stakes fields. However, the applicability of DL models is often hampered by their substantial data requirements and confinement to specific training domains. This prevents them from transitioning to new tasks where data is scarce. Therefore, domain adaptation emerges as a critical element for the practical implementation… ▽ More Deep Learning (DL) has achieved robust competency assessment in various high-stakes fields. However, the applicability of DL models is often hampered by their substantial data requirements and confinement to specific training domains. This prevents them from transitioning to new tasks where data is scarce. Therefore, domain adaptation emerges as a critical element for the practical implementation of DL in real-world scenarios. Herein, we introduce A-VBANet, a novel meta-learning model capable of delivering domain-agnostic skill assessment via one-shot learning. Our methodology has been tested by assessing surgical skills on five laparoscopic and robotic simulators and real-life laparoscopic cholecystectomy. Our model successfully adapted with accuracies up to 99.5% in one-shot and 99.9% in few-shot settings for simulated tasks and 89.7% for laparoscopic cholecystectomy. This study marks the first instance of a domain-agnostic methodology for skill assessment in critical fields setting a precedent for the broad application of DL across diverse real-life domains with limited data. △ Less

Submitted 19 April, 2024; v1 submitted 15 December, 2022; originally announced January 2023.

Comments: 23 pages (Main Manuscript + Supplementary Materials + Arxiv Logs), 4 figures (+2 Supplementary Figures), 2 tables (+5 Supplementary Tables)

Video-based Formative and Summative Assessment of Surgical Tasks using Deep Learning

Authors: Erim Yanik, Uwe Kruger, Xavier Intes, Rahul Rahul, Suvranu De

Abstract: To ensure satisfactory clinical outcomes, surgical skill assessment must be objective, time-efficient, and preferentially automated - none of which is currently achievable. Video-based assessment (VBA) is being deployed in intraoperative and simulation settings to evaluate technical skill execution. However, VBA remains manually- and time-intensive and prone to subjective interpretation and poor i… ▽ More To ensure satisfactory clinical outcomes, surgical skill assessment must be objective, time-efficient, and preferentially automated - none of which is currently achievable. Video-based assessment (VBA) is being deployed in intraoperative and simulation settings to evaluate technical skill execution. However, VBA remains manually- and time-intensive and prone to subjective interpretation and poor inter-rater reliability. Herein, we propose a deep learning (DL) model that can automatically and objectively provide a high-stakes summative assessment of surgical skill execution based on video feeds and low-stakes formative assessment to guide surgical skill acquisition. Formative assessment is generated using heatmaps of visual features that correlate with surgical performance. Hence, the DL model paves the way to the quantitative and reproducible evaluation of surgical tasks from videos with the potential for broad dissemination in surgical training, certification, and credentialing. △ Less

Submitted 17 March, 2022; originally announced March 2022.

Comments: 20 pages, 4 figures, 4 extended data figures, 3 tables, 4 extended data tables. Supplementary information is available

Deep Learning in fNIRS: A review

Authors: Condell Eastmond, Aseem Subedi, Suvranu De, Xavier Intes

Abstract: Significance: Optical neuroimaging has become a well-established clinical and research tool to monitor cortical activations in the human brain. It is notable that outcomes of functional Near-InfraRed Spectroscopy (fNIRS) studies depend heavily on the data processing pipeline and classification model employed. Recently, Deep Learning (DL) methodologies have demonstrated fast and accurate performanc… ▽ More Significance: Optical neuroimaging has become a well-established clinical and research tool to monitor cortical activations in the human brain. It is notable that outcomes of functional Near-InfraRed Spectroscopy (fNIRS) studies depend heavily on the data processing pipeline and classification model employed. Recently, Deep Learning (DL) methodologies have demonstrated fast and accurate performances in data processing and classification tasks across many biomedical fields. Aim: We aim to review the emerging DL applications in fNIRS studies. Approach: We first introduce some of the commonly used DL techniques. Then the review summarizes current DL work in some of the most active areas of this field, including brain-computer interface, neuro-impairment diagnosis, and neuroscience discovery. Results: Of the 63 papers considered in this review, 32 report a comparative study of deep learning techniques to traditional machine learning techniques where 26 have been shown outperforming the latter in terms of classification accuracy. Additionally, 8 studies also utilize deep learning to reduce the amount of preprocessing typically done with fNIRS data or increase the amount of data via data augmentation. Conclusions: The application of DL techniques to fNIRS studies has shown to mitigate many of the hurdles present in fNIRS studies such as lengthy data preprocessing or small sample sizes while achieving comparable or improved classification accuracy. △ Less

Submitted 15 July, 2022; v1 submitted 31 January, 2022; originally announced January 2022.

Comments: 41 pages, 9 figures

Deep learning in biomedical optics

Authors: Lei Tian, Brady Hunt, Muyinatu A. Lediju Bell, Ji Yi, Jason T. Smith, Marien Ochoa, Xavier Intes, Nicholas J. Durr

Abstract: This article reviews deep learning applications in biomedical optics with a particular emphasis on image formation. The review is organized by imaging domains within biomedical optics and includes microscopy, fluorescence lifetime imaging, in vivo microscopy, widefield endoscopy, optical coherence tomography, photoacoustic imaging, diffuse tomography, and functional optical brain imaging. For each… ▽ More This article reviews deep learning applications in biomedical optics with a particular emphasis on image formation. The review is organized by imaging domains within biomedical optics and includes microscopy, fluorescence lifetime imaging, in vivo microscopy, widefield endoscopy, optical coherence tomography, photoacoustic imaging, diffuse tomography, and functional optical brain imaging. For each of these domains, we summarize how deep learning has been applied and highlight methods by which deep learning can enable new capabilities for optics in medicine. Challenges and opportunities to improve translation and adoption of deep learning in biomedical optics are also summarized. △ Less

Submitted 23 May, 2021; originally announced May 2021.

Deep Neural Networks for the Assessment of Surgical Skills: A Systematic Review

Authors: Erim Yanik, Xavier Intes, Uwe Kruger, Pingkun Yan, David Miller, Brian Van Voorst, Basiel Makled, Jack Norfleet, Suvranu De

Abstract: Surgical training in medical school residency programs has followed the apprenticeship model. The learning and assessment process is inherently subjective and time-consuming. Thus, there is a need for objective methods to assess surgical skills. Here, we use the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines to systematically survey the literature on the use… ▽ More Surgical training in medical school residency programs has followed the apprenticeship model. The learning and assessment process is inherently subjective and time-consuming. Thus, there is a need for objective methods to assess surgical skills. Here, we use the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines to systematically survey the literature on the use of Deep Neural Networks for automated and objective surgical skill assessment, with a focus on kinematic data as putative markers of surgical competency. There is considerable recent interest in deep neural networks (DNN) due to the availability of powerful algorithms, multiple datasets, some of which are publicly available, as well as efficient computational hardware to train and host them. We have reviewed 530 papers, of which we selected 25 for this systematic review. Based on this review, we concluded that DNNs are powerful tools for automated, objective surgical skill assessment using both kinematic and video data. The field would benefit from large, publicly available, annotated datasets that are representative of the surgical trainee and expert demographics and multimodal data beyond kinematics and videos. △ Less

Submitted 3 March, 2021; originally announced March 2021.

Comments: 23 pages, 3 figures, 3 tables, Journal Paper

Decreasing the Surgical Errors by Neurostimulation of Primary Motor Cortex and the Associated Brain Activation via Neuroimaging

Authors: Yuanyuan Gao, Lora Cavuoto, Anirban Dutta, Uwe Kruger, Pingkun Yan, Arun Nemani, Jack E. Norfleet, Basiel A. Makled, Jessica Silvestri, Steven Schwaitzberg, Xavier Intes, Suvranu De

Abstract: Acquisition of fine motor skills is a time-consuming process as it requires frequent repetitions. Transcranial electrical stimulation is a promising means of enhancing simple motor skill development via neuromodulatory mechanisms. Here, we report that non-invasive neurostimulation facilitates the learning of complex fine bimanual motor skills associated with a surgical task. During the training of… ▽ More Acquisition of fine motor skills is a time-consuming process as it requires frequent repetitions. Transcranial electrical stimulation is a promising means of enhancing simple motor skill development via neuromodulatory mechanisms. Here, we report that non-invasive neurostimulation facilitates the learning of complex fine bimanual motor skills associated with a surgical task. During the training of 17 medical students on the Fundamentals of Laparoscopic Surgery (FLS) pattern cutting task over a period of 12 days, we observed that transcranial direct current stimulation (tDCS) decreased the error level and the variability in performance, compared to the Sham group. By concurrently monitoring the cortical activations of the subjects via functional near-infrared spectroscopy (fNIRS), our study showed that the cortical activation significantly stimulated by tDCS. The lowered performance error and the increased brain activation were retained after one-month post-training. This work supports the use of tDCS to enhance performance accuracy in fine bimanual motor tasks. △ Less

Submitted 8 October, 2020; originally announced October 2020.

Comments: 5 figures

MSC Class: 92-05

Deep Compressive Macroscopic Fluorescence Lifetime Imaging

Authors: Ruoyang Yao, Marien Ochoa, Xavier Intes, Pingkun Yan

Abstract: Compressive Macroscopic Fluorescence Lifetime Imaging (MFLI) is a novel technical implementation that enables monitoring multiple molecular interactions in macroscopic scale. Especially, we reported recently on the development of a hyperspectral wide-field time-resolved single-pixel imaging platform that facilitates whole-body in vivo lifetime imaging in less than 14 minutes. However, despite effi… ▽ More Compressive Macroscopic Fluorescence Lifetime Imaging (MFLI) is a novel technical implementation that enables monitoring multiple molecular interactions in macroscopic scale. Especially, we reported recently on the development of a hyperspectral wide-field time-resolved single-pixel imaging platform that facilitates whole-body in vivo lifetime imaging in less than 14 minutes. However, despite efficient data acquisition, the data processing of a Compressed Sensing (CS) based inversion plus lifetime fitting remain very time consuming. Herein, we propose to investigate the potential of deep learning for fast and accurate image formation. More precisely we developed a Convolutional Neural Network (CNN) called Net-FLICS (Network for Fluorescence Lifetime Imaging with Compressive Sensing) that reconstructs both intensity and lifetime images directly from raw CS measurements. Results show that better quality reconstruction can be obtained using Net-FLICS, for both simulation and experimental dataset, with almost negligible time compared to the traditional analytic methods. This first investigation suggests that Net-FLICS may be a powerful tool to enable CS-based lifetime imaging for real-time applications. △ Less

Submitted 22 February, 2018; v1 submitted 16 November, 2017; originally announced November 2017.

Comments: To present at IEEE ISBI 2018

Reconstruction of optical parameters for molecular tomographic imaging

Authors: Wenxiang Cong, Xavier Intes, Ge Wang

Abstract: Optical molecular tomographic imaging is to reconstruct the concentration distribution of photon-molecular probes in a small animal from measured photon fluence rates. The localization and quantification of molecular probes is related to tissue optical properties. The optical parameters can be reconstructed by the optimization method, in which the objective function is not convex, and there are mu… ▽ More Optical molecular tomographic imaging is to reconstruct the concentration distribution of photon-molecular probes in a small animal from measured photon fluence rates. The localization and quantification of molecular probes is related to tissue optical properties. The optical parameters can be reconstructed by the optimization method, in which the objective function is not convex, and there are multiple local optimal solutions. The gradient-based optimization algorithm could converge prematurely to an inaccurate solution if the initial guess is not close enough to the true solution. In this paper, we develop a hybrid algorithm combined global optimization and gradient-based optimization methods to reconstruct optical parameters. Differential evolution is an effective global optimizer, and could yield an estimated solution near the global optimal region to eliminate the need of initial guess of the optical parameters. The gradient-based optimization method is further performed with initial values obtained from DE to obtain local optimal solution for more accurate reconstruction of optical parameters. This proposed reconstruction method is evaluated by comprehensive numerical simulation experiments. The results show that the proposed method is stable and accurate for the reconstruction of optical parameters. △ Less

Submitted 15 July, 2017; v1 submitted 4 July, 2017; originally announced July 2017.

Optical Tomographic Imaging for Breast Cancer Detection

Authors: Wenxiang Cong, Xavier Intes, Ge Wang

Abstract: Diffuse optical breast imaging utilizes near-infrared (NIR) light propagation through tissues to assess the optical properties of tissue for the identification of abnormal tissue. This optical imaging approach is sensitive, cost-effective, and does not involve any ionizing radiation. However, the image reconstruction of diffuse optical tomography (DOT) is a nonlinear inverse problem and suffers fr… ▽ More Diffuse optical breast imaging utilizes near-infrared (NIR) light propagation through tissues to assess the optical properties of tissue for the identification of abnormal tissue. This optical imaging approach is sensitive, cost-effective, and does not involve any ionizing radiation. However, the image reconstruction of diffuse optical tomography (DOT) is a nonlinear inverse problem and suffers from severe ill-posedness, especially in the cases of strong noise and incomplete data. In this paper, a novel image reconstruction method is proposed for the detection of breast cancer. This method split the image reconstruction problem into the localization of abnormal tissues and quantification of absorption variations. The localization of abnormal tissues is performed based on a new well-posed optimization model, which can be solved via differential evolution optimization method to achieve a stable image reconstruction. The quantification of abnormal absorption variations is then determined in localized regions of relatively small extents, which are potentially tumors. Consequently, the number of unknown absorption variables can be greatly reduced to overcome the underdetermined nature of diffuse optical tomography (DOT), allowing for accurate and stable reconstruction of the abnormal absorption variations in the breast. Numerical simulation experiments show that the image reconstruction method is stable and accurate for the identification of abnormal tissues, and robust against measurement noise of data. △ Less

Submitted 16 April, 2017; originally announced April 2017.