-
Unlocking Real-Time Fluorescence Lifetime Imaging: Multi-Pixel Parallelism for FPGA-Accelerated Processing
Authors:
Ismail Erbas,
Aporva Amarnath,
Vikas Pandey,
Karthik Swaminathan,
Naigang Wang,
Xavier Intes
Abstract:
Fluorescence lifetime imaging (FLI) is a widely used technique in the biomedical field for measuring the decay times of fluorescent molecules, providing insights into metabolic states, protein interactions, and ligand-receptor bindings. However, its broader application in fast biological processes, such as dynamic activity monitoring, and clinical use, such as in guided surgery, is limited by long…
▽ More
Fluorescence lifetime imaging (FLI) is a widely used technique in the biomedical field for measuring the decay times of fluorescent molecules, providing insights into metabolic states, protein interactions, and ligand-receptor bindings. However, its broader application in fast biological processes, such as dynamic activity monitoring, and clinical use, such as in guided surgery, is limited by long data acquisition times and computationally demanding data processing. While deep learning has reduced post-processing times, time-resolved data acquisition remains a bottleneck for real-time applications. To address this, we propose a method to achieve real-time FLI using an FPGA-based hardware accelerator. Specifically, we implemented a GRU-based sequence-to-sequence (Seq2Seq) model on an FPGA board compatible with time-resolved cameras. The GRU model balances accurate processing with the resource constraints of FPGAs, which have limited DSP units and BRAM. The limited memory and computational resources on the FPGA require efficient scheduling of operations and memory allocation to deploy deep learning models for low-latency applications. We address these challenges by using STOMP, a queue-based discrete-event simulator that automates and optimizes task scheduling and memory management on hardware. By integrating a GRU-based Seq2Seq model and its compressed version, called Seq2SeqLite, generated through knowledge distillation, we were able to process multiple pixels in parallel, reducing latency compared to sequential processing. We explore various levels of parallelism to achieve an optimal balance between performance and resource utilization. Our results indicate that the proposed techniques achieved a 17.7x and 52.0x speedup over manual scheduling for the Seq2Seq model and the Seq2SeqLite model, respectively.
△ Less
Submitted 9 October, 2024;
originally announced October 2024.
-
Compressing Recurrent Neural Networks for FPGA-accelerated Implementation in Fluorescence Lifetime Imaging
Authors:
Ismail Erbas,
Vikas Pandey,
Aporva Amarnath,
Naigang Wang,
Karthik Swaminathan,
Stefan T. Radev,
Xavier Intes
Abstract:
Fluorescence lifetime imaging (FLI) is an important technique for studying cellular environments and molecular interactions, but its real-time application is limited by slow data acquisition, which requires capturing large time-resolved images and complex post-processing using iterative fitting algorithms. Deep learning (DL) models enable real-time inference, but can be computationally demanding d…
▽ More
Fluorescence lifetime imaging (FLI) is an important technique for studying cellular environments and molecular interactions, but its real-time application is limited by slow data acquisition, which requires capturing large time-resolved images and complex post-processing using iterative fitting algorithms. Deep learning (DL) models enable real-time inference, but can be computationally demanding due to complex architectures and large matrix operations. This makes DL models ill-suited for direct implementation on field-programmable gate array (FPGA)-based camera hardware. Model compression is thus crucial for practical deployment for real-time inference generation. In this work, we focus on compressing recurrent neural networks (RNNs), which are well-suited for FLI time-series data processing, to enable deployment on resource-constrained FPGA boards. We perform an empirical evaluation of various compression techniques, including weight reduction, knowledge distillation (KD), post-training quantization (PTQ), and quantization-aware training (QAT), to reduce model size and computational load while preserving inference accuracy. Our compressed RNN model, Seq2SeqLite, achieves a balance between computational efficiency and prediction accuracy, particularly at 8-bit precision. By applying KD, the model parameter size was reduced by 98\% while retaining performance, making it suitable for concurrent real-time FLI analysis on FPGA during data capture. This work represents a big step towards integrating hardware-accelerated real-time FLI analysis for fast biological processes.
△ Less
Submitted 1 October, 2024;
originally announced October 2024.
-
A Neuro-Symbolic Approach to Multi-Agent RL for Interpretability and Probabilistic Decision Making
Authors:
Chitra Subramanian,
Miao Liu,
Naweed Khan,
Jonathan Lenchner,
Aporva Amarnath,
Sarathkrishna Swaminathan,
Ryan Riegel,
Alexander Gray
Abstract:
Multi-agent reinforcement learning (MARL) is well-suited for runtime decision-making in optimizing the performance of systems where multiple agents coexist and compete for shared resources. However, applying common deep learning-based MARL solutions to real-world problems suffers from issues of interpretability, sample efficiency, partial observability, etc. To address these challenges, we present…
▽ More
Multi-agent reinforcement learning (MARL) is well-suited for runtime decision-making in optimizing the performance of systems where multiple agents coexist and compete for shared resources. However, applying common deep learning-based MARL solutions to real-world problems suffers from issues of interpretability, sample efficiency, partial observability, etc. To address these challenges, we present an event-driven formulation, where decision-making is handled by distributed co-operative MARL agents using neuro-symbolic methods. The recently introduced neuro-symbolic Logical Neural Networks (LNN) framework serves as a function approximator for the RL, to train a rules-based policy that is both logical and interpretable by construction. To enable decision-making under uncertainty and partial observability, we developed a novel probabilistic neuro-symbolic framework, Probabilistic Logical Neural Networks (PLNN), which combines the capabilities of logical reasoning with probabilistic graphical models. In PLNN, the upward/downward inference strategy, inherited from LNN, is coupled with belief bounds by setting the activation function for the logical operator associated with each neural network node to a probability-respecting generalization of the Fréchet inequalities. These PLNN nodes form the unifying element that combines probabilistic logic and Bayes Nets, permitting inference for variables with unobserved states. We demonstrate our contributions by addressing key MARL challenges for power sharing in a system-on-chip application.
△ Less
Submitted 20 February, 2024;
originally announced February 2024.
-
HetSched: Quality-of-Mission Aware Scheduling for Autonomous Vehicle SoCs
Authors:
Aporva Amarnath,
Subhankar Pal,
Hiwot Kassa,
Augusto Vega,
Alper Buyuktosunoglu,
Hubertus Franke,
John-David Wellman,
Ronald Dreslinski,
Pradip Bose
Abstract:
Systems-on-Chips (SoCs) that power autonomous vehicles (AVs) must meet stringent performance and safety requirements prior to deployment. With increasing complexity in AV applications, the system needs to meet these real-time demands of multiple safety-critical applications simultaneously. A typical AV-SoC is a heterogeneous multiprocessor consisting of accelerators supported by general-purpose co…
▽ More
Systems-on-Chips (SoCs) that power autonomous vehicles (AVs) must meet stringent performance and safety requirements prior to deployment. With increasing complexity in AV applications, the system needs to meet these real-time demands of multiple safety-critical applications simultaneously. A typical AV-SoC is a heterogeneous multiprocessor consisting of accelerators supported by general-purpose cores. Such heterogeneity, while needed for power-performance efficiency, complicates the art of task scheduling.
In this paper, we demonstrate that hardware heterogeneity impacts the scheduler's effectiveness and that optimizing for only the real-time aspect of applications is not sufficient in AVs. Therefore, a more holistic approach is required -- one that considers global Quality-of-Mission (QoM) metrics, as defined in the paper. We then propose HetSched, a multi-step scheduler that leverages dynamic runtime information about the underlying heterogeneous hardware platform, along with the applications' real-time constraints and the task traffic in the system to optimize overall mission performance. HetSched proposes two scheduling policies: MSstat and MSdyn and scheduling optimizations like task pruning, hybrid heterogeneous ranking and rank update. HetSched improves overall mission performance on average by 4.6x, 2.6x and 2.6x when compared against CPATH, ADS and 2lvl-EDF (state-of-the-art real-time schedulers built for heterogeneous systems), respectively, and achieves an average of 53.3% higher hardware utilization, while meeting 100% critical deadlines for real-world applications of autonomous vehicles. Furthermore, when used as part of an SoC design space exploration loop, in comparison to prior schedulers, HetSched reduces the number of processing elements required by an SoC to safely complete AV's missions by 35% on average while achieving 2.7x lower energy-mission time product.
△ Less
Submitted 24 March, 2022;
originally announced March 2022.
-
STOMP: A Tool for Evaluation of Scheduling Policies in Heterogeneous Multi-Processors
Authors:
Augusto Vega,
Aporva Amarnath,
John-David Wellman,
Hiwot Kassa,
Subhankar Pal,
Hubertus Franke,
Alper Buyuktosunoglu,
Ronald Dreslinski,
Pradip Bose
Abstract:
The proliferation of heterogeneous chip multiprocessors in recent years has reached unprecedented levels. Traditional homogeneous platforms have shown fundamental limitations when it comes to enabling high-performance yet-ultra-low-power computing, in particular in application domains with real-time execution deadlines or criticality constraints. By combining the right set of general purpose cores…
▽ More
The proliferation of heterogeneous chip multiprocessors in recent years has reached unprecedented levels. Traditional homogeneous platforms have shown fundamental limitations when it comes to enabling high-performance yet-ultra-low-power computing, in particular in application domains with real-time execution deadlines or criticality constraints. By combining the right set of general purpose cores and hardware accelerators together, along with proper chip interconnects and memory technology, heterogeneous chip multiprocessors have become an effective high-performance and low-power computing alternative.
One of the challenges of heterogeneous architectures relates to efficient scheduling of application tasks (processes, threads) across the variety of options in the chip. As a result, it is key to provide tools to enable early-stage prototyping and evaluation of new scheduling policies for heterogeneous platforms. In this paper, we present STOMP (Scheduling Techniques Optimization in heterogeneous Multi-Processors), a simulator for fast implementation and evaluation of task scheduling policies in multi-core/multi-processor systems with a convenient interface for "plugging" in new scheduling policies in a simple manner. Thorough validation of STOMP exhibits small relative errors when compared against closed-formed equivalent models during steady-state analysis.
△ Less
Submitted 28 July, 2020;
originally announced July 2020.
-
Improved Bounds for Uniform Hypergraphs without Property B
Authors:
Sachin Aglave,
V. A. Amarnath,
Saswata Shannigrahi,
Shwetank Singh
Abstract:
A hypergraph is said to be properly 2-colorable if there exists a 2-coloring of its vertices such that no hyperedge is monochromatic. On the other hand, a hypergraph is called non-2-colorable if there exists at least one monochromatic hyperedge in each of the possible 2-colorings of its vertex set. Let m(n) denote the minimum number of hyperedges in a non-2-colorable n-uniform hypergraph. Establis…
▽ More
A hypergraph is said to be properly 2-colorable if there exists a 2-coloring of its vertices such that no hyperedge is monochromatic. On the other hand, a hypergraph is called non-2-colorable if there exists at least one monochromatic hyperedge in each of the possible 2-colorings of its vertex set. Let m(n) denote the minimum number of hyperedges in a non-2-colorable n-uniform hypergraph. Establishing the lower and upper bounds on m(n) is a well-studied research direction over several decades. In this paper, we present new constructions for non-2-colorable n-uniform hypergraphs. These constructions improve the upper bounds for m(8), m(13), m(14), m(16) and m(17). We also improve the lower bound for m(5).
△ Less
Submitted 11 June, 2019; v1 submitted 31 January, 2016;
originally announced February 2016.
