-
MFIT: Multi-Fidelity Thermal Modeling for 2.5D and 3D Multi-Chiplet Architectures
Authors:
Lukas Pfromm,
Alish Kanani,
Harsh Sharma,
Parth Solanki,
Eric Tervo,
Jaehyun Park,
Janardhan Rao Doppa,
Partha Pratim Pande,
Umit Y. Ogras
Abstract:
Rapidly evolving artificial intelligence and machine learning applications require ever-increasing computational capabilities, while monolithic 2D design technologies approach their limits. Heterogeneous integration of smaller chiplets using a 2.5D silicon interposer and 3D packaging has emerged as a promising paradigm to address this limit and meet performance demands. These approaches offer a si…
▽ More
Rapidly evolving artificial intelligence and machine learning applications require ever-increasing computational capabilities, while monolithic 2D design technologies approach their limits. Heterogeneous integration of smaller chiplets using a 2.5D silicon interposer and 3D packaging has emerged as a promising paradigm to address this limit and meet performance demands. These approaches offer a significant cost reduction and higher manufacturing yield than monolithic 2D integrated circuits. However, the compact arrangement and high compute density exacerbate the thermal management challenges, potentially compromising performance. Addressing these thermal modeling challenges is critical, especially as system sizes grow and different design stages require varying levels of accuracy and speed. Since no single thermal modeling technique meets all these needs, this paper introduces MFIT, a range of multi-fidelity thermal models that effectively balance accuracy and speed. These multi-fidelity models can enable efficient design space exploration and runtime thermal management. Our extensive testing on systems with 16, 36, and 64 2.5D integrated chiplets and 16x3 3D integrated chiplets demonstrates that these models can reduce execution times from days to mere seconds and milliseconds with negligible loss in accuracy.
△ Less
Submitted 11 October, 2024;
originally announced October 2024.
-
Machine Learning-based Low Overhead Congestion Control Algorithm for Industrial NoCs
Authors:
Shruti Yadav Narayana,
Sumit K. Mandal,
Raid Ayoub,
Michael Kishinevsky,
Umit Y. Ogras
Abstract:
Network-on-Chip (NoC) congestion builds up during heavy traffic load and cripples the system performance by stalling the cores. Moreover, congestion leads to wasted link bandwidth due to blocked buffers and bouncing packets. Existing approaches throttle the cores after congestion is detected, reducing efficiency and wasting line bandwidth unnecessarily. In contrast, we propose a lightweight machin…
▽ More
Network-on-Chip (NoC) congestion builds up during heavy traffic load and cripples the system performance by stalling the cores. Moreover, congestion leads to wasted link bandwidth due to blocked buffers and bouncing packets. Existing approaches throttle the cores after congestion is detected, reducing efficiency and wasting line bandwidth unnecessarily. In contrast, we propose a lightweight machine learning-based technique that helps predict congestion in the network. Specifically, our proposed technique collects the features related to traffic at each destination. Then, it labels the features using a novel time reversal approach. The labeled data is used to design a low overhead and an explainable decision tree model used at runtime congestion control. Experimental evaluations with synthetic and real traffic on industrial 6$\times$6 NoC show that the proposed approach increases fairness and memory read bandwidth by up to 114\% with respect to existing congestion control technique while incurring less than 0.01\% of overhead.
△ Less
Submitted 24 February, 2023;
originally announced February 2023.
-
PD-MORL: Preference-Driven Multi-Objective Reinforcement Learning Algorithm
Authors:
Toygun Basaklar,
Suat Gumussoy,
Umit Y. Ogras
Abstract:
Multi-objective reinforcement learning (MORL) approaches have emerged to tackle many real-world problems with multiple conflicting objectives by maximizing a joint objective function weighted by a preference vector. These approaches find fixed customized policies corresponding to preference vectors specified during training. However, the design constraints and objectives typically change dynamical…
▽ More
Multi-objective reinforcement learning (MORL) approaches have emerged to tackle many real-world problems with multiple conflicting objectives by maximizing a joint objective function weighted by a preference vector. These approaches find fixed customized policies corresponding to preference vectors specified during training. However, the design constraints and objectives typically change dynamically in real-life scenarios. Furthermore, storing a policy for each potential preference is not scalable. Hence, obtaining a set of Pareto front solutions for the entire preference space in a given domain with a single training is critical. To this end, we propose a novel MORL algorithm that trains a single universal network to cover the entire preference space scalable to continuous robotic tasks. The proposed approach, Preference-Driven MORL (PD-MORL), utilizes the preferences as guidance to update the network parameters. It also employs a novel parallelization approach to increase sample efficiency. We show that PD-MORL achieves up to 25% larger hypervolume for challenging continuous control tasks and uses an order of magnitude fewer trainable parameters compared to prior approaches.
△ Less
Submitted 29 May, 2023; v1 submitted 16 August, 2022;
originally announced August 2022.
-
COIN: Communication-Aware In-Memory Acceleration for Graph Convolutional Networks
Authors:
Sumit K. Mandal,
Gokul Krishnan,
A. Alper Goksoy,
Gopikrishnan Ravindran Nair,
Yu Cao,
Umit Y. Ogras
Abstract:
Graph convolutional networks (GCNs) have shown remarkable learning capabilities when processing graph-structured data found inherently in many application areas. GCNs distribute the outputs of neural networks embedded in each vertex over multiple iterations to take advantage of the relations captured by the underlying graphs. Consequently, they incur a significant amount of computation and irregul…
▽ More
Graph convolutional networks (GCNs) have shown remarkable learning capabilities when processing graph-structured data found inherently in many application areas. GCNs distribute the outputs of neural networks embedded in each vertex over multiple iterations to take advantage of the relations captured by the underlying graphs. Consequently, they incur a significant amount of computation and irregular communication overheads, which call for GCN-specific hardware accelerators. To this end, this paper presents a communication-aware in-memory computing architecture (COIN) for GCN hardware acceleration. Besides accelerating the computation using custom compute elements (CE) and in-memory computing, COIN aims at minimizing the intra- and inter-CE communication in GCN operations to optimize the performance and energy efficiency. Experimental evaluations with widely used datasets show up to 105x improvement in energy consumption compared to state-of-the-art GCN accelerator.
△ Less
Submitted 15 May, 2022;
originally announced May 2022.
-
Fast and Scalable Human Pose Estimation using mmWave Point Cloud
Authors:
Sizhe An,
Umit Y. Ogras
Abstract:
Millimeter-Wave (mmWave) radar can enable high-resolution human pose estimation with low cost and computational requirements. However, mmWave data point cloud, the primary input to processing algorithms, is highly sparse and carries significantly less information than other alternatives such as video frames. Furthermore, the scarce labeled mmWave data impedes the development of machine learning (M…
▽ More
Millimeter-Wave (mmWave) radar can enable high-resolution human pose estimation with low cost and computational requirements. However, mmWave data point cloud, the primary input to processing algorithms, is highly sparse and carries significantly less information than other alternatives such as video frames. Furthermore, the scarce labeled mmWave data impedes the development of machine learning (ML) models that can generalize to unseen scenarios. We propose a fast and scalable human pose estimation (FUSE) framework that combines multi-frame representation and meta-learning to address these challenges. Experimental evaluations show that FUSE adapts to the unseen scenarios 4$\times$ faster than current supervised learning approaches and estimates human joint coordinates with about 7 cm mean absolute error.
△ Less
Submitted 29 April, 2022;
originally announced May 2022.
-
tinyMAN: Lightweight Energy Manager using Reinforcement Learning for Energy Harvesting Wearable IoT Devices
Authors:
Toygun Basaklar,
Yigit Tuncel,
Umit Y. Ogras
Abstract:
Advances in low-power electronics and machine learning techniques lead to many novel wearable IoT devices. These devices have limited battery capacity and computational power. Thus, energy harvesting from ambient sources is a promising solution to power these low-energy wearable devices. They need to manage the harvested energy optimally to achieve energy-neutral operation, which eliminates rechar…
▽ More
Advances in low-power electronics and machine learning techniques lead to many novel wearable IoT devices. These devices have limited battery capacity and computational power. Thus, energy harvesting from ambient sources is a promising solution to power these low-energy wearable devices. They need to manage the harvested energy optimally to achieve energy-neutral operation, which eliminates recharging requirements. Optimal energy management is a challenging task due to the dynamic nature of the harvested energy and the battery energy constraints of the target device. To address this challenge, we present a reinforcement learning-based energy management framework, tinyMAN, for resource-constrained wearable IoT devices. The framework maximizes the utilization of the target device under dynamic energy harvesting patterns and battery constraints. Moreover, tinyMAN does not rely on forecasts of the harvested energy which makes it a prediction-free approach. We deployed tinyMAN on a wearable device prototype using TensorFlow Lite for Micro thanks to its small memory footprint of less than 100 KB. Our evaluations show that tinyMAN achieves less than 2.36 ms and 27.75 $μ$J while maintaining up to 45% higher utility compared to prior approaches.
△ Less
Submitted 18 March, 2022; v1 submitted 18 February, 2022;
originally announced February 2022.
-
Performant, Multi-objective Scheduling of Highly Interleaved Task Graphs on Heterogeneous System on Chip Devices
Authors:
Joshua Mack,
Samet E. Arda,
Umit Y. Ogras,
Ali Akoglu
Abstract:
Performance-, power-, and energy-aware scheduling techniques play an essential role in optimally utilizing processing elements (PEs) of heterogeneous systems. List schedulers, a class of low-complexity static schedulers, have commonly been used in static execution scenarios. However, list schedulers are not suitable for runtime decision making, particularly when multiple concurrent applications ar…
▽ More
Performance-, power-, and energy-aware scheduling techniques play an essential role in optimally utilizing processing elements (PEs) of heterogeneous systems. List schedulers, a class of low-complexity static schedulers, have commonly been used in static execution scenarios. However, list schedulers are not suitable for runtime decision making, particularly when multiple concurrent applications are interleaved dynamically. For such cases, the static task execution times and expectation of idle PEs assumed by list schedulers lead to inefficient system utilization and poor performance. To address this problem, we present techniques for optimizing execution of list scheduling algorithms in dynamic runtime scenarios via a family of algorithms inspired by the well-known heterogeneous earliest finish time (HEFT) list scheduler. Through dynamically arriving, realistic workload scenarios that are simulated in an open-source discrete event heterogeneous SoC simulator, we exhaustively evaluate each of the proposed algorithms across two SoCs modeled after the Xilinx Zynq Ultrascale+ ZCU102 and O-Droid XU3 development boards. Altogether, depending on the chosen variant in this family of algorithms, we are able to achieve an up to 39% execution time improvement, up to 7.24x algorithmic speedup, or up to 30% energy consumption improvement compared to the baseline HEFT implementation.
△ Less
Submitted 16 December, 2021;
originally announced December 2021.
-
DAS: Dynamic Adaptive Scheduling for Energy-Efficient Heterogeneous SoCs
Authors:
A. Alper Goksoy,
Anish Krishnakumar,
Md Sahil Hassan,
Allen J. Farcas,
Ali Akoglu,
Radu Marculescu,
Umit Y. Ogras
Abstract:
Domain-specific systems-on-chip (DSSoCs) aim at bridging the gap between application-specific integrated circuits (ASICs) and general-purpose processors. Traditional operating system (OS) schedulers can undermine the potential of DSSoCs since their execution times can be orders of magnitude larger than the execution time of the task itself. To address this problem, we propose a dynamic adaptive sc…
▽ More
Domain-specific systems-on-chip (DSSoCs) aim at bridging the gap between application-specific integrated circuits (ASICs) and general-purpose processors. Traditional operating system (OS) schedulers can undermine the potential of DSSoCs since their execution times can be orders of magnitude larger than the execution time of the task itself. To address this problem, we propose a dynamic adaptive scheduling (DAS) framework that combines the benefits of a fast (low-overhead) scheduler and a slow (sophisticated, high-performance but high-overhead) scheduler. Experiments with five real-world streaming applications show that DAS consistently outperforms both the fast and slow schedulers. For 40 different workloads, DAS achieves on average 1.29x speedup and 45% lower EDP compared to the sophisticated scheduler at low data rates and 1.28x speedup and 37% lower EDP than the fast scheduler when the workload complexity increases.
△ Less
Submitted 22 September, 2021;
originally announced September 2021.
-
Theoretical Analysis and Evaluation of NoCs with Weighted Round-Robin Arbitration
Authors:
Sumit K. Mandal,
Jie Tong,
Raid Ayoub,
Michael Kishinevsky,
Ahmed Abousamra,
Umit Y. Ogras
Abstract:
Fast and accurate performance analysis techniques are essential in early design space exploration and pre-silicon evaluations, including software eco-system development. In particular, on-chip communication continues to play an increasingly important role as the many-core processors scale up. This paper presents the first performance analysis technique that targets networks-on-chip (NoCs) that emp…
▽ More
Fast and accurate performance analysis techniques are essential in early design space exploration and pre-silicon evaluations, including software eco-system development. In particular, on-chip communication continues to play an increasingly important role as the many-core processors scale up. This paper presents the first performance analysis technique that targets networks-on-chip (NoCs) that employ weighted round-robin (WRR) arbitration. Besides fairness, WRR arbitration provides flexibility in allocating bandwidth proportionally to the importance of the traffic classes, unlike basic round-robin and priority-based arbitration. The proposed approach first estimates the effective service time of the packets in the queue due to WRR arbitration. Then, it uses the effective service time to compute the average waiting time of the packets. Next, we incorporate a decomposition technique to extend the analytical model to handle NoC of any size. The proposed approach achieves less than 5% error while executing real applications and 10% error under challenging synthetic traffic with different burstiness levels.
△ Less
Submitted 11 August, 2023; v1 submitted 21 August, 2021;
originally announced August 2021.
-
SIAM: Chiplet-based Scalable In-Memory Acceleration with Mesh for Deep Neural Networks
Authors:
Gokul Krishnan,
Sumit K. Mandal,
Manvitha Pannala,
Chaitali Chakrabarti,
Jae-sun Seo,
Umit Y. Ogras,
Yu Cao
Abstract:
In-memory computing (IMC) on a monolithic chip for deep learning faces dramatic challenges on area, yield, and on-chip interconnection cost due to the ever-increasing model sizes. 2.5D integration or chiplet-based architectures interconnect multiple small chips (i.e., chiplets) to form a large computing system, presenting a feasible solution beyond a monolithic IMC architecture to accelerate large…
▽ More
In-memory computing (IMC) on a monolithic chip for deep learning faces dramatic challenges on area, yield, and on-chip interconnection cost due to the ever-increasing model sizes. 2.5D integration or chiplet-based architectures interconnect multiple small chips (i.e., chiplets) to form a large computing system, presenting a feasible solution beyond a monolithic IMC architecture to accelerate large deep learning models. This paper presents a new benchmarking simulator, SIAM, to evaluate the performance of chiplet-based IMC architectures and explore the potential of such a paradigm shift in IMC architecture design. SIAM integrates device, circuit, architecture, network-on-chip (NoC), network-on-package (NoP), and DRAM access models to realize an end-to-end system. SIAM is scalable in its support of a wide range of deep neural networks (DNNs), customizable to various network structures and configurations, and capable of efficient design space exploration. We demonstrate the flexibility, scalability, and simulation speed of SIAM by benchmarking different state-of-the-art DNNs with CIFAR-10, CIFAR-100, and ImageNet datasets. We further calibrate the simulation results with a published silicon result, SIMBA. The chiplet-based IMC architecture obtained through SIAM shows 130$\times$ and 72$\times$ improvement in energy-efficiency for ResNet-50 on the ImageNet dataset compared to Nvidia V100 and T4 GPUs.
△ Less
Submitted 14 August, 2021;
originally announced August 2021.
-
FLASH: Fast Neural Architecture Search with Hardware Optimization
Authors:
Guihong Li,
Sumit K. Mandal,
Umit Y. Ogras,
Radu Marculescu
Abstract:
Neural architecture search (NAS) is a promising technique to design efficient and high-performance deep neural networks (DNNs). As the performance requirements of ML applications grow continuously, the hardware accelerators start playing a central role in DNN design. This trend makes NAS even more complicated and time-consuming for most real applications. This paper proposes FLASH, a very fast NAS…
▽ More
Neural architecture search (NAS) is a promising technique to design efficient and high-performance deep neural networks (DNNs). As the performance requirements of ML applications grow continuously, the hardware accelerators start playing a central role in DNN design. This trend makes NAS even more complicated and time-consuming for most real applications. This paper proposes FLASH, a very fast NAS methodology that co-optimizes the DNN accuracy and performance on a real hardware platform. As the main theoretical contribution, we first propose the NN-Degree, an analytical metric to quantify the topological characteristics of DNNs with skip connections (e.g., DenseNets, ResNets, Wide-ResNets, and MobileNets). The newly proposed NN-Degree allows us to do training-free NAS within one second and build an accuracy predictor by training as few as 25 samples out of a vast search space with more than 63 billion configurations. Second, by performing inference on the target hardware, we fine-tune and validate our analytical models to estimate the latency, area, and energy consumption of various DNN architectures while executing standard ML datasets. Third, we construct a hierarchical algorithm based on simplicial homology global optimization (SHGO) to optimize the model-architecture co-design process, while considering the area, latency, and energy consumption of the target hardware. We demonstrate that, compared to the state-of-the-art NAS approaches, our proposed hierarchical SHGO-based algorithm enables more than four orders of magnitude speedup (specifically, the execution time of the proposed algorithm is about 0.1 seconds). Finally, our experimental evaluations show that FLASH is easily transferable to different hardware architectures, thus enabling us to do NAS on a Raspberry Pi-3B processor in less than 3 seconds.
△ Less
Submitted 1 August, 2021;
originally announced August 2021.
-
Impact of On-Chip Interconnect on In-Memory Acceleration of Deep Neural Networks
Authors:
Gokul Krishnan,
Sumit K. Mandal,
Chaitali Chakrabarti,
Jae-sun Seo,
Umit Y. Ogras,
Yu Cao
Abstract:
With the widespread use of Deep Neural Networks (DNNs), machine learning algorithms have evolved in two diverse directions -- one with ever-increasing connection density for better accuracy and the other with more compact sizing for energy efficiency. The increase in connection density increases on-chip data movement, which makes efficient on-chip communication a critical function of the DNN accel…
▽ More
With the widespread use of Deep Neural Networks (DNNs), machine learning algorithms have evolved in two diverse directions -- one with ever-increasing connection density for better accuracy and the other with more compact sizing for energy efficiency. The increase in connection density increases on-chip data movement, which makes efficient on-chip communication a critical function of the DNN accelerator. The contribution of this work is threefold. First, we illustrate that the point-to-point (P2P)-based interconnect is incapable of handling a high volume of on-chip data movement for DNNs. Second, we evaluate P2P and network-on-chip (NoC) interconnect (with a regular topology such as a mesh) for SRAM- and ReRAM-based in-memory computing (IMC) architectures for a range of DNNs. This analysis shows the necessity for the optimal interconnect choice for an IMC DNN accelerator. Finally, we perform an experimental evaluation for different DNNs to empirically obtain the performance of the IMC architecture with both NoC-tree and NoC-mesh. We conclude that, at the tile level, NoC-tree is appropriate for compact DNNs employed at the edge, and NoC-mesh is necessary to accelerate DNNs with high connection density. Furthermore, we propose a technique to determine the optimal choice of interconnect for any given DNN. In this technique, we use analytical models of NoC to evaluate end-to-end communication latency of any given DNN. We demonstrate that the interconnect optimization in the IMC architecture results in up to 6$\times$ improvement in energy-delay-area product for VGG-19 inference compared to the state-of-the-art ReRAM-based IMC architectures.
△ Less
Submitted 5 July, 2021;
originally announced July 2021.
-
Hypervector Design for Efficient Hyperdimensional Computing on Edge Devices
Authors:
Toygun Basaklar,
Yigit Tuncel,
Shruti Yadav Narayana,
Suat Gumussoy,
Umit Y. Ogras
Abstract:
Hyperdimensional computing (HDC) has emerged as a new light-weight learning algorithm with smaller computation and energy requirements compared to conventional techniques. In HDC, data points are represented by high-dimensional vectors (hypervectors), which are mapped to high-dimensional space (hyperspace). Typically, a large hypervector dimension ($\geq1000$) is required to achieve accuracies com…
▽ More
Hyperdimensional computing (HDC) has emerged as a new light-weight learning algorithm with smaller computation and energy requirements compared to conventional techniques. In HDC, data points are represented by high-dimensional vectors (hypervectors), which are mapped to high-dimensional space (hyperspace). Typically, a large hypervector dimension ($\geq1000$) is required to achieve accuracies comparable to conventional alternatives. However, unnecessarily large hypervectors increase hardware and energy costs, which can undermine their benefits. This paper presents a technique to minimize the hypervector dimension while maintaining the accuracy and improving the robustness of the classifier. To this end, we formulate the hypervector design as a multi-objective optimization problem for the first time in the literature. The proposed approach decreases the hypervector dimension by more than $32\times$ while maintaining or increasing the accuracy achieved by conventional HDC. Experiments on a commercial hardware platform show that the proposed approach achieves more than one order of magnitude reduction in model size, inference time, and energy consumption. We also demonstrate the trade-off between accuracy and robustness to noise and provide Pareto front solutions as a design parameter in our hypervector design.
△ Less
Submitted 8 March, 2021;
originally announced March 2021.
-
Online Adaptive Learning for Runtime Resource Management of Heterogeneous SoCs
Authors:
Sumit K. Mandal,
Umit Y. Ogras,
Janardhan Rao Doppa,
Raid Z. Ayoub,
Michael Kishinevsky,
Partha P. Pande
Abstract:
Dynamic resource management has become one of the major areas of research in modern computer and communication system design due to lower power consumption and higher performance demands. The number of integrated cores, level of heterogeneity and amount of control knobs increase steadily. As a result, the system complexity is increasing faster than our ability to optimize and dynamically manage th…
▽ More
Dynamic resource management has become one of the major areas of research in modern computer and communication system design due to lower power consumption and higher performance demands. The number of integrated cores, level of heterogeneity and amount of control knobs increase steadily. As a result, the system complexity is increasing faster than our ability to optimize and dynamically manage the resources. Moreover, offline approaches are sub-optimal due to workload variations and large volume of new applications unknown at design time. This paper first reviews recent online learning techniques for predicting system performance, power, and temperature. Then, we describe the use of predictive models for online control using two modern approaches: imitation learning (IL) and an explicit nonlinear model predictive control (NMPC). Evaluations on a commercial mobile platform with 16 benchmarks show that the IL approach successfully adapts the control policy to unknown applications. The explicit NMPC provides 25% energy savings compared to a state-of-the-art algorithm for multi-variable power management of modern GPU sub-systems.
△ Less
Submitted 21 August, 2020;
originally announced August 2020.
-
Performance Analysis of Priority-Aware NoCs with Deflection Routing under Traffic Congestion
Authors:
Sumit K. Mandal,
Anish Krishnakumar,
Raid Ayoub,
Michael Kishinevsky,
Umit Y. Ogras
Abstract:
Priority-aware networks-on-chip (NoCs) are used in industry to achieve predictable latency under different workload conditions. These NoCs incorporate deflection routing to minimize queuing resources within routers and achieve low latency during low traffic load. However, deflected packets can exacerbate congestion during high traffic load since they consume the NoC bandwidth. State-of-the-art ana…
▽ More
Priority-aware networks-on-chip (NoCs) are used in industry to achieve predictable latency under different workload conditions. These NoCs incorporate deflection routing to minimize queuing resources within routers and achieve low latency during low traffic load. However, deflected packets can exacerbate congestion during high traffic load since they consume the NoC bandwidth. State-of-the-art analytical models for priority-aware NoCs ignore deflected traffic despite its significant latency impact during congestion. This paper proposes a novel analytical approach to estimate end-to-end latency of priority-aware NoCs with deflection routing under bursty and heavy traffic scenarios. Experimental evaluations show that the proposed technique outperforms alternative approaches and estimates the average latency for real applications with less than 8% error compared to cycle-accurate simulations.
△ Less
Submitted 8 November, 2020; v1 submitted 10 August, 2020;
originally announced August 2020.
-
Analytical Performance Modeling of NoCs under Priority Arbitration and Bursty Traffic
Authors:
Sumit K. Mandal,
Raid Ayoub,
Michael Kishinevsky,
Mohammad M. Islam,
Umit Y. Ogras
Abstract:
Networks-on-Chip (NoCs) used in commercial many-core processors typically incorporate priority arbitration. Moreover, they experience bursty traffic due to application workloads. However, most state-of-the-art NoC analytical performance analysis techniques assume fair arbitration and simple traffic models. To address these limitations, we propose an analytical modeling technique for priority-aware…
▽ More
Networks-on-Chip (NoCs) used in commercial many-core processors typically incorporate priority arbitration. Moreover, they experience bursty traffic due to application workloads. However, most state-of-the-art NoC analytical performance analysis techniques assume fair arbitration and simple traffic models. To address these limitations, we propose an analytical modeling technique for priority-aware NoCs under bursty traffic. Experimental evaluations with synthetic and bursty traffic show that the proposed approach has less than 10% modeling error with respect to cycle-accurate NoC simulator.
△ Less
Submitted 27 July, 2020;
originally announced July 2020.
-
Runtime Task Scheduling using Imitation Learning for Heterogeneous Many-Core Systems
Authors:
Anish Krishnakumar,
Samet E. Arda,
A. Alper Goksoy,
Sumit K. Mandal,
Umit Y. Ogras,
Anderson L. Sartor,
Radu Marculescu
Abstract:
Domain-specific systems-on-chip, a class of heterogeneous many-core systems, are recognized as a key approach to narrow down the performance and energy-efficiency gap between custom hardware accelerators and programmable processors. Reaching the full potential of these architectures depends critically on optimally scheduling the applications to available resources at runtime. Existing optimization…
▽ More
Domain-specific systems-on-chip, a class of heterogeneous many-core systems, are recognized as a key approach to narrow down the performance and energy-efficiency gap between custom hardware accelerators and programmable processors. Reaching the full potential of these architectures depends critically on optimally scheduling the applications to available resources at runtime. Existing optimization-based techniques cannot achieve this objective at runtime due to the combinatorial nature of the task scheduling problem. As the main theoretical contribution, this paper poses scheduling as a classification problem and proposes a hierarchical imitation learning (IL)-based scheduler that learns from an Oracle to maximize the performance of multiple domain-specific applications. Extensive evaluations with six streaming applications from wireless communications and radar domains show that the proposed IL-based scheduler approximates an offline Oracle policy with more than 99% accuracy for performance- and energy-based optimization objectives. Furthermore, it achieves almost identical performance to the Oracle with a low runtime overhead and successfully adapts to new applications, many-core system configurations, and runtime variations in application characteristics.
△ Less
Submitted 6 August, 2020; v1 submitted 18 July, 2020;
originally announced July 2020.
-
User-Space Emulation Framework for Domain-Specific SoC Design
Authors:
Joshua Mack,
Nirmal Kumbhare,
Anish NK,
Umit Y. Ogras,
Ali Akoglu
Abstract:
In this work, we propose a portable, Linux-based emulation framework to provide an ecosystem for hardware-software co-design of Domain-specific SoCs (DSSoCs) and enable their rapid evaluation during the pre-silicon design phase. This framework holistically targets three key challenges of DSSoC design: accelerator integration, resource management, and application development. We address these chall…
▽ More
In this work, we propose a portable, Linux-based emulation framework to provide an ecosystem for hardware-software co-design of Domain-specific SoCs (DSSoCs) and enable their rapid evaluation during the pre-silicon design phase. This framework holistically targets three key challenges of DSSoC design: accelerator integration, resource management, and application development. We address these challenges via a flexible and lightweight user-space runtime environment that enables easy integration of new accelerators, scheduling heuristics, and user applications, and we illustrate the utility of each through various case studies. With signal processing (WiFi and RADAR) as the target domain, we use our framework to evaluate the performance of various dynamic workloads on hypothetical DSSoC hardware configurations composed of mixtures of CPU cores and FFT accelerators using a Zynq UltraScale+TM MPSoC. We show the portability of this framework by conducting a similar study on an Odroid platform composed of big.LITTLE ARM clusters. Finally, we introduce a prototype compilation toolchain that enables automatic mapping of unlabeled C code to DSSoC platforms. Taken together, this environment offers a unique ecosystem to rapidly perform functional verification and obtain performance and utilization estimates that help accelerate convergence towards a final DSSoC design.
△ Less
Submitted 11 April, 2020; v1 submitted 1 April, 2020;
originally announced April 2020.
-
Analysis and Control of Power-Temperature Dynamics in Heterogeneous Multiprocessors
Authors:
Ganapati Bhat,
Suat Gumussoy,
Umit Y. Ogras
Abstract:
Virtually all electronic systems try to optimize a fundamental trade-off between higher performance and lower power consumption. The latter becomes critical in mobile computing systems, such as smartphones, which rely on passive cooling. Otherwise, the heat concentrated in a small area drives both the junction and skin temperatures up. High junction temperatures degrade the reliability, while skin…
▽ More
Virtually all electronic systems try to optimize a fundamental trade-off between higher performance and lower power consumption. The latter becomes critical in mobile computing systems, such as smartphones, which rely on passive cooling. Otherwise, the heat concentrated in a small area drives both the junction and skin temperatures up. High junction temperatures degrade the reliability, while skin temperature deteriorates the user experience. Therefore, there is a strong need for a formal analysis of power consumption-temperature dynamics and predictive thermal management algorithms. This paper presents a theoretical power-temperature analysis of multiprocessor systems, which are modeled as multi-input multi-output dynamic systems. We analyze the conditions under which the system converges to a stable steady-state temperature. Then, we use these models to design a control algorithm that manages the temperature of the system without affecting the performance of the application. Experiments on the Odroid-XU3 board show that the control algorithm is able to regulate the temperature with a minimal loss in performance when compared to the default thermal governors.
△ Less
Submitted 24 March, 2020;
originally announced March 2020.
-
An Energy-Aware Online Learning Framework for Resource Management in Heterogeneous Platforms
Authors:
Sumit K. Mandal,
Ganapati Bhat,
Janardhan Rao Doppa,
Partha Pratim Pande,
Umit Y. Ogras
Abstract:
Mobile platforms must satisfy the contradictory requirements of fast response time and minimum energy consumption as a function of dynamically changing applications. To address this need, system-on-chips (SoC) that are at the heart of these devices provide a variety of control knobs, such as the number of active cores and their voltage/frequency levels. Controlling these knobs optimally at runtime…
▽ More
Mobile platforms must satisfy the contradictory requirements of fast response time and minimum energy consumption as a function of dynamically changing applications. To address this need, system-on-chips (SoC) that are at the heart of these devices provide a variety of control knobs, such as the number of active cores and their voltage/frequency levels. Controlling these knobs optimally at runtime is challenging for two reasons. First, the large configuration space prohibits exhaustive solutions. Second, control policies designed offline are at best sub-optimal since many potential new applications are unknown at design-time. We address these challenges by proposing an online imitation learning approach. Our key idea is to construct an offline policy and adapt it online to new applications to optimize a given metric (e.g., energy). The proposed methodology leverages the supervision enabled by power-performance models learned at runtime. We demonstrate its effectiveness on a commercial mobile platform with 16 diverse benchmarks. Our approach successfully adapts the control policy to an unknown application after executing less than 25% of its instructions.
△ Less
Submitted 20 March, 2020;
originally announced March 2020.
-
DS3: A System-Level Domain-Specific System-on-Chip Simulation Framework
Authors:
Samet E. Arda,
Anish NK,
A. Alper Goksoy,
Nirmal Kumbhare,
Joshua Mack,
Anderson L. Sartor,
Ali Akoglu,
Radu Marculescu,
Umit Y. Ogras
Abstract:
Heterogeneous systems-on-chip (SoCs) are highly favorable computing platforms due to their superior performance and energy efficiency potential compared to homogeneous architectures. They can be further tailored to a specific domain of applications by incorporating processing elements (PEs) that accelerate frequently used kernels in these applications. However, this potential is contingent upon op…
▽ More
Heterogeneous systems-on-chip (SoCs) are highly favorable computing platforms due to their superior performance and energy efficiency potential compared to homogeneous architectures. They can be further tailored to a specific domain of applications by incorporating processing elements (PEs) that accelerate frequently used kernels in these applications. However, this potential is contingent upon optimizing the SoC for the target domain and utilizing its resources effectively at runtime. To this end, system-level design - including scheduling, power-thermal management algorithms and design space exploration studies - plays a crucial role. This paper presents a system-level domain-specific SoC simulation (DS3) framework to address this need. DS3 enables both design space exploration and dynamic resource management for power-performance optimization of domain applications. We showcase DS3 using six real-world applications from wireless communications and radar processing domain. DS3, as well as the reference applications, is shared as open-source software to stimulate research in this area.
△ Less
Submitted 19 March, 2020;
originally announced March 2020.
-
Work-in-Progress: A Simulation Framework for Domain-Specific System-on-Chips
Authors:
Samet E. Arda,
Anish NK,
A. Alper Goksoy,
Joshua Mack,
Nirmal Kumbhare,
Anderson L. Sartor,
Ali Akoglu,
Radu Marculescu,
Umit Y. Ogras
Abstract:
Heterogeneous system-on-chips (SoCs) have become the standard embedded computing platforms due to their potential to deliver superior performance and energy efficiency compared to homogeneous architectures. They can be particularly suited to target a specific domain of applications. However, this potential is contingent upon optimizing the SoC for the target domain and utilizing its resources effe…
▽ More
Heterogeneous system-on-chips (SoCs) have become the standard embedded computing platforms due to their potential to deliver superior performance and energy efficiency compared to homogeneous architectures. They can be particularly suited to target a specific domain of applications. However, this potential is contingent upon optimizing the SoC for the target domain and utilizing its resources effectively at run-time. Cycle-accurate instruction set simulators are not suitable for this optimization, since meaningful temperature and power consumption evaluations require simulating seconds, if not minutes, of workloads.
This paper presents a system-level domain-specific SoC simulation (DS3) framework to address this need. DS3 enables both design space exploration and dynamic resource management for power-performance optimization for domain applications with$~600\times$ speedup compared to commonly used gem5 simulator. We showcase DS3 using five applications from wireless communications and radar processing domain. DS3, as well as the reference applications, will be shared as open-source software to stimulate research in this area.
△ Less
Submitted 9 August, 2019;
originally announced August 2019.
-
Analytical Performance Models for NoCs with Multiple Priority Traffic Classes
Authors:
Sumit K. Mandal,
Raid Ayoub,
Michael Kishinevsky,
Umit Y. Ogras
Abstract:
Networks-on-chip (NoCs) have become the standard for interconnect solutions in industrial designs ranging from client CPUs to many-core chip-multiprocessors. Since NoCs play a vital role in system performance and power consumption, pre-silicon evaluation environments include cycle-accurate NoC simulators. Long simulations increase the execution time of evaluation frameworks, which are already noto…
▽ More
Networks-on-chip (NoCs) have become the standard for interconnect solutions in industrial designs ranging from client CPUs to many-core chip-multiprocessors. Since NoCs play a vital role in system performance and power consumption, pre-silicon evaluation environments include cycle-accurate NoC simulators. Long simulations increase the execution time of evaluation frameworks, which are already notoriously slow, and prohibit design-space exploration. Existing analytical NoC models, which assume fair arbitration, cannot replace these simulations since industrial NoCs typically employ priority schedulers and multiple priority classes. To address this limitation, we propose a systematic approach to construct priority-aware analytical performance models using micro-architecture specifications and input traffic. Our approach consists of developing two novel transformations of queuing system and designing an algorithm which iteratively uses these two transformations to estimate end-to-end latency. Our approach decomposes the given NoC into individual queues with modified service time to enable accurate and scalable latency computations. Specifically, we introduce novel transformations along with an algorithm that iteratively applies these transformations to decompose the queuing system. Experimental evaluations using real architectures and applications show high accuracy of 97% and up to 2.5x speedup in full-system simulation.
△ Less
Submitted 3 January, 2020; v1 submitted 6 August, 2019;
originally announced August 2019.
-
Power and Thermal Analysis of Commercial Mobile Platforms: Experiments and Case Studies
Authors:
Ganapati Bhat,
Suat Gumussoy,
Umit Y. Ogras
Abstract:
State-of-the-art mobile processors can deliver fast response time and high throughput to maximize the user experience. However, high performance comes at the expense of larger power density, which leads to higher skin temperatures. Since this can degrade the user experience, there is a strong need for power consumption and thermal analysis in mobile processors. In this paper, we first perform expe…
▽ More
State-of-the-art mobile processors can deliver fast response time and high throughput to maximize the user experience. However, high performance comes at the expense of larger power density, which leads to higher skin temperatures. Since this can degrade the user experience, there is a strong need for power consumption and thermal analysis in mobile processors. In this paper, we first perform experiments on the Nexus 6P phone to study the power, performance and thermal behavior of modern smartphones. Using the insight from these experiments, we propose a control algorithm that throttles select applications without affecting other apps. We demonstrate our governor on the Exynos 5422 processor employed in the Odroid-XU3 board.
△ Less
Submitted 19 March, 2019;
originally announced April 2019.
-
OpenHealth: Open Source Platform for Wearable Health Monitoring
Authors:
Ganapati Bhat,
Ranadeep Deb,
Umit Y. Ogras
Abstract:
Movement disorders are becoming one of the leading causes of functional disability due to aging populations and extended life expectancy. Wearable health monitoring is emerging as an effective way to augment clinical care for movement disorders. However, wearable devices face a number of adaptation and technical challenges that hinder their widespread adoption. To address these challenges, we intr…
▽ More
Movement disorders are becoming one of the leading causes of functional disability due to aging populations and extended life expectancy. Wearable health monitoring is emerging as an effective way to augment clinical care for movement disorders. However, wearable devices face a number of adaptation and technical challenges that hinder their widespread adoption. To address these challenges, we introduce OpenHealth, an open source platform for wearable health monitoring. OpenHealth aims to design a standard set of hardware/software and wearable devices that can enable autonomous collection of clinically relevant data. The OpenHealth platform includes a wearable device, standard software interfaces and reference implementations of human activity and gesture recognition applications.
△ Less
Submitted 16 March, 2019; v1 submitted 18 February, 2019;
originally announced March 2019.
-
REAP: Runtime Energy-Accuracy Optimization for Energy Harvesting IoT Devices
Authors:
Ganapati Bhat,
Kunal Bagewadi,
Hyung Gyu Lee,
Umit Y. Ogras
Abstract:
The use of wearable and mobile devices for health monitoring and activity recognition applications is increasing rapidly. These devices need to maximize their accuracy and active time under a tight energy budget imposed by battery and small form-factor constraints. This paper considers energy harvesting devices that run on a limited energy budget to recognize user activities over a given period. W…
▽ More
The use of wearable and mobile devices for health monitoring and activity recognition applications is increasing rapidly. These devices need to maximize their accuracy and active time under a tight energy budget imposed by battery and small form-factor constraints. This paper considers energy harvesting devices that run on a limited energy budget to recognize user activities over a given period. We propose a technique to co-optimize the accuracy and active time by utilizing multiple design points with different energy-accuracy trade-offs. The proposed technique switches between these design points at runtime to maximize a generalized objective function under tight harvested energy budget constraints. We evaluate the proposed approach experimentally using a custom hardware prototype and fourteen user studies. The proposed approach achieves both 46% higher expected accuracy and 66% longer active time compared to the highest performance design point.
△ Less
Submitted 22 February, 2019; v1 submitted 2 February, 2019;
originally announced February 2019.
-
Online Human Activity Recognition using Low-Power Wearable Devices
Authors:
Ganapati Bhat,
Ranadeep Deb,
Vatika Vardhan Chaurasia,
Holly Shill,
Umit Y. Ogras
Abstract:
Human activity recognition~(HAR) has attracted significant research interest due to its applications in health monitoring and patient rehabilitation. Recent research on HAR focuses on using smartphones due to their widespread use. However, this leads to inconvenient use, limited choice of sensors and inefficient use of resources, since smartphones are not designed for HAR. This paper presents the…
▽ More
Human activity recognition~(HAR) has attracted significant research interest due to its applications in health monitoring and patient rehabilitation. Recent research on HAR focuses on using smartphones due to their widespread use. However, this leads to inconvenient use, limited choice of sensors and inefficient use of resources, since smartphones are not designed for HAR. This paper presents the first HAR framework that can perform both online training and inference. The proposed framework starts with a novel technique that generates features using the fast Fourier and discrete wavelet transforms of a textile-based stretch sensor and accelerometer. Using these features, we design an artificial neural network classifier which is trained online using the policy gradient algorithm. Experiments on a low power IoT device (TI-CC2650 MCU) with nine users show 97.7% accuracy in identifying six activities and their transitions with less than 12.5 mW power consumption.
△ Less
Submitted 4 February, 2019; v1 submitted 26 August, 2018;
originally announced August 2018.
-
Power-Temperature Stability and Safety Analysis for Multiprocessor Systems
Authors:
Ganapati Bhat,
Suat Gumussoy,
Umit Y. Ogras
Abstract:
Modern multiprocessor system-on-chips (SoCs) integrate multiple heterogeneous cores to achieve high energy efficiency. The power consumption of each core contributes to an increase in the temperature across the chip floorplan. In turn, higher temperature increases the leakage power exponentially, and leads to a positive feedback with nonlinear dynamics. This paper presents a power-temperature stab…
▽ More
Modern multiprocessor system-on-chips (SoCs) integrate multiple heterogeneous cores to achieve high energy efficiency. The power consumption of each core contributes to an increase in the temperature across the chip floorplan. In turn, higher temperature increases the leakage power exponentially, and leads to a positive feedback with nonlinear dynamics. This paper presents a power-temperature stability and safety analysis technique for multiprocessor systems. This analysis reveals the conditions under which the power-temperature trajectory converges to a stable fixed point. We also present a simple formula to compute the stable fixed point and maximum thermally-safe power consumption at runtime. Hardware measurements on a state-of-the-art mobile processor show that our analytical formulation can predict the stable fixed point with an average error of 2.6%. Hence, our approach can be used at runtime to ensure thermally safe operation and guard against thermal threats.
△ Less
Submitted 16 June, 2018;
originally announced June 2018.
-
Energy- and Performance-Driven NoC Communication Architecture Synthesis Using a Decomposition Approach
Authors:
Umit Y. Ogras,
Radu Marculescu
Abstract:
In this paper, we present a methodology for customized communication architecture synthesis that matches the communication requirements of the target application. This is an important problem, particularly for network-based implementations of complex applications. Our approach is based on using frequently encountered generic communication primitives as an alphabet capable of characterizing any g…
▽ More
In this paper, we present a methodology for customized communication architecture synthesis that matches the communication requirements of the target application. This is an important problem, particularly for network-based implementations of complex applications. Our approach is based on using frequently encountered generic communication primitives as an alphabet capable of characterizing any given communication pattern. The proposed algorithm searches through the entire design space for a solution that minimizes the system total energy consumption, while satisfying the other design constraints. Compared to the standard mesh architecture, the customized architecture generated by the newly proposed approach shows about 36% throughput increase and 51% reduction in the energy required to encrypt 128 bits of data with a standard encryption algorithm.
△ Less
Submitted 25 October, 2007;
originally announced October 2007.
