-
SimplePIM: A Software Framework for Productive and Efficient Processing-in-Memory
Authors:
Jinfan Chen,
Juan Gómez-Luna,
Izzat El Hajj,
Yuxin Guo,
Onur Mutlu
Abstract:
Data movement between memory and processors is a major bottleneck in modern computing systems. The processing-in-memory (PIM) paradigm aims to alleviate this bottleneck by performing computation inside memory chips. Real PIM hardware (e.g., the UPMEM system) is now available and has demonstrated potential in many applications. However, programming such real PIM hardware remains a challenge for man…
▽ More
Data movement between memory and processors is a major bottleneck in modern computing systems. The processing-in-memory (PIM) paradigm aims to alleviate this bottleneck by performing computation inside memory chips. Real PIM hardware (e.g., the UPMEM system) is now available and has demonstrated potential in many applications. However, programming such real PIM hardware remains a challenge for many programmers.
This paper presents a new software framework, SimplePIM, to aid programming real PIM systems. The framework processes arrays of arbitrary elements on a PIM device by calling iterator functions from the host and provides primitives for communication among PIM cores and between PIM and the host system. We implement SimplePIM for the UPMEM PIM system and evaluate it on six major applications. Our results show that SimplePIM enables 66.5% to 83.1% reduction in lines of code in PIM programs. The resulting code leads to higher performance (between 10% and 37% speedup) than hand-optimized code in three applications and provides comparable performance in three others. SimplePIM is fully and freely available at https://github.com/CMU-SAFARI/SimplePIM.
△ Less
Submitted 3 October, 2023;
originally announced October 2023.
-
Predicting the Performance-Cost Trade-off of Applications Across Multiple Systems
Authors:
Amir Nassereldine,
Safaa Diab,
Mohammed Baydoun,
Kenneth Leach,
Maxim Alt,
Dejan Milojicic,
Izzat El Hajj
Abstract:
In modern computing environments, users may have multiple systems accessible to them such as local clusters, private clouds, or public clouds. This abundance of choices makes it difficult for users to select the system and configuration for running an application that best meet their performance and cost objectives. To assist such users, we propose a prediction tool that predicts the full performa…
▽ More
In modern computing environments, users may have multiple systems accessible to them such as local clusters, private clouds, or public clouds. This abundance of choices makes it difficult for users to select the system and configuration for running an application that best meet their performance and cost objectives. To assist such users, we propose a prediction tool that predicts the full performance-cost trade-off space of an application across multiple systems. Our tool runs and profiles a submitted application on a small number of configurations from some of the systems, and uses that information to predict the application's performance on all configurations in all systems. The prediction models are trained offline with data collected from running a large number of applications on a wide variety of configurations. Notable aspects of our tool include: providing different scopes of prediction with varying online profiling requirements, automating the selection of the small number of configurations and systems used for online profiling, performing online profiling using partial runs thereby make predictions for applications without running them to completion, employing a classifier to distinguish applications that scale well from those that scale poorly, and predicting the sensitivity of applications to interference from other users. We evaluate our tool using 69 data analytics and scientific computing benchmarks executing on three different single-node CPU systems with 8-9 configurations each and show that it can achieve low prediction error with modest profiling overhead.
△ Less
Submitted 4 April, 2023;
originally announced April 2023.
-
Asynchronous Persistence with ASAP
Authors:
Ahmed Abulila,
Izzat El Hajj,
Myoungsoo Jung,
Nam Sung Kim
Abstract:
Supporting atomic durability of updates for persistent memories is typically achieved with Write-Ahead Logging (WAL). WAL flushes log entries to persistent memory before making the actual data persistent to ensure that a consistent state can be recovered if a crash occurs. Performing WAL in hardware is attractive because it makes most aspects of log management transparent to software, and it compl…
▽ More
Supporting atomic durability of updates for persistent memories is typically achieved with Write-Ahead Logging (WAL). WAL flushes log entries to persistent memory before making the actual data persistent to ensure that a consistent state can be recovered if a crash occurs. Performing WAL in hardware is attractive because it makes most aspects of log management transparent to software, and it completes log persist operations (LPs) and data persist operations (DPs) in the background, overlapping them with the execution of other instructions.
Prior hardware logging solutions commit atomic regions synchronously. Once the end of a region is reached, all outstanding persist operations required for the region to commit must be completed before instruction execution may proceed. For undo logging, LPs and DPs are both performed synchronously to ensure that the region commits synchronously. For redo logging, DPs can be performed asynchronously, but LPs are performed synchronously to ensure that the region commits synchronously. In both cases, waiting for synchronous persist operations (LP or DP) at the end of an atomic region causes atomic regions to incur high latency.
To tackle this limitation, we propose ASAP, a hardware logging solution that allows atomic regions to commit asynchronously. That is, once the end of an atomic region is reached, instruction execution may proceed without waiting for outstanding persist operations to complete. As such, both LPs and DPs can be performed asynchronously. The challenge with allowing atomic regions to commit asynchronously is that it can lead to control and data dependence violations in the commit order of the atomic regions, leaving data in an unrecoverable state in case of a crash. To address this issue, ASAP tracks and enforces control and data dependencies between atomic regions in hardware to ensure that the regions commit in the proper order.
△ Less
Submitted 26 February, 2023;
originally announced February 2023.
-
Efficient algorithms to solve atom reconfiguration problems. II. The assignment-rerouting-ordering (aro) algorithm
Authors:
Remy El Sabeh,
Jessica Bohm,
Zhiqian Ding,
Stephanie Maaz,
Naomi Nishimura,
Izzat El Hajj,
Amer E. Mouawad,
Alexandre Cooper
Abstract:
Programmable arrays of optical traps enable the assembly of configurations of single atoms to perform controlled experiments on quantum many-body systems. Finding the sequence of control operations to transform an arbitrary configuration of atoms into a predetermined one requires solving an atom reconfiguration problem quickly and efficiently. A typical approach to solve atom reconfiguration probl…
▽ More
Programmable arrays of optical traps enable the assembly of configurations of single atoms to perform controlled experiments on quantum many-body systems. Finding the sequence of control operations to transform an arbitrary configuration of atoms into a predetermined one requires solving an atom reconfiguration problem quickly and efficiently. A typical approach to solve atom reconfiguration problems is to use an assignment algorithm to determine which atoms to move to which traps. This approach results in control protocols that exactly minimize the number of displacement operations; however, this approach does not optimize for the number of displaced atoms nor the number of times each atom is displaced, resulting in unnecessary control operations that increase the execution time and failure rate of the control protocol. In this work, we propose the assignment-rerouting-ordering (aro) algorithm to improve the performance of assignment-based algorithms in solving atom reconfiguration problems. The aro algorithm uses an assignment subroutine to minimize the total distance traveled by all atoms, a rerouting subroutine to reduce the number of displaced atoms, and an ordering subroutine to guarantee that each atom is displaced at most once. The ordering subroutine relies on the existence of a partial ordering of moves that can be obtained using a polynomial-time algorithm that we introduce within the formal framework of graph theory. We numerically quantify the performance of the aro algorithm in the presence and in the absence of loss, and show that it outperforms the exact, approximation, and heuristic algorithms that we use as benchmarks. Our results are useful for assembling large configurations of atoms with high success probability and fast preparation time, as well as for designing and benchmarking novel atom reconfiguration algorithms.
△ Less
Submitted 11 December, 2022;
originally announced December 2022.
-
Efficient algorithms to solve atom reconfiguration problems. I. The redistribution-reconfiguration (red-rec) algorithm
Authors:
Barry Cimring,
Remy El Sabeh,
Marc Bacvanski,
Stephanie Maaz,
Izzat El Hajj,
Naomi Nishimura,
Amer E. Mouawad,
Alexandre Cooper
Abstract:
We propose the redistribution-reconfiguration~(red-rec) algorithm to efficiently compute control protocols to assemble compact-centered configurations of atoms in two-dimensional arrays of optical traps with lattice geometries. The red-rec algorithm redistributes atoms among pairs of donor-receiver columns and reconfigures each column using an exact displacement-minimizing algorithm, harnessing pa…
▽ More
We propose the redistribution-reconfiguration~(red-rec) algorithm to efficiently compute control protocols to assemble compact-centered configurations of atoms in two-dimensional arrays of optical traps with lattice geometries. The red-rec algorithm redistributes atoms among pairs of donor-receiver columns and reconfigures each column using an exact displacement-minimizing algorithm, harnessing parallel control operations that simultaneously actuate multiple traps to reduce the execution time. We numerically quantify the performance of the red-rec algorithm, both in the absence and in the presence of loss, using realistic physical parameters and operational constraints. We show that the number of traps required to prepare a compact-centered configuration of atoms on a grid with a mean success probability of one half scales as the 3/2 power of the number of desired atoms, highlighting the challenges of assembling configurations of tens of thousands of atoms. We further demonstrate that faster preparation times can be achieved by rejecting configurations of atoms containing fewer atoms than a given threshold. The red-rec algorithm admits an efficient implementation that can readily be deployed on real-time control systems to assemble large configurations of atoms with high mean success probability and fast preparation times.
△ Less
Submitted 7 December, 2022;
originally announced December 2022.
-
Parallelizing Maximal Clique Enumeration on GPUs
Authors:
Mohammad Almasri,
Yen-Hsiang Chang,
Izzat El Hajj,
Rakesh Nagi,
Jinjun Xiong,
Wen-mei Hwu
Abstract:
We present a GPU solution for exact maximal clique enumeration (MCE) that performs a search tree traversal following the Bron-Kerbosch algorithm. Prior works on parallelizing MCE on GPUs perform a breadth-first traversal of the tree, which has limited scalability because of the explosion in the number of tree nodes at deep levels. We propose to parallelize MCE on GPUs by performing depth-first tra…
▽ More
We present a GPU solution for exact maximal clique enumeration (MCE) that performs a search tree traversal following the Bron-Kerbosch algorithm. Prior works on parallelizing MCE on GPUs perform a breadth-first traversal of the tree, which has limited scalability because of the explosion in the number of tree nodes at deep levels. We propose to parallelize MCE on GPUs by performing depth-first traversal of independent subtrees in parallel. Since MCE suffers from high load imbalance and memory capacity requirements, we propose a worker list for dynamic load balancing, as well as partial induced subgraphs and a compact representation of excluded vertex sets to regulate memory consumption. Our evaluation shows that our GPU implementation on a single GPU outperforms the state-of-the-art parallel CPU implementation by a geometric mean of 4.9x (up to 16.7x), and scales efficiently to multiple GPUs. Our code has been open-sourced to enable further research on accelerating MCE.
△ Less
Submitted 25 October, 2023; v1 submitted 2 December, 2022;
originally announced December 2022.
-
A Framework for High-throughput Sequence Alignment using Real Processing-in-Memory Systems
Authors:
Safaa Diab,
Amir Nassereldine,
Mohammed Alser,
Juan Gómez-Luna,
Onur Mutlu,
Izzat El Hajj
Abstract:
Sequence alignment is a memory bound computation whose performance in modern systems is limited by the memory bandwidth bottleneck. Processing-in-memory architectures alleviate this bottleneck by providing the memory with computing competencies. We propose Alignment-in-Memory (AIM), a framework for high-throughput sequence alignment using processing-in-memory, and evaluate it on UPMEM, the first p…
▽ More
Sequence alignment is a memory bound computation whose performance in modern systems is limited by the memory bandwidth bottleneck. Processing-in-memory architectures alleviate this bottleneck by providing the memory with computing competencies. We propose Alignment-in-Memory (AIM), a framework for high-throughput sequence alignment using processing-in-memory, and evaluate it on UPMEM, the first publicly-available general-purpose programmable processing-in-memory system.
Our evaluation shows that a real processing-in-memory system can substantially outperform server-grade multi-threaded CPU systems running at full-scale when performing sequence alignment for a variety of algorithms, read lengths, and edit distance thresholds. We hope that our findings inspire more work on creating and accelerating bioinformatics algorithms for such real processing-in-memory systems.
Our code is available at https://github.com/safaad/aim.
△ Less
Submitted 27 March, 2023; v1 submitted 2 August, 2022;
originally announced August 2022.
-
Parallel Vertex Cover Algorithms on GPUs
Authors:
Peter Yamout,
Karim Barada,
Adnan Jaljuli,
Amer E. Mouawad,
Izzat El Hajj
Abstract:
Finding small vertex covers in a graph has applications in numerous domains. Two common formulations of the problem include: Minimum Vertex Cover, which finds the smallest vertex cover in a graph, and Parameterized Vertex Cover, which finds a vertex cover whose size is less than or equal to some parameter $k$. Algorithms for both formulations traverse a search tree, which grows exponentially with…
▽ More
Finding small vertex covers in a graph has applications in numerous domains. Two common formulations of the problem include: Minimum Vertex Cover, which finds the smallest vertex cover in a graph, and Parameterized Vertex Cover, which finds a vertex cover whose size is less than or equal to some parameter $k$. Algorithms for both formulations traverse a search tree, which grows exponentially with the size of the graph or the value of $k$.
Parallelizing the traversal of the vertex cover search tree on GPUs is challenging for multiple reasons. First, the search tree is a narrow binary tree which makes it difficult to extract enough sub-trees to process in parallel to fully utilize the GPU's resources. Second, the search tree is highly imbalanced which makes load balancing across a massive number of parallel GPU workers challenging. Third, keeping around all the intermediate state needed to traverse many sub-trees in parallel puts high pressure on the GPU's memory resources and may act as a limiting factor to parallelism.
To address these challenges, we propose an approach to traverse the vertex cover search tree in parallel using GPUs while handling dynamic load balancing. Each thread block traverses a different sub-tree using a local stack, however, we also use a global worklist to balance load. Blocks contribute branches of their sub-trees to the global worklist on an as-needed basis, while blocks that finish their sub-trees get new ones from the global worklist. We use degree arrays to represent intermediate graphs so that the representation is compact in memory to avoid limiting parallelism, but self-contained which is necessary for load balancing. Our evaluation shows that compared to prior work, our hybrid approach of using local stacks and a global worklist substantially improves performance and reduces load imbalance, especially on difficult instances of the problem.
△ Less
Submitted 21 April, 2022;
originally announced April 2022.
-
High-throughput Pairwise Alignment with the Wavefront Algorithm using Processing-in-Memory
Authors:
Safaa Diab,
Amir Nassereldine,
Mohammed Alser,
Juan Gómez Luna,
Onur Mutlu,
Izzat El Hajj
Abstract:
We show that the wavefront algorithm can achieve higher pairwise read alignment throughput on a UPMEM PIM system than on a server-grade multi-threaded CPU system.
We show that the wavefront algorithm can achieve higher pairwise read alignment throughput on a UPMEM PIM system than on a server-grade multi-threaded CPU system.
△ Less
Submitted 23 April, 2022; v1 submitted 5 April, 2022;
originally announced April 2022.
-
A Compiler Framework for Optimizing Dynamic Parallelism on GPUs
Authors:
Mhd Ghaith Olabi,
Juan Gómez Luna,
Onur Mutlu,
Wen-mei Hwu,
Izzat El Hajj
Abstract:
Dynamic parallelism on GPUs allows GPU threads to dynamically launch other GPU threads. It is useful in applications with nested parallelism, particularly where the amount of nested parallelism is irregular and cannot be predicted beforehand. However, prior works have shown that dynamic parallelism may impose a high performance penalty when a large number of small grids are launched. The large num…
▽ More
Dynamic parallelism on GPUs allows GPU threads to dynamically launch other GPU threads. It is useful in applications with nested parallelism, particularly where the amount of nested parallelism is irregular and cannot be predicted beforehand. However, prior works have shown that dynamic parallelism may impose a high performance penalty when a large number of small grids are launched. The large number of launches results in high launch latency due to congestion, and the small grid sizes result in hardware underutilization.
To address this issue, we propose a compiler framework for optimizing the use of dynamic parallelism in applications with nested parallelism. The framework features three key optimizations: thresholding, coarsening, and aggregation. Thresholding involves launching a grid dynamically only if the number of child threads exceeds some threshold, and serializing the child threads in the parent thread otherwise. Coarsening involves executing the work of multiple thread blocks by a single coarsened block to amortize the common work across them. Aggregation involves combining multiple child grids into a single aggregated grid.
Our evaluation shows that our compiler framework improves the performance of applications with nested parallelism by a geometric mean of 43.0x over applications that use dynamic parallelism, 8.7x over applications that do not use dynamic parallelism, and 3.6x over applications that use dynamic parallelism with aggregation alone as proposed in prior work.
△ Less
Submitted 8 January, 2022;
originally announced January 2022.
-
Benchmarking Memory-Centric Computing Systems: Analysis of Real Processing-in-Memory Hardware
Authors:
Juan Gómez-Luna,
Izzat El Hajj,
Ivan Fernandez,
Christina Giannoula,
Geraldo F. Oliveira,
Onur Mutlu
Abstract:
Many modern workloads such as neural network inference and graph processing are fundamentally memory-bound. For such workloads, data movement between memory and CPU cores imposes a significant overhead in terms of both latency and energy. A major reason is that this communication happens through a narrow bus with high latency and limited bandwidth, and the low data reuse in memory-bound workloads…
▽ More
Many modern workloads such as neural network inference and graph processing are fundamentally memory-bound. For such workloads, data movement between memory and CPU cores imposes a significant overhead in terms of both latency and energy. A major reason is that this communication happens through a narrow bus with high latency and limited bandwidth, and the low data reuse in memory-bound workloads is insufficient to amortize the cost of memory access. Fundamentally addressing this data movement bottleneck requires a paradigm where the memory system assumes an active role in computing by integrating processing capabilities. This paradigm is known as processing-in-memory (PIM). Recent research explores different forms of PIM architectures, motivated by the emergence of new technologies that integrate memory with a logic layer, where processing elements can be easily placed. Past works evaluate these architectures in simulation or, at best, with simplified hardware prototypes. In contrast, the UPMEM company has designed and manufactured the first publicly-available real-world PIM architecture. The UPMEM PIM architecture combines traditional DRAM memory arrays with general-purpose in-order cores, called DRAM Processing Units (DPUs), integrated in the same chip. This paper presents key takeaways from the first comprehensive analysis of the first publicly-available real-world PIM architecture. We provide four key takeaways about the UPMEM PIM architecture, which stem from our study. More insights about suitability of different workloads to the PIM system, programming recommendations for software designers, and suggestions and hints for hardware and architecture designers of future PIM systems are available in arXiv:2105.03814
△ Less
Submitted 3 April, 2023; v1 submitted 4 October, 2021;
originally announced October 2021.
-
Benchmarking a New Paradigm: An Experimental Analysis of a Real Processing-in-Memory Architecture
Authors:
Juan Gómez-Luna,
Izzat El Hajj,
Ivan Fernandez,
Christina Giannoula,
Geraldo F. Oliveira,
Onur Mutlu
Abstract:
Many modern workloads, such as neural networks, databases, and graph processing, are fundamentally memory-bound. For such workloads, the data movement between main memory and CPU cores imposes a significant overhead in terms of both latency and energy. A major reason is that this communication happens through a narrow bus with high latency and limited bandwidth, and the low data reuse in memory-bo…
▽ More
Many modern workloads, such as neural networks, databases, and graph processing, are fundamentally memory-bound. For such workloads, the data movement between main memory and CPU cores imposes a significant overhead in terms of both latency and energy. A major reason is that this communication happens through a narrow bus with high latency and limited bandwidth, and the low data reuse in memory-bound workloads is insufficient to amortize the cost of main memory access. Fundamentally addressing this data movement bottleneck requires a paradigm where the memory system assumes an active role in computing by integrating processing capabilities. This paradigm is known as processing-in-memory (PIM).
Recent research explores different forms of PIM architectures, motivated by the emergence of new 3D-stacked memory technologies that integrate memory with a logic layer where processing elements can be easily placed. Past works evaluate these architectures in simulation or, at best, with simplified hardware prototypes. In contrast, the UPMEM company has designed and manufactured the first publicly-available real-world PIM architecture.
This paper provides the first comprehensive analysis of the first publicly-available real-world PIM architecture. We make two key contributions. First, we conduct an experimental characterization of the UPMEM-based PIM system using microbenchmarks to assess various architecture limits such as compute throughput and memory bandwidth, yielding new insights. Second, we present PrIM, a benchmark suite of 16 workloads from different application domains (e.g., linear algebra, databases, graph processing, neural networks, bioinformatics).
△ Less
Submitted 4 May, 2022; v1 submitted 8 May, 2021;
originally announced May 2021.
-
Parallel K-Clique Counting on GPUs
Authors:
Mohammad Almasri,
Izzat El Hajj,
Rakesh Nagi,
Jinjun Xiong,
Wen-mei Hwu
Abstract:
Counting k-cliques in a graph is an important problem in graph analysis with many applications such as community detection and graph partitioning. Counting k-cliques is typically done by traversing search trees starting at each vertex in the graph. Parallelizing k-clique counting has been well-studied on CPUs and many solutions exist. However, there are no performant solutions for k-clique countin…
▽ More
Counting k-cliques in a graph is an important problem in graph analysis with many applications such as community detection and graph partitioning. Counting k-cliques is typically done by traversing search trees starting at each vertex in the graph. Parallelizing k-clique counting has been well-studied on CPUs and many solutions exist. However, there are no performant solutions for k-clique counting on GPUs.
Parallelizing k-clique counting on GPUs comes with numerous challenges such as the need for extracting fine-grain multi-level parallelism, sensitivity to load imbalance, and constrained physical memory capacity. While there has been work on related problems such as finding maximal cliques and generalized sub-graph matching on GPUs, k-clique counting in particular has yet to be explored in depth. In this paper, we present the first parallel GPU solution specialized for the k-clique counting problem. Our solution supports both graph orientation and pivoting for eliminating redundant clique discovery. It incorporates both vertex-centric and edge-centric parallelization schemes for distributing work across thread blocks, and further partitions work within each thread block to extract fine-grain multi-level parallelism while tolerating load imbalance. It also includes optimizations such as binary encoding of induced sub-graphs and sub-warp partitioning to limit memory consumption and improve the utilization of execution resources.
Our evaluation shows that our best GPU implementation outperforms the best state-of-the-art parallel CPU implementation by a geometric mean of 12.39x, 6.21x, and 18.99x for k=4, 7, and 10, respectively. We also perform a detailed evaluation of the trade-offs involved in the choice of parallelization scheme, and the incremental speedup of each optimization to provide an in-depth understanding of the optimization space. ...
△ Less
Submitted 6 June, 2022; v1 submitted 27 April, 2021;
originally announced April 2021.
-
PANTHER: A Programmable Architecture for Neural Network Training Harnessing Energy-efficient ReRAM
Authors:
Aayush Ankit,
Izzat El Hajj,
Sai Rahul Chalamalasetti,
Sapan Agarwal,
Matthew Marinella,
Martin Foltin,
John Paul Strachan,
Dejan Milojicic,
Wen-mei Hwu,
Kaushik Roy
Abstract:
The wide adoption of deep neural networks has been accompanied by ever-increasing energy and performance demands due to the expensive nature of training them. Numerous special-purpose architectures have been proposed to accelerate training: both digital and hybrid digital-analog using resistive RAM (ReRAM) crossbars. ReRAM-based accelerators have demonstrated the effectiveness of ReRAM crossbars a…
▽ More
The wide adoption of deep neural networks has been accompanied by ever-increasing energy and performance demands due to the expensive nature of training them. Numerous special-purpose architectures have been proposed to accelerate training: both digital and hybrid digital-analog using resistive RAM (ReRAM) crossbars. ReRAM-based accelerators have demonstrated the effectiveness of ReRAM crossbars at performing matrix-vector multiplication operations that are prevalent in training. However, they still suffer from inefficiency due to the use of serial reads and writes for performing the weight gradient and update step. A few works have demonstrated the possibility of performing outer products in crossbars, which can be used to realize the weight gradient and update step without the use of serial reads and writes. However, these works have been limited to low precision operations which are not sufficient for typical training workloads. Moreover, they have been confined to a limited set of training algorithms for fully-connected layers only. To address these limitations, we propose a bit-slicing technique for enhancing the precision of ReRAM-based outer products, which is substantially different from bit-slicing for matrix-vector multiplication only. We incorporate this technique into a crossbar architecture with three variants catered to different training algorithms. To evaluate our design on different types of layers in neural networks (fully-connected, convolutional, etc.) and training algorithms, we develop PANTHER, an ISA-programmable training accelerator with compiler support. Our evaluation shows that PANTHER achieves up to $8.02\times$, $54.21\times$, and $103\times$ energy reductions as well as $7.16\times$, $4.02\times$, and $16\times$ execution time reductions compared to digital accelerators, ReRAM-based accelerators, and GPUs, respectively.
△ Less
Submitted 24 December, 2019;
originally announced December 2019.
-
PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inference
Authors:
Aayush Ankit,
Izzat El Hajj,
Sai Rahul Chalamalasetti,
Geoffrey Ndu,
Martin Foltin,
R. Stanley Williams,
Paolo Faraboschi,
Wen-mei Hwu,
John Paul Strachan,
Kaushik Roy,
Dejan S Milojicic
Abstract:
Memristor crossbars are circuits capable of performing analog matrix-vector multiplications, overcoming the fundamental energy efficiency limitations of digital logic. They have been shown to be effective in special-purpose accelerators for a limited set of neural network applications.
We present the Programmable Ultra-efficient Memristor-based Accelerator (PUMA) which enhances memristor crossba…
▽ More
Memristor crossbars are circuits capable of performing analog matrix-vector multiplications, overcoming the fundamental energy efficiency limitations of digital logic. They have been shown to be effective in special-purpose accelerators for a limited set of neural network applications.
We present the Programmable Ultra-efficient Memristor-based Accelerator (PUMA) which enhances memristor crossbars with general purpose execution units to enable the acceleration of a wide variety of Machine Learning (ML) inference workloads. PUMA's microarchitecture techniques exposed through a specialized Instruction Set Architecture (ISA) retain the efficiency of in-memory computing and analog circuitry, without compromising programmability.
We also present the PUMA compiler which translates high-level code to PUMA ISA. The compiler partitions the computational graph and optimizes instruction scheduling and register allocation to generate code for large and complex workloads to run on thousands of spatial cores.
We have developed a detailed architecture simulator that incorporates the functionality, timing, and power models of PUMA's components to evaluate performance and energy consumption. A PUMA accelerator running at 1 GHz can reach area and power efficiency of $577~GOPS/s/mm^2$ and $837~GOPS/s/W$, respectively. Our evaluation of diverse ML applications from image recognition, machine translation, and language modelling (5M-800M synapses) shows that PUMA achieves up to $2,446\times$ energy and $66\times$ latency improvement for inference compared to state-of-the-art GPUs. Compared to an application-specific memristor-based accelerator, PUMA incurs small energy overheads at similar inference latency and added programmability.
△ Less
Submitted 29 January, 2019; v1 submitted 29 January, 2019;
originally announced January 2019.