research-article

Public Access

MR-iNet Gym: Framework for Edge Deployment of Deep Reinforcement Learning on Embedded Software Defined Radio

Authors:

Jithin Jagannath,

Collin Farquhar,

Keyvan Ramezanpour,

Anu JagannathAuthors Info & Claims

WiseML '22: Proceedings of the 2022 ACM Workshop on Wireless Security and Machine Learning

Pages 51 - 56

https://doi.org/10.1145/3522783.3529530

Published: 16 May 2022 Publication History

Abstract

Dynamic resource allocation plays a critical role in the next generation of intelligent wireless communication systems. Machine learning has been leveraged as a powerful tool to make strides in this domain. In most cases, the progress has been limited to simulations due to the challenging nature of hardware deployment of these solutions. In this paper, for the first time, we design and deploy deep reinforcement learning (DRL)-based power control agents on the GPU embedded software defined radios (SDRs). To this end, we propose an end-to-end framework (MR-iNet Gym) where the simulation suite and the embedded SDR development work cohesively to overcome real-world implementation hurdles. To prove feasibility, we consider the problem of distributed power control for code-division multiple access (DS-CDMA)-based LPI/D transceivers. We first build a DS-CDMA ns3 module that interacts with the OpenAI Gym environment. Next, we train the power control DRL agents in this ns3-gym simulation environment in a scenario that replicates our hardware testbed. Next, for edge (embedded on-device) deployment, the trained models are optimized for real-time operation without loss of performance. Hardware-based evaluation verifies the efficiency of DRL agents over traditional distributed constrained power control (DCPC) algorithm. More significantly, as the primary goal, this is the first work that has established the feasibility of deploying DRL to provide optimized distributed resource allocation for next-generation of GPU-embedded radios.

References

[1]

Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. 2016. OpenAI Gym. arxiv: 1606.01540 [cs.LG]

[2]

Tamer ElBatt and Anthony Ephremides. 2004. Joint scheduling and power control for wireless ad hoc networks. IEEE Transactions on Wireless communications, Vol. 3, 1 (2004), 74--85.

Digital Library

[3]

Piotr Gawłowicz and Anatolij Zubow. 2019. ns-3 meets OpenAI Gym: The Playground for Machine Learning in Networking Research. In ACM Intl. Conf. on Modeling, Analysis and Sim. of Wireless and Mobile Systems (Miami Beach, USA).

Digital Library

[4]

Sudheer A Grandhi, Jens Zander, and Roy Yates. 1994. Constrained power control. Wireless Personal Communications, Vol. 1, 4 (1994), 257--270.

[5]

Ade Hermawan, Rizki Ginanjar, Dong-Seong Kim, and Jae-Min Lee. 2020. CNN-Based Automatic Modulation Classification for Beyond 5G Communications. IEEE Communications Letters, Vol. 24, 5 (2020), 1038--1041.

[6]

Md Habibul Islam, Ying-Chang Liang, and Anh Tuan Hoang. 2007. Distributed power and admission control for cognitive radio networks using antenna arrays. In 2007 2nd IEEE International Symposium on New Frontiers in Dynamic Spectrum Access Networks. IEEE, 250--253.

Digital Library

[7]

Anu Jagannath, Jithin Jagannath, and Tommaso Melodia. 2021. Redefining Wireless Communication for 6G: Signal Processing Meets Deep Learning with Deep Unfolding . IEEE Transaction on Artificial Intelligence (2021).

[8]

Jithin Jagannath, Nicholas Polosky, Dan Connor, Lakshmi Theagarajan, Brendan Sheaffer, Svetlana Foulke, and Pramod Varshney. 2018. Artificial Neural Network based Automatic Modulation Classifier for Software Defined Radios. In Proc. of IEEE Intl, Conf. on Communications (ICC). Kansas City, USA.

[9]

Jithin Jagannath, Nicholas Polosky, Anu Jagannath, Francesco Restuccia, and Tommaso Melodia. 2019. Machine Learning for Wireless Communications in the Internet of Things: A Comprehensive Survey. Ad Hoc Networks (Elsevier) (2019).

[10]

Jithin Jagannath, Nicholas Polosky, Anu Jagannath, Francesco Restuccia, and Tommaso Melodia. 2020. Neural Networks for Signal Intelligence: Theory and Practice, . In Machine Learning for Future Wireless Communications, F.L. Luo (Ed.). John Wiley & Sons, Limited. 2019029933

[11]

Xingjian Li, Jun Fang, Wen Cheng, Huiping Duan, Zhi Chen, and Hongbin Li. 2018. Intelligent power control for spectrum sharing in cognitive radios: A deep reinforcement learning approach. IEEE access, Vol. 6 (2018).

[12]

Nguyen Cong Luong, Dinh Thai Hoang, Shimin Gong, Dusit Niyato, Ping Wang, Ying-Chang Liang, and Dong In Kim. 2019. Applications of Deep Reinforcement Learning in Communications and Networking: A Survey. IEEE Communications Surveys Tutorials, Vol. 21, 4 (2019), 3133--3174.

Digital Library

[13]

Yasar Sinan Nasir and Dongning Guo. 2020. Deep Actor-Critic Learning for Distributed Power Control in Wireless Mobile Networks. In 2020 54th Asilomar Conference on Signals, Systems, and Computers. IEEE, 398--402.

[14]

Keyvan Ramezanpour and Jithin Jagannath. 2022. Intelligent Zero Trust Architecture for 5G/6G Networks: Principles, Challenges, and the Role of Machine Learning in the context of O-RAN . arXiv preprint arXiv:2105.01478 (2022).

[15]

Kaiming Shen and Wei Yu. 2018. Fractional programming for communication systems-Part I: Power control and beamforming. IEEE Transactions on Signal Processing, Vol. 66, 10 (2018), 2616--2630.

[16]

Haoran Sun, Xiangyi Chen, Qingjiang Shi, Mingyi Hong, Xiao Fu, and Nicholas D Sidiropoulos. 2018. Learning to optimize: Training deep neural networks for interference management. IEEE Transactions on Signal Processing (2018).

[17]

John Tadrous, Ahmed Sultan, Mohammed Nafie, and Amr El-Keyi. 2010. Power control for constrained throughput maximization in spectrum shared networks. In IEEE Global Communications Conference (GLOBECOM) .

[18]

Chen Tessler, Nadav Merlis, and Shie Mannor. 2019. Stabilizing off-policy reinforcement learning with conservative policy gradients. (2019).

[19]

Mingbo Xiao, Ness B Shroff, and Edwin KP Chong. 2003. A utility-based power-control scheme in wireless cellular systems. IEEE/ACM Transactions On Networking, Vol. 11, 2 (2003), 210--221.

Digital Library

Cited By

Rajasekhar IMonisha M(2024)Research direction towards exploring software control radio using machine learning algorithms for smart hotel applications with future aspectsAustralian Journal of Electrical and Electronics Engineering10.1080/1448837X.2024.2398276(1-24)Online publication date: 13-Sep-2024
https://doi.org/10.1080/1448837X.2024.2398276
Kafle SJagannath JKane ZBiswas NKumar PJagannath A(2023)Generalization of Deep Reinforcement Learning for Jammer-Resilient Frequency and Power AllocationIEEE Communications Letters10.1109/LCOMM.2023.327459427:7(1789-1793)Online publication date: Jul-2023
https://doi.org/10.1109/LCOMM.2023.3274594
Yahmed AAllah Abbassi ANikanjam ALi HKhomh F(2023)Deploying Deep Reinforcement Learning Systems: A Taxonomy of Challenges2023 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME58846.2023.00015(26-38)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICSME58846.2023.00015

Index Terms

MR-iNet Gym: Framework for Edge Deployment of Deep Reinforcement Learning on Embedded Software Defined Radio
1. Computing methodologies
  1. Machine learning
2. Networks
  1. Network protocols

Recommendations

Marconi-Rosenblatt Framework for Intelligent Networks (MR-iNet Gym): For Rapid Design and Implementation of Distributed Multi-agent Reinforcement Learning Solutions for Wireless Networks
Abstract
We present the Marconi-Rosenblatt Framework for Intelligent Networks (MR-iNet Gym) an open-source architecture designed for accelerating research and development of novel reinforcement learning applied to distributed wireless networks. To ensure ...
IoT Sensor Gym: Training Autonomous IoT Devices with Deep Reinforcement Learning
IoT '19: Proceedings of the 9th International Conference on the Internet of Things

We describe IoT Sensor Gym, a framework to train the behavior of constrained IoT devices using deep reinforcement learning. We focus on the main architectural choices to align problems from the IoT domain with cutting-edge reinforcement learning ...
Deep Reinforcement Learning based reliable spectrum sensing under SSDF attacks in Cognitive Radio networks
Abstract
In Spectrum Sensing Data Falsification (SSDF) attacks, malicious Secondary Users (SUs) actively send erroneous local sensing results to the Fusion Centre (FC) that influence the actual outcomes of Cooperative Spectrum Sensing (CSS). ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WiseML '22: Proceedings of the 2022 ACM Workshop on Wireless Security and Machine Learning

May 2022

93 pages

ISBN:9781450392778

DOI:10.1145/3522783

General Chair:
Murtuza Jadliwala
University of Texas at San Antonio, San Antonio, Texas, USA

Copyright © 2022 ACM.

Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 16 May 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

U.S. Army

Conference

WiSec '22

Sponsor:

WiSec '22: 15th ACM Conference on Security and Privacy in Wireless and Mobile Networks

May 19, 2022

TX, San Antonio, USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
242
Total Downloads

Downloads (Last 12 months)100
Downloads (Last 6 weeks)13

Reflects downloads up to 23 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Rajasekhar IMonisha M(2024)Research direction towards exploring software control radio using machine learning algorithms for smart hotel applications with future aspectsAustralian Journal of Electrical and Electronics Engineering10.1080/1448837X.2024.2398276(1-24)Online publication date: 13-Sep-2024
https://doi.org/10.1080/1448837X.2024.2398276
Kafle SJagannath JKane ZBiswas NKumar PJagannath A(2023)Generalization of Deep Reinforcement Learning for Jammer-Resilient Frequency and Power AllocationIEEE Communications Letters10.1109/LCOMM.2023.327459427:7(1789-1793)Online publication date: Jul-2023
https://doi.org/10.1109/LCOMM.2023.3274594
Yahmed AAllah Abbassi ANikanjam ALi HKhomh F(2023)Deploying Deep Reinforcement Learning Systems: A Taxonomy of Challenges2023 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME58846.2023.00015(26-38)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICSME58846.2023.00015
Rosen DRochez IMcIrvin CLee JD'Alessandro KWiecek MHoang NSaffarini RPhilips SJones VIvey WHarris-Smart ZHarris-Smart ZChin ZJohnson AJones AHeadley W(2023)RFRL Gym: A Reinforcement Learning Testbed for Cognitive Radio Applications2023 International Conference on Machine Learning and Applications (ICMLA)10.1109/ICMLA58977.2023.00046(279-286)Online publication date: 15-Dec-2023
https://doi.org/10.1109/ICMLA58977.2023.00046

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents