-
SafeEmbodAI: a Safety Framework for Mobile Robots in Embodied AI Systems
Authors:
Wenxiao Zhang,
Xiangrui Kong,
Thomas Braunl,
Jin B. Hong
Abstract:
Embodied AI systems, including AI-powered robots that autonomously interact with the physical world, stand to be significantly advanced by Large Language Models (LLMs), which enable robots to better understand complex language commands and perform advanced tasks with enhanced comprehension and adaptability, highlighting their potential to improve embodied AI capabilities. However, this advancement…
▽ More
Embodied AI systems, including AI-powered robots that autonomously interact with the physical world, stand to be significantly advanced by Large Language Models (LLMs), which enable robots to better understand complex language commands and perform advanced tasks with enhanced comprehension and adaptability, highlighting their potential to improve embodied AI capabilities. However, this advancement also introduces safety challenges, particularly in robotic navigation tasks. Improper safety management can lead to failures in complex environments and make the system vulnerable to malicious command injections, resulting in unsafe behaviours such as detours or collisions. To address these issues, we propose \textit{SafeEmbodAI}, a safety framework for integrating mobile robots into embodied AI systems. \textit{SafeEmbodAI} incorporates secure prompting, state management, and safety validation mechanisms to secure and assist LLMs in reasoning through multi-modal data and validating responses. We designed a metric to evaluate mission-oriented exploration, and evaluations in simulated environments demonstrate that our framework effectively mitigates threats from malicious commands and improves performance in various environment settings, ensuring the safety of embodied AI systems. Notably, In complex environments with mixed obstacles, our method demonstrates a significant performance increase of 267\% compared to the baseline in attack scenarios, highlighting its robustness in challenging conditions.
△ Less
Submitted 3 September, 2024;
originally announced September 2024.
-
A Study on Prompt Injection Attack Against LLM-Integrated Mobile Robotic Systems
Authors:
Wenxiao Zhang,
Xiangrui Kong,
Conan Dewitt,
Thomas Braunl,
Jin B. Hong
Abstract:
The integration of Large Language Models (LLMs) like GPT-4o into robotic systems represents a significant advancement in embodied artificial intelligence. These models can process multi-modal prompts, enabling them to generate more context-aware responses. However, this integration is not without challenges. One of the primary concerns is the potential security risks associated with using LLMs in…
▽ More
The integration of Large Language Models (LLMs) like GPT-4o into robotic systems represents a significant advancement in embodied artificial intelligence. These models can process multi-modal prompts, enabling them to generate more context-aware responses. However, this integration is not without challenges. One of the primary concerns is the potential security risks associated with using LLMs in robotic navigation tasks. These tasks require precise and reliable responses to ensure safe and effective operation. Multi-modal prompts, while enhancing the robot's understanding, also introduce complexities that can be exploited maliciously. For instance, adversarial inputs designed to mislead the model can lead to incorrect or dangerous navigational decisions. This study investigates the impact of prompt injections on mobile robot performance in LLM-integrated systems and explores secure prompt strategies to mitigate these risks. Our findings demonstrate a substantial overall improvement of approximately 30.8% in both attack detection and system performance with the implementation of robust defence mechanisms, highlighting their critical role in enhancing security and reliability in mission-oriented tasks.
△ Less
Submitted 8 September, 2024; v1 submitted 6 August, 2024;
originally announced August 2024.
-
Expectations Versus Reality: Evaluating Intrusion Detection Systems in Practice
Authors:
Jake Hesford,
Daniel Cheng,
Alan Wan,
Larry Huynh,
Seungho Kim,
Hyoungshick Kim,
Jin B. Hong
Abstract:
Our paper provides empirical comparisons between recent IDSs to provide an objective comparison between them to help users choose the most appropriate solution based on their requirements. Our results show that no one solution is the best, but is dependent on external variables such as the types of attacks, complexity, and network environment in the dataset. For example, BoT_IoT and Stratosphere I…
▽ More
Our paper provides empirical comparisons between recent IDSs to provide an objective comparison between them to help users choose the most appropriate solution based on their requirements. Our results show that no one solution is the best, but is dependent on external variables such as the types of attacks, complexity, and network environment in the dataset. For example, BoT_IoT and Stratosphere IoT datasets both capture IoT-related attacks, but the deep neural network performed the best when tested using the BoT_IoT dataset while HELAD performed the best when tested using the Stratosphere IoT dataset. So although we found that a deep neural network solution had the highest average F1 scores on tested datasets, it is not always the best-performing one. We further discuss difficulties in using IDS from literature and project repositories, which complicated drawing definitive conclusions regarding IDS selection.
△ Less
Submitted 28 March, 2024; v1 submitted 26 March, 2024;
originally announced March 2024.
-
Model-based Cybersecurity Analysis: Past Work and Future Directions
Authors:
Simon Yusuf Enoch,
Mengmeng Ge,
Jin B. Hong,
Dong Seong Kim
Abstract:
Model-based evaluation in cybersecurity has a long history. Attack Graphs (AGs) and Attack Trees (ATs) were the earlier developed graphical security models for cybersecurity analysis. However, they have limitations (e.g., scalability problem, state-space explosion problem, etc.) and lack the ability to capture other security features (e.g., countermeasures). To address the limitations and to cope…
▽ More
Model-based evaluation in cybersecurity has a long history. Attack Graphs (AGs) and Attack Trees (ATs) were the earlier developed graphical security models for cybersecurity analysis. However, they have limitations (e.g., scalability problem, state-space explosion problem, etc.) and lack the ability to capture other security features (e.g., countermeasures). To address the limitations and to cope with various security features, a graphical security model named attack countermeasure tree (ACT) was developed to perform security analysis by taking into account both attacks and countermeasures. In our research, we have developed different variants of a hierarchical graphical security model to solve the complexity, dynamicity, and scalability issues involved with security models in the security analysis of systems. In this paper, we summarize and classify security models into the following; graph-based, tree-based, and hybrid security models. We discuss the development of a hierarchical attack representation model (HARM) and different variants of the HARM, its applications, and usability in a variety of domains including the Internet of Things (IoT), Cloud, Software-Defined Networking, and Moving Target Defenses. We provide the classification of the security metrics, including their discussions. Finally, we highlight existing problems and suggest future research directions in the area of graphical security models and applications. As a result of this work, a decision-maker can understand which type of HARM will suit their network or security analysis requirements.
△ Less
Submitted 24 May, 2021; v1 submitted 18 May, 2021;
originally announced May 2021.
-
Composite Metrics for Network Security Analysis
Authors:
Simon Yusuf Enoch,
Jin B. Hong,
Mengmeng Ge,
Dong Seong Kim
Abstract:
Security metrics present the security level of a system or a network in both qualitative and quantitative ways. In general, security metrics are used to assess the security level of a system and to achieve security goals. There are a lot of security metrics for security analysis, but there is no systematic classification of security metrics that are based on network reachability information. To ad…
▽ More
Security metrics present the security level of a system or a network in both qualitative and quantitative ways. In general, security metrics are used to assess the security level of a system and to achieve security goals. There are a lot of security metrics for security analysis, but there is no systematic classification of security metrics that are based on network reachability information. To address this, we propose a systematic classification of existing security metrics based on network reachability information. Mainly, we classify the security metrics into host-based and network-based metrics. The host-based metrics are classified into metrics ``without probability" and "with probability", while the network-based metrics are classified into "path-based" and "non-path based". Finally, we present and describe an approach to develop composite security metrics and it's calculations using a Hierarchical Attack Representation Model (HARM) via an example network. Our novel classification of security metrics provides a new methodology to assess the security of a system.
△ Less
Submitted 17 July, 2020; v1 submitted 7 July, 2020;
originally announced July 2020.
-
CloudSafe: A Tool for an Automated Security Analysis for Cloud Computing
Authors:
Seoungmo An,
Taehoon Eom,
Jong Sou Park,
Jin B. Hong,
Armstrong Nhlabatsi,
Noora Fetais,
Khaled M. Khan,
Dong Seong Kim
Abstract:
Cloud computing has been adopted widely, providing on-demand computing resources to improve perfornance and reduce the operational costs. However, these new functionalities also bring new ways to exploit the cloud computing environment. To assess the security of the cloud, graphical security models can be used, such as Attack Graphs and Attack Trees. However, existing models do not consider all ty…
▽ More
Cloud computing has been adopted widely, providing on-demand computing resources to improve perfornance and reduce the operational costs. However, these new functionalities also bring new ways to exploit the cloud computing environment. To assess the security of the cloud, graphical security models can be used, such as Attack Graphs and Attack Trees. However, existing models do not consider all types of threats, and also automating the security assessment functions are difficult. In this paper, we propose a new security assessment tool for the cloud named CloudSafe, an automated security assessment for the cloud. The CloudSafe tool collates various tools and frameworks to automate the security assessment process. To demonstrate the applicability of the CloudSafe, we conducted security assessment in Amazon AWS, where our experimental results showed that we can effectively gather security information of the cloud and carry out security assessment to produce security reports. Users and cloud service providers can use the security report generated by the CloudSafe to understand the security posture of the cloud being used/provided.
△ Less
Submitted 11 March, 2019;
originally announced March 2019.