-
Automated Configuration Synthesis for Machine Learning Models: A git-Based Requirement and Architecture Management System
Authors:
Abdullatif AlShriaf,
Hans-Martin Heyn,
Eric Knauss
Abstract:
This work introduces a tool for generating runtime configurations automatically from textual requirements stored as artifacts in git repositories (a.k.a. T-Reqs) alongside the software code. The tool leverages T-Reqs-modelled architectural description to identify relevant configuration properties for the deployment of artificial intelligence (AI)-enabled software systems. This enables traceable co…
▽ More
This work introduces a tool for generating runtime configurations automatically from textual requirements stored as artifacts in git repositories (a.k.a. T-Reqs) alongside the software code. The tool leverages T-Reqs-modelled architectural description to identify relevant configuration properties for the deployment of artificial intelligence (AI)-enabled software systems. This enables traceable configuration generation, taking into account both functional and non-functional requirements. The resulting configuration specification also includes the dynamic properties that need to be adjusted and the rationale behind their adjustment. We show that this intermediary format can be directly used by the system or adapted for specific targets, for example in order to achieve runtime optimisations in term of ML model size before deployment.
△ Less
Submitted 26 April, 2024;
originally announced April 2024.
-
VEDLIoT -- Next generation accelerated AIoT systems and applications
Authors:
Kevin Mika,
René Griessl,
Nils Kucza,
Florian Porrmann,
Martin Kaiser,
Lennart Tigges,
Jens Hagemeyer,
Pedro Trancoso,
Muhammad Waqar Azhar,
Fareed Qararyah,
Stavroula Zouzoula,
Jämes Ménétrey,
Marcelo Pasin,
Pascal Felber,
Carina Marcus,
Oliver Brunnegard,
Olof Eriksson,
Hans Salomonsson,
Daniel Ödman,
Andreas Ask,
Antonio Casimiro,
Alysson Bessani,
Tiago Carvalho,
Karol Gugala,
Piotr Zierhoffer
, et al. (7 additional authors not shown)
Abstract:
The VEDLIoT project aims to develop energy-efficient Deep Learning methodologies for distributed Artificial Intelligence of Things (AIoT) applications. During our project, we propose a holistic approach that focuses on optimizing algorithms while addressing safety and security challenges inherent to AIoT systems. The foundation of this approach lies in a modular and scalable cognitive IoT hardware…
▽ More
The VEDLIoT project aims to develop energy-efficient Deep Learning methodologies for distributed Artificial Intelligence of Things (AIoT) applications. During our project, we propose a holistic approach that focuses on optimizing algorithms while addressing safety and security challenges inherent to AIoT systems. The foundation of this approach lies in a modular and scalable cognitive IoT hardware platform, which leverages microserver technology to enable users to configure the hardware to meet the requirements of a diverse array of applications. Heterogeneous computing is used to boost performance and energy efficiency. In addition, the full spectrum of hardware accelerators is integrated, providing specialized ASICs as well as FPGAs for reconfigurable computing. The project's contributions span across trusted computing, remote attestation, and secure execution environments, with the ultimate goal of facilitating the design and deployment of robust and efficient AIoT systems. The overall architecture is validated on use-cases ranging from Smart Home to Automotive and Industrial IoT appliances. Ten additional use cases are integrated via an open call, broadening the range of application areas.
△ Less
Submitted 9 May, 2023;
originally announced May 2023.
-
Automotive Perception Software Development: An Empirical Investigation into Data, Annotation, and Ecosystem Challenges
Authors:
Hans-Martin Heyn,
Khan Mohammad Habibullah,
Eric Knauss,
Jennifer Horkoff,
Markus Borg,
Alessia Knauss,
Polly Jing Li
Abstract:
Software that contains machine learning algorithms is an integral part of automotive perception, for example, in driving automation systems. The development of such software, specifically the training and validation of the machine learning components, require large annotated datasets. An industry of data and annotation services has emerged to serve the development of such data-intensive automotive…
▽ More
Software that contains machine learning algorithms is an integral part of automotive perception, for example, in driving automation systems. The development of such software, specifically the training and validation of the machine learning components, require large annotated datasets. An industry of data and annotation services has emerged to serve the development of such data-intensive automotive software components. Wide-spread difficulties to specify data and annotation needs challenge collaborations between OEMs (Original Equipment Manufacturers) and their suppliers of software components, data, and annotations. This paper investigates the reasons for these difficulties for practitioners in the Swedish automotive industry to arrive at clear specifications for data and annotations. The results from an interview study show that a lack of effective metrics for data quality aspects, ambiguities in the way of working, unclear definitions of annotation quality, and deficits in the business ecosystems are causes for the difficulty in deriving the specifications. We provide a list of recommendations that can mitigate challenges when deriving specifications and we propose future research opportunities to overcome these challenges. Our work contributes towards the on-going research on accountability of machine learning as applied to complex software systems, especially for high-stake applications such as automated driving.
△ Less
Submitted 10 March, 2023;
originally announced March 2023.
-
Requirements Engineering for Automotive Perception Systems: an Interview Study
Authors:
Khan Mohammad Habibullah,
Hans-Martin Heyn,
Gregory Gay,
Jennifer Horkoff,
Eric Knauss,
Markus Borg,
Alessia Knauss,
Håkan Sivencrona,
Polly Jing Li
Abstract:
Background: Driving automation systems (DAS), including autonomous driving and advanced driver assistance, are an important safety-critical domain. DAS often incorporate perceptions systems that use machine learning (ML) to analyze the vehicle environment. Aims: We explore new or differing requirements engineering (RE) topics and challenges that practitioners experience in this domain. Method: We…
▽ More
Background: Driving automation systems (DAS), including autonomous driving and advanced driver assistance, are an important safety-critical domain. DAS often incorporate perceptions systems that use machine learning (ML) to analyze the vehicle environment. Aims: We explore new or differing requirements engineering (RE) topics and challenges that practitioners experience in this domain. Method: We have conducted an interview study with 19 participants across five companies and performed thematic analysis. Results: Practitioners have difficulty specifying upfront requirements, and often rely on scenarios and operational design domains (ODDs) as RE artifacts. Challenges relate to ODD detection and ODD exit detection, realistic scenarios, edge case specification, breaking down requirements, traceability, creating specifications for data and annotations, and quantifying quality requirements. Conclusions: Our findings contribute to understanding how RE is practiced for DAS perception systems and the collected challenges can drive future research for DAS and other ML-enabled systems.
△ Less
Submitted 23 February, 2023;
originally announced February 2023.
-
An investigation of challenges encountered when specifying training data and runtime monitors for safety critical ML applications
Authors:
Hans-Martin Heyn,
Eric Knauss,
Iswarya Malleswaran,
Shruthi Dinakaran
Abstract:
Context and motivation: The development and operation of critical software that contains machine learning (ML) models requires diligence and established processes. Especially the training data used during the development of ML models have major influences on the later behaviour of the system. Runtime monitors are used to provide guarantees for that behaviour. Question / problem: We see major uncer…
▽ More
Context and motivation: The development and operation of critical software that contains machine learning (ML) models requires diligence and established processes. Especially the training data used during the development of ML models have major influences on the later behaviour of the system. Runtime monitors are used to provide guarantees for that behaviour. Question / problem: We see major uncertainty in how to specify training data and runtime monitoring for critical ML models and by this specifying the final functionality of the system. In this interview-based study we investigate the underlying challenges for these difficulties. Principal ideas/results: Based on ten interviews with practitioners who develop ML models for critical applications in the automotive and telecommunication sector, we identified 17 underlying challenges in 6 challenge groups that relate to the challenge of specifying training data and runtime monitoring. Contribution: The article provides a list of the identified underlying challenges related to the difficulties practitioners experience when specifying training data and runtime monitoring for ML models. Furthermore, interconnection between the challenges were found and based on these connections recommendation proposed to overcome the root causes for the challenges.
△ Less
Submitted 31 January, 2023;
originally announced January 2023.
-
A Compositional Approach to Creating Architecture Frameworks with an Application to Distributed AI Systems
Authors:
Hans-Martin Heyn,
Eric Knauss,
Patrizio Pelliccione
Abstract:
Artificial intelligence (AI) in its various forms finds more and more its way into complex distributed systems. For instance, it is used locally, as part of a sensor system, on the edge for low-latency high-performance inference, or in the cloud, e.g. for data mining. Modern complex systems, such as connected vehicles, are often part of an Internet of Things (IoT). To manage complexity, architectu…
▽ More
Artificial intelligence (AI) in its various forms finds more and more its way into complex distributed systems. For instance, it is used locally, as part of a sensor system, on the edge for low-latency high-performance inference, or in the cloud, e.g. for data mining. Modern complex systems, such as connected vehicles, are often part of an Internet of Things (IoT). To manage complexity, architectures are described with architecture frameworks, which are composed of a number of architectural views connected through correspondence rules. Despite some attempts, the definition of a mathematical foundation for architecture frameworks that are suitable for the development of distributed AI systems still requires investigation and study. In this paper, we propose to extend the state of the art on architecture framework by providing a mathematical model for system architectures, which is scalable and supports co-evolution of different aspects for example of an AI system. Based on Design Science Research, this study starts by identifying the challenges with architectural frameworks. Then, we derive from the identified challenges four rules and we formulate them by exploiting concepts from category theory. We show how compositional thinking can provide rules for the creation and management of architectural frameworks for complex systems, for example distributed systems with AI. The aim of the paper is not to provide viewpoints or architecture models specific to AI systems, but instead to provide guidelines based on a mathematical formulation on how a consistent framework can be built up with existing, or newly created, viewpoints. To put in practice and test the approach, the identified and formulated rules are applied to derive an architectural framework for the EU Horizon 2020 project ``Very efficient deep learning in the IoT" (VEDLIoT) in the form of a case study.
△ Less
Submitted 27 December, 2022;
originally announced December 2022.
-
VEDLIoT: Very Efficient Deep Learning in IoT
Authors:
Martin Kaiser,
Rene Griessl,
Nils Kucza,
Carola Haumann,
Lennart Tigges,
Kevin Mika,
Jens Hagemeyer,
Florian Porrmann,
Ulrich Rückert,
Micha vor dem Berge,
Stefan. Krupop,
Mario Porrmann,
Marco Tassemeier,
Pedro Trancoso,
Fareed Quararyah,
Stavroula Zouzoula,
Antonio Casimiro,
Alysson Bessani,
Jose Cecilio,
Stefan Andersson,
Oliver Brunnegard,
Olof Eriksson,
Roland Weiss,
Franz Meierhöfer,
Hans Salomonsson
, et al. (11 additional authors not shown)
Abstract:
The VEDLIoT project targets the development of energy-efficient Deep Learning for distributed AIoT applications. A holistic approach is used to optimize algorithms while also dealing with safety and security challenges. The approach is based on a modular and scalable cognitive IoT hardware platform. Using modular microserver technology enables the user to configure the hardware to satisfy a wide r…
▽ More
The VEDLIoT project targets the development of energy-efficient Deep Learning for distributed AIoT applications. A holistic approach is used to optimize algorithms while also dealing with safety and security challenges. The approach is based on a modular and scalable cognitive IoT hardware platform. Using modular microserver technology enables the user to configure the hardware to satisfy a wide range of applications. VEDLIoT offers a complete design flow for Next-Generation IoT devices required for collaboratively solving complex Deep Learning applications across distributed systems. The methods are tested on various use-cases ranging from Smart Home to Automotive and Industrial IoT appliances. VEDLIoT is an H2020 EU project which started in November 2020. It is currently in an intermediate stage with the first results available.
△ Less
Submitted 1 July, 2022;
originally announced July 2022.
-
Setting AI in context: A case study on defining the context and operational design domain for automated driving
Authors:
Hans-Martin Heyn,
Padmini Subbiash,
Jennifer Linder,
Eric Knauss,
Olof Eriksson
Abstract:
[Context and motivation] For automated driving systems, the operational context needs to be known in order to state guarantees on performance and safety. The operational design domain (ODD) is an abstraction of the operational context, and its definition is an integral part of the system development process. [Question / problem] There are still major uncertainties in how to clearly define and doc…
▽ More
[Context and motivation] For automated driving systems, the operational context needs to be known in order to state guarantees on performance and safety. The operational design domain (ODD) is an abstraction of the operational context, and its definition is an integral part of the system development process. [Question / problem] There are still major uncertainties in how to clearly define and document the operational context in a diverse and distributed development environment such as the automotive industry. This case study investigates the challenges with context definitions for the development of perception functions that use machine learning for automated driving. [Principal ideas/results] Based on qualitative analysis of data from semi-structured interviews, the case study shows that there is a lack of standardisation for context definitions across the industry, ambiguities in the processes that lead to deriving the ODD, missing documentation of assumptions about the operational context, and a lack of involvement of function developers in the context definition. [Contribution] The results outline challenges experienced by an automotive supplier company when defining the operational context for systems using machine learning. Furthermore, the study collected ideas for potential solutions from the perspective of practitioners.
△ Less
Submitted 27 January, 2022;
originally announced January 2022.
-
Requirement Engineering Challenges for AI-intense Systems Development
Authors:
Hans-Martin Heyn,
Eric Knauss,
Amna Pir Muhammad,
Olof Eriksson,
Jennifer Linder,
Padmini Subbiah,
Shameer Kumar Pradhan,
Sagar Tungal
Abstract:
Availability of powerful computation and communication technology as well as advances in artificial intelligence enable a new generation of complex, AI-intense systems and applications. Such systems and applications promise exciting improvements on a societal level, yet they also bring with them new challenges for their development. In this paper we argue that significant challenges relate to defi…
▽ More
Availability of powerful computation and communication technology as well as advances in artificial intelligence enable a new generation of complex, AI-intense systems and applications. Such systems and applications promise exciting improvements on a societal level, yet they also bring with them new challenges for their development. In this paper we argue that significant challenges relate to defining and ensuring behaviour and quality attributes of such systems and applications. We specifically derive four challenge areas from relevant use cases of complex, AI-intense systems and applications related to industry, transportation, and home automation: understanding, determining, and specifying (i) contextual definitions and requirements, (ii) data attributes and requirements, (iii) performance definition and monitoring, and (iv) the impact of human factors on system acceptance and success. Solving these challenges will imply process support that integrates new requirements engineering methods into development approaches for complex, AI-intense systems and applications. We present these challenges in detail and propose a research roadmap.
△ Less
Submitted 22 March, 2021; v1 submitted 18 March, 2021;
originally announced March 2021.
