-
A Hybrid Spiking-Convolutional Neural Network Approach for Advancing Machine Learning Models
Authors:
Sanaullah,
Kaushik Roy,
Ulrich Rückert,
Thorsten Jungeblut
Abstract:
In this article, we propose a novel standalone hybrid Spiking-Convolutional Neural Network (SC-NN) model and test on using image inpainting tasks. Our approach uses the unique capabilities of SNNs, such as event-based computation and temporal processing, along with the strong representation learning abilities of CNNs, to generate high-quality inpainted images. The model is trained on a custom data…
▽ More
In this article, we propose a novel standalone hybrid Spiking-Convolutional Neural Network (SC-NN) model and test on using image inpainting tasks. Our approach uses the unique capabilities of SNNs, such as event-based computation and temporal processing, along with the strong representation learning abilities of CNNs, to generate high-quality inpainted images. The model is trained on a custom dataset specifically designed for image inpainting, where missing regions are created using masks. The hybrid model consists of SNNConv2d layers and traditional CNN layers. The SNNConv2d layers implement the leaky integrate-and-fire (LIF) neuron model, capturing spiking behavior, while the CNN layers capture spatial features. In this study, a mean squared error (MSE) loss function demonstrates the training process, where a training loss value of 0.015, indicates accurate performance on the training set and the model achieved a validation loss value as low as 0.0017 on the testing set. Furthermore, extensive experimental results demonstrate state-of-the-art performance, showcasing the potential of integrating temporal dynamics and feature extraction in a single network for image inpainting.
△ Less
Submitted 11 July, 2024;
originally announced July 2024.
-
Design-Space Exploration of SNN Models using Application-Specific Multi-Core Architectures
Authors:
Sanaullah,
Shamini Koravuna,
Ulrich Rückert,
Thorsten Jungeblut
Abstract:
With the motivation and the difficulties that currently exist in comprehending and utilizing the promising features of SNNs, we proposed a novel run-time multi-core architecture-based simulator called "RAVSim" (Runtime Analysis and Visualization Simulator), a cutting-edge SNN simulator, developed using LabVIEW and it is publicly available on their website as an official module. RAVSim is a runtime…
▽ More
With the motivation and the difficulties that currently exist in comprehending and utilizing the promising features of SNNs, we proposed a novel run-time multi-core architecture-based simulator called "RAVSim" (Runtime Analysis and Visualization Simulator), a cutting-edge SNN simulator, developed using LabVIEW and it is publicly available on their website as an official module. RAVSim is a runtime virtual simulation environment tool that enables the user to interact with the model, observe its behavior of output concentration, and modify the set of parametric values at any time while the simulation is in execution. Recently some popular tools have been presented, but we believe that none of the tools allow users to interact with the model simulation in run time.
△ Less
Submitted 25 March, 2024; v1 submitted 7 February, 2024;
originally announced March 2024.
-
Bidirectional UWB Localization: A Review on an Elastic Positioning Scheme for GNSS-deprived Zones
Authors:
Cung Lian Sang,
Michael Adams,
Marc Hesse,
Ulrich Rückert
Abstract:
A bidirectional Ultra-Wideband (UWB) localization scheme is one of the three widely adopted design integration processes commonly used in time-based UWB positioning systems. The key property of bidirectional UWB localization is its ability to serve both navigation and tracking tasks within a single localization scheme on demand. Traditionally, navigation and tracking in wireless localization syste…
▽ More
A bidirectional Ultra-Wideband (UWB) localization scheme is one of the three widely adopted design integration processes commonly used in time-based UWB positioning systems. The key property of bidirectional UWB localization is its ability to serve both navigation and tracking tasks within a single localization scheme on demand. Traditionally, navigation and tracking in wireless localization systems were treated as separate entities due to distinct applicable use-cases and methodological needs in each implementation process. Therefore, the ability to flexibly or elastically combine two unique positioning perspectives (navigation and tracking) within a single scheme can be regarded as a paradigm shift in the way location-based services are conventionally observed. This article reviews the mentioned bidirectional UWB localization from the perspective of a flexible and versatile positioning topology and highlights its potential in the field. In this regard, the article comprehensively describes the complete system model of the bidirectional UWB localization scheme using modular processes. It also discusses the demonstrative evaluation of two system integration processes and conducts a SWOT (Strengths, Weaknesses, Opportunities, and Threats) analysis of the scheme. Furthermore, the prospect of the presented bidirectional localization scheme for achieving precise location estimation in 5G/6G wireless mobile networks, as well as in Wi-Fi fine-time measurement-based positioning systems was briefly discussed.
△ Less
Submitted 21 December, 2023; v1 submitted 15 February, 2023;
originally announced February 2023.
-
VEDLIoT: Very Efficient Deep Learning in IoT
Authors:
Martin Kaiser,
Rene Griessl,
Nils Kucza,
Carola Haumann,
Lennart Tigges,
Kevin Mika,
Jens Hagemeyer,
Florian Porrmann,
Ulrich Rückert,
Micha vor dem Berge,
Stefan. Krupop,
Mario Porrmann,
Marco Tassemeier,
Pedro Trancoso,
Fareed Quararyah,
Stavroula Zouzoula,
Antonio Casimiro,
Alysson Bessani,
Jose Cecilio,
Stefan Andersson,
Oliver Brunnegard,
Olof Eriksson,
Roland Weiss,
Franz Meierhöfer,
Hans Salomonsson
, et al. (11 additional authors not shown)
Abstract:
The VEDLIoT project targets the development of energy-efficient Deep Learning for distributed AIoT applications. A holistic approach is used to optimize algorithms while also dealing with safety and security challenges. The approach is based on a modular and scalable cognitive IoT hardware platform. Using modular microserver technology enables the user to configure the hardware to satisfy a wide r…
▽ More
The VEDLIoT project targets the development of energy-efficient Deep Learning for distributed AIoT applications. A holistic approach is used to optimize algorithms while also dealing with safety and security challenges. The approach is based on a modular and scalable cognitive IoT hardware platform. Using modular microserver technology enables the user to configure the hardware to satisfy a wide range of applications. VEDLIoT offers a complete design flow for Next-Generation IoT devices required for collaboratively solving complex Deep Learning applications across distributed systems. The methods are tested on various use-cases ranging from Smart Home to Automotive and Industrial IoT appliances. VEDLIoT is an H2020 EU project which started in November 2020. It is currently in an intermediate stage with the first results available.
△ Less
Submitted 1 July, 2022;
originally announced July 2022.
-
A Unified Approach to Entity-Centric Context Tracking in Social Conversations
Authors:
Ulrich Rückert,
Srinivas Sunkara,
Abhinav Rastogi,
Sushant Prakash,
Pranav Khaitan
Abstract:
In human-human conversations, Context Tracking deals with identifying important entities and keeping track of their properties and relationships. This is a challenging problem that encompasses several subtasks such as slot tagging, coreference resolution, resolving plural mentions and entity linking. We approach this problem as an end-to-end modeling task where the conversational context is repres…
▽ More
In human-human conversations, Context Tracking deals with identifying important entities and keeping track of their properties and relationships. This is a challenging problem that encompasses several subtasks such as slot tagging, coreference resolution, resolving plural mentions and entity linking. We approach this problem as an end-to-end modeling task where the conversational context is represented by an entity repository containing the entity references mentioned so far, their properties and the relationships between them. The repository is updated turn-by-turn, thus making training and inference computationally efficient even for long conversations. This paper lays the groundwork for an investigation of this framework in two ways. First, we release Contrack, a large scale human-human conversation corpus for context tracking with people and location annotations. It contains over 7000 conversations with an average of 11.8 turns, 5.8 entities and 15.2 references per conversation. Second, we open-source a neural network architecture for context tracking. Finally we compare this network to state-of-the-art approaches for the subtasks it subsumes and report results on the involved tradeoffs.
△ Less
Submitted 26 April, 2022; v1 submitted 28 January, 2022;
originally announced January 2022.
-
Benchmarking Deep Spiking Neural Networks on Neuromorphic Hardware
Authors:
Christoph Ostrau,
Jonas Homburg,
Christian Klarhorst,
Michael Thies,
Ulrich Rückert
Abstract:
With more and more event-based neuromorphic hardware systems being developed at universities and in industry, there is a growing need for assessing their performance with domain specific measures. In this work, we use the methodology of converting pre-trained non-spiking to spiking neural networks to evaluate the performance loss and measure the energy-per-inference for three neuromorphic hardware…
▽ More
With more and more event-based neuromorphic hardware systems being developed at universities and in industry, there is a growing need for assessing their performance with domain specific measures. In this work, we use the methodology of converting pre-trained non-spiking to spiking neural networks to evaluate the performance loss and measure the energy-per-inference for three neuromorphic hardware systems (BrainScaleS, Spikey, SpiNNaker) and common simulation frameworks for CPU (NEST) and CPU/GPU (GeNN). For analog hardware we further apply a re-training technique known as hardware-in-the-loop training to cope with device mismatch. This analysis is performed for five different networks, including three networks that have been found by an automated optimization with a neural architecture search framework. We demonstrate that the conversion loss is usually below one percent for digital implementations, and moderately higher for analog systems with the benefit of much lower energy-per-inference costs.
△ Less
Submitted 26 October, 2020; v1 submitted 3 April, 2020;
originally announced April 2020.
-
Learning to Navigate the Web
Authors:
Izzeddin Gur,
Ulrich Rueckert,
Aleksandra Faust,
Dilek Hakkani-Tur
Abstract:
Learning in environments with large state and action spaces, and sparse rewards, can hinder a Reinforcement Learning (RL) agent's learning through trial-and-error. For instance, following natural language instructions on the Web (such as booking a flight ticket) leads to RL settings where input vocabulary and number of actionable elements on a page can grow very large. Even though recent approache…
▽ More
Learning in environments with large state and action spaces, and sparse rewards, can hinder a Reinforcement Learning (RL) agent's learning through trial-and-error. For instance, following natural language instructions on the Web (such as booking a flight ticket) leads to RL settings where input vocabulary and number of actionable elements on a page can grow very large. Even though recent approaches improve the success rate on relatively simple environments with the help of human demonstrations to guide the exploration, they still fail in environments where the set of possible instructions can reach millions. We approach the aforementioned problems from a different perspective and propose guided RL approaches that can generate unbounded amount of experience for an agent to learn from. Instead of learning from a complicated instruction with a large vocabulary, we decompose it into multiple sub-instructions and schedule a curriculum in which an agent is tasked with a gradually increasing subset of these relatively easier sub-instructions. In addition, when the expert demonstrations are not available, we propose a novel meta-learning framework that generates new instruction following tasks and trains the agent more effectively. We train DQN, deep reinforcement learning agent, with Q-value function approximated with a novel QWeb neural network architecture on these smaller, synthetic instructions. We evaluate the ability of our agent to generalize to new instructions on World of Bits benchmark, on forms with up to 100 elements, supporting 14 million possible instructions. The QWeb agent outperforms the baseline without using any human demonstration achieving 100% success rate on several difficult environments.
△ Less
Submitted 21 December, 2018;
originally announced December 2018.
-
Coordinated Heterogeneous Distributed Perception based on Latent Space Representation
Authors:
Timo Korthals,
Jürgen Leitner,
Ulrich Rückert
Abstract:
We investigate a reinforcement approach for distributed sensing based on the latent space derived from multi-modal deep generative models. Our contribution provides insights to the following benefits: Detections can be exchanged effectively between robots equipped with uni-modal sensors due to a shared latent representation of information that is trained by a Variational Auto Encoder (VAE). Sensor…
▽ More
We investigate a reinforcement approach for distributed sensing based on the latent space derived from multi-modal deep generative models. Our contribution provides insights to the following benefits: Detections can be exchanged effectively between robots equipped with uni-modal sensors due to a shared latent representation of information that is trained by a Variational Auto Encoder (VAE). Sensor-fusion can be applied asynchronously due to the generative feature of the VAE. Deep Q-Networks (DQNs) are trained to minimize uncertainty in latent space by coordinating robots to a Point-of-Interest (PoI) where their sensor modality can provide beneficial information about the PoI. Additionally, we show that the decrease in uncertainty can be defined as the direct reward signal for training the DQN.
△ Less
Submitted 12 September, 2018;
originally announced September 2018.
-
Towards Inverse Sensor Mapping in Agriculture
Authors:
Timo Korthals,
Mikkel Kragh,
Peter Christiansen,
Ulrich Rückert
Abstract:
In recent years, the drive of the Industry 4.0 initiative has enriched industrial and scientific approaches to build self-driving cars or smart factories. Agricultural applications benefit from both advances, as they are in reality mobile driving factories which process the environment. Therefore, acurate perception of the surrounding is a crucial task as it involves the goods to be processed, in…
▽ More
In recent years, the drive of the Industry 4.0 initiative has enriched industrial and scientific approaches to build self-driving cars or smart factories. Agricultural applications benefit from both advances, as they are in reality mobile driving factories which process the environment. Therefore, acurate perception of the surrounding is a crucial task as it involves the goods to be processed, in contrast to standard indoor production lines. Environmental processing requires accurate and robust quantification in order to correctly adjust processing parameters and detect hazardous risks during the processing. While today approaches still implement functional elements based on a single particular set of sensors, it may become apparent that a unified representation of the environment compiled from all available information sources would be more versatile, sufficient, and cost effective. The key to this approach is the means of developing a common information language from the data provided. In this paper, we introduce and discuss techniques to build so called inverse sensor models that create a common information language among different, but typically agricultural, information providers. These can be current live sensor data, farm management systems, or long term information generated from previous processing, drones, or satellites. In the context of Industry 4.0, this enables the interoperability of different agricultural systems and allows information transparency.
△ Less
Submitted 22 May, 2018;
originally announced May 2018.
-
Development of Energy Models for Design Space Exploration of Embedded Many-Core Systems
Authors:
Christian Klarhorst,
Martin Flasskamp,
Johannes Ax,
Thorsten Jungeblut,
Wayne Kelly,
Mario Porrmann,
Ulrich Rückert
Abstract:
This paper introduces a methodology to develop energy models for the design space exploration of embedded many-core systems. The design process of such systems can benefit from sophisticated models. Software and hardware can be specifically optimized based on comprehensive knowledge about application scenario and hardware behavior. The contribution of our work is an automated framework to estimate…
▽ More
This paper introduces a methodology to develop energy models for the design space exploration of embedded many-core systems. The design process of such systems can benefit from sophisticated models. Software and hardware can be specifically optimized based on comprehensive knowledge about application scenario and hardware behavior. The contribution of our work is an automated framework to estimate the energy consumption at an arbitrary abstraction level without the need to provide further information about the system. We validated our framework with the configurable many-core system CoreVA-MPSoC. Compared to a simulation of the CoreVA-MPSoC on gate level in a 28nm FD-SOI standard cell technology, our framework shows an average estimation error of about 4%.
△ Less
Submitted 15 January, 2018; v1 submitted 12 January, 2018;
originally announced January 2018.
-
A Unifying View of Multiple Kernel Learning
Authors:
Marius Kloft,
Ulrich Rückert,
Peter L. Bartlett
Abstract:
Recent research on multiple kernel learning has lead to a number of approaches for combining kernels in regularized risk minimization. The proposed approaches include different formulations of objectives and varying regularization strategies. In this paper we present a unifying general optimization criterion for multiple kernel learning and show how existing formulations are subsumed as special ca…
▽ More
Recent research on multiple kernel learning has lead to a number of approaches for combining kernels in regularized risk minimization. The proposed approaches include different formulations of objectives and varying regularization strategies. In this paper we present a unifying general optimization criterion for multiple kernel learning and show how existing formulations are subsumed as special cases. We also derive the criterion's dual representation, which is suitable for general smooth optimization algorithms. Finally, we evaluate multiple kernel learning in this framework analytically using a Rademacher complexity bound on the generalization error and empirically in a set of experiments.
△ Less
Submitted 4 May, 2010;
originally announced May 2010.
