skip to main content
short-paper

Poster: Twins, a Middleware for Adaptive Streaming Provenance at the Edge

Published: 05 January 2021 Publication History

Abstract

Data streaming applications process continuous flows of data to detect unusual/critical events. When it is beneficial to further analyze the source data leading to such events, fine-grained streaming provenance can be used to link each event back to its contributing data. Existing provenance tools, though, (i) can be computationally heavy, especially for applications deployed on resource-constrained devices at the edge of Cyber-Physical Systems, and (ii) cannot activate/deactivate provenance recording based on user-defined rules. To cover such gaps, we present Twins, a new adaptive provenance tool that leverages APIs found in state-of-the-art streaming frameworks to allow for custom conditions to activate/deactivate provenance recording. Our preliminary results, based on an implementation on top of Apache Flink and GeneaLog show that Twins can match, during the periods in which provenance is inactive, the performance of queries that do not record provenance at all.

References

[1]
Daniel J Abadi, Yanif Ahmad, Magdalena Balazinska, Ugur Çetintemel, Mitch Cherniack, Jeong-Hyon Hwang, Wolfgang Lindner, Anurag Maskey, Alex Rasin, Esther Ryvkina, Nesime Tatbul, Ying Xing, and Stanley B Zdonik. 2005. The Design of the Borealis Stream Processing Engine. In Cidr. 277–289. http://www.cs.harvard.edu/ mdw/course/cs260r/papers/borealis-cidr05.pdf
[2]
Tyler Akidau, Robert Bradshaw, Craig Chambers, Slava Chernyak, Rafael J Fer Andez-Moctezuma, Reuven Lax, Sam Mcveety, Daniel Mills, Frances Perry, Eric Schmidt, and Sam Whittle Google. 2015. The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing. Vldb 8, 12 (2015), 1792–1803.
[3]
Apache. 2020. Flink. http://flink.apache.org//
[4]
Stefania Costache, Vincenzo Gulisano, and Marina Papatriantafilou. 2016. Understanding the data-processing challenges in Intelligent Vehicular Systems. In 2016 IEEE Intelligent Vehicles Symposium (IV). IEEE, 611–618.
[5]
Boris Glavic, Kyumars Sheykh Esmaili, Peter M. Fischer, and Nesime Tatbul. 2014. Efficient Stream Provenance via Operator Instrumentation. ACM Transactions on Internet Technology 14, 1 (2014), 1–26. https://doi.org/10.1145/2633689
[6]
Dimitris Palyvos-Giannas, Vincenzo Gulisano, and Marina Papatriantafilou. 2018. GeneaLog: Fine-Grained Data Streaming Provenance at the Edge. In Proceedings of the 19th International Middleware Conference(Middleware ’18). ACM, New York, NY, USA, 227–238. https://doi.org/10.1145/3274808.3274826
[7]
Ivan Walulya, Dimitris Palyvos-Giannas, Yiannis Nikolakopoulos, Vincenzo Gulisano, Marina Papatriantafilou, and Philippas Tsigas. 2018. Viper: A module for communication-layer determinism and scaling in low-latency stream processing. Future Generation Computer Systems 88 (2018), 297–308.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ICDCN '21: Proceedings of the 22nd International Conference on Distributed Computing and Networking
January 2021
252 pages
ISBN:9781450389334
DOI:10.1145/3427796
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 January 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Fine-grained data provenance
  2. Middleware
  3. Stream processing

Qualifiers

  • Short-paper
  • Research
  • Refereed limited

Funding Sources

Conference

ICDCN '21

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 72
    Total Downloads
  • Downloads (Last 12 months)8
  • Downloads (Last 6 weeks)4
Reflects downloads up to 19 Oct 2024

Other Metrics

Citations

View Options

Get Access

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media