skip to main content
research-article

FrUITeR: a framework for evaluating UI test reuse

Published: 08 November 2020 Publication History

Abstract

UI testing is tedious and time-consuming due to the manual effort required. Recent research has explored opportunities for reusing existing UI tests from an app to automatically generate new tests for other apps. However, the evaluation of such techniques currently remains manual, unscalable, and unreproducible, which can waste effort and impede progress in this emerging area. We introduce FrUITeR, a framework that automatically evaluates UI test reuse in a reproducible way. We apply FrUITeR to existing test-reuse techniques on a uniform benchmark we established, resulting in 11,917 test reuse cases from 20 apps. We report several key findings aimed at improving UI test reuse that are missed by existing work.

Supplementary Material

Auxiliary Teaser Video (fse20main-p309-p-teaser.mp4)
This is the presentation of the paper "FrUITeR: A Framework for Evaluating UI Test Reuse" that was accepted in the research track at ESEC/FSE 2020. Slides: https://speakerdeck.com/yixue_zhao/fruiter-a-framework-for-evaluating-ui-test-reuse Abstract: UI testing is tedious and time-consuming due to the manual effort required. Recent research has explored opportunities for reusing existing UI tests from an app to automatically generate new tests for other apps. However, the evaluation of such techniques currently remains manual, unscalable, and unreproducible, which can waste effort and impede progress in this emerging area. We introduce FrUITeR, a framework that automatically evaluates UI test reuse in a reproducible way. We apply FrUITeR to existing test-reuse techniques on a uniform benchmark we established, resulting in 11,917 test reuse cases from 20 apps. We report several key findings aimed at improving UI test reuse that are missed by existing work.
Auxiliary Presentation Video (fse20main-p309-p-video.mp4)
This is the presentation of the paper "FrUITeR: A Framework for Evaluating UI Test Reuse" that was accepted in the research track at ESEC/FSE 2020. Slides: https://speakerdeck.com/yixue_zhao/fruiter-a-framework-for-evaluating-ui-test-reuse Abstract: UI testing is tedious and time-consuming due to the manual effort required. Recent research has explored opportunities for reusing existing UI tests from an app to automatically generate new tests for other apps. However, the evaluation of such techniques currently remains manual, unscalable, and unreproducible, which can waste effort and impede progress in this emerging area. We introduce FrUITeR, a framework that automatically evaluates UI test reuse in a reproducible way. We apply FrUITeR to existing test-reuse techniques on a uniform benchmark we established, resulting in 11,917 test reuse cases from 20 apps. We report several key findings aimed at improving UI test reuse that are missed by existing work.

References

[1]
2019. AppFlow's source code and artifacts. https://github.com/columbia/appflow.
[2]
2019. Appium: Mobile App Automation Made Awesome. http://appium.io
[3]
2019. ATM's source code and artifacts. https://sites.google.com/view/ apptestmigrator/.
[4]
2019. Click-Appium. http://appium.io/docs/en/commands/element/actions/ click
[5]
2019. CraftDroid's source code and artifacts. https://sites.google.com/view/ craftdroid/.
[6]
2019. Espresso. https://developer.android.com/training/testing/espresso.
[7]
2019. Find Elements-Appium. http://appium.io/docs/en/commands/element/ ifnd-elements
[8]
2019. GTM's source code and artifacts. https://sites.google.com/view/ testmigration/.
[9]
2019. Introduction to Activities | Android Developers. https://developer.android. com/guide/components/activities/intro-activities
[10]
2019. Send Keys-Appium. http://appium.io/docs/en/commands/element/ actions/send-keys
[11]
2019. Soot-A Java optimization framework. https://github.com/Sable/soot.
[12]
2020. FrUITeR's website. https://felicitia.github.io/FrUITeR/.
[13]
2020. Gherkin Syntax-Cucumber Documentation. https://cucumber.io/docs/ gherkin
[14]
2020. How to locate an element on the page-Web Performance. https://www.webperformance. com/load-testing-tools/blog/articles/realbrowser-manual/building-a-testcase/how-locate-element-the-page
[15]
2020. JSON-Wikipedia. https://en.wikipedia.org/wiki/JSON
[16]
2020. SeleniumHQ Browser Automation. https://www.selenium.dev
[17]
2020. Web element :: Documentation for Selenium. https://selenium.dev/ documentation/en/webdriver/web_element
[18]
admin. 2019. Chapter-4: Appium Locator Finding Strategies-Kobiton. Kobiton (Apr 2019 ). https://kobiton.com/book/chapter-4-appium-locator-findingstrategies
[19]
Frances E. Allen and John Cocke. 1976. A program data flow analysis procedure. Commun. ACM 19, 3 ( 1976 ), 137.
[20]
Farnaz Behrang and Alessandro Orso. 2018. Test migration for eficient largescale assessment of mobile app coding assignments. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis.
[21]
Farnaz Behrang and Alessandro Orso. 2019. Test Migration Between Mobile Apps with Similar Functionality. In 34th International Conference on Automated Software Engineering (ASE 2019 ).
[22]
Shauvik Roy Choudhary, Alessandra Gorla, and Alessandro Orso. 2015. Automated test input generation for android: Are we there yet? (E). In 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).
[23]
Gang Hu, Linjie Zhu, and Junfeng Yang. 2018. AppFlow: using machine learning to synthesize robust, reusable UI tests. In Proceedings of the 2018 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM, 269-282.
[24]
Vladimir I Levenshtein. 1966. Binary codes capable of correcting deletions, insertions, and reversals. In Soviet physics doklady, Vol. 10. 707-710.
[25]
Jun-Wei Lin, Reyhaneh Jabbarvand, and Sam Malek. 2019. Test Transfer Across Mobile Apps Through Semantic Mapping. In 34th International Conference on Automated Software Engineering (ASE 2019 ).
[26]
Mario Linares-Vásquez, Carlos Bernal-Cárdenas, Kevin Moran, and Denys Poshyvanyk. 2017. How do developers test android applications?. In 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME).
[27]
Christopher D Manning, Prabhakar Raghavan, and Hinrich Schütze. 2008. Introduction to information retrieval. Cambridge university press.
[28]
Andreas Rau, Jenny Hotzkow, and Andreas Zeller. 2018. Transferring Tests Across Web Applications. In Web Engineering, Tommi Mikkonen, Ralf Klamma, and Juan Hernández (Eds.). Springer International Publishing, Cham, 50-64.

Cited By

View all
  • (2024)MUT: Human-in-the-Loop Unit Test MigrationProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639124(1-12)Online publication date: 20-May-2024
  • (2023)Flaky Tests in UI: Understanding Causes and Applying Correction StrategiesProceedings of the XXXVII Brazilian Symposium on Software Engineering10.1145/3613372.3613406(398-406)Online publication date: 25-Sep-2023
  • (2023)Property-Based Fuzzing for Finding Data Manipulation Errors in Android AppsProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616286(1088-1100)Online publication date: 30-Nov-2023
  • Show More Cited By

Index Terms

  1. FrUITeR: a framework for evaluating UI test reuse

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ESEC/FSE 2020: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering
    November 2020
    1703 pages
    ISBN:9781450370431
    DOI:10.1145/3368089
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 08 November 2020

    Permissions

    Request permissions for this article.

    Check for updates

    Badges

    Author Tags

    1. Mobile Application
    2. Open Science
    3. Software Testing
    4. Test Reuse

    Qualifiers

    • Research-article

    Funding Sources

    • U.S. National Science Foundation
    • U.S. Office of Naval Research

    Conference

    ESEC/FSE '20
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 112 of 543 submissions, 21%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)63
    • Downloads (Last 6 weeks)5
    Reflects downloads up to 19 Oct 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)MUT: Human-in-the-Loop Unit Test MigrationProceedings of the IEEE/ACM 46th International Conference on Software Engineering10.1145/3597503.3639124(1-12)Online publication date: 20-May-2024
    • (2023)Flaky Tests in UI: Understanding Causes and Applying Correction StrategiesProceedings of the XXXVII Brazilian Symposium on Software Engineering10.1145/3613372.3613406(398-406)Online publication date: 25-Sep-2023
    • (2023)Property-Based Fuzzing for Finding Data Manipulation Errors in Android AppsProceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3611643.3616286(1088-1100)Online publication date: 30-Nov-2023
    • (2023)JTestMigBench and JTestMigTax: A benchmark and taxonomy for unit test migration2023 IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER56733.2023.00077(713-717)Online publication date: Mar-2023
    • (2023)ReSPlay: Improving Cross-Platform Record-and-Replay with GUI Sequence Matching2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE)10.1109/ISSRE59848.2023.00056(439-450)Online publication date: 9-Oct-2023
    • (2023)RIDA: Cross-App Record and Replay for Android2023 IEEE Conference on Software Testing, Verification and Validation (ICST)10.1109/ICST57152.2023.00031(246-257)Online publication date: Apr-2023
    • (2023)Read It, Don't Watch It: Captioning Bug Recordings Automatically2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)10.1109/ICSE48619.2023.00197(2349-2361)Online publication date: May-2023
    • (2023)Avgust: A Tool for Generating Usage-Based Tests from Videos of App Executions2023 IEEE/ACM 45th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion)10.1109/ICSE-Companion58688.2023.00030(83-87)Online publication date: May-2023
    • (2023)Image Understanding of GUI Widgets for Test Reuse2023 3rd International Conference on Pervasive Computing and Social Networking (ICPCSN)10.1109/ICPCSN58827.2023.00100(572-579)Online publication date: Jun-2023
    • (2023)To Share, or Not to Share: Exploring Test-Case Reusability in Fork Ecosystems2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE)10.1109/ASE56229.2023.00191(837-849)Online publication date: 11-Sep-2023
    • Show More Cited By

    View Options

    Get Access

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media