research-article

High Costs and Small Benefits: A Field Study of How Users Experience Operating System Upgrades

Authors:

Francesco Vitale,

Joanna McGrenere,

Aurélien Tabard,

Michel Beaudouin-Lafon,

Wendy E. MackayAuthors Info & Claims

CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

Pages 4242 - 4253

https://doi.org/10.1145/3025453.3025509

Published: 02 May 2017 Publication History

Abstract

Users must manage frequent software and operating system upgrades across multiple computing devices. While current research focuses primarily on the security aspect, we investigate the user's perspective of upgrading software. Our first study (n=65) found that users delay major upgrades by an average of 80 days. We then ran a field study (n=14), beginning with in-depth observations during an operating system upgrade, followed by a four-week diary study. Very few participants prepared for upgrades (e.g., backing up files), and over half had negative reactions to the upgrade process and other changes (e.g., bugs, lost settings, unwanted features). During the upgrade process, waiting times were too long, feedback was confusing or misleading, and few had clear mental models of what was happening. Users almost never mentioned security as a concern or reason for upgrading. By contrast, interviews (n=3) with technical staff responsible for one organization's upgrades focused only on security and licensing, not user interface changes. We conclude with recommendations to improve the user's upgrade experience.

Supplementary Material

suppl.mov (pn1311p.mp4)

Supplemental video

Download
1.20 MB

References

[1]

Ron Amadeo. 2016. Android N borrows Chrome OS code for "seamless" update installation | Ars Technica. http: //arstechnica.com/gadgets/2016/05/android-n-borrows-c hrome-os-code-for-seamless-update-installation/. (May 2016). (Accessed on August 2, 2016).

[2]

Apple. 2016. Get help with over-the-air iOS updates Apple Support. https://support.apple.com/en-us/HT201435. (June 2016). (Accessed on August 2, 2016).

[3]

Harry Brignull. 2013. Dark Patterns: inside the interfaces designed to trick you | The Verge. http://www.theverge.com/2013/8/29/4640308/dark-patte rns-inside-the-interfaces-designed-to-trick-you. (Aug. 2013). (Accessed on July 28, 2016).

[4]

Brad Chacos. 2016. Forced Windows 10 upgrades push users to dangerously disable Windows Update. http://www.pcworld.com/article/3075729/windows/fearin g-forced-windows-10-upgrades-users-are-disabling-cri tical-updates-at-their-own-risk.html. (May 2016). (Accessed on August 1, 2016).

[5]

David Coburn. 2014. How Apple's OS X Yosemite tracks you. https://www.washingtonpost.com/posttv/business/t echnology/how-apples-os-x-yosemite-tracks-you/2014/ 10/22/66df4386--59f1--11e4--9d6c-756a229d8b18_video.html. (Oct. 2014). (Accessed on August 8, 2016).

[6]

Shane Dingman. 2016. Considering updating to Windows 10? The choice may be out of your hands - The Globe and Mail. http://www.theglobeandmail.com/technology/t ech-news/considering-updating-to-windows-10-the-choic e-may-be-out-of-your-hands/article28533377/. (Feb. 2016). (Accessed on August 1, 2016).

[7]

Thomas Duebendorfer and Stefan Frei. 2009. Why silent updates boost security. TIK, ETH Zurich, Tech. Rep 302 (2009).

[8]

Kevin Dunn. 2004. Automatic update risks: can patching let a hacker in Network Security 2004, 7 (2004), 5--8.

[9]

Michael Fagan, Mohammad Maifi Hasan Khan, and Ross Buck. 2015. A study of users' experiences and beliefs about software update messages. Computers in Human Behavior 51 (2015), 504--519.

Digital Library

[10]

Michael Fagan, Mohammad Maifi Hasan Khan, and Nhan Nguyen. 2015. How does this message make you feel? A study of user perspectives on software update/warning message design. Human-centric Computing and Information Sciences 5, 1 (2015), 1.

Digital Library

[11]

Marvin Fleischmann, Miglena Amirpur, Tillmann Grupp, Alexander Benlian, and Thomas Hess. 2016. The role of software updates in information systems continuance-An experimental study from a user perspective. Decision Support Systems 83 (2016), 83--96.

Digital Library

[12]

Marita Franzke and John Rieman. 1993. Natural training wheels: Learning and transfer between two versions of a computer application. In Human Computer Interaction. Springer, 315--328.

[13]

Barbara L Fredrickson and Daniel Kahneman. 1993. Duration neglect in retrospective evaluations of affective episodes. Journal of personality and social psychology 65, 1 (1993), 45.

[14]

Stefan Frei, Thomas Duebendorfer, and Bernhard Plattner. 2008. Firefox (in) security update dynamics exposed. ACM SIGCOMM Computer Communication Review 39, 1 (2008), 16--22.

Digital Library

[15]

Samuel Gibbs. 2016. Windows 10: Microsoft launches intrusive full-screen upgrade reminder | Technology | The Guardian. https://www.theguardian.com/technology/2016/ jul/04/microsoft-windows-10-full-screen-upgrade-notif ication-pop-up-reminder. (July 2016). (Accessed on August 1, 2016).

[16]

Rob Griffiths. 2016. A useless analysis of OS X release dates | The Robservatory. http://robservatory.com/a-use less-analysis-of-os-x-release-dates/. (July 2016). (Accessed on August 1, 2016).

[17]

Iulia Ion, Rob Reeder, and Sunny Consolvo. 2015. "... no one can hack my mind": Comparing Expert and Non-Expert Security Practices. In Eleventh Symposium on Usable Privacy and Security (SOUPS 2015). 327--346.

[18]

Daniel Kahneman, Barbara L Fredrickson, Charles A Schreiber, and Donald A Redelmeier. 1993. When more pain is preferred to less: Adding a better end. Psychological science 4, 6 (1993), 401--405.

[19]

Moazzam Khan, Zehui Bi, and John A Copeland. 2012. Software updates as a security metric: Passive identification of update trends and effect on machine infection. In MILCOM 2012--2012 IEEE Military Communications Conference. IEEE, 1--6.

[20]

Huoy Min Khoo and Daniel Robey. 2007. Deciding to upgrade packaged software: a comparative case study of motives, contingencies and dependencies. European Journal of Information Systems 16, 5 (2007), 555--567.

[21]

Huoy Min Khoo, Daniel Robey, and Srinivasan Venkoba Rao. 2011. An exploratory study of the impacts of upgrading packaged software: a stakeholder perspective. Journal of Information technology 26, 3 (2011), 153--169.

[22]

Wendy E. Mackay. 1991. Triggers and Barriers to Customizing Software. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '91). ACM, NY, NY, USA, 153--160.

Digital Library

[23]

Wendy E Mackay. 2000. Responding to cognitive overload: Co-adaptation between users and technology. Intellectica 30, 1 (2000), 177--193.

[24]

Andrew Martonik. 2015. Hey Google, don't just change fundamental parts of my launcher without warning | Android Central. http://www.androidcentral.com/googl e-dont-just-change-my-launcher-without-warning. (September 2015). (Accessed on September, 10 2016).

[25]

Joanna McGrenere, Ronald M. Baecker, and Kellogg S. Booth. 2002. An Evaluation of a Multiple Interface Design Solution for Bloated Software. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '02). ACM, NY, NY, USA, 164--170.

Digital Library

[26]

Joanna McGrenere and Gale Moore. 2000. Are we all in the same" bloat"?. In Graphics interface, Vol. 2000. 187--196.

[27]

Microsoft. 2016. Upgrade to Windows 10: FAQ. https://support.microsoft.com/en-us/help/12435/window s-10-upgrade-faq. (July 2016). (Accessed on August 2, 2016).

[28]

Andreas Möller, Florian Michahelles, Stefan Diewald, Luis Roalter, and Matthias Kranz. 2012. Update behavior in app markets and security implications: A case study in google play. In Proc. of the 3rd Intl. Workshop on Research in the Large. Held in Conjunction with Mobile HCI. Citeseer, 3--6.

[29]

Brad A. Myers. 1985. The Importance of Percent-done Progress Indicators for Computer-human Interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '85). ACM, NY, NY, USA, 11--17.

Digital Library

[30]

Antonio Nappa, Richard Johnson, Leyla Bilge, Juan Caballero, and Tudor Dumitras. 2015. The attack of the clones: a study of the impact of shared code on vulnerability patching. In 2015 IEEE Symposium on Security and Privacy. IEEE, 692--708.

Digital Library

[31]

Jakob Nielsen. 1995. 10 Heuristics for User Interface Design. https: //www.nngroup.com/articles/ten-usability-heuristics/. (Jan. 1995). (Accessed on August 12, 2016).

[32]

Donald A Norman. 1983. Some observations on mental models. Mental models 7, 112 (1983), 7--14.

[33]

Robert Plutchik. 2001. The Nature of Emotions Human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice. American Scientist 89, 4 (2001), 344--350.

[34]

Jonathan Posner, James A Russell, and Bradley S Peterson. 2005. The circumplex model of affect: An integrative approach to affective neuroscience, cognitive development, and psychopathology. Development and psychopathology 17, 03 (2005), 715--734.

[35]

John Siracusa. 2014. OS X 10.10 Yosemite: The Ars Technica Review | Ars Technica. http://arstechnica.co m/apple/2014/10/os-x-10--10/9/#spotlight. (Oct. 2014). (Accessed on August 1, 2016).

[36]

Ashkan Soltani and Craig Timberg. 2014. Apple's Mac computers can automatically collect your location information - The Washington Post. https://www.washingtonpost.com/news/the-switch/wp /2014/10/20/apples-mac-computers-can-automatically-c ollect-your-location-information/. (Oct. 2014). (Accessed on August 1, 2016).

[37]

Franck Tétard and Mikael Collan. 2009. Lazy user theory: A dynamic model to understand user selection of products and services. In System Sciences, 2009. HICSS'09. 42nd Hawaii International Conference on. IEEE, 1--9.

[38]

Yuan Tian, Bin Liu, Weisi Dai, Blase Ur, Patrick Tague, and Lorrie Faith Cranor. 2015. Supporting Privacy-Conscious App Update Decisions with User Reviews. In Proceedings of the 5th Annual ACM CCS Workshop on Security and Privacy in Smartphones and Mobile Devices. ACM, 51--61.

Digital Library

[39]

Kami Vaniea and Yasmeen Rashidi. 2016. Tales of Software Updates: The Process of Updating Software. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 3215--3226.

Digital Library

[40]

Kami E. Vaniea, Emilee Rader, and Rick Wash. 2014. Betrayed by Updates: How Negative Experiences Affect Future Security. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14). ACM, NY, NY, USA, 2671--2674.

Digital Library

[41]

Rick Wash, Emilee J Rader, Kami Vaniea, and Michelle Rizor. 2014. Out of the Loop: How Automated Software Updates Cause Unintended Security Consequences. In SOUPS. 89--104.

[42]

Lance Whitney. 2016. Microsoft's Windows 10 upgrades are getting even more sneaky-pushy - CNET. http://www.cnet.com/news/microsoft-windows-10-upgrade s-get-more-sneaky-pushy/. (May 2016). (Accessed on August 1, 2016).

[43]

Timothy D Wilson and Daniel T Gilbert. 2005. Affective forecasting knowing what to want. Current Directions in Psychological Science 14, 3 (2005), 131--134.

Cited By

Burke WStranieri AOseni TGondal I(2024)The need for cybersecurity self-evaluation in healthcareBMC Medical Informatics and Decision Making10.1186/s12911-024-02551-x24:1Online publication date: 23-May-2024
https://doi.org/10.1186/s12911-024-02551-x
Komatsu TXie CYamada S(2024)Waiting Time Perceptions for Faster Count-downs/ups Are More Sensitive Than Slower Ones: Experimental Investigation and Its ApplicationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641942(1-13)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641942
Chalhoub GMartin A(2023)But is it exploitable? Exploring how Router Vendors Manage and Patch Security Vulnerabilities in Consumer-Grade RoutersProceedings of the 2023 European Symposium on Usable Security10.1145/3617072.3617110(277-295)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3617072.3617110
Show More Cited By

Index Terms

High Costs and Small Benefits: A Field Study of How Users Experience Operating System Upgrades

Recommendations

Driven to distraction

Display Omitted An observational study examined the apparent purpose of interruptions in critical care.Information seeking and team decision-making are among frequent interruption types.Improved information displays and tools for team communication can ...
The psychological functions of avatars and alt(s)

Multiple identity systems involve a physical self and a set of virtual identities.An 8-item model of avatar functions within a multiple identity system was identified.The vast majority of participants noted all or most functions in their interviews.The ...
Older adults' perception of costs and benefits of web-based and mobile PHR technologies: a focus group approach
USAB'11: Proceedings of the 7th conference on Workgroup Human-Computer Interaction and Usability Engineering of the Austrian Computer Society: information Quality in e-Health

The goal of the study was to explore older adults perceived benefits and costs of relatively new personal health records (PHR) technologies: web-based PHRs versus mobile PHRs. Twenty-six older adults (ages 63-79 years) participated in a focus group. The ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems

May 2017

7138 pages

ISBN:9781450346559

DOI:10.1145/3025453

General Chairs:
Gloria Mark
University of California Irvine
,
Susan Fussell
Cornell University
,
Program Chairs:
Cliff Lampe
University of Michigan
,
m.c. schraefel
University of Southampton
,
Juan Pablo Hourcade
University of Iowa
,
Caroline Appert
Université Paris-Sud
,
Daniel Wigdor
University of Toronto

Copyright © 2017 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

European Research Council

Conference

CHI '17

Sponsor:

SIGCHI

CHI '17: CHI Conference on Human Factors in Computing Systems

May 6 - 11, 2017

Colorado, Denver, USA

Acceptance Rates

CHI '17 Paper Acceptance Rate 600 of 2,400 submissions, 25%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
601
Total Downloads

Downloads (Last 12 months)69
Downloads (Last 6 weeks)3

Reflects downloads up to 21 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Burke WStranieri AOseni TGondal I(2024)The need for cybersecurity self-evaluation in healthcareBMC Medical Informatics and Decision Making10.1186/s12911-024-02551-x24:1Online publication date: 23-May-2024
https://doi.org/10.1186/s12911-024-02551-x
Komatsu TXie CYamada S(2024)Waiting Time Perceptions for Faster Count-downs/ups Are More Sensitive Than Slower Ones: Experimental Investigation and Its ApplicationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641942(1-13)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3641942
Chalhoub GMartin A(2023)But is it exploitable? Exploring how Router Vendors Manage and Patch Security Vulnerabilities in Consumer-Grade RoutersProceedings of the 2023 European Symposium on Usable Security10.1145/3617072.3617110(277-295)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1145/3617072.3617110
Ma TChen SWu YDeng ESong ZChen QGuo MAamodt TJerger NSwift M(2023)Efficient Scheduler Live Update for Linux Kernel with ModularizationProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 310.1145/3582016.3582054(194-207)Online publication date: 25-Mar-2023
https://dl.acm.org/doi/10.1145/3582016.3582054
Ebbers F(2023)A Large-Scale Analysis of IoT Firmware Version Distribution in the WildIEEE Transactions on Software Engineering10.1109/TSE.2022.316396949:2(816-830)Online publication date: 1-Feb-2023
https://doi.org/10.1109/TSE.2022.3163969
Mosesso LMaudet NNano EThibault TTabard A(2023)Obsolescence Paths: Living with Aging Devices2023 International Conference on ICT for Sustainability (ICT4S)10.1109/ICT4S58814.2023.00011(13-23)Online publication date: 5-Jun-2023
https://doi.org/10.1109/ICT4S58814.2023.00011
Craig KGrover VThatcher J(2023)Integrating the perceptions that cause Resistance to IT: Expanding the nomological network of the IT identity ThreatJournal of Information Technology Case and Application Research10.1080/15228053.2023.226578626:1(35-69)Online publication date: 18-Oct-2023
https://doi.org/10.1080/15228053.2023.2265786
DeKoven LRandall AMirian AAkiwate GBlume ASaul LSchulman AVoelker GSavage S(2022)Measuring security practicesCommunications of the ACM10.1145/354713365:9(93-102)Online publication date: 19-Aug-2022
https://dl.acm.org/doi/10.1145/3547133
Tchernavskij PBødker S(2022)Entangled Artifacts: The Meeting Between a Volunteer-run Citizen Science Project and a Biodiversity Data PlatformNordic Human-Computer Interaction Conference10.1145/3546155.3546682(1-13)Online publication date: 8-Oct-2022
https://dl.acm.org/doi/10.1145/3546155.3546682
Blázquez EPastrana SFeal ÁGamba JKotzias PVallina-Rodriguez NTapiador J(2021)Trouble Over-The-Air: An Analysis of FOTA Apps in the Android Ecosystem2021 IEEE Symposium on Security and Privacy (SP)10.1109/SP40001.2021.00095(1606-1622)Online publication date: May-2021
https://doi.org/10.1109/SP40001.2021.00095
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents