research-article

Evaluating a tool for the exploratory analysis of usability information using a cognitive walkthrough method

Authors:

Christa Womser-HackerAuthors Info & Claims

IIiX '14: Proceedings of the 5th Information Interaction in Context Symposium

Pages 243 - 246

https://doi.org/10.1145/2637002.2637033

Published: 26 August 2014 Publication History

Abstract

Results of empirical usability evaluations in large software-development organizations constitute a valuable asset for these companies. Information needs of usability professionals in these organizations are diverse, and involve both qualitative findings and quantitative data from diverse research methods and sources. Therefore, usability specialists need support for organizing, retrieving, assessing, and analyzing the internal results of usability research. This paper focuses on a method used to evaluate a prototype of a faceted retrieval tool that specifically supports usability specialists accessing a collection of usability results. The evaluation (n=11) was conducted using a primarily qualitative, scenario-based approach. Because of this, it was possible to direct evaluation towards conceptual issues instead of examining details of the surface of the interface. In addition, a survey collected answers to standardized items on the usefulness and ease of use of the system in combination with more domain specific questions. Together, these results provide a valid foundation for the assessment of the usefulness and the relative priority of features.

References

[1]

Andre, T. S., Hartson, H. R., Belz, S. M., and McCreary, F. A. The user action framework: a reliable foundation for usability engineering support tools. International Journal of Human Computer Studies 54, 1 (2001), 107--136.

Digital Library

[2]

Blandford, A. and Attfield, S. Interacting with Information. Morgan & Claypool, 2010.

Digital Library

[3]

Borlund, P. The IIR evaluation model: a framework for evaluation of interactive information retrieval systems. Information Research 8, 3 (2003), 8--3.

[4]

Diriye, A., Blandford, A., and Tombros, A. Exploring the impact of search interface features on search tasks. Research and Advanced Technology for Digital Libraries, (2010), 184--195.

Digital Library

[5]

Grigoreanu, V. and Mohanna, M. Informal cognitive walkthroughs (ICW): paring down and pairing up for an agile world. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM (2013), 3093--3096.

Digital Library

[6]

Hollingsed, T. and Novick, D. G. Usability inspection methods after 15 years of research and practice. Proc. ACM international conference on design of communication, ACM (2007), 249--255.

Digital Library

[7]

Hughes, M. A Pattern Language Approach to Usability Knowledge Management. Journal of Usability Studies 1, 2 (2006), 76--90.

Digital Library

[8]

Hughes-Morgan, K. and Wilson, M. L. Information vs Interaction - examining different interaction models over consistent metadata. Proc. IIiX 2012, (2012).

Digital Library

[9]

Hwang, Y. Measuring information behaviour performance inside a company: a case study. Information Research 16, 2 (2011).

[10]

ISO. ISO/IEC TR 25060 - Common Industry Format (CIF) for usability - General framework for usability-related information. 2010.

[11]

Kelly, D. Methods for Evaluating Interactive Information Retrieval Systems with Users. Found. Trends Inf. Retr. 3, 1--2 (2009), 1--224.

Digital Library

[12]

LaRosa, M., Poole, D., and Schusteritsch, R. Designing and deploying usetube, google's global user experience observation and recording system. Proc. SIGCHI EA '09, ACM (2009), 2971--2986.

Digital Library

[13]

Lewis, C. and Wharton, C. Cognitive walkthroughs. Handbook of human-computer interaction 2, (1997), 717--732.

[14]

Tsakonas, G. and Papatheodorou, C. Exploring usefulness and usability in the evaluation of open access digital libraries. Information Processing & Management 44, 3 (2008), 1234--1250.

Digital Library

[15]

Venkatesh, V. and Bala, H. Technology acceptance model 3 and a research agenda on interventions. Decision sciences 39, 2 (2008), 273--315.

[16]

Vilbergsdottir, S. G., Hvannberg, E. T., and Law, E. L.-C. Assessing the reliability, validity and acceptance of a classification scheme of usability problems (CUP). Journal of Systems and Software 87, (2014), 18--37.

Digital Library

[17]

Wilson, M. L., schraefel, M. C., and White, R. W. Evaluating Advanced Search Interfaces using Established Information-Seeking Models. JASIST, (2009).

Digital Library

[18]

Zhang, X., Liu, J., Li, Y., and Zhang, Y. How usable are operational digital libraries: a usability evaluation of system interactions. Proc. SIGCHI symposium on engineering interactive computing systems, ACM (2009), 177--186.

Digital Library

Cited By

Garza‐Morales GNizamis Kden Haan RGosselink GBonnema G(2024)Aligning technical and project management through participatory approaches: An industrial case studyINCOSE International Symposium10.1002/iis2.1323034:1(1646-1673)Online publication date: 7-Sep-2024
https://doi.org/10.1002/iis2.13230
Mahdi MAhmad ANatiq HSubhi MQassim Q(2021)Comprehensive Review and Future Research Directions on Dynamic Faceted SearchApplied Sciences10.3390/app1117811311:17(8113)Online publication date: 31-Aug-2021
https://doi.org/10.3390/app11178113

Index Terms

Evaluating a tool for the exploratory analysis of usability information using a cognitive walkthrough method
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods

Recommendations

Evaluating Usability: a two-fold assessment
CERI '16: Proceedings of the 4th Spanish Conference on Information Retrieval

This paper presents the progressive application of methods used in usability assessment to a single system, Searchit, a meta-search engine used by some academic libraries in the UK to allow a search across a range of databases as well as in the library'...
Using the cognitive walkthrough in iterative design
CHI '92: Posters and Short Talks of the 1992 SIGCHI Conference on Human Factors in Computing Systems

The primary goal of the Cognitive Walkthrough (CW) is to make a formal user interface evaluation method accessible to individuals who are directly involved in the design process of a new system, but do not necessarily have expertise in UI issues. As ...
Predictive usability evaluation: aligning HCI and software engineering practices
IHM '14: Proceedings of the 26th Conference on l'Interaction Homme-Machine

Can we - software developers, usability experts, user interface designers - predict usability from the early user interface (UI) design artifacts and models? Can we define predictive measures to evaluate usability without a concrete UI? These questions ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

IIiX '14: Proceedings of the 5th Information Interaction in Context Symposium

August 2014

368 pages

ISBN:9781450329767

DOI:10.1145/2637002

Conference Chairs:
David Elsweiler
University of Regensburg
,
Bernd Ludwig
University of Regensburg
,
Program Chairs:
Leif Azzopardi
University of Glasgow
,
Max L. Wilson
University of Nottingham

Copyright © 2014 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

University of Regensburg: University of Regensburg

In-Cooperation

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 August 2014

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

European Union Funds for Regional Development (EFRE)

Conference

IIiX '14

Sponsor:

University of Regensburg

IIiX '14: Fifth Information Interaction in Context Symposium

August 26 - 30, 2014

Regensburg, Germany

Acceptance Rates

IIiX '14 Paper Acceptance Rate 21 of 45 submissions, 47%;

Overall Acceptance Rate 21 of 45 submissions, 47%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
249
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)2

Reflects downloads up to 23 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Garza‐Morales GNizamis Kden Haan RGosselink GBonnema G(2024)Aligning technical and project management through participatory approaches: An industrial case studyINCOSE International Symposium10.1002/iis2.1323034:1(1646-1673)Online publication date: 7-Sep-2024
https://doi.org/10.1002/iis2.13230
Mahdi MAhmad ANatiq HSubhi MQassim Q(2021)Comprehensive Review and Future Research Directions on Dynamic Faceted SearchApplied Sciences10.3390/app1117811311:17(8113)Online publication date: 31-Aug-2021
https://doi.org/10.3390/app11178113

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents