Methodology for Evaluating the Usability of Public Equipment for Physical Activity: An Approach to Interface with Blind and Low Vision Individuals

The activities that promote the social inclusion of people with special needs by the public administration are still scarce. Constantly in Brazil increases the number of visually impaired people, including people with blindness and low vision.

In this context, the Brazilian public gyms for physical activity appliances consist of no possibility of load adjustments, i.e., using only your body weight for activities such as stretching, flexibility, muscle strength and movement. Currently, Brazil has various academies on the national territory. Only in the city of Curitiba, capital of Paraná State there are over one hundred equipment available for use in the community. This is an initiative of the government to promote recreation, health and quality of life. The equipment may be used by anyone, working as a fitness outdoors. For correct use, just follow the simple guidelines set out in the near panel (Fig. 1).

The Brazilian public gyms for physical activity (Source: City of Curitiba, 2012)

Full size image

The equipment outdoor fitness are public. Thus, should be utilized to most people, or must meet the most different individual capacities. Given the above in theory should meet minimum requirements of Universal Design. However, this study proposes a methodology for evaluation of equipment capable of use by blind and low vision people, noting that the equipment must meet the basic criteria for use of normal users and also for the use of individuals with visual constraints, seeking by a better physical condition. In this context, this research arises from the following question: How to evaluate the usability, user experience and emotion of blind and low vision individuals in the interface with public exercise equipment?

To answer the research question we present a methodology for measuring the user experience. The methodology was adapted for people with visual constraints and emphasizes the user interaction with the products. Assessment procedures of equipment that were adopted provide evaluation criteria from ergonomic, usability and emotional factors. This methodology serves as a support for evaluation in the context of real use of fitness equipment public.

The methodology for this study includes: familiarity test with products, perception of pain or discomfort, formal perception and intuitiveness of use, evaluation of the experience [1–3] emotional factors and evaluation with a focus group after context use by all members of the group.

The result include techniques for collecting data, relating to usability specifically adapted for blind and people with low vision. This paper also reaches a pilot experiment to validate the method. The experiment was conducted with a blind person the Paraná Institute for the Blind, located in the city of Curitiba, Brazil.

This research is a study of ergonomics and usability lab of the Federal University of Paraná (UFPR), is embedded in the research group of Ergonomics (UFPR) and it was a search of the Pos-Graduate Program in Design. The project has partnered with the government of Curitiba, which reviewed the project and issued an ethical opinion for testing the city’s public spaces. Partner is still with the Paraná Institute of the Blind and the Center for Visual Support Area, located in the town of Araucaria, also in the state of Paraná, Brazil.

The resulting methodology of this research can contribute to ergonomists and designers evaluate the usability of products commonly used by individuals who are blind and low vision to find directions for design, improving user experience and to improve the configuration industrial products.

Tullis and Albert [3] present some usability metrics that reveal aspects about the user experience and interaction between the user and the object emphasizing: effectiveness, efficiency and satisfaction [4]. The authors note that the metrics are focused on users and assess behavioral aspects and attitudes, not the product alone.

Metrics are not final conclusions about the products, but means to assist the designer in decisions from data obtained through usability testing with users. Thus, the results serve during the product development process. The evaluation metrics widespread by Tullis and Albert [3] in the book “Measuring the User Experience” are:

• Performance: Evaluates aspects of specific tasks already planned in the use test. Evaluates aspects such as efficiency and effectiveness.

• Based Usability Issues: Assist the detection and identification of problems in usability. Expressed by users during testing products or systems.

• Self- Reports: Data collection occurs from reports of users.

• Behavioral and Physiological: The data collected depart from information not explicitly said or consciously planned.

Therefore, the user experience measurement metrics have been taken into account in the formulation of evaluation protocols for field-testing of this research.

2.1 Pain or Discomfort Perception

The Diagram Corlett [5, 6], adapted from Corlett and Manenica (1980), is an ergonomic assessment protocol of the user experience. It is divided into left and right body regions, each of which allows for five levels intensity of discomfort or pain: none (1); little discomfort (2); moderate (3); enough (4); excessive (5). It can be applied by the investigator or by the person, which will indicate which regions feel discomfort or pain during or after performing some activity and which side: right or left. Still, the people can report which feels discomfort level indicating some number. It is a technique for assessing postural discomfort (Fig. 2).

Corlett Diagram (Source: Humanics Ergonomics, 2015)

Full size image

2.2 Circle of Emotions

The Circle of Emoticons [7] can be used to measure the emotions of users through self-reports or tag manifest their feelings about a product or system. It requests that the individuals to report any emotion, according to the circle (Fig. 3).

Circle of Emotions by Desmet, 2007; adapted from Russel,1980 (Source: International Journal of Design, 2015).

Full size image

The experiment aimed to measure the experience of the blind and low vision users in the interface with the products of Outdoor Fitness. Before the experiments with individuals, was the application of a “Statement of Consent”. During the pilot experiment with a user were collected self-reports that expressed perceptions regarding the use of each device. During the execution of tasks on each product, questionnaires were applied on intuitiveness of use, satisfaction, comfort, pleasantness, accessibility and use of motivation. Also were collected behavioral and physiological manifestations through the audio and video recording.

The user’s performance in each of the devices was performed using the diagnostic task completion of each product, and counting the number of errors. In the course of testing the user reported their perceptions about the interface of each device, suggesting usability trends for improvement of the equipment with respect to comfort, intuitiveness and communication task.

An expert (physical coach) coordinated the prescription of tasks; the explanation of the exercises repetitions series in each unit; training objectives of each device and orientation of rest breaks. The instructions (pre-use) the physical guiding allowance had the information signs of orientation arranged in public spaces.

During the experiment the test supervisors asked the participant verbalization of self-reports, about your experience with the products concerned. At this stage, we applied some protocols about the intuitiveness, ease of performing the task, comfort, motivation and use of satisfaction.

The processes involved for the implementation of usability testing the equipment were:

• Defining the objective of the pilot test: measure the comfort, intuitive use, attractiveness of the shape of the products, learning, feedback, satisfaction and user experience through self-reports during the task on each product. Also measure performance through the diagnosis of successful completion and error number count.

• Preparation of the necessary devices: cameras, tripods, video cameras for filming and audio recording, thus allowing registering behavioral manifestations.

• Application of Questionnaires Pre- test: consent form signed by the experiment participant, confirming their agreement to participate in the pilot test. Form of Fitness for Physical Activity, which attested to the health conditions and physical necessary to participate in the test.

• Familiarity Questionnaire with the equipment, which aimed to verify whether the person had any previous contact with the products of outdoor gyms or used at some point the product often. If used, was asked if the person felt some difficulty, pain or discomfort during the course of exercise on equipment.

• Measurement Protocols of User Experience: Self-reports regarding the perception of satisfaction; pleasantness; difficulties in tasks; pain or discomfort; motivation; ease of use, learning ability; product intuitiveness (the shape of the product induced the function performed the task or not, and even if the shape was easy recognition associated with exercise); product feedback; signaling of the activity on the product itself. Still, forms were completed by the applicators of the tests, allowed to note the presence of errors during the activities in each equipment and measuring timed exercise performance in each unit.

The assessment protocols were based on usability metrics: performance, self-reports, behavioral and physiological and usability problems. During the test scenario were recorded pictures, sound recording and video for the collection of non-verbal expressions and user behavior. There has also been the completion of the task successfully and the number of errors in performing the task on each physical activity equipment.

In the pilot test, the user is encouraged to report usability issues and put suggestions for improvements that could be applied to products. Thus, the tests aimed to verify if the equipment meets the usage requirements and meet the expectations of users and the Brazilian technical standards (Figs. 4 and 5).

User using the walk simulator equipment during the pilot test (Source: The Authors, 2015).

Full size image

User using the extender equipment during the pilot test (Source: The Authors, 2015)

Full size image

Observing apparatus for physical exercise of public spaces in Brazil, we found that many of them do not include accessibility requirements in the technical and current standards in law.

Still, from the observation that many blind and low- vision users a Brazilian city practice physical activities in the equipment, note that such products can meet both people without restrictions as visually impaired.

Thus, this work proposes a methodology that evaluates the user experience of public gyms equipment in order to identify whether the products meet basic principles of universal design, as they are objects of collective use, arranged in public places. This method can be replicated in other contexts, to assess common use of products that can be used both by normal people as by disabled people.