• Nu S-Au Găsit Rezultate

Proceedings of the 13

N/A
N/A
Protected

Academic year: 2022

Share "Proceedings of the 13"

Copied!
129
0
0

Text complet

(1)

ROCHI – INTERNATIONAL CONFERENCE ON HUMAN-COMPUTER INTERACTION

Proceedings of the 13

th

International Conference on Human- Computer Interaction RoCHI’2016, 8-9 September, Iasi, Romania

Edited by:

Adrian Iftene

A.I. Cuza University of Iasi, Romania

and

Jean Vanderdonckt

Université catholique de Louvain, Belgium

MATRIX ROM

Bucharest, 2016

(2)

Publisher

MATRIX ROM

C.P. 16-162

062510 – Bucureşti, România Tel.: 021 4113617, Fax: 021 4114280

E-mail: [email protected] www.matrixrom.ro

MatrixRom publishing house is a certified publisher by the National Council for Scientific Research in Higher Education (Consiliul Naţional al Cercetării Ştiinţifice din Învăţământul Superior)

Cover graphic design: Sabin-Corneliu Buraga, A.I.Cuza University, Iasi Graphic identity: Adrian Mironescu, Idegrafo

© Copyright 2016

All rights of the current edition are reserved by MATRIX ROM.

No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, without permission in writing from the publisher.

RoCHI – International Conference on Human Computer Interaction (2016)

http://rochi.utcluj.ro/proceedings/en/index.php

ISSN 2501-9422

ISSN-L 2501-9422

(3)

TABLE OF CONTENTS

Organization Forword

Invited paper

 “The design is right and the right design” – How to measure and theorize “right” in HCI and Software Engineering? ………...

Ahmed Seffah

Usability and accessibility evaluation

 Implementation of a polyglot text-to-speech synthesis in two assistive technologies……..

Paul Fogarassy-Neszly, Aurel Patru, Dragos Daniel Iordache and Costin Pribeanu

Comments on the reliability and validity of UMUX and UMUX-LITE short scales...

Costin Pribeanu

 Automated Evaluation of Menu by Guidelines Review ...

Sara Bouzit, Gaëlle Calvary, Denis Chêne and Jean Vanderdonckt

 Perceived aesthetics of user-modifiable layouts: a comparison between an unspecified design and a GUI ...

Stefan Trausan-Matu, Brahma Dathan

User interface development

 Enactment of User Interface Development Methods in Software Life Cycles………

Iyad Khaddam, Hanaa Barakat and Jean Vanderdonckt

 Increasing the Accuracy of Indoor Localization Applications by Using Predefined Markers and the Phone's Camera ……….…

Szabolcs Orban, Teodor Stefanut

 Using WSN and Mobile Apps for Home and Office Ambient Monitoring and Control...….

Catalin Damian, Lenuta Alboaie and Adrian Iftene

 Sphero - Multiplayer Augmented Game (SMAUG) ...

Marian-Nicolae Pinzariu, Adrian Iftene

Interaction Techniques

 Reducing Gestural Interaction Physical Stress in Virtual Environments: an Experiment...

Sobhi Ahmed, Laure Leroy and Ari Bouaniche

 Oculus Rift 3D Interaction and Nicotine Craving: Results from a Pilot Study………

Ioana-Monica Ciolan, Sabin Buraga and Ion Dafinoiu

 Brain Computer Interface using Machine Learning ……….…

Cristian-Valeriu Soare

 Methodology for Identification and Evaluation of Web Application Performance Oriented Usability Issues...

Mihaela Ciugudean, Dorian Gorgan

v ix

1

2

6

11

22

26

36

42

46

50

58

65

69

(4)

HCI in e-learning systems

 Testing the technology acceptance model with Romanian university students...

Alexandru Balog, Costin Pribeanu

Measuring the negative effects of the Facebook dependence on the students’ university work………..……….……….

Gabriel Gorghiu, Valentina Iuliana Manea, Dragoş Daniel Iordache and Costin Pribeanu

 Evaluation of the Tesys e-Learning platform’s interface ………..

Paul Stefan Popescu, Cristian Mihaescu, Mihai Mocanu and Costel Ionascu

 MOOCBuddy: a Chatbot for personalized learning with MOOCs ………

Carmen Holotescu

Interactive applications

 ReaderBench goes Online: A Comprehension-Centered Framework for Educational Purposes………...

Marius-Gabriel Gutu, Mihai Dascalu, Stefan Trausan-Matu, Philippe Dessus

 Event detection in Tweets ..………..

Andrei Bogdan Baran, Adrian Iftene

 Rhythm analysis of texts using Natural Language Processing...

Irina-Diana Niculescu and Stefan Trausan-Matu

 Terrain Synthesis from Crude Heightmaps ...

Alexandre Philippe Mangra, Adrian Sabou, Dorian Gorgan

Author Index ...

76

81

86

91

95

103

107

113

119

(5)

ORGANIZERS

Conference Chair

Adrian Iftene, Alexandru Ioan Cuza University, Iasi, Romania

Program Committee Chair

Jean Vanderdonckt, Université catholique de Louvain, Belgium

Organizing Committee Chair

Lenuța Alboaie, Alexandru Ioan Cuza University, Iasi, Romania

RoCHI Conference Website Administrator

Teodor Stefanut, Technical University Cluj-Napoca, Romania

ORGANIZING COMMITTEE

Andrei Arusoaie, Alexandru Ioan Cuza University, Iaşi, Romania Corina Forăscu, Alexandru Ioan Cuza University, Iaşi, Romania Daniela Gîfu, Alexandru Ioan Cuza University, Iaşi, Romania Alex Moruz, Alexandru Ioan Cuza University, Iaşi, Romania Vlad Rădulescu, Alexandru Ioan Cuza University, Iaşi, Romania Cosmin Vârlan, Alexandru Ioan Cuza University, Iaşi, Romania

(6)

PROGRAM COMMITTEE

Alexandru Balog, ICI Bucharest, Romania

Sabin-Corneliu Buraga, Alexandru Ioan Cuza University, Iaşi, Romania Gilbert Cockton, Northumbria University, United Kingdom

Grigoreta Sofia Cojocar, Babeş-Bolyai University, Cluj-Napoca, Romania Karin Coninx, Hasselt University, Belgium

Marian Dârdală, University of Economic Studies, Bucharest, Romania Mihai Dascălu, University Politehnică of Bucharest, Romania Philippe Dessus, Université Grenobles Alpes, France

Bruno Dumas. Université de Namur, Belgium

Paul Fogarassy-Neszly, BAUM Engineering, Arad, Romania Peter Forbrig, University of Rostock, Germany

Alfonso Gracia Frey, Luxembourg Institute of Science and Technology, Luxembourg Vivian Genaro Motti, George Mason University, United States

Juan Gonzalez Calleros, Benemérita Universidad Autónoma de Puebla, Mexico Dorian Gorgan, Technical University Cluj-Napoca, Romania

Jan Gulliksen, KTH Royal Institute of Technology, Sweden

Adriana-Mihaela Guran, Universitatea Babeş-Bolyai, Cluj-Napoca, Romania Adrian Iftene, Alexandru Ioan Cuza University, Iaşi, Romania

Dragoş Daniel Iordache, ICI Bucharest, Romania David Lamas, Tallinn University, Estonia

Alin Moldoveanu, University Politehnică of Bucharest, Romania Philippe Palanque, IRIT, Université Paul Sabatier de Toulouse, France Oscar Pastor, Polytechnic University of Valencia, Spain

Lyn Pemberton, Brighton University, United Kingdom

Dorin Mircea Popovici, Universitaty Ovidius, Constanta, Romania Costin Pribeanu, ICI Bucharest, Romania

Traian Eugen Rebedea, University Politehnică of Bucharest, Romania Adriana Elena Reveiu, University of Economic Studies, Bucharest, Romania Cristian Rusu, Pontificia Universidad Católica de Valparaíso, Chile Carmen Santoro. ISTI-CNR, Pisa, Italy

Dominique Scapin, INRIA, France

Ahmed Seffah, Lappeeranta University of Technology, Finland Marcin Sikorski, Gdansk University, Poland

Jeam-Sebastien Sotted, Luxembourg Institute of Science and Technology, Luxembourg Christian Stary, University of Linz, Austria

Teodor Stefănuţ, Technical University Cluj-Napoca, Romania Ştefan Trăuşan-Matu, University Politehnică of Bucharest, Romania Jean Vanderdonckt, Universite catholique de Louvain, Belgium Radu Daniel Vatavu, University of Suceava, Romania

Panayiotis Zaphiris, Cyprus University of Technology, Cyprus

(7)

ADDITIONAL REVIEWERS

Lenuţa Alboaie, Alexandru Ioan Cuza University, Iaşi, Romania Felix Albu, Valahia University of Târgovişt, Romania

Adriana Alexandru, ICI Bucharest, Romania Dragos Catalin Barbu, ICI Bucharest, Romania Verginica Barbu-Mititelu, ICIA Bucharest, Romania Oana Balan, University Politehnică of Bucharest, Romania Elena Băutu, Universiy Ovidius, Constanta, Romania Victor Bâcu, Technical University Cluj-Napoca, Romania Tiberiu Boroş, ICIA Bucharest, Romania

Alexandru Butean, University Politehnică of Bucharest, Romania

Dumitru Clementin Cercel, University Politehnică of Bucharest, Romania Mihaela Colhon, University of Craiova, Romania

Liviu Cotfas, University of Economic Studies, Bucharest, Romania Titus Felix Furtună, University of Economic Studies, Bucharest, Romania Florin Girbacia, University Transilvania, Brasov, Romania

Gabriela Grosseck, University of West, Timisoara, Romania Carmen Holotescu, Ioan Slavici University, Timisoara, Romania Vincentas Lamanauskas, Siauliai University, Lithuania

Valentina Marinescu, University ofBucharest, Romania Cornelia Melenti, Technical University Cluj-Napoca, Romania Vasile Dănuţ Mihon, Technical University Cluj-Napoca, Romania Cristian Mihaescu, University of Craiova, Romania

Delia Mitrea, Technical University Cluj-Napoca, Romania

Florica Moldoveanu, University Politehnică of Bucharest, Romania Gabriel Neagu, ICI Bucharest, Romania

Mihaela Ordean, iQuest Technologies Cluj-Napoca

Marian Pădure, Babeş-Bolyai University, Cluj-Napoca, Romania Camelia Mihaela Popa, IFPAR, Bucharest, Romania

Elvira Popescu, University of Craiova, Romania Dan Rotar, University of Bacau, Romania

Corina Sas, University of Lancaster, United Kingdom Emil Stănescu, ICI Bucharest, Romania

Serban Ungureanu, Alexandru Ioan Cuza University, Iasi, Romania

(8)

viii

Conference organized with the support of:

Alexandru Ioan Cuza University of Iasi, Romania

Université catholique de Louvain, Belgium

Louvain School of Management, Belgium

International Business Machines – IBM Belgium

RoCHI – ACM SIGCHI România

This volume was published with the support of the:

National Authority for Scientific Research and Innovation (ANCSI)

(9)

Foreword

In 2016, the International Conference on Human-Computer – RoCHI has reached its 13th edition. Like all the previous editions, RoCHI 2016 provides an inter-disciplinary forum for the exchange of ideas, expertise and research results in the field of human-computer interaction.

The conference is publishing within this proceedings an invited paper and 20 regular papers organized in five thematic sessions. These papers were selected from a total of 29 submissions that were sent to RoCHI 2016. After a careful review process, each submission being assigned to a minimum of three and maximum of six reviewers in a peer-reviewed fashion (double blind), a total number of 20 short and long papers were accepted, which represents an acceptance rate of 68%.

The invited paper was entitled “The design is right and the right design” – How to measure and theorize “right” in HCI and Software Engineering? It was presented by Professor Ahmed Seffah from Lappeenranta University of Technology, Finland. It is about bridging the gap between classical design for usable user interfaces and sustainable design so that it could be measured.

The five sessions were entitled: Usability and accessibility evaluation, User interface development, Interaction techniques, HCI in e-learning systems, and Interactive applications.

The first session in the proceedings and in the conference program is Usability and accessibility evaluation. The session contains papers on topics that are at the core of HCI research, such as usability studies, automated evaluation, reliability and models for software design and GUI. The papers from this section present: (1) how can assistive technologies help people with disabilities (Implementation of a polyglot text-to-speech synthesis in two assistive technologies – Paul Fogarassy-Neszly, Aurel Patru, Dragos Daniel Iordache and Costin Pribeanu), (2) how was UMUX and UMUX-LITE scales been tested in the context of Facebook use by university students (Comments on the reliability and validity of UMUX and UMUX-LITE short scales – Costin Pribeanu), (3) how it is built ERGOSIM, a software that automatically evaluate the design of menu bars, pull-down menus, and sub-menus of a graphical user interface by reviewing usability guidelines related to menu design (Automated Evaluation of Menu by Guidelines Review – Sara Bouzit, Gaëlle Calvary, Denis Chêne and Jean Vanderdonckt) and (4) what are the results of an experiment in which students are asked to reposition rectangular shapes in a window in two cases: when they are or they are not told that the window should be considered the configuration of a graphical user interface (GUI) (Perceived aesthetics of user-modifiable layouts: a comparison between an unspecified design and a GUI – Stefan Trausan-Matu, Brahma Dathan).

The second section, User interface development, contains four papers related to user interfaces: (1) UIDLC Manager, a software that provides user interface designers and developers with methodological guidance throughout user interface development life cycle (Enactment of User Interface Development Methods in Software Life Cycles – Iyad Khaddam, Hanaa Barakat, Jean Vanderdonckt), (2) an Android application which offers the possibility to navigate through a 3D model using the movement of a mobile phone, determined by its own sensors (Increasing the Accuracy of Indoor Localization Applications by Using Predefined Markers and the Phone’s Camera – Szabolcs Orban, Teodor Stefanut), (3) a complex system (hardware and software) that is used to monitor and control some ambient parameters (temperature, humidity, light intensity, etc.) (Using WSN and Mobile Apps for Home and Office Ambient Monitoring and Control – Catalin Damian, Lenuta Alboaie and Adrian Iftene) and (4) SMAUG is a game based on Sphero 2.0 robotic ball, where multiple players share the same driving control of the robot (Sphero – Multiplayer Augmented Game (SMAUG) – Marian- Nicolae Pinzariu, Adrian Iftene).

The third section, Interaction Techniques, addresses last technologies used in new types of interaction: (1) a method with aim to improve interaction in virtual environments and make it less stressful and more effortless (Reducing Gestural Interaction Physical Stress in Virtual Environments: an Experiment – Sobhi Ahmed, Laure Leroy and Ari Bouaniche), (2) a study with aim to present Oculus Rift, used to assess craving of Romanian smokers (Oculus Rift 3D Interaction and Nicotine Craving: Results from a Pilot Study – Ioana-Monica Ciolan, Sabin Buraga and Ion Dafinoiu), (3) the design and development of a complete hardware and software solution for a brain computer interface (Brain Computer Interface using Machine Learning – Cristian-Valeriu Soare) and (4) a methodology for identifying and assessing a set of performance issues encountered in a particular web application, with impact on the usability level (Visual techniques for identification and evaluation of a web application usability issues – Mihaela Ciugudean, Dorian Gorgan).

Forth section, HCI in e-learning system, contains papers on novel and popular topics in HCI. The papers are presenting:

(1) a case study in testing technology acceptance model to explain the adoption of Facebook by Romanian university students (Testing the technology acceptance model with Romanian university students – Alexandru Balog, Costin Pribeanu), (2) a case study about the relationship between the Facebook dependence and the negative consequences of the excessive use of Facebook on the students’ university work (Measuring the negative effects of the Facebook dependence on the students’ university work – Gorghiu Gabriel, Manea Valentina Iuliana, Iordache Dragoş Daniel and Pribeanu Costin), (3) the methodology and results obtained after the evaluation of the Tesys e-learning platform (Evaluation of the Tesys e-Learning platform’s interface – Paul Stefan Popescu, Cristian Mihaescu, Mihai Mocanu and Costel Ionascu) and

(10)

x

(4) MOOCBuddy, an innovative project aiming to make known the Romanian initiatives related to MOOCs (MOOCBuddy: a Chatbot for personalized learning with MOOCs – Carmen Holotescu).

In the last section, Interactive applications, are four papers covering the following themes: (1) ReaderBench framework, which includes multi-lingual comprehension-centered web services designed to address a wide range of individual and collaborative learning scenarios (ReaderBench goes Online: A Comprehension-Centered Framework for Educational Purposes – Marius-Gabriel Gutu, Mihai Dascalu, Stefan Trausan-Matu, Philippe Dessus), (2) how the tweets can be processed in order to obtain valuable information in real time, based on user preferences and different search criteria (Event detection in Tweets – Andrei Bogdan Baran, Adrian Iftene), (3) a study about rhythm in various genres of texts (Rhythm analysis of texts using Natural Language Processing – Irina-Diana Niculescu and Stefan Trausan-Matu) and (4) an approach to terrain synthesis from minimal-detail user-provided heightmaps (Terrain Synthesis from Crude Heightmaps – Alexandre Philippe Mangra, Adrian Sabou, Dorian Gorgan).

We cannot end this preface without expressing our appreciation to the members of the scientific committee and to the volunteer reviewers from the RoCHI group who helped for selecting the best papers to be presented at the conference.

Moreover, we acknowledge the efforts of all the persons involved in the organization of the RoCHI 2016 Conference and thank them for their efforts!

Iasi, August 8th, 2016

Editors, Adrian Iftene Jean Vanderdonckt

(11)

“The design is right and the right design”

How to measure and theorize “right” in HCI and Software Engineering?

Ahmed Seffah

Human-centric Software Engineering

[email protected]

ABSTRACT

In this keynote, we will show how to improve the practices of interactive software and user interfaces design, by applying lessons from various fields of design to the co-creation of interactive software products, systems and services. The goal is to create software that works – really works, meaning we can measure it works, usable, profitable yet secure, resilient and sustainable – A software is being appropriate and effective for people who live in the world that the software creates.

Beyond the “cook-books of the HCI gurus” and the large diversity of HCI design patterns, guidelines and principles, the talk will highlight the the importance of the qualities, properties and quantities that quantify the quality of software systems from the human perspective. How to measure the “right”, “the old user friendly”, the past usability, the today quality of user experience and the tomorrow sustainability. How to ground measures of HCI in the general theory of software quality and measurement. What we can learn from the successful stories and history of measurement in many fields: medical sciences, engineering, and even from natural and hearth sciences?

ABOUT

Ahmed Seffah is a professor of human-centric software engineering and HCI at Lappeenranta University of Technology, Finland. Previously, he was a professor and Concordia research chair on human-centered software engineering at Concordia University as well a visiting professor in more than 10 universities and research Centre including IBM, University of Lausanne, Daimler Chrysler and the Computer research institute of Montreal. Professor Seffah co-authored five books, the last one on the “Patterns of HCI Design Patterns and the HCI Design of Patterns”. His main research is to understanding those HCI and interactive software systems design, software usability, user-centric engineering, UX design practices and all similar ones within the wider software and systems engineering processes. Most visible contributions of his work are on the gaps and bridges between design science practices and software engineering methodologies such as agile, model-driven and service-oriented while building a theories of human-centric software engineering.

(12)

2

Implementation of a polyglot text-to-speech synthesis in two assistive technologies

Paul Fogarassy-Neszly, Aurel Patru BAUM Engineering

Str. Traian Mosoiu 8, Arad, Romania [email protected], [email protected]

Dragoş Daniel Iordache, Costin Pribeanu

National Institute for R & D in Informatics - ICI Bucharest

Bd. Averescu 8-10, Bucucharest, Romania [email protected], [email protected]

ABSTRACT

The text-to-speech (TTS) synthesis is widely used in the area of assistive technologies for visually impaired people as well as for people with dyslexia or related learning disabilities. In the case of multilingual text-to-speech synthesis, both the language identification and voice switching are desirable. This paper presents the implementation of a multilingual text-to-speech in two assistive technologies: the automatic reading machine POET and the portable organizer for blind people Pronto.

Both implementations have been tested with visually impaired users. The evaluation results show that the implementation of this software component makes these devices more easy and comfortable to use.

Author Keywords

Accessibility, assistive technologies, usability, technology acceptance, TTS, visually impaired users.

ACM Classification Keywords

D.2.2: Design tools and techniques. H5.2 User interfaces.

INTRODUCTION

A Text-To-Speech (TTS) system is a software component able to produce speech output, whether directly from text input or from a scanned document page previously submitted to an Optical Character Recognition (OCR) system. Many assistive technologies for visually impaired people are using text-to-speech (TTS). TTS synthesis is also used by people with reading disabilities (dyslectic, illiterate, or with learning disability) in order to make the electronic or printed documents accessible. Examples of assistive technologies using TTS synthesis are: screen readers, automatic reading machines, portable computers with voice interface, smartphones, GPS systems, as well as many other gadgets or self-voiced software.

Usually, the synthetic voice corresponding to the native (or preferred) language of the user is currently selected. If the text is written in another language, then the user has to manually select a corresponding voice for that language.

In recent years, there is an increasing interest in the applications that are able to process texts written in two or more languages. There are many application areas that need polyglot text-to-speech, such as education for all and multi-cultural contexts [8, 11, 12, 13]. In this case, both a multilingual (polyglot) text-to-speech synthesis and voice switching are needed. This requires to analyze the text in order to detect the language and then to select the voice available for that language. Several approaches for the multilingual TTS exist that differ with regard to the solutions adopted for the text analysis and speech synthesis [1, 3, 9, 10, 11].

In a previous work, a software component for multilingual text-to-speech has been presented that performs both the automatic language identification and voice switching.

The component has been developed during the research project iT2V that has been carried on in a consortium of three partners: BAUM Engineering, ETA Automatizari Industriale, and National Institute for Research and Development in Informatics – ICI Bucharest.

The language recognition component plays the role of an intermediate layer, voice independent, between the application and the synthesis process. Language identification is based on statistical analysis and trigrams frequency evaluation for envisaged languages [2]. The development followed four steps: alpha version (proof-of- concept, functional version (beta), commercial version, and implementation in several applications. The development cycle of iT2V is illustrated in Figure 1.

Figure 1. iT2V development cycle.

The goal of the alpha version was to test the language detection algorithms [5]. In the second step, a preliminary functional version (beta01) was developed and tested with four candidate languages [7]. The evaluation revealed a major usability issue: switching the voice in the middle of a sentence. Therefore, is an improved functional version has been developed and tested again. Based on the testing results [6], the commercial version has been developed and then used to implement the software component. The iT2V component plays the role of a special voice (SAPIiT2V) that identifies the language and selects the corresponding synthetic voice.

This paper presents the implementation of the multilingual text-to-speech in two assistive technologies: the automatic reading machine POET and Pronto, the portable electronic organizer for blind and visually impaired users. The implementations have been evaluated for usability and technology acceptance with visually impaired users. The rest of this paper is organized as follows. Next section briefly presents the automatic reading machine POET and the portable organizer for blind people Pronto. Then the evaluation method is presented. In the next section the evaluation results of the implementations in these devices are presented and discussed. The paper ends with conclusion and future research directions.

(13)

IMPLEMENTATIONS OF IT2V Implementation in a reading machine

The first implementation has been done on the automatic reading machine POET (trade mark of BAUM Retec AG).

This type of assistive technology integrates a computer and a scanner (including an Optical Character Recognition software - OCR - with italics adjustment) in one single device. The device is simple and is easy to use. The basic version of POET (Compact 2) has only two buttons (Start and Stop) and two control knobs to adjust the volume and the reading speed. Figure 2 presents the reading device POET Compact 2. The user can read periodicals, books, or magazines. The text could be written on one or several columns.

Figure 2. Automatic reading device POET Compact 2.

The text could be saved and then used for a new lecture or transferred to another device, such as an MP3 player. The model Compact 2+ has three buttons and a menu keypad and has been designed for users that prefer is a model for more advanced users who prefer more features and full control over the functionality. For Braille literate and deaf and blind users a Braille display can be connected to the Poet. As Braille tables are language dependent, prior language detection is necessary before producing Braille output. This is a not foreseen application of language detection.

Implementation in a portable device

The second implementation has been done on the organizer for blind people Pronto (trade mark of BAUM Retec AG).

Pronto is a portable organizer for visually impaired people that runs under the operating system Window CE. The device includes a Braille display and speech output.

Pronto supports various applications, like a regular PDA (personal digital assistant). As such is more complex than other assistive technologies. Figure 3 presents the organizer Pronto. The main functions are: taking notes with a text editor, agenda & organizer, playing music, reading books in text or Braille format, Internet access.

The device is easy to handle and easy to use. It provides a keyboard for 8-dot Braille input, an 18-cell Braille display with integrated cursor routing, four function keys for fast access to applications, and a navistick for an easy and comfortable operation.

Figure 3. Organizer for blind people Pronto.

METHOD

Evaluation techniques

The evaluation has two goals: usability and acceptance.

First, a usability inspection has been carried on by three experts. Then, each implementation has been tested with visually impaired users for usability and acceptance. After testing the device, participants were asked to answer a questionnaire as regards the actors that are influencing the technology acceptance. The users were also asked to mention the most important positive and negative aspects as regards each device and to weight the relative importance of three factors (ease of use, usefulness, and enjoyment).

Measures

Following measures were collected: number of usability problems by severity degree (major, moderate, and minor), description of unique usability problems, number of positive and negative aspects, the ratings of each item of the questionnaire, and the relative importance of each factor. The items in the questionnaire are presented in Table 1.

Item Statement

PEU1 It is easy to understand how to use iT2V with this device PEU2 This device is easier to use with iT2V

PU1 iT2V is a useful facility for me

PU2 The capabilities of iT2V correspond to my requirements PE1 I prefer to use this device having iT2V

SAT1 I am satisfied with this device having iT2V INT1 If would like to use this device with iT2V

INT2 I will recommend other people this device having iT2V Table 1. Questionnaire.

The questions are inspired from the technology acceptance model [4]. Since all the users were visually impaired, the answers have been collected via interview (ratings of each item on a 5-point Likert scale).

Participants and tasks

Seven people participated in user testing (six men and a woman). The mean age of participants is 40.3 years (SD=9.23) with a minimum of 23 and a maximum of 48 years. The user testing took place in Arad, at the Local Branch of Romanian Association of the Blind (Filiala Arad a Asociației Nevăzătorilor din România). Except for one user (university student), all are retired for medical reasons. All participants graduated a high school. The disability degree is severe (first degree - legally blind).

(14)

4

The familiarity with the assistive technologies is variable.

However, all are familiar with the lecture on the Internet.

The most used information and communication technologies are the computer, mobile phone, tablet, and scanner + OCR. Most used applications are the Internet browsers, Skype, and Facebook. The usual goals are related to information, lecture, entertainment (games), and socialization. The POET device has been tested with three tasks.

The goal of the first task was to read two documents, each of them written in two languages (Romanian and English), without iT2V. This means to stop the lecture and manually change the voices from the device option menu. In the first document the text was written in one column. In the second document, the same text was written in two columns.

The second task was identical, but with the iT2V feature selected. The third task was to read several pages from a magazine with iT2V feature selected. The Pronto device has been tested with a task performed under two conditions. The task was to open and read a document written in two languages (Romanian and English). The condition is to have the iT2V selected.

RESULTS

Reading machine POET

The usability inspection identified several usability problems for the first task. The problems are related to the navigation in the menu in order to manually change the voice. No usability problem has been detected at the second and third tasks. The accuracy of language identification was 100%. The average time saving for a voice switching was 2 minutes. The user testing confirmed the results of the usability inspection: no usability problem has been detected after selecting the iT2V feature.

Item Factor Mean SD

PEU1 Ease of use 4.71 0.49 PEU2 Ease of use 4.43 0.79 PU1 Usefulness 4.29 1.50 PU2 Usefulness 4.29 1.50

PE1 Enjoyment 4.57 0.79

SAT1 Satisfaction 5.00 0.00 INT1 Intention to use 5.00 0.00 INT2 Intention to use 4.71 0.76 Table 2. Descriptives for the POET device.

Most of the users mentioned that the device is accessible, simple, easy to use, useful, and compact. They appreciated the ease of reading and the multilingual feature. As regards the negative aspects, the users mentioned the price (too high), the dimensions (too big) and the lack of the translation.

The users considered that the most important factor for the acceptance of the reading device is the usefulness (45%), followed by the ease of use (29%), and usefulness (26%).

The descriptive statistics for the items in the questionnaire is given in Table 2. The mean value of items related to the ease of use is 4.57 (SD = 0.45) and of the items related to usefulness 4.29 (SD = 1.47). An analysis of correlation

based on Spearman coefficient shows a marginally significant relationship between the enjoyment and the satisfaction (rho = 0.68, p = 0.091).

The organizer for blind people Pronto

The usability inspection identified few usability problems when performing the task without iT2V. The problems are related to the navigation in the menu in order to manually change the voice. The problems are related to the device itself and somehow inherent, given the rich functionality and the lack of familiarity with the device. The user testing confirmed the results of the usability inspection: no usability problem has been detected after selecting the iT2V feature.

Overall, the usability evaluation results were similar for the two implementations. Most frequently mentioned positive aspects were the portability, the Braille feature, flexibility of use (both voice and Braille), and the multilingual feature. As regards the negative aspects, they mentioned the price and the difficulty to use.

The users considered that the most important factor for the acceptance of the reading device is the ease of use (44%), followed by the usefulness (36%), and enjoyment (20%).

The descriptive statistics for the items in the questionnaire is given in Table 2.

Item Factor Mean SD

PEU1 Ease of use 3.71 0.76 PEU2 Ease of use 3.86 0.90 PU1 Usefulness 4.43 1.13 PU2 Usefulness 4.43 0.53 PE1 Enjoyment 4.43 1.13 SAT1 Satisfaction 4.57 0.53 INT1 Intention to use 5.00 0.00 INT2 Intention to use 4.86 0.38 Table 3. Descriptives for the Prontop device

The mean value of items related to the ease of use is 3.79 (SD = 0.39) and of the items related to usefulness 4.43 (SD = 0.53). The analysis of correlation showed a marginally significant relationship between the perceived enjoyment and the satisfaction (rho = 0.72, p = 0.068) and a significant relationship between the perceived enjoyment and the perceived usefulness (rho = 0.84, p = 0.018).

Discussion

The evaluation has been focused on the implementation of the iT2V component and not on the device itself. In this respect, no usability problem has been identified. In both cases, the accuracy of voice switching was excellent.

The component saves time and make the use of these assistive technologies much simpler. The answers to the questionnaires show a difference between the ease of use of the two devices. Pronto has a more rich functionality which makes it more difficult to use. Nevertheless, in both cases the users have been satisfied and expressed the intention to use these technologies.

There are several limitations of this work. First of all, the number of users is small, so the results could be seen as

(15)

only exploratory. Second, the users were not familiar with these devices. A reason is the price: six out of seven users said that they can’t afford these technologies. Third, the time for evaluation was limited, so only few tasks were assigned. The reasons were the availability of users and the time needed to explain how to use a new device.

CONCLUSION AND FUTURE WORK

In this paper two implementations of the iT2V software component for multilingual text-to-speech and automatic voice switching have been presented.

The evaluation results showed that iT2V is usable, useful and enjoyable. A device with iT2V is easier to use. Since the users are visually impaired, manually changing the voice could be done via an audio menu, which is both difficult to use and time-consuming.

In the next future the language recognition will be implemented in COBRA, the screen reader software. This will be very useful, especially for internet browsing, when beside pages in local language, pages in English should be accessed. Also, in countries with two or more official languages this facility will be useful, as well as for any polyglot user who often are switching between documents in different languages.

Also, beside the language dependent speech synthesis switch, the Braille table will be selected accordingly, taking into account that not only synthetic voices, but also Braille tables are language dependent. This will allow the user to access documents in different languages, both through speech synthesis and Braille display, without any option menu exploration.

ACKNOWLEDGMENTS

This work is partly supported by the IT2V research project (29DPST/2013), financed by UEFISCDI under the PNCDI II Innovation Program.

REFERENCES

1. Bourlard, H., Dines, J., Magimai-Doss, M., Garner, P.

N., Imseng, D., Motlicek, P. and Valente, F. Current trends in multilingual speech processing, Sadhana 36(5), (2011), 885-915.

2. Cavnar, W. and Trenkle, J. N-gram-based text categorization. Proceedings Document Analysis and Information Retrieval (SDAIR-94), (1994).

3. Chen, C. P., Huang, Y. C., Wu, C. H. and Lee K. D.

Polyglot speech synthesis based on cross-lingual frame

selection using auditory and articulatory features.

IEEE/ACM TASLP 22 (10), (2014), 1558-1570.

4. Davis, F.D., Bagozzi, R.P. and Warshaw, P.R.

Extrinsic and intrinsic motivation to use computers in the workplace. Journal of Applied Social Psychology, 22 (14), (1992), 1111-1132.

5. Fogarassy-Neszly, P. and Gherhes, V. Applications for dynamic language identification. Proceedings of RoCHI 2014 Conference, Popovici D., M. & Iordache D.D. (Eds.), (2014), 51-54.

6. Fogarassy-Neszly, P., Zinveliu, Z. and Pribeanu, C. A software component for polyglot text-to-speech synthesis: user interface and beta testing results.

Proceedings of RoCHI 2015 Conference, Dardala, M., Rebedea, T.E. (Eds.), (2015), 145-148.

7. Pribeanu, C. and Fogarassy-Neszly, P. Beta testing of a dynamic language identification software component - preliminary results. Revista Romana de Interactiune Om-Calculator 7(3), (2014), 259-272.

8. Ramani, B., Actlin Jeeva, M.P., Vijayalaksmi, P., and Nagarajan, T. Cross-lingual voice conversion-based polyglot speech synthesizer for Indian languages.

Proceedings INTERSPEECH, (2014), 775-779.

9. Romsdorfer, H. and Pfister, B. Text analysis and language identification for polyglot text-to-speech synthesis. Speech Communication 49, (2007), 697-724.

10. Shiga, Y. and Kawai, H. Multilingual speech synthesis system. Journal of the National Institute of information and Communication Technology 59(3/4), (2012), 21- 28.

11. Traber, C., Huber, K., Nedir, K., Pfister, B., Keller, E.

and Zellner, B. From multilingual to polyglot speech synthesis. Proceedings of EUROSPEECH, Budapest, Hungary, (1999), 835–838.

12. Tripathi, M., and Shukla, A. Use of assistive technologies in academic libraries: A survey. Assistive Technology, 26(2), (2014), 105-118.

13. Turunen, M. and Hakulinen, J. Mailman-a multilingual speech-only e-mail client based on an adaptive speech application framework. In Proceedings of Workshop on Multilingual Speech Communication - MSC 2000, (2000), 7-12.

14. Udvari-Solner, A. and Thousand, J. S. Creating a responsive curriculum for inclusive schools. Remedial and special education, 17(3), (1996), 182-191.

(16)

6

Comments on the reliability and validity of UMUX and UMUX-LITE short scales

Costin Pribeanu

National Institute for Research and Development in Informatics - ICI Bucharest

Blvd. Maresal Averescu, nr.8-10, Bucharest, Romania [email protected]

ABSTRACT

Recent work on usability and user experience shows several concerns on the validity of evaluation instruments.

There is a debate on the use of standardized scales versus short scales, such as UMUX and UMUX-LITE or even a single-item measure of usability. Nevertheless, there are relatively few papers reporting the testing of these scales together for reliability and validity. This paper aims at discussing the UMUX and UMUX-LITE scales that have been tested in the context of Facebook use by university students. From a theoretical point of view, both scales are questionable. From an empirical point of view, the testing results confirmed a lack of unidimensionality as well as a poor reliability and convergent validity of these scales.

Author Keywords

Usability scales, UMUX, UMUX-LITE, factor analysis, validity, Facebook use.

INTRODUCTION

Recent work on usability and user experience shows several concerns on the reliability and validity of the scales used to measure the perceived usability. There is currently a debate on the use of standardized scales versus short scales, such as UMUX [6] and UMUX-LITE [14] or even a single-item measure of usability [13]. There are many pros and cons as regards the reliability and validity as well as the practical benefits. Nevertheless, there are relatively few papers reporting the testing of these scales for reliability and validity.

This paper aims at discussing the UMUX and UMUX- LITE scales from both a theoretical and an empirical point of view. Additionally, the perceived ease of use (PEU) is analyzed that is a widely used concept in the context of technology acceptance [5]. Since PEU is a short scale taping on several key usability aspects, it could be a better alternative than UMUX and UMUX-LITE.

Theoretically, the analysis is following the scale development recommendations. Empirically, the analysis is focused on the scale testing that has been carried on by using two samples collected during a larger study on Facebook use by university students [9]. The first sample is from the pilot study and is used to assess the UMUX and UMUX-LITE scales. The second sample is from a subsequent study using a revised evaluation instrument and is used to assess the UMUX-LITE and PEU scales.

The rest of this paper is organized as follows. The following section briefly presents recent approaches in the area of scale development with an emphasis on the scale development process and the existing usability scales. In

section 3, the analysis of UMUX and UMUX-LITE is presented based on two empirical studies. The same assessment criteria are used to analyze the PEOU scale.

The paper ends with conclusion and future research directions.

RELATED WORK Scale development

The interest in developing rating scales increased after the adoption of the ISO 9241-11 standard that included satisfaction as a key usability aspect. As Lindgaard &

Kirakowski [15] pointed out, the landscape of scale development in HCI shows many usability scales, many approaches, as well as many opinions as regards the scale reliability and validity.

When analyzing the reliability and validity of the usability scales, two aspects are usually neglected: the theoretical meaning and the multidimensional nature of the usability concept. Psychometrics is not a favorite area of expertise in HCI [15]. As such, the scale development process is not well understood as regards both the ordering of steps to be carried on and the suitable techniques that should be used in each step.

Long time ago, Gerbing & Anderson [7] outlined an updated paradigm for scale development that includes a confirmatory factor analysis (CFA) to assess the scale unidimensionality. They underlined that, only after achieving an acceptable unidimensionality level, the reliability could be assessed. This precondition is usually ignored in the existing papers reporting the development and testing of usability scales. The authors rely on the traditional approach that only includes the Cronbach’s alpha coefficient, the item-to-total correlations, and the exploratory factor analysis (EFA).

More recently, MacKenzie et al. [16] emphasized the importance of the conceptualization as a first step in the scale development process. They noticed that an adequate conceptualization is difficult and requires a review of the literature on the meaning of related constructs, aspects these constructs refer to, dimensionality and preliminary research with domain experts or practitioners. Another important issue is to specify if the construct is measured reflectively or formatively.

Usability in the ISO standards

ISO 9241-11 standard defined usability as the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use [11]. The ISO standard 9126-1 defined usability as the capability of a

(17)

software product to be understood, learned, used, and attractive to the user, when used under specified conditions [10]. Later on, both definitions were integrated in the ISO standard 25010 [12], under two key terms:

 Quality in use: the degree to which a product used by specific users meets their needs to achieve specific goals with effectiveness, efficiency, safety, and satisfaction in specific contexts of use.

 Usability: the degree to which a software product is able to satisfy the following needs when used under specified conditions: appropriateness recognizability, learnability, operability, error protection, user interface aesthetics, and accessibility.

As pointed out by Bevan et al. [1], the quality in use defines usability as a high level concept, focusing on the outcomes of the interaction rather than on the characteristics that make a product usable. Unfortunately, this distinction is rarely made in the mainstream of HCI literature.

Measurement scales for the perceived usability A well-known usability scale is System Usability Scale (SUS) that has been developed by Brooke as a simple,

“quick and dirty” scale [3]. SUS has been widely used and is considered an industrial standard [2, 14].

Several authors noticed that SUS lacks unidimensionality.

For example, Borsci et al. [2] found that a learnability dimension of SUS might emerge under certain conditions (when administrated to experienced users). SUS has been also criticized for using both positive and negative wording, since this may lead to mistakes (made by respondents) and mis-coding (made by researchers) [17].

More recently, Finstad proposed the UMUX (Usability Metrics for User Experience) as a shorter alternative to SUS. UMUX have been criticized for dimensionality [4, 14] and for using negative wording [14, 17]. Lewis et al.

[14] found that UMUX has a bi-factorial structure with positive tones aligning with one factor and negative tones with the second factor.

An even shorter scale that is based on UMUX has been proposed by Lewis et al. [14]. UMUX-LITE was intended as a very quick, two-item scale, that uses the first and the third item from UMUX. The authors found that UMUX- LITE is unidimensional and has acceptable reliability.

However, they recommended using this scale with caution until it will be validated across a wider variety of systems.

A well-known scale measuring the perceived ease of use (PEU) has been developed and tested in the context of technology acceptance studies. The technology acceptance model (TAM) has been developed by Davis et al. [5], in order to explain and predict the technology acceptance on a large variety of technologies. Although it is a short scale with a widely recognized psychometric quality, PEU has been rarely used in the HCI studies.

ANALYSIS OF UMUX AND UMUX-LITE Method

The analysis follows the recommendations in the literature for scale development and assessment of dimensionality, reliability, and validity [7, 8, 16]. The first step is to analyze the conceptualization based on the definition of concepts in the literature. Then, the dimensionality is assessed via exploratory and confirmatory factor analysis.

After demonstrating that the construct is unidimensional, the reliability could be analyzed checking the magnitude of Cronbach’s alpha and the item-to-total correlations. The convergent validity is assessed by the examination of the composite reliability (CR), and average variance extracted (AVE).

Empirical studies

UMUX and UMUX-LITE have been tested in a larger study on the use of Facebook (FB) by university students.

Two samples collected during these studies are used for the analysis of the psychometric quality of UMUX and UMUX-LITE. The respondents were asked to answer questions related to demographics, enrollment, FB usage (the size of their FB network, frequency of use, minutes per day), and to evaluate items on a 7-point Likert scale.

The first sample has been collected in 2014 and consists of 152 students (110 female, 42 male) from two universities in Lithuania. The negatively worded items in Table 1were recoded. The first and third items in Table 1 represent the UMUX-LITE scale.

Item Statement M SD

U1 FB’s capabilities meet my requirements 4.02 1.38 U2 Using FB is a frustrating experience 3.44 1.64

U3 FB is easy to use 4.91 1.15

U4 I have to spend too much time correcting things with FB

4.12 1.63

Table 1. Descriptives for UMUX scale (N=152).

The second sample has been collected in 2015 and consists of 414 students (258 female, 156 male) from a Romanian university. Since the testing results from the first study revealed poor psychometric properties of UMUX, the scale has been removed from the evaluation instrument and replaced with PEU. However, the first item has been preserved in order to test again UMUX-LITE.

Item Statement M SD

U1 FB’s capabilities meet my requirements

4.36 1.51 PEU1 It is easy to learn how to use FB 6.10 1.27 PEU2 / U3 FB is easy to use 6.21 1.17 PEU3 My interaction with FB is clear and

understandable

5.69 1.38 Table 2. Descriptives for UMUX-LITE and PEU (N=414).

(18)

8

The PEU scale has been developed by adapting items from the existing scales. The three items tap on three usability attributes: understandability, learnability, and operability.

As it could be observed, the item U3 in Table 1 is identical with the item PEU2 in Table 2.

Conceptualization

In most studies discussing UMUX and UMUX-LITE the reliability and validity is limited at dimensionality which is assessed with exploratory factor analysis techniques, reliability, and correlation with other usability scales. The main shortcoming of UMUX (and, consequently, of UMUX-LITE) is the poor conceptualization which is due, on one side, to the underlying usability definition and, on another side, to the misunderstanding of the nature of the measurement model.

Both scales take the roots from the quality in use concept and try to measure the user’s subjective satisfaction.

Nevertheless, the measured variable is not the satisfaction (there is no item saying “I am satisfied with…”. Rather, the operationalization of the construct is done with a mix of items measuring various aspects of user experience and usability. As such, the conceptualization is ambiguous and is not clear what UMUX is actually measuring.

The first item is the most ambiguous and undermines the conceptualization of both UMUX and UMUX-LITE. The fit between the user’s needs and requirements could refer to anything: ease of use, aesthetics, flexibility, robustness, safety, usefulness, enjoyment, etc.

The second shortcoming is the lack of a clear definition of the nature of the measurement model. The measurement model describes the relationship between a construct and its measures [7, 16]. According to the direction of the causal relationship, the constructs could be reflective (from the construct to its measures) or formative (from measures to construct). It is also possible to define multidimensional constructs where the dimensions are specified as first order constructs.

Failure to adequately specify the measurement model leads to a poor operationalization and a lack of validity.

The point is that the conceptualization and validation recommendations are different for reflective and formative measurement models. Unidimensionality and inter-item correlation are required for reflective constructs, since all items are supposed to measure the same thing. For a more detailed discussion on the scale development of reflective and formative constructs, see MacKenzie et al. [16].

Therefore, neither UMUX nor UMUX-LITE could be adequately assessed as measurement scales, since both of them suffer from a lack of clear definition of the construct domain. As regards the measurement model, although is not specified, it is assumed to be reflective according to the assessment techniques used by the authors.

Dimensionality

The dimensionality of UMUX and UMUX-LITE has been analyzed within the two empirical studies. The first study enabled testing of UMUX and UMUX-LITE.

The principal component analysis with Varimax rotation for UMUX resulted in two factors explaining 34.56%, respectively 26.69% of the variance. The same analysis for UMUX-LITE resulted in one factor explaining 60.06

% of the variance.

A confirmatory factor analysis has been then carried on.

The results revealed the lack of dimensionality for both constructs. The loadings of the underlying construct on its measures (α regression coefficients) is below the cutoff value of 0.60 [8]. The results are presented in Table 3.

Item

UMUX UMUX-LITE

EFA CFA (α) EFA

CFA (α)

1 2 1

U1 .707 -.364 0.66 .775 0.65 U2 .700 .332 -0.38

U3 .609 0.31 .775 0.34 U4 .919 0.32

Table 3. Dimensionality of UMUX and UMUX-LITE (N=152).

The second study enabled the analysis of dimensionality of UMUX-LITE and PEU. The principal component analysis with Varimax rotation for UMUX-LITE resulted in one factor explaining 68% of the variance. The same analysis for PEU resulted in one factor explaining 80.61%

of the variance.

The confirmatory factor analysis for UMUX-LITE revealed a low item loading of U1 (α=0.49). The same analysis for PEU confirmed its unidimensionality (item loadings: 0.86, 0.92, and 0.73).

The results of the two empirical studies demonstrate the lack of dimensionality for the UMUX and UMUX-LITE short scales as well as the limitations of the exploratory factor analysis for testing the dimensionality.

Reliability

The Cronbach’s alpha was unacceptable low in the first study: 0.213 for UMUX and 0.331 for UMUX-LITE. The item-to-total correlations were in the range of 0.05-0.23for UMUX, respectively 0.20 for UMUX-LITE.

In the second study, the Cronbach’s alpha for UMUX- LITE was low (0.517) and the item-to-total correlation also low (0.36). Cronbach’s alpha for PEU was 0.874 and the item-to-total correlation in the range of 0.68-0.82.

Convergent validity

Convergent validity refers to the degree to which the measures of a construct that are supposed to be related, are in fact related.

In the first study, the low item loadings make no sense to test the convergent validity of UMUX. For UMUX-LITE, the composite reliability of 0.412 and the average variance extracted of 0.279 demonstrate the lack of convergent validity. The second study confirmed the poor convergent validity of UMUX-LITE (CR=0.552, AVE=0.390). The

(19)

convergent validity for PEU was very good (CR=0.885, AVE=0.722).

Interpretation of scores

The final step in scale development is to provide the prospective researchers with some recommendations for the interpretation of scores. Since this step is beyond the purpose of this study, it will not be discussed. However, is important to note that the correlation of a scale under consideration with other existing scales does not ensure the scale validity. It is expected that two scales pointing to similar usability aspects correlate. The problem is that if the scale under consideration and the reference scale are not unidimensional, then a comparison leads to ambiguous if not erroneous conclusions.

DISCUSSION AND CONCLUSION

In the area of HCI, several misconceptions exist as regards the scale development and the validity criteria. The two short scales analyzed in this paper suffer from an ambiguous definition of the construct. An adequate conceptualization should include the specification of the dimensionality and the nature of the construct (reflectively vs. formatively measured).

It seems that the relationship between the definition of the target construct and the criteria for its assessment are not well understood: if a usability scale is not unidimensional, then more than one thing (e.g. usability) is measured. In other words, what is actually measured is not what has been supposed to be measured.

A problem with the conceptualization of existing short scales in HCI, such as SUS, UMUX, and UMUX-LITE is the confusion between usability and quality in use. The quality in use is a multidimensional construct since it taps on different concepts. Another problem is the overlapping between two HCI concepts: usability and user experience.

It is advisable to keep apart the scales measuring the pragmatic and hedonic aspects (each scale should undergo a separate validation procedure). For example, the perceived ease of use refers to pragmatic aspects, while the perceived enjoyment refers to hedonic aspects. Both scales have been widely used and validated in technology acceptance studies.

The empirical studies confirmed the recommendation of Gerbing and Anderson to use confirmatory factor analysis to assess the dimensionality [7]. The exploratory factor analysis is clearly not enough.

The two empirical studies show that both UMUX and its shorter version, UMUX-LITE, suffer from poor reliability, lack of unidimensionality, and poor convergent validity.

The correlation with other usability scales, which is frequently mentioned as an argument for reliability, is a poor surrogate when the candidate scale and the reference scale does not measure the same thing.

This paper does not deny the practical value of the short questionnaires which are less expensive and could provide a useful feedback for the developers. However, these should not be referred as usability or UX scales.

As it was shown, the PEU scale is unidimensional and reliable. PEU provides with a useful feedback on some key aspects of usability and could be combined with other short scales pointing to other usability / user experience aspects. This approach enables a step-by-step development of valid and reliable evaluation instruments and a flexible choice of scales, according to the objectives of the evaluation.

ACKNOWLEDGMENTS

This work was supported by the Romanian grant financed by ANCS under COGNOTIC 1609 0101 / 2016.

REFERENCES

1. Bevan, N., Carter, J. and Harker, S. ISO 2041-11 revised: What have we learned about usability since 1998? Human-Computer Interaction Design and Evaluation, Proc. HCI International Conference, LNCS 9169, Springer, (2015), 143-151.

2. Borsci, S., Federici, S., Bacci, S., Gnaldi, M. and Bartolucci, F. Assessing User Satisfaction in the Era of User Experience: Comparison of the SUS, UMUX, and UMUX-LITE as a Function of Product Experience.

International Journal of Human-Computer Interaction, 31(8), (2015), 484-495.

3. Brooke, J. SUS-A quick and dirty usability scale. Usability evaluation in industry, 189 (194), (1996), 4-7.

4. Cairns, P. A Commentary on short questionnaires for assessing usability. Interacting with Computers 25(4), (2013), 312-316.

5. Davis, F.D., Bagozzi, R.P. and Warshaw, P.R. User acceptance of computer technology: A comparison of two theoretical models, Management Science, 35 (8), (1989), 982-1003.

6. Finstad, K. The usability metric for user experience. Interacting with Computers 22(5), (2010), 323-327.

7. Gerbing, D.W. and Anderson, J.C. An updated paradigm for scale development incorporating unidimensionality and its assesment. Journal of Marketing Research 25(2), (1988), 86-192.

8. Hair, J.F., Black, W.C., Babin, B.J., Anderson, R.E.

and Tatham, R.L. Multivariate Data Analysis. 6th ed., Prentice Hall, 2006.

9. Iordache, D.D., Pribeanu, C., Lamanauskas, V. and Raguliene, L. Usage of Facebook by university students in Romania and Lithuania: a comparative study. Informatica Economica 19(1), (2015), 46-54.

10. ISO/IEC 9126-1, Software Engineering –Product quality. Part I: Quality Model, 2001.

11. ISO 9241-11, Ergonomic requirements for office work with visual display terminals (VDTs) - Part 11 Guidance on usability, 1998.

12. ISO/IEC 25010, Systems and software engineering – Systems and software product Quality Requirements and Evaluation (SQuaRE) – System and software quality models, 2011.

(20)

10

13. Konradt, U., Wandke, H., Balazs, B. and Christophersen, T. Usability in online shops: scale construction, validation and the influence on the buyers’ intention and decision. Behaviour and Information Technology 22, (2003), 165–174.

14. Lewis, J. R., Utesch, B. S. and Maher, D. E. UMUX- LITE: when there’s no time for the SUS. Proceedings of CHI 2013, ACM, (2013), 2099-2102.

15. Lindgaard, G. and Kirakowski, J. Introduction to the Special Issue: The tricky landscape of developing

rating scales in HCI. Interacting with Computers 25 (4), (2013), 271-277.

16. MacKenzie, S. B., Podsakoff, P. M., and Podsakoff, N.

P. Construct measurement and validation procedures in MIS and behavioral research: Integrating new and existing techniques. MIS quarterly, 35(2), (2011), 293- 334.

17. Sauro, J. and Lewis, J. When designing usability questionnaires, does it hurt to be positive? Proceedings of the CHI 2011, ACM, (2011), 2215-2223.

Referințe

DOCUMENTE SIMILARE

However, this paper agreed that the Act covered digital innovations from inference drawn from section 1(1), section 7 and section 8 of the Act which conferred protection to

The Constitution of the Republic of Albania regulates three situations that require extraordinary measures: war situation, state of emergency and state of natural

De¸si ˆın ambele cazuri de mai sus (S ¸si S ′ ) algoritmul Perceptron g˘ ase¸ste un separator liniar pentru datele de intrare, acest fapt nu este garantat ˆın gazul general,

Next, we shall present the principles on which the formalization of the Romanian language phonology is based, in order to realize the phonological transcription of a written text,

The classic definition of prosody refers to the speech features whose domain is not a single phonetic segment, but larger units of more than one segment, possibly whole

Thus, if Don Quixote is the idealist, Casanova the adventurous seducer, Werther the suicidal hero, Wilhelm Meister the apprentice, Jesus Christ will be, in the audacious and

According to our previous investigations it seems that tolerance, whether regarded as a political practice or a philosophical or moral principle, is a strategy (or tactics) of one

Two young medical doctors (1 year CEUS experience), two experts and the CAD prototype, reevaluated 50 FLLs CEUS videos (diagnosis of benign vs. malignant) first blinded to