ROCHI – INTERNATIONAL CONFERENCE ON HUMAN-COMPUTER INTERACTION
Proceedings of the 13
thInternational Conference on Human- Computer Interaction RoCHI’2016, 8-9 September, Iasi, Romania
Edited by:
Adrian Iftene
A.I. Cuza University of Iasi, Romania
and
Jean Vanderdonckt
Université catholique de Louvain, Belgium
MATRIX ROM
Bucharest, 2016
Publisher
MATRIX ROM
C.P. 16-162
062510 – Bucureşti, România Tel.: 021 4113617, Fax: 021 4114280
E-mail: [email protected] www.matrixrom.ro
MatrixRom publishing house is a certified publisher by the National Council for Scientific Research in Higher Education (Consiliul Naţional al Cercetării Ştiinţifice din Învăţământul Superior)
Cover graphic design: Sabin-Corneliu Buraga, A.I.Cuza University, Iasi Graphic identity: Adrian Mironescu, Idegrafo
© Copyright 2016
All rights of the current edition are reserved by MATRIX ROM.
No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, without permission in writing from the publisher.
RoCHI – International Conference on Human Computer Interaction (2016)
http://rochi.utcluj.ro/proceedings/en/index.php
ISSN 2501-9422
ISSN-L 2501-9422
TABLE OF CONTENTS
Organization Forword
Invited paper
“The design is right and the right design” – How to measure and theorize “right” in HCI and Software Engineering? ………...
Ahmed Seffah
Usability and accessibility evaluation
Implementation of a polyglot text-to-speech synthesis in two assistive technologies……..
Paul Fogarassy-Neszly, Aurel Patru, Dragos Daniel Iordache and Costin Pribeanu
Comments on the reliability and validity of UMUX and UMUX-LITE short scales...
Costin Pribeanu
Automated Evaluation of Menu by Guidelines Review ...
Sara Bouzit, Gaëlle Calvary, Denis Chêne and Jean Vanderdonckt
Perceived aesthetics of user-modifiable layouts: a comparison between an unspecified design and a GUI ...
Stefan Trausan-Matu, Brahma Dathan
User interface development
Enactment of User Interface Development Methods in Software Life Cycles………
Iyad Khaddam, Hanaa Barakat and Jean Vanderdonckt
Increasing the Accuracy of Indoor Localization Applications by Using Predefined Markers and the Phone's Camera ……….…
Szabolcs Orban, Teodor Stefanut
Using WSN and Mobile Apps for Home and Office Ambient Monitoring and Control...….
Catalin Damian, Lenuta Alboaie and Adrian Iftene
Sphero - Multiplayer Augmented Game (SMAUG) ...
Marian-Nicolae Pinzariu, Adrian Iftene
Interaction Techniques
Reducing Gestural Interaction Physical Stress in Virtual Environments: an Experiment...
Sobhi Ahmed, Laure Leroy and Ari Bouaniche
Oculus Rift 3D Interaction and Nicotine Craving: Results from a Pilot Study………
Ioana-Monica Ciolan, Sabin Buraga and Ion Dafinoiu
Brain Computer Interface using Machine Learning ……….…
Cristian-Valeriu Soare
Methodology for Identification and Evaluation of Web Application Performance Oriented Usability Issues...
Mihaela Ciugudean, Dorian Gorgan
v ix
1
2
6
11
22
26
36
42
46
50
58
65
69
HCI in e-learning systems
Testing the technology acceptance model with Romanian university students...
Alexandru Balog, Costin Pribeanu
Measuring the negative effects of the Facebook dependence on the students’ university work………..……….……….
Gabriel Gorghiu, Valentina Iuliana Manea, Dragoş Daniel Iordache and Costin Pribeanu
Evaluation of the Tesys e-Learning platform’s interface ………..
Paul Stefan Popescu, Cristian Mihaescu, Mihai Mocanu and Costel Ionascu
MOOCBuddy: a Chatbot for personalized learning with MOOCs ………
Carmen Holotescu
Interactive applications
ReaderBench goes Online: A Comprehension-Centered Framework for Educational Purposes………...
Marius-Gabriel Gutu, Mihai Dascalu, Stefan Trausan-Matu, Philippe Dessus
Event detection in Tweets ..………..
Andrei Bogdan Baran, Adrian Iftene
Rhythm analysis of texts using Natural Language Processing...
Irina-Diana Niculescu and Stefan Trausan-Matu
Terrain Synthesis from Crude Heightmaps ...
Alexandre Philippe Mangra, Adrian Sabou, Dorian Gorgan
Author Index ...
76
81
86
91
95
103
107
113
119
ORGANIZERS
Conference Chair
Adrian Iftene, Alexandru Ioan Cuza University, Iasi, Romania
Program Committee Chair
Jean Vanderdonckt, Université catholique de Louvain, Belgium
Organizing Committee Chair
Lenuța Alboaie, Alexandru Ioan Cuza University, Iasi, Romania
RoCHI Conference Website Administrator
Teodor Stefanut, Technical University Cluj-Napoca, Romania
ORGANIZING COMMITTEE
Andrei Arusoaie, Alexandru Ioan Cuza University, Iaşi, Romania Corina Forăscu, Alexandru Ioan Cuza University, Iaşi, Romania Daniela Gîfu, Alexandru Ioan Cuza University, Iaşi, Romania Alex Moruz, Alexandru Ioan Cuza University, Iaşi, Romania Vlad Rădulescu, Alexandru Ioan Cuza University, Iaşi, Romania Cosmin Vârlan, Alexandru Ioan Cuza University, Iaşi, Romania
PROGRAM COMMITTEE
Alexandru Balog, ICI Bucharest, Romania
Sabin-Corneliu Buraga, Alexandru Ioan Cuza University, Iaşi, Romania Gilbert Cockton, Northumbria University, United Kingdom
Grigoreta Sofia Cojocar, Babeş-Bolyai University, Cluj-Napoca, Romania Karin Coninx, Hasselt University, Belgium
Marian Dârdală, University of Economic Studies, Bucharest, Romania Mihai Dascălu, University Politehnică of Bucharest, Romania Philippe Dessus, Université Grenobles Alpes, France
Bruno Dumas. Université de Namur, Belgium
Paul Fogarassy-Neszly, BAUM Engineering, Arad, Romania Peter Forbrig, University of Rostock, Germany
Alfonso Gracia Frey, Luxembourg Institute of Science and Technology, Luxembourg Vivian Genaro Motti, George Mason University, United States
Juan Gonzalez Calleros, Benemérita Universidad Autónoma de Puebla, Mexico Dorian Gorgan, Technical University Cluj-Napoca, Romania
Jan Gulliksen, KTH Royal Institute of Technology, Sweden
Adriana-Mihaela Guran, Universitatea Babeş-Bolyai, Cluj-Napoca, Romania Adrian Iftene, Alexandru Ioan Cuza University, Iaşi, Romania
Dragoş Daniel Iordache, ICI Bucharest, Romania David Lamas, Tallinn University, Estonia
Alin Moldoveanu, University Politehnică of Bucharest, Romania Philippe Palanque, IRIT, Université Paul Sabatier de Toulouse, France Oscar Pastor, Polytechnic University of Valencia, Spain
Lyn Pemberton, Brighton University, United Kingdom
Dorin Mircea Popovici, Universitaty Ovidius, Constanta, Romania Costin Pribeanu, ICI Bucharest, Romania
Traian Eugen Rebedea, University Politehnică of Bucharest, Romania Adriana Elena Reveiu, University of Economic Studies, Bucharest, Romania Cristian Rusu, Pontificia Universidad Católica de Valparaíso, Chile Carmen Santoro. ISTI-CNR, Pisa, Italy
Dominique Scapin, INRIA, France
Ahmed Seffah, Lappeeranta University of Technology, Finland Marcin Sikorski, Gdansk University, Poland
Jeam-Sebastien Sotted, Luxembourg Institute of Science and Technology, Luxembourg Christian Stary, University of Linz, Austria
Teodor Stefănuţ, Technical University Cluj-Napoca, Romania Ştefan Trăuşan-Matu, University Politehnică of Bucharest, Romania Jean Vanderdonckt, Universite catholique de Louvain, Belgium Radu Daniel Vatavu, University of Suceava, Romania
Panayiotis Zaphiris, Cyprus University of Technology, Cyprus
ADDITIONAL REVIEWERS
Lenuţa Alboaie, Alexandru Ioan Cuza University, Iaşi, Romania Felix Albu, Valahia University of Târgovişt, Romania
Adriana Alexandru, ICI Bucharest, Romania Dragos Catalin Barbu, ICI Bucharest, Romania Verginica Barbu-Mititelu, ICIA Bucharest, Romania Oana Balan, University Politehnică of Bucharest, Romania Elena Băutu, Universiy Ovidius, Constanta, Romania Victor Bâcu, Technical University Cluj-Napoca, Romania Tiberiu Boroş, ICIA Bucharest, Romania
Alexandru Butean, University Politehnică of Bucharest, Romania
Dumitru Clementin Cercel, University Politehnică of Bucharest, Romania Mihaela Colhon, University of Craiova, Romania
Liviu Cotfas, University of Economic Studies, Bucharest, Romania Titus Felix Furtună, University of Economic Studies, Bucharest, Romania Florin Girbacia, University Transilvania, Brasov, Romania
Gabriela Grosseck, University of West, Timisoara, Romania Carmen Holotescu, Ioan Slavici University, Timisoara, Romania Vincentas Lamanauskas, Siauliai University, Lithuania
Valentina Marinescu, University ofBucharest, Romania Cornelia Melenti, Technical University Cluj-Napoca, Romania Vasile Dănuţ Mihon, Technical University Cluj-Napoca, Romania Cristian Mihaescu, University of Craiova, Romania
Delia Mitrea, Technical University Cluj-Napoca, Romania
Florica Moldoveanu, University Politehnică of Bucharest, Romania Gabriel Neagu, ICI Bucharest, Romania
Mihaela Ordean, iQuest Technologies Cluj-Napoca
Marian Pădure, Babeş-Bolyai University, Cluj-Napoca, Romania Camelia Mihaela Popa, IFPAR, Bucharest, Romania
Elvira Popescu, University of Craiova, Romania Dan Rotar, University of Bacau, Romania
Corina Sas, University of Lancaster, United Kingdom Emil Stănescu, ICI Bucharest, Romania
Serban Ungureanu, Alexandru Ioan Cuza University, Iasi, Romania
viii
Conference organized with the support of:
Alexandru Ioan Cuza University of Iasi, Romania
Université catholique de Louvain, Belgium
Louvain School of Management, Belgium
International Business Machines – IBM Belgium
RoCHI – ACM SIGCHI România
This volume was published with the support of the:
National Authority for Scientific Research and Innovation (ANCSI)
Foreword
In 2016, the International Conference on Human-Computer – RoCHI has reached its 13th edition. Like all the previous editions, RoCHI 2016 provides an inter-disciplinary forum for the exchange of ideas, expertise and research results in the field of human-computer interaction.
The conference is publishing within this proceedings an invited paper and 20 regular papers organized in five thematic sessions. These papers were selected from a total of 29 submissions that were sent to RoCHI 2016. After a careful review process, each submission being assigned to a minimum of three and maximum of six reviewers in a peer-reviewed fashion (double blind), a total number of 20 short and long papers were accepted, which represents an acceptance rate of 68%.
The invited paper was entitled “The design is right and the right design” – How to measure and theorize “right” in HCI and Software Engineering? It was presented by Professor Ahmed Seffah from Lappeenranta University of Technology, Finland. It is about bridging the gap between classical design for usable user interfaces and sustainable design so that it could be measured.
The five sessions were entitled: Usability and accessibility evaluation, User interface development, Interaction techniques, HCI in e-learning systems, and Interactive applications.
The first session in the proceedings and in the conference program is Usability and accessibility evaluation. The session contains papers on topics that are at the core of HCI research, such as usability studies, automated evaluation, reliability and models for software design and GUI. The papers from this section present: (1) how can assistive technologies help people with disabilities (Implementation of a polyglot text-to-speech synthesis in two assistive technologies – Paul Fogarassy-Neszly, Aurel Patru, Dragos Daniel Iordache and Costin Pribeanu), (2) how was UMUX and UMUX-LITE scales been tested in the context of Facebook use by university students (Comments on the reliability and validity of UMUX and UMUX-LITE short scales – Costin Pribeanu), (3) how it is built ERGOSIM, a software that automatically evaluate the design of menu bars, pull-down menus, and sub-menus of a graphical user interface by reviewing usability guidelines related to menu design (Automated Evaluation of Menu by Guidelines Review – Sara Bouzit, Gaëlle Calvary, Denis Chêne and Jean Vanderdonckt) and (4) what are the results of an experiment in which students are asked to reposition rectangular shapes in a window in two cases: when they are or they are not told that the window should be considered the configuration of a graphical user interface (GUI) (Perceived aesthetics of user-modifiable layouts: a comparison between an unspecified design and a GUI – Stefan Trausan-Matu, Brahma Dathan).
The second section, User interface development, contains four papers related to user interfaces: (1) UIDLC Manager, a software that provides user interface designers and developers with methodological guidance throughout user interface development life cycle (Enactment of User Interface Development Methods in Software Life Cycles – Iyad Khaddam, Hanaa Barakat, Jean Vanderdonckt), (2) an Android application which offers the possibility to navigate through a 3D model using the movement of a mobile phone, determined by its own sensors (Increasing the Accuracy of Indoor Localization Applications by Using Predefined Markers and the Phone’s Camera – Szabolcs Orban, Teodor Stefanut), (3) a complex system (hardware and software) that is used to monitor and control some ambient parameters (temperature, humidity, light intensity, etc.) (Using WSN and Mobile Apps for Home and Office Ambient Monitoring and Control – Catalin Damian, Lenuta Alboaie and Adrian Iftene) and (4) SMAUG is a game based on Sphero 2.0 robotic ball, where multiple players share the same driving control of the robot (Sphero – Multiplayer Augmented Game (SMAUG) – Marian- Nicolae Pinzariu, Adrian Iftene).
The third section, Interaction Techniques, addresses last technologies used in new types of interaction: (1) a method with aim to improve interaction in virtual environments and make it less stressful and more effortless (Reducing Gestural Interaction Physical Stress in Virtual Environments: an Experiment – Sobhi Ahmed, Laure Leroy and Ari Bouaniche), (2) a study with aim to present Oculus Rift, used to assess craving of Romanian smokers (Oculus Rift 3D Interaction and Nicotine Craving: Results from a Pilot Study – Ioana-Monica Ciolan, Sabin Buraga and Ion Dafinoiu), (3) the design and development of a complete hardware and software solution for a brain computer interface (Brain Computer Interface using Machine Learning – Cristian-Valeriu Soare) and (4) a methodology for identifying and assessing a set of performance issues encountered in a particular web application, with impact on the usability level (Visual techniques for identification and evaluation of a web application usability issues – Mihaela Ciugudean, Dorian Gorgan).
Forth section, HCI in e-learning system, contains papers on novel and popular topics in HCI. The papers are presenting:
(1) a case study in testing technology acceptance model to explain the adoption of Facebook by Romanian university students (Testing the technology acceptance model with Romanian university students – Alexandru Balog, Costin Pribeanu), (2) a case study about the relationship between the Facebook dependence and the negative consequences of the excessive use of Facebook on the students’ university work (Measuring the negative effects of the Facebook dependence on the students’ university work – Gorghiu Gabriel, Manea Valentina Iuliana, Iordache Dragoş Daniel and Pribeanu Costin), (3) the methodology and results obtained after the evaluation of the Tesys e-learning platform (Evaluation of the Tesys e-Learning platform’s interface – Paul Stefan Popescu, Cristian Mihaescu, Mihai Mocanu and Costel Ionascu) and
x
(4) MOOCBuddy, an innovative project aiming to make known the Romanian initiatives related to MOOCs (MOOCBuddy: a Chatbot for personalized learning with MOOCs – Carmen Holotescu).
In the last section, Interactive applications, are four papers covering the following themes: (1) ReaderBench framework, which includes multi-lingual comprehension-centered web services designed to address a wide range of individual and collaborative learning scenarios (ReaderBench goes Online: A Comprehension-Centered Framework for Educational Purposes – Marius-Gabriel Gutu, Mihai Dascalu, Stefan Trausan-Matu, Philippe Dessus), (2) how the tweets can be processed in order to obtain valuable information in real time, based on user preferences and different search criteria (Event detection in Tweets – Andrei Bogdan Baran, Adrian Iftene), (3) a study about rhythm in various genres of texts (Rhythm analysis of texts using Natural Language Processing – Irina-Diana Niculescu and Stefan Trausan-Matu) and (4) an approach to terrain synthesis from minimal-detail user-provided heightmaps (Terrain Synthesis from Crude Heightmaps – Alexandre Philippe Mangra, Adrian Sabou, Dorian Gorgan).
We cannot end this preface without expressing our appreciation to the members of the scientific committee and to the volunteer reviewers from the RoCHI group who helped for selecting the best papers to be presented at the conference.
Moreover, we acknowledge the efforts of all the persons involved in the organization of the RoCHI 2016 Conference and thank them for their efforts!
Iasi, August 8th, 2016
Editors, Adrian Iftene Jean Vanderdonckt
“The design is right and the right design”
How to measure and theorize “right” in HCI and Software Engineering?
Ahmed Seffah
Human-centric Software Engineering
ABSTRACT
In this keynote, we will show how to improve the practices of interactive software and user interfaces design, by applying lessons from various fields of design to the co-creation of interactive software products, systems and services. The goal is to create software that works – really works, meaning we can measure it works, usable, profitable yet secure, resilient and sustainable – A software is being appropriate and effective for people who live in the world that the software creates.
Beyond the “cook-books of the HCI gurus” and the large diversity of HCI design patterns, guidelines and principles, the talk will highlight the the importance of the qualities, properties and quantities that quantify the quality of software systems from the human perspective. How to measure the “right”, “the old user friendly”, the past usability, the today quality of user experience and the tomorrow sustainability. How to ground measures of HCI in the general theory of software quality and measurement. What we can learn from the successful stories and history of measurement in many fields: medical sciences, engineering, and even from natural and hearth sciences?
ABOUT
Ahmed Seffah is a professor of human-centric software engineering and HCI at Lappeenranta University of Technology, Finland. Previously, he was a professor and Concordia research chair on human-centered software engineering at Concordia University as well a visiting professor in more than 10 universities and research Centre including IBM, University of Lausanne, Daimler Chrysler and the Computer research institute of Montreal. Professor Seffah co-authored five books, the last one on the “Patterns of HCI Design Patterns and the HCI Design of Patterns”. His main research is to understanding those HCI and interactive software systems design, software usability, user-centric engineering, UX design practices and all similar ones within the wider software and systems engineering processes. Most visible contributions of his work are on the gaps and bridges between design science practices and software engineering methodologies such as agile, model-driven and service-oriented while building a theories of human-centric software engineering.
2
Implementation of a polyglot text-to-speech synthesis in two assistive technologies
Paul Fogarassy-Neszly, Aurel Patru BAUM Engineering
Str. Traian Mosoiu 8, Arad, Romania [email protected], [email protected]
Dragoş Daniel Iordache, Costin Pribeanu
National Institute for R & D in Informatics - ICI Bucharest
Bd. Averescu 8-10, Bucucharest, Romania [email protected], [email protected]
ABSTRACT
The text-to-speech (TTS) synthesis is widely used in the area of assistive technologies for visually impaired people as well as for people with dyslexia or related learning disabilities. In the case of multilingual text-to-speech synthesis, both the language identification and voice switching are desirable. This paper presents the implementation of a multilingual text-to-speech in two assistive technologies: the automatic reading machine POET and the portable organizer for blind people Pronto.
Both implementations have been tested with visually impaired users. The evaluation results show that the implementation of this software component makes these devices more easy and comfortable to use.
Author Keywords
Accessibility, assistive technologies, usability, technology acceptance, TTS, visually impaired users.
ACM Classification Keywords
D.2.2: Design tools and techniques. H5.2 User interfaces.
INTRODUCTION
A Text-To-Speech (TTS) system is a software component able to produce speech output, whether directly from text input or from a scanned document page previously submitted to an Optical Character Recognition (OCR) system. Many assistive technologies for visually impaired people are using text-to-speech (TTS). TTS synthesis is also used by people with reading disabilities (dyslectic, illiterate, or with learning disability) in order to make the electronic or printed documents accessible. Examples of assistive technologies using TTS synthesis are: screen readers, automatic reading machines, portable computers with voice interface, smartphones, GPS systems, as well as many other gadgets or self-voiced software.
Usually, the synthetic voice corresponding to the native (or preferred) language of the user is currently selected. If the text is written in another language, then the user has to manually select a corresponding voice for that language.
In recent years, there is an increasing interest in the applications that are able to process texts written in two or more languages. There are many application areas that need polyglot text-to-speech, such as education for all and multi-cultural contexts [8, 11, 12, 13]. In this case, both a multilingual (polyglot) text-to-speech synthesis and voice switching are needed. This requires to analyze the text in order to detect the language and then to select the voice available for that language. Several approaches for the multilingual TTS exist that differ with regard to the solutions adopted for the text analysis and speech synthesis [1, 3, 9, 10, 11].
In a previous work, a software component for multilingual text-to-speech has been presented that performs both the automatic language identification and voice switching.
The component has been developed during the research project iT2V that has been carried on in a consortium of three partners: BAUM Engineering, ETA Automatizari Industriale, and National Institute for Research and Development in Informatics – ICI Bucharest.
The language recognition component plays the role of an intermediate layer, voice independent, between the application and the synthesis process. Language identification is based on statistical analysis and trigrams frequency evaluation for envisaged languages [2]. The development followed four steps: alpha version (proof-of- concept, functional version (beta), commercial version, and implementation in several applications. The development cycle of iT2V is illustrated in Figure 1.
Figure 1. iT2V development cycle.
The goal of the alpha version was to test the language detection algorithms [5]. In the second step, a preliminary functional version (beta01) was developed and tested with four candidate languages [7]. The evaluation revealed a major usability issue: switching the voice in the middle of a sentence. Therefore, is an improved functional version has been developed and tested again. Based on the testing results [6], the commercial version has been developed and then used to implement the software component. The iT2V component plays the role of a special voice (SAPIiT2V) that identifies the language and selects the corresponding synthetic voice.
This paper presents the implementation of the multilingual text-to-speech in two assistive technologies: the automatic reading machine POET and Pronto, the portable electronic organizer for blind and visually impaired users. The implementations have been evaluated for usability and technology acceptance with visually impaired users. The rest of this paper is organized as follows. Next section briefly presents the automatic reading machine POET and the portable organizer for blind people Pronto. Then the evaluation method is presented. In the next section the evaluation results of the implementations in these devices are presented and discussed. The paper ends with conclusion and future research directions.
IMPLEMENTATIONS OF IT2V Implementation in a reading machine
The first implementation has been done on the automatic reading machine POET (trade mark of BAUM Retec AG).
This type of assistive technology integrates a computer and a scanner (including an Optical Character Recognition software - OCR - with italics adjustment) in one single device. The device is simple and is easy to use. The basic version of POET (Compact 2) has only two buttons (Start and Stop) and two control knobs to adjust the volume and the reading speed. Figure 2 presents the reading device POET Compact 2. The user can read periodicals, books, or magazines. The text could be written on one or several columns.
Figure 2. Automatic reading device POET Compact 2.
The text could be saved and then used for a new lecture or transferred to another device, such as an MP3 player. The model Compact 2+ has three buttons and a menu keypad and has been designed for users that prefer is a model for more advanced users who prefer more features and full control over the functionality. For Braille literate and deaf and blind users a Braille display can be connected to the Poet. As Braille tables are language dependent, prior language detection is necessary before producing Braille output. This is a not foreseen application of language detection.
Implementation in a portable device
The second implementation has been done on the organizer for blind people Pronto (trade mark of BAUM Retec AG).
Pronto is a portable organizer for visually impaired people that runs under the operating system Window CE. The device includes a Braille display and speech output.
Pronto supports various applications, like a regular PDA (personal digital assistant). As such is more complex than other assistive technologies. Figure 3 presents the organizer Pronto. The main functions are: taking notes with a text editor, agenda & organizer, playing music, reading books in text or Braille format, Internet access.
The device is easy to handle and easy to use. It provides a keyboard for 8-dot Braille input, an 18-cell Braille display with integrated cursor routing, four function keys for fast access to applications, and a navistick for an easy and comfortable operation.
Figure 3. Organizer for blind people Pronto.
METHOD
Evaluation techniques
The evaluation has two goals: usability and acceptance.
First, a usability inspection has been carried on by three experts. Then, each implementation has been tested with visually impaired users for usability and acceptance. After testing the device, participants were asked to answer a questionnaire as regards the actors that are influencing the technology acceptance. The users were also asked to mention the most important positive and negative aspects as regards each device and to weight the relative importance of three factors (ease of use, usefulness, and enjoyment).
Measures
Following measures were collected: number of usability problems by severity degree (major, moderate, and minor), description of unique usability problems, number of positive and negative aspects, the ratings of each item of the questionnaire, and the relative importance of each factor. The items in the questionnaire are presented in Table 1.
Item Statement
PEU1 It is easy to understand how to use iT2V with this device PEU2 This device is easier to use with iT2V
PU1 iT2V is a useful facility for me
PU2 The capabilities of iT2V correspond to my requirements PE1 I prefer to use this device having iT2V
SAT1 I am satisfied with this device having iT2V INT1 If would like to use this device with iT2V
INT2 I will recommend other people this device having iT2V Table 1. Questionnaire.
The questions are inspired from the technology acceptance model [4]. Since all the users were visually impaired, the answers have been collected via interview (ratings of each item on a 5-point Likert scale).
Participants and tasks
Seven people participated in user testing (six men and a woman). The mean age of participants is 40.3 years (SD=9.23) with a minimum of 23 and a maximum of 48 years. The user testing took place in Arad, at the Local Branch of Romanian Association of the Blind (Filiala Arad a Asociației Nevăzătorilor din România). Except for one user (university student), all are retired for medical reasons. All participants graduated a high school. The disability degree is severe (first degree - legally blind).
4
The familiarity with the assistive technologies is variable.
However, all are familiar with the lecture on the Internet.
The most used information and communication technologies are the computer, mobile phone, tablet, and scanner + OCR. Most used applications are the Internet browsers, Skype, and Facebook. The usual goals are related to information, lecture, entertainment (games), and socialization. The POET device has been tested with three tasks.
The goal of the first task was to read two documents, each of them written in two languages (Romanian and English), without iT2V. This means to stop the lecture and manually change the voices from the device option menu. In the first document the text was written in one column. In the second document, the same text was written in two columns.
The second task was identical, but with the iT2V feature selected. The third task was to read several pages from a magazine with iT2V feature selected. The Pronto device has been tested with a task performed under two conditions. The task was to open and read a document written in two languages (Romanian and English). The condition is to have the iT2V selected.
RESULTS
Reading machine POET
The usability inspection identified several usability problems for the first task. The problems are related to the navigation in the menu in order to manually change the voice. No usability problem has been detected at the second and third tasks. The accuracy of language identification was 100%. The average time saving for a voice switching was 2 minutes. The user testing confirmed the results of the usability inspection: no usability problem has been detected after selecting the iT2V feature.
Item Factor Mean SD
PEU1 Ease of use 4.71 0.49 PEU2 Ease of use 4.43 0.79 PU1 Usefulness 4.29 1.50 PU2 Usefulness 4.29 1.50
PE1 Enjoyment 4.57 0.79
SAT1 Satisfaction 5.00 0.00 INT1 Intention to use 5.00 0.00 INT2 Intention to use 4.71 0.76 Table 2. Descriptives for the POET device.
Most of the users mentioned that the device is accessible, simple, easy to use, useful, and compact. They appreciated the ease of reading and the multilingual feature. As regards the negative aspects, the users mentioned the price (too high), the dimensions (too big) and the lack of the translation.
The users considered that the most important factor for the acceptance of the reading device is the usefulness (45%), followed by the ease of use (29%), and usefulness (26%).
The descriptive statistics for the items in the questionnaire is given in Table 2. The mean value of items related to the ease of use is 4.57 (SD = 0.45) and of the items related to usefulness 4.29 (SD = 1.47). An analysis of correlation
based on Spearman coefficient shows a marginally significant relationship between the enjoyment and the satisfaction (rho = 0.68, p = 0.091).
The organizer for blind people Pronto
The usability inspection identified few usability problems when performing the task without iT2V. The problems are related to the navigation in the menu in order to manually change the voice. The problems are related to the device itself and somehow inherent, given the rich functionality and the lack of familiarity with the device. The user testing confirmed the results of the usability inspection: no usability problem has been detected after selecting the iT2V feature.
Overall, the usability evaluation results were similar for the two implementations. Most frequently mentioned positive aspects were the portability, the Braille feature, flexibility of use (both voice and Braille), and the multilingual feature. As regards the negative aspects, they mentioned the price and the difficulty to use.
The users considered that the most important factor for the acceptance of the reading device is the ease of use (44%), followed by the usefulness (36%), and enjoyment (20%).
The descriptive statistics for the items in the questionnaire is given in Table 2.
Item Factor Mean SD
PEU1 Ease of use 3.71 0.76 PEU2 Ease of use 3.86 0.90 PU1 Usefulness 4.43 1.13 PU2 Usefulness 4.43 0.53 PE1 Enjoyment 4.43 1.13 SAT1 Satisfaction 4.57 0.53 INT1 Intention to use 5.00 0.00 INT2 Intention to use 4.86 0.38 Table 3. Descriptives for the Prontop device
The mean value of items related to the ease of use is 3.79 (SD = 0.39) and of the items related to usefulness 4.43 (SD = 0.53). The analysis of correlation showed a marginally significant relationship between the perceived enjoyment and the satisfaction (rho = 0.72, p = 0.068) and a significant relationship between the perceived enjoyment and the perceived usefulness (rho = 0.84, p = 0.018).
Discussion
The evaluation has been focused on the implementation of the iT2V component and not on the device itself. In this respect, no usability problem has been identified. In both cases, the accuracy of voice switching was excellent.
The component saves time and make the use of these assistive technologies much simpler. The answers to the questionnaires show a difference between the ease of use of the two devices. Pronto has a more rich functionality which makes it more difficult to use. Nevertheless, in both cases the users have been satisfied and expressed the intention to use these technologies.
There are several limitations of this work. First of all, the number of users is small, so the results could be seen as
only exploratory. Second, the users were not familiar with these devices. A reason is the price: six out of seven users said that they can’t afford these technologies. Third, the time for evaluation was limited, so only few tasks were assigned. The reasons were the availability of users and the time needed to explain how to use a new device.
CONCLUSION AND FUTURE WORK
In this paper two implementations of the iT2V software component for multilingual text-to-speech and automatic voice switching have been presented.
The evaluation results showed that iT2V is usable, useful and enjoyable. A device with iT2V is easier to use. Since the users are visually impaired, manually changing the voice could be done via an audio menu, which is both difficult to use and time-consuming.
In the next future the language recognition will be implemented in COBRA, the screen reader software. This will be very useful, especially for internet browsing, when beside pages in local language, pages in English should be accessed. Also, in countries with two or more official languages this facility will be useful, as well as for any polyglot user who often are switching between documents in different languages.
Also, beside the language dependent speech synthesis switch, the Braille table will be selected accordingly, taking into account that not only synthetic voices, but also Braille tables are language dependent. This will allow the user to access documents in different languages, both through speech synthesis and Braille display, without any option menu exploration.
ACKNOWLEDGMENTS
This work is partly supported by the IT2V research project (29DPST/2013), financed by UEFISCDI under the PNCDI II Innovation Program.
REFERENCES
1. Bourlard, H., Dines, J., Magimai-Doss, M., Garner, P.
N., Imseng, D., Motlicek, P. and Valente, F. Current trends in multilingual speech processing, Sadhana 36(5), (2011), 885-915.
2. Cavnar, W. and Trenkle, J. N-gram-based text categorization. Proceedings Document Analysis and Information Retrieval (SDAIR-94), (1994).
3. Chen, C. P., Huang, Y. C., Wu, C. H. and Lee K. D.
Polyglot speech synthesis based on cross-lingual frame
selection using auditory and articulatory features.
IEEE/ACM TASLP 22 (10), (2014), 1558-1570.
4. Davis, F.D., Bagozzi, R.P. and Warshaw, P.R.
Extrinsic and intrinsic motivation to use computers in the workplace. Journal of Applied Social Psychology, 22 (14), (1992), 1111-1132.
5. Fogarassy-Neszly, P. and Gherhes, V. Applications for dynamic language identification. Proceedings of RoCHI 2014 Conference, Popovici D., M. & Iordache D.D. (Eds.), (2014), 51-54.
6. Fogarassy-Neszly, P., Zinveliu, Z. and Pribeanu, C. A software component for polyglot text-to-speech synthesis: user interface and beta testing results.
Proceedings of RoCHI 2015 Conference, Dardala, M., Rebedea, T.E. (Eds.), (2015), 145-148.
7. Pribeanu, C. and Fogarassy-Neszly, P. Beta testing of a dynamic language identification software component - preliminary results. Revista Romana de Interactiune Om-Calculator 7(3), (2014), 259-272.
8. Ramani, B., Actlin Jeeva, M.P., Vijayalaksmi, P., and Nagarajan, T. Cross-lingual voice conversion-based polyglot speech synthesizer for Indian languages.
Proceedings INTERSPEECH, (2014), 775-779.
9. Romsdorfer, H. and Pfister, B. Text analysis and language identification for polyglot text-to-speech synthesis. Speech Communication 49, (2007), 697-724.
10. Shiga, Y. and Kawai, H. Multilingual speech synthesis system. Journal of the National Institute of information and Communication Technology 59(3/4), (2012), 21- 28.
11. Traber, C., Huber, K., Nedir, K., Pfister, B., Keller, E.
and Zellner, B. From multilingual to polyglot speech synthesis. Proceedings of EUROSPEECH, Budapest, Hungary, (1999), 835–838.
12. Tripathi, M., and Shukla, A. Use of assistive technologies in academic libraries: A survey. Assistive Technology, 26(2), (2014), 105-118.
13. Turunen, M. and Hakulinen, J. Mailman-a multilingual speech-only e-mail client based on an adaptive speech application framework. In Proceedings of Workshop on Multilingual Speech Communication - MSC 2000, (2000), 7-12.
14. Udvari-Solner, A. and Thousand, J. S. Creating a responsive curriculum for inclusive schools. Remedial and special education, 17(3), (1996), 182-191.
6
Comments on the reliability and validity of UMUX and UMUX-LITE short scales
Costin Pribeanu
National Institute for Research and Development in Informatics - ICI Bucharest
Blvd. Maresal Averescu, nr.8-10, Bucharest, Romania [email protected]
ABSTRACT
Recent work on usability and user experience shows several concerns on the validity of evaluation instruments.
There is a debate on the use of standardized scales versus short scales, such as UMUX and UMUX-LITE or even a single-item measure of usability. Nevertheless, there are relatively few papers reporting the testing of these scales together for reliability and validity. This paper aims at discussing the UMUX and UMUX-LITE scales that have been tested in the context of Facebook use by university students. From a theoretical point of view, both scales are questionable. From an empirical point of view, the testing results confirmed a lack of unidimensionality as well as a poor reliability and convergent validity of these scales.
Author Keywords
Usability scales, UMUX, UMUX-LITE, factor analysis, validity, Facebook use.
INTRODUCTION
Recent work on usability and user experience shows several concerns on the reliability and validity of the scales used to measure the perceived usability. There is currently a debate on the use of standardized scales versus short scales, such as UMUX [6] and UMUX-LITE [14] or even a single-item measure of usability [13]. There are many pros and cons as regards the reliability and validity as well as the practical benefits. Nevertheless, there are relatively few papers reporting the testing of these scales for reliability and validity.
This paper aims at discussing the UMUX and UMUX- LITE scales from both a theoretical and an empirical point of view. Additionally, the perceived ease of use (PEU) is analyzed that is a widely used concept in the context of technology acceptance [5]. Since PEU is a short scale taping on several key usability aspects, it could be a better alternative than UMUX and UMUX-LITE.
Theoretically, the analysis is following the scale development recommendations. Empirically, the analysis is focused on the scale testing that has been carried on by using two samples collected during a larger study on Facebook use by university students [9]. The first sample is from the pilot study and is used to assess the UMUX and UMUX-LITE scales. The second sample is from a subsequent study using a revised evaluation instrument and is used to assess the UMUX-LITE and PEU scales.
The rest of this paper is organized as follows. The following section briefly presents recent approaches in the area of scale development with an emphasis on the scale development process and the existing usability scales. In
section 3, the analysis of UMUX and UMUX-LITE is presented based on two empirical studies. The same assessment criteria are used to analyze the PEOU scale.
The paper ends with conclusion and future research directions.
RELATED WORK Scale development
The interest in developing rating scales increased after the adoption of the ISO 9241-11 standard that included satisfaction as a key usability aspect. As Lindgaard &
Kirakowski [15] pointed out, the landscape of scale development in HCI shows many usability scales, many approaches, as well as many opinions as regards the scale reliability and validity.
When analyzing the reliability and validity of the usability scales, two aspects are usually neglected: the theoretical meaning and the multidimensional nature of the usability concept. Psychometrics is not a favorite area of expertise in HCI [15]. As such, the scale development process is not well understood as regards both the ordering of steps to be carried on and the suitable techniques that should be used in each step.
Long time ago, Gerbing & Anderson [7] outlined an updated paradigm for scale development that includes a confirmatory factor analysis (CFA) to assess the scale unidimensionality. They underlined that, only after achieving an acceptable unidimensionality level, the reliability could be assessed. This precondition is usually ignored in the existing papers reporting the development and testing of usability scales. The authors rely on the traditional approach that only includes the Cronbach’s alpha coefficient, the item-to-total correlations, and the exploratory factor analysis (EFA).
More recently, MacKenzie et al. [16] emphasized the importance of the conceptualization as a first step in the scale development process. They noticed that an adequate conceptualization is difficult and requires a review of the literature on the meaning of related constructs, aspects these constructs refer to, dimensionality and preliminary research with domain experts or practitioners. Another important issue is to specify if the construct is measured reflectively or formatively.
Usability in the ISO standards
ISO 9241-11 standard defined usability as the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency, and satisfaction in a specified context of use [11]. The ISO standard 9126-1 defined usability as the capability of a
software product to be understood, learned, used, and attractive to the user, when used under specified conditions [10]. Later on, both definitions were integrated in the ISO standard 25010 [12], under two key terms:
Quality in use: the degree to which a product used by specific users meets their needs to achieve specific goals with effectiveness, efficiency, safety, and satisfaction in specific contexts of use.
Usability: the degree to which a software product is able to satisfy the following needs when used under specified conditions: appropriateness recognizability, learnability, operability, error protection, user interface aesthetics, and accessibility.
As pointed out by Bevan et al. [1], the quality in use defines usability as a high level concept, focusing on the outcomes of the interaction rather than on the characteristics that make a product usable. Unfortunately, this distinction is rarely made in the mainstream of HCI literature.
Measurement scales for the perceived usability A well-known usability scale is System Usability Scale (SUS) that has been developed by Brooke as a simple,
“quick and dirty” scale [3]. SUS has been widely used and is considered an industrial standard [2, 14].
Several authors noticed that SUS lacks unidimensionality.
For example, Borsci et al. [2] found that a learnability dimension of SUS might emerge under certain conditions (when administrated to experienced users). SUS has been also criticized for using both positive and negative wording, since this may lead to mistakes (made by respondents) and mis-coding (made by researchers) [17].
More recently, Finstad proposed the UMUX (Usability Metrics for User Experience) as a shorter alternative to SUS. UMUX have been criticized for dimensionality [4, 14] and for using negative wording [14, 17]. Lewis et al.
[14] found that UMUX has a bi-factorial structure with positive tones aligning with one factor and negative tones with the second factor.
An even shorter scale that is based on UMUX has been proposed by Lewis et al. [14]. UMUX-LITE was intended as a very quick, two-item scale, that uses the first and the third item from UMUX. The authors found that UMUX- LITE is unidimensional and has acceptable reliability.
However, they recommended using this scale with caution until it will be validated across a wider variety of systems.
A well-known scale measuring the perceived ease of use (PEU) has been developed and tested in the context of technology acceptance studies. The technology acceptance model (TAM) has been developed by Davis et al. [5], in order to explain and predict the technology acceptance on a large variety of technologies. Although it is a short scale with a widely recognized psychometric quality, PEU has been rarely used in the HCI studies.
ANALYSIS OF UMUX AND UMUX-LITE Method
The analysis follows the recommendations in the literature for scale development and assessment of dimensionality, reliability, and validity [7, 8, 16]. The first step is to analyze the conceptualization based on the definition of concepts in the literature. Then, the dimensionality is assessed via exploratory and confirmatory factor analysis.
After demonstrating that the construct is unidimensional, the reliability could be analyzed checking the magnitude of Cronbach’s alpha and the item-to-total correlations. The convergent validity is assessed by the examination of the composite reliability (CR), and average variance extracted (AVE).
Empirical studies
UMUX and UMUX-LITE have been tested in a larger study on the use of Facebook (FB) by university students.
Two samples collected during these studies are used for the analysis of the psychometric quality of UMUX and UMUX-LITE. The respondents were asked to answer questions related to demographics, enrollment, FB usage (the size of their FB network, frequency of use, minutes per day), and to evaluate items on a 7-point Likert scale.
The first sample has been collected in 2014 and consists of 152 students (110 female, 42 male) from two universities in Lithuania. The negatively worded items in Table 1were recoded. The first and third items in Table 1 represent the UMUX-LITE scale.
Item Statement M SD
U1 FB’s capabilities meet my requirements 4.02 1.38 U2 Using FB is a frustrating experience 3.44 1.64
U3 FB is easy to use 4.91 1.15
U4 I have to spend too much time correcting things with FB
4.12 1.63
Table 1. Descriptives for UMUX scale (N=152).
The second sample has been collected in 2015 and consists of 414 students (258 female, 156 male) from a Romanian university. Since the testing results from the first study revealed poor psychometric properties of UMUX, the scale has been removed from the evaluation instrument and replaced with PEU. However, the first item has been preserved in order to test again UMUX-LITE.
Item Statement M SD
U1 FB’s capabilities meet my requirements
4.36 1.51 PEU1 It is easy to learn how to use FB 6.10 1.27 PEU2 / U3 FB is easy to use 6.21 1.17 PEU3 My interaction with FB is clear and
understandable
5.69 1.38 Table 2. Descriptives for UMUX-LITE and PEU (N=414).
8
The PEU scale has been developed by adapting items from the existing scales. The three items tap on three usability attributes: understandability, learnability, and operability.
As it could be observed, the item U3 in Table 1 is identical with the item PEU2 in Table 2.
Conceptualization
In most studies discussing UMUX and UMUX-LITE the reliability and validity is limited at dimensionality which is assessed with exploratory factor analysis techniques, reliability, and correlation with other usability scales. The main shortcoming of UMUX (and, consequently, of UMUX-LITE) is the poor conceptualization which is due, on one side, to the underlying usability definition and, on another side, to the misunderstanding of the nature of the measurement model.
Both scales take the roots from the quality in use concept and try to measure the user’s subjective satisfaction.
Nevertheless, the measured variable is not the satisfaction (there is no item saying “I am satisfied with…”. Rather, the operationalization of the construct is done with a mix of items measuring various aspects of user experience and usability. As such, the conceptualization is ambiguous and is not clear what UMUX is actually measuring.
The first item is the most ambiguous and undermines the conceptualization of both UMUX and UMUX-LITE. The fit between the user’s needs and requirements could refer to anything: ease of use, aesthetics, flexibility, robustness, safety, usefulness, enjoyment, etc.
The second shortcoming is the lack of a clear definition of the nature of the measurement model. The measurement model describes the relationship between a construct and its measures [7, 16]. According to the direction of the causal relationship, the constructs could be reflective (from the construct to its measures) or formative (from measures to construct). It is also possible to define multidimensional constructs where the dimensions are specified as first order constructs.
Failure to adequately specify the measurement model leads to a poor operationalization and a lack of validity.
The point is that the conceptualization and validation recommendations are different for reflective and formative measurement models. Unidimensionality and inter-item correlation are required for reflective constructs, since all items are supposed to measure the same thing. For a more detailed discussion on the scale development of reflective and formative constructs, see MacKenzie et al. [16].
Therefore, neither UMUX nor UMUX-LITE could be adequately assessed as measurement scales, since both of them suffer from a lack of clear definition of the construct domain. As regards the measurement model, although is not specified, it is assumed to be reflective according to the assessment techniques used by the authors.
Dimensionality
The dimensionality of UMUX and UMUX-LITE has been analyzed within the two empirical studies. The first study enabled testing of UMUX and UMUX-LITE.
The principal component analysis with Varimax rotation for UMUX resulted in two factors explaining 34.56%, respectively 26.69% of the variance. The same analysis for UMUX-LITE resulted in one factor explaining 60.06
% of the variance.
A confirmatory factor analysis has been then carried on.
The results revealed the lack of dimensionality for both constructs. The loadings of the underlying construct on its measures (α regression coefficients) is below the cutoff value of 0.60 [8]. The results are presented in Table 3.
Item
UMUX UMUX-LITE
EFA CFA (α) EFA
CFA (α)
1 2 1
U1 .707 -.364 0.66 .775 0.65 U2 .700 .332 -0.38
U3 .609 0.31 .775 0.34 U4 .919 0.32
Table 3. Dimensionality of UMUX and UMUX-LITE (N=152).
The second study enabled the analysis of dimensionality of UMUX-LITE and PEU. The principal component analysis with Varimax rotation for UMUX-LITE resulted in one factor explaining 68% of the variance. The same analysis for PEU resulted in one factor explaining 80.61%
of the variance.
The confirmatory factor analysis for UMUX-LITE revealed a low item loading of U1 (α=0.49). The same analysis for PEU confirmed its unidimensionality (item loadings: 0.86, 0.92, and 0.73).
The results of the two empirical studies demonstrate the lack of dimensionality for the UMUX and UMUX-LITE short scales as well as the limitations of the exploratory factor analysis for testing the dimensionality.
Reliability
The Cronbach’s alpha was unacceptable low in the first study: 0.213 for UMUX and 0.331 for UMUX-LITE. The item-to-total correlations were in the range of 0.05-0.23for UMUX, respectively 0.20 for UMUX-LITE.
In the second study, the Cronbach’s alpha for UMUX- LITE was low (0.517) and the item-to-total correlation also low (0.36). Cronbach’s alpha for PEU was 0.874 and the item-to-total correlation in the range of 0.68-0.82.
Convergent validity
Convergent validity refers to the degree to which the measures of a construct that are supposed to be related, are in fact related.
In the first study, the low item loadings make no sense to test the convergent validity of UMUX. For UMUX-LITE, the composite reliability of 0.412 and the average variance extracted of 0.279 demonstrate the lack of convergent validity. The second study confirmed the poor convergent validity of UMUX-LITE (CR=0.552, AVE=0.390). The
convergent validity for PEU was very good (CR=0.885, AVE=0.722).
Interpretation of scores
The final step in scale development is to provide the prospective researchers with some recommendations for the interpretation of scores. Since this step is beyond the purpose of this study, it will not be discussed. However, is important to note that the correlation of a scale under consideration with other existing scales does not ensure the scale validity. It is expected that two scales pointing to similar usability aspects correlate. The problem is that if the scale under consideration and the reference scale are not unidimensional, then a comparison leads to ambiguous if not erroneous conclusions.
DISCUSSION AND CONCLUSION
In the area of HCI, several misconceptions exist as regards the scale development and the validity criteria. The two short scales analyzed in this paper suffer from an ambiguous definition of the construct. An adequate conceptualization should include the specification of the dimensionality and the nature of the construct (reflectively vs. formatively measured).
It seems that the relationship between the definition of the target construct and the criteria for its assessment are not well understood: if a usability scale is not unidimensional, then more than one thing (e.g. usability) is measured. In other words, what is actually measured is not what has been supposed to be measured.
A problem with the conceptualization of existing short scales in HCI, such as SUS, UMUX, and UMUX-LITE is the confusion between usability and quality in use. The quality in use is a multidimensional construct since it taps on different concepts. Another problem is the overlapping between two HCI concepts: usability and user experience.
It is advisable to keep apart the scales measuring the pragmatic and hedonic aspects (each scale should undergo a separate validation procedure). For example, the perceived ease of use refers to pragmatic aspects, while the perceived enjoyment refers to hedonic aspects. Both scales have been widely used and validated in technology acceptance studies.
The empirical studies confirmed the recommendation of Gerbing and Anderson to use confirmatory factor analysis to assess the dimensionality [7]. The exploratory factor analysis is clearly not enough.
The two empirical studies show that both UMUX and its shorter version, UMUX-LITE, suffer from poor reliability, lack of unidimensionality, and poor convergent validity.
The correlation with other usability scales, which is frequently mentioned as an argument for reliability, is a poor surrogate when the candidate scale and the reference scale does not measure the same thing.
This paper does not deny the practical value of the short questionnaires which are less expensive and could provide a useful feedback for the developers. However, these should not be referred as usability or UX scales.
As it was shown, the PEU scale is unidimensional and reliable. PEU provides with a useful feedback on some key aspects of usability and could be combined with other short scales pointing to other usability / user experience aspects. This approach enables a step-by-step development of valid and reliable evaluation instruments and a flexible choice of scales, according to the objectives of the evaluation.
ACKNOWLEDGMENTS
This work was supported by the Romanian grant financed by ANCS under COGNOTIC 1609 0101 / 2016.
REFERENCES
1. Bevan, N., Carter, J. and Harker, S. ISO 2041-11 revised: What have we learned about usability since 1998? Human-Computer Interaction Design and Evaluation, Proc. HCI International Conference, LNCS 9169, Springer, (2015), 143-151.
2. Borsci, S., Federici, S., Bacci, S., Gnaldi, M. and Bartolucci, F. Assessing User Satisfaction in the Era of User Experience: Comparison of the SUS, UMUX, and UMUX-LITE as a Function of Product Experience.
International Journal of Human-Computer Interaction, 31(8), (2015), 484-495.
3. Brooke, J. SUS-A quick and dirty usability scale. Usability evaluation in industry, 189 (194), (1996), 4-7.
4. Cairns, P. A Commentary on short questionnaires for assessing usability. Interacting with Computers 25(4), (2013), 312-316.
5. Davis, F.D., Bagozzi, R.P. and Warshaw, P.R. User acceptance of computer technology: A comparison of two theoretical models, Management Science, 35 (8), (1989), 982-1003.
6. Finstad, K. The usability metric for user experience. Interacting with Computers 22(5), (2010), 323-327.
7. Gerbing, D.W. and Anderson, J.C. An updated paradigm for scale development incorporating unidimensionality and its assesment. Journal of Marketing Research 25(2), (1988), 86-192.
8. Hair, J.F., Black, W.C., Babin, B.J., Anderson, R.E.
and Tatham, R.L. Multivariate Data Analysis. 6th ed., Prentice Hall, 2006.
9. Iordache, D.D., Pribeanu, C., Lamanauskas, V. and Raguliene, L. Usage of Facebook by university students in Romania and Lithuania: a comparative study. Informatica Economica 19(1), (2015), 46-54.
10. ISO/IEC 9126-1, Software Engineering –Product quality. Part I: Quality Model, 2001.
11. ISO 9241-11, Ergonomic requirements for office work with visual display terminals (VDTs) - Part 11 Guidance on usability, 1998.
12. ISO/IEC 25010, Systems and software engineering – Systems and software product Quality Requirements and Evaluation (SQuaRE) – System and software quality models, 2011.
10
13. Konradt, U., Wandke, H., Balazs, B. and Christophersen, T. Usability in online shops: scale construction, validation and the influence on the buyers’ intention and decision. Behaviour and Information Technology 22, (2003), 165–174.
14. Lewis, J. R., Utesch, B. S. and Maher, D. E. UMUX- LITE: when there’s no time for the SUS. Proceedings of CHI 2013, ACM, (2013), 2099-2102.
15. Lindgaard, G. and Kirakowski, J. Introduction to the Special Issue: The tricky landscape of developing
rating scales in HCI. Interacting with Computers 25 (4), (2013), 271-277.
16. MacKenzie, S. B., Podsakoff, P. M., and Podsakoff, N.
P. Construct measurement and validation procedures in MIS and behavioral research: Integrating new and existing techniques. MIS quarterly, 35(2), (2011), 293- 334.
17. Sauro, J. and Lewis, J. When designing usability questionnaires, does it hurt to be positive? Proceedings of the CHI 2011, ACM, (2011), 2215-2223.