While the user can still control the character quite well, the benchmark program will increase the visual complexity of the display

(1)

BENCHMARKING COGNITIVE ABILITIES OF THE BRAIN WITH THE EVENT OF LOSING THE CHARACTER IN

COMPUTER GAMES

NORBERT BÁTFAI, DÁVID PAPP, RENÁTÓ BESENCZI, GERGŐ BOGACSOVICS, DÁVID VERES

Abstract. Most computer game players have experienced the sensation of temporarily losing their character in a given gameplay situation when they cannot control the character, simply because they temporarily cannot see it. The main reasons for this sensation may be due to the interplay of the following factors: (1) the visual complexity of the game is unexpectedly increased compared with the previous time period as more and more game objects and effects are rendered on the display; (2) and/or the game is lagging; (3) and finally, it is also possible that the players don’t have sufficient experience with controlling the character. This paper focuses on the first reason. We have developed a benchmark program which allows its user to experience the sensation of losing character. While the user can still control the character quite well, the benchmark program will increase the visual complexity of the display. Conversely, if the user loses the character then the program will decrease the complexity until the user finds the character again, and so on. The complexity is measured based on the number of changed pixels between two consecutive display images. Our measurements show that the average of bit per second values of losing and finding pairs describes the user well. The final goal of this research is to further develop our benchmark to a standard psychological test.

Received by the editors: December 22, 2018.

2010Mathematics Subject Classification. 91E10.

1998 CR Categories and Descriptors. H.5.2 [INFORMATION INTERFACES AND PRESENTATION]: User Interfaces – Benchmarking; H.1.2 [INFORMATION INTERFACES AND PRESENTATION]: User/Machine Systems – Software psychology.

Key words and phrases. esport, talent search, benchmark program, visual complexity, losing character, psychology test.

This work was supported by the construction EFOP-3.6.3-VEKOP-16-2017-00002. The project was supported by the European Union, co-financed by the European Social Fund.

The work/publication is supported by the GINOP-2.3.2-15-2016-00062 project.

15

(2)

1. Introduction

Losing the control of the character in a given gameplay situation is a very common sensation that is well known among gamers. In this situation, players cannot control their character, simply because they temporarily cannot see it due to one or more of the following reasons: the visual complexity of the display is unexpectedly increased, and/or the game is lagging and, finally, the player simply isn’t experience enough to control the character. In this paper, we introduce our benchmark computer program called BrainB Test Series 6 that can abstract this sensation of losing sight of a character. In this test, game objects are symbolized by boxes as shown in Fig. 1. All box movement is determined by random walks. There is a distinguished box labeled by the name Samu Entropy. It represents the character controlled by the player.

The benchmark test lasts for 10 minutes. During the test, the user must continuously hold and drag the mouse button on the center of Samu Entropy.

If the user succeeds in this task then the benchmark program will increase the visual complexity of the display. It will draw more and more overlapping boxes which will move faster and faster. Otherwise, if the mouse pointer cannot follow the center of Samu Entropy then the visual complexity will be decreased. The test will delete more and more boxes and the remaining boxes move slower and slower until the user finds Samu Entropy again, i.e., clicks on Samu Entropy.

The BrainB Series 1 to 4 were developed in the family setting of the first author¹. Then, in our university environment, we did a preliminary study [3]

on the next version (BrainB Series 5). Some of its measurements were streamed live on Twitch². The main research goal of this study is to show that players lose the character on a higher complexity level of the display and they find it on a relatively lower complexity level.

1.1. Psychological Background. The cognitive ability of attention is a sig- nificant factor in everyday life. The research of vigilance is an important topic in Psychology from 1970 to the present day. The first method used to measure vigilance was the Mackworth Clock [9]. Another method is the Toulouse-Piéron test [19], in which participants have to follow a given scheme to separate right and wrong signs. The Yerkes-Dodson law [21] says that for achieving the best performance there is an optimal arousal level, which level is higher in simpler tasks, and lower in complex activities. We must not forget that as in some other things, in the attentional system there are also personal differences that should be taken into consideration while researching the subject [6]. Witkin et al. did research on perception [20], and from this work, they created a theory

1For example, seehttps://www.twitch.tv/videos/139186614

2For example, seehttps://www.twitch.tv/videos/206478952

(3)

Figure 1. A screenshot of BrainB Test Series 6 in action. The greater the visual complexity of the screen, the greater the prob- ability of losing the character.

about two different cognitive styles. Sagan wanted to calculate the information processing speed of the brain, to do so, he based his calculation on the example of looking at the moon, and from this example he drew the consequence, that the brain can process about 5000 bit/sec at its peak performance [17]. In a modern project, called Building 8, the main thought is to make the brain into a computer [11]. Based on this project, the information processing speed of the brain is about a terabyte/sec, which far exceeds the speed estimated by Sagan. These type of tests and experiments are common tools in the science of psychology [16], [15], [14]. Repeatedly performing the same experiment or test with the same participants could affect the results. Previously, as we specified, repeatedly using the same method could cause the lowering of its validity, and the results could be distorted. Participants can learn and adapt to certain methods, even if it just means a small percentage of difference. The current test takes 10 minutes to complete, in this 10 minutes the participant’s full attention and concentration is needed. We should keep in mind, that the negative effects of fatigue could balance the positive effects of practice, in a direct way through the use of repeated examinations. It is therefore proposed to perform our benchmark test in a competitive environment trying to beat friends, family members, colleagues or ourselves.

(4)

1.2. Informatics Background. Since computer games have a relatively short history and their effects on cognitive skills have only recently started to be re- searched, there are plenty of questions to be answered. In [8], the authors reported an increase in executive functions in school students after playing computer games. Moisala et al. in [10] shows that enhancements in speed and performance accuracy of working memory tasks is related to daily gaming activity. In [5], authors present an analysis of the impact of action video games on cognitive skills. Using computer games to measure cognitive abilities has a short history, but a promising future. Most research try to measure the pres- ence or severity of a certain cognitive disease such as dementia or Alzheimer’s disease. In [1], authors show how a long-term use of video games can reduce the costs of multitasking in older adults. Geyer et al. in [7] show that the change of the score of an online game is in connection with the age-related changes in working memory. Seldom can we find applications that have been developed for the measurement of cognitive abilities. One such application is reported in [13] and [12]; it is a framework that has been developed to measure cognitive abilities and its change in the elderly with computer games. This framework is able to log and analyze scores achieved in various online computer games.

From the viewpoint of information theory and HCI (Human-Computer Inter- action), the Hick’s law [18] could be an interesting aspect. This law states that the response time of the brain increases with the logarithm of the size of the input. For our purposes, it seems to present an interesting question: how can we apply the Hick’s law (or other information theory figure) in our benchmark software?

1.3. Losing The Character. We have experienced the sensation of losing the character during playing several games³. Now we share our thoughts about the phenomenon of “losing the character” and give some examples to illustrate it from the game called League of Legends. As we head into the mid and late game, teams start fights more often with more people, even with all of them.

This is what we call “teamfights”. These are harder to handle, because a lot of things can appear on our screen at the same time including: the champions who participate in the fight, optionally minions or jungle monsters, and the visual effects of the spells, summoner spells, and the active or passive abilities of the items. Besides those effects, we see a lot of other things and we still have to make sure that we fulfill our ingame role properly: position well, attack the

3For example League of Legends, https://na.leagueoflegends.com, Clash of Clans, http://supercell.com/en/games/clashofclans/, Clash Royale, http://supercell.com/

en/games/clashroyale/, Heroes of the Storm, https://heroesofthestorm.com, Dota 2, https://www.dota2.com, World of Warcraft, https://worldofwarcraft.com or Cabal, http://cabal.playthisgame.com.

(5)

proper target, or defend our teammates. We have to handle a lot of information at a blink of an eye, so it is completely natural, that sometimes we do not know where to look or what to do. Consequently, we can lose our own character, which can end with our death, or: we can lose the target character, and it can survive; or we can lose the character that we wanted to protect, thus an important member of the team can die. An example ingame footage can be viewed athttps://youtu.be/wdy3KUm1454, starting at 2:12.

2. Brain Benchmarking Series 6

BrainB is a Qt C++ desktop application that uses the OpenCV library. It is developed as an open source project that is available on GitHub [2]. Its source can be built easily on GNU/Linux systems. But the latest (6.0.3) Windows binary release can also be downloaded as a ZIP file from http://smartcity.

inf.unideb.hu/~norbi/BrainBSeries6/. The code snippet shown in Listing 1 is the heart of our benchmark program. It is a simplified version of the original source code that can be found in the GitHub repository⁴. This code is executed at every 100 milliseconds that is ten times per second. First, as shown in Line 1, it computes the distance between the mouse pointer and the center of the box of Samu Entropy and the result is stored in the variable called dist that holds the square of the Euclidean distance. If the distance is larger than121pixels (11 is the square root of121) and if it reoccurs 12consecutive times or more in a row (that means at least a time interval of 1.2 seconds) and it is also true that the player was controlling the character well in the previous time slices (that is in Line 6 thestateis equal tof ound) then we say that the user has lost the character Samu Entropy and the visual complexity of the display will be saved in Line 7. The sequence of these losing values and the symmetrical finding values saved in Line 16 are shown in Fig. 2. The complexity is computed in bits per second (bps) units that is based on the number of changed pixels between two consecutive rectangular environments of the character with a given width and height.

Finally, it should be noticed that the magic-numbers such as 121 or 12 in the code snippet Listing 1 are based on systematic tryouts. Using these values allows players to experience the sensation of losing the character during the 10 minutes of the test. The code snippets Listing 1 and 2 do not include ad- vanced C++ language specific elements so it can be considered as pseudocode.

However using the original source code gives the possibility to investigate the algorithm of BrainB as precise as possible.

4https://github.com/nbatfai/esport-talent-search/blob/master/BrainBWin.

cpp#L65

(6)

Listing 1. The algorithm for administration of losing and finding the character.

1 int d i s t = ( this - > m o u s e _ x - x ) * ( this - > m o u s e _ x - x ) 2 + ( this - > m o u s e _ y - y ) * ( this - > m o u s e _ y - y );

3 if ( d i s t > 121 ) {

4 ++ n o f L o s t ; n o f F o u n d = 0;

5 if ( n o f L o s t > 12 ) {

6 if ( s t a t e == f o u n d && f i r s t L o s t ) {

7 f o u n d 2 l o s t . p u s h _ b a c k ( b r a i n B T h r e a d - > g e t _ b p s ( ) ) ;

8 }

9 f i r s t L o s t = t r u e ;

10 s t a t e = l o s t ; n o f L o s t = 0; b r a i n B T h r e a d - > d e c C o m p ();

11 }

12 } e l s e {

13 ++ n o f F o u n d ; n o f L o s t = 0;

14 if ( n o f F o u n d > 12 ) {

15 if ( s t a t e == l o s t && f i r s t L o s t ) {

16 l o s t 2 f o u n d . p u s h _ b a c k ( b r a i n B T h r e a d - > g e t _ b p s ( ) ) ;

17 }

18 s t a t e = f o u n d ;

19 n o f F o u n d = 0; b r a i n B T h r e a d - > i n c C o m p ();

20 }

21 }

The final result printed by the benchmark after it ends in the form “U R about 5.92902 Kilobytes” is the mean of upper bounds for the bps values of the display measured when the variable state changes from found to lost (in Listing 1 from Line 6 to 9) and vice versa, when the variable state changes from lost to found (in Listing 1 from Lines 15 to 18). The simple calculation of this final result is shown in Listing 2.

Listing 2. The calculation of the final result of the benchmark.

1 int m1 = m e a n ( l o s t 2 f o u n d ) , m2 = m e a n ( f o u n d 2 l o s t );

2 d o u b l e res = ( ( ( ( d o u b l e ) m1

3 + ( d o u b l e ) m2 ) / 2 . 0 ) / 8 . 0 ) / 1 0 2 4 . 0 ;

4 t e x t S t r e a m < < " U ␣ R ␣ a b o u t ␣ " < < res < < " ␣ K i l o b y t e s \ n " ;

2.1. First Measurements. As concluded in our former preliminary study [3], one of the further developments of Series 5 is changing to full screen from fixed- size window. This modification affects the basic operation of the benchmark, so the first objective was to verify that whether the sensation of losing the character still appears correctly or not. On Windows systems there were no problems. One of the first experiments using default settings on Windows 10 can be seen in Fig. 2 and Fig. 1.

(7)

●

●●

●

0 10 20 30 40 50 60

20000400006000080000

Index

bps

FL F

L

F L

FL

F L

F

Figure 2. The bps values associated to events of losing and finding. The first element of this sequence is the first element of the lost2found (shown in Listing 1 Line 6) sequence. The second element is the first element of the found2lost, and so on. It should be noticed that the losing (labeled by L) and finding (F) events are mixed, see, for example the 13th event on the x axis where the complexity of finding is greater than the complexity of losing in this individual measurement.

2.2. Systematic Measurements with Series 6. The BrainB Series 6 was measured in two groups: UDPROG and DEAC-Hackers. The first one is a Facebook community of the BSc course of “High Level Programming Lan- guages” at the University of Debrecen. The second one is an esport department of the University of Debrecen’s Athletic Club. Participation in the BrainB Se- ries 6 survey was voluntary in both groups. In the UDPROG community 33 members send back their results including the PNG screenshot and the pro- duced text file within 2 days from the date of announcement (20 August 2018).

(8)

The averaged losing and finding curve for all members is shown in Fig. 3. It should be noticed that x-axis is not the time when the losing or finding events were occurred but only the order of events. The index denotes the temporal order of events that occurred. This simple averaged curve has shown that the consecutive losing and finding events has been precisely separated. We believe this means that our notion of losing and finding is strong enough to capture the investigated phenomenon “losing the character”. In addition, the easily understandable averaged curve allows us to avoid application of mathematical statistics hypothesis testing in this case.

All anonymized raw data measured in this experiment can be found athttp:

//smartcity.inf.unideb.hu/~norbi/BrainBSeries6/measurements (in the DEAC-Hackers community 12 esport players send back their results).

3. Conclusion

Our research hypothesis was that the mean of the complexity of changing lost to found is less than the mean of the changing found to lost. Fig. 3 shows the fulfillment of this hypothesis. It seems very clear in these figures that the averaged losing and finding curve has precisely separated the losing and finding events. Intuitively, this result shows that we lose the character on a higher complexity level then we find it on a relatively lower level again.

This simple hypothesis has been proved by the results of this study. In order to further strengthen the completion of our benchmark test in a competitive way in the following versions we are going to offer to test subjects a little more liberty of fine-tuning the settings, for example with regard to the fine-tuning of mouse settings and to using custom colors. The next research objective will be to verify the satisfaction of Hick’s law. To achieve this goal it is simple enough to compare the complexity of the finding and losing events with the time differences for each. Unfortunately, the actual version of the BrainB benchmark does not record these timestamps. The BrainB Series 7 will contain this feature. Our long-term research goal is to further develop our benchmark to a standard psychological test that can be used for talent search in esport.

4. Acknowledgment

Thanks to the students of the BSc course titled “High Level Programming Languages” at the University of Debrecen, to the members of the NEMESPOR mailing listshttps://groups.google.com/forum/#!members/nemespor, to the members of the UDPROG Facebook communityhttps://www.facebook.com/

groups/udprogand to the members of the DEAC-Hackers esport department http://deac.hu/szakosztalyok/esportfor their interest and for performing

(9)

●

0 10 20 30 40 50 60 70

20000400006000080000

Index

bps

F L

F

Figure 3. Measurements in the community UDPROG. The arithmetic mean of the final results of UDPROG participants is 4.95345. This figure shows the averaged losing and finding curve for all UDPROG participants where the losing (L) and finding (F) events are also indicated. The anonymized data can be found at http://smartcity.inf.unideb.hu/~norbi/

BrainBSeries6/measurements/UDPROG/.

the BrainB Test Series 6. Special thanks to Roland Paszerbovics, Gergő Ha- jzer and Péter Rozsos for their interest and support and to Nándor Benjámin Bátfai for performing the BrainB Test Series 6 on several occasions. Finally, our thanks go to Dr. Péter Jeszenszky and Mr. Louis Mattia for reading the manuscript and suggesting improvements. A broader version of this paper can be found as Arxiv preprint [4]. Author contributions were the following: N. B.

conceived the idea, developed the benchmark program, collected the data from the UDPROG community and analyzed the measurements. R. B. collected the data from the DEAC-Hackers. All authors wrote the manuscript.

(10)

References

[1] J. A. Anguera, J. Boccanfuso, J. L. Rintoul, O. Al-Hashimi, F. Faraji, J. Janowich, E. Kong, Y. Larraburo, C. Rolle, E. Johnston, and A. Gazzaley. Video game training enhances cognitive control in older adults.Nature, 501(7465):97–101, 2013.

[2] N. Bátfai. esport-talent-search. GitHub repository, 2017. https://github.com/

nbatfai/esport-talent-search(visited: 2018-09-09).

[3] N. Bátfai, G. Bogacsovics, R. Paszerbovics, A. Antal, I. Czevár, V. Kelemen, and R. Besenczi. E-sportolók mérése (Measuring esport athletes).Információs Társadalom, 18(1):146–155, 2018. Original document in Hungarian: http://real.mtak.hu/79216/

1/it_2018_1_10_batfai_et_al.pdf(visited: 2018-09-09).

[4] N. Bátfai, D. Papp, R. Besenczi, G. Bogacsovics, and D. Veres. Benchmarking Cognitive Abilities of the Brain with Computer Games.ArXiv e-prints, 2018.https://arxiv.org/

abs/1809.00172(visited: 2018-09-09).

[5] B. Bediou, D. M. Adams, R. E. Mayer, E. Tipton, C. S. Green, and D. Bavelier. Meta- analysis of action video game impact on perceptual, attentional, and cognitive skills.

Psychological bulletin, 144(1):77–110, 2018.

[6] V. Csépe, M. Győri, and A. Ragó.Általános pszichológia 1. - Észlelés és figyelem. Osiris Kiadó, 2007.

[7] J. Geyer, P. Insel, F. Farzin, D. Sternberg, J. L. Hardy, M. Scanlon, D. Mungas, J. Kramer, R. S. Mackin, and M. W. Weiner. Evidence for age-associated cognitive decline from internet game scores.Alzheimer’s & Dementia: Diagnosis, Assessment &

Disease Monitoring, 1(2):260–267, 2015.

[8] B. D. Homer, J. L. Plass, C. Raffaele, T. M. Ober, and A. Ali. Improving high school students’ executive functions through digital game play.Computers & Education, 117:50–

58, 2018.

[9] N. H. Mackworth. The breakdown of vigilance during prolonged visual search.Quarterly Journal of Experimental Psychology, 1(1):6–21, 1948.

[10] M. Moisala, V. Salmela, L. Hietajärvi, S. Carlson, V. Vuontela, K. Lonka, K. Hakkarainen, K. Salmela-Aro, and K. Alho. Gaming is related to enhanced working memory performance and task-related cortical activity.Brain Research, 1655:204–215, 2017.

[11] R. Nieva. Facebook’s moonshots: making brains type and skin hear, 2017.

https://www.cnet.com/news/facebook-f8-building-8-moonshot-projects- zuckerberg-regina-dugan/(visited: 2018-09-09).

[12] B. Pataki, P. Hanák, and G. Csukly. Computer games for older adults beyond entertainment and training: possible tools for early warnings-concept and proof of concept.

InICT4AgeingWell, pages 285–294, 2015.

[13] B. Pataki, P. Hanák, and G. Csukly. Surpassing entertainment with computer games:

online tools for early warnings of mild cognitive impairment. In M. Helfert, A. Holzinger, M. Ziefle, A. Fred, J. O’Donoghue, and C. Röcker, editors,Information and Communi- cation Technologies for Ageing Well and e-Health, pages 217–237. Springer International Publishing, 2015.

[14] D. E. Powers. Test preparation for the GRE analytical ability measure: differential effects for subgroups of GRE test takers.ETS Research Report Series, 86(2):1–19, 1986.

[15] D. E. Powers and S. S. Swinton. Effects of self-study for coachable test item types.

Journal of Educational Psychology, 76(2):266–278, 1984.

(11)

[16] S. Rózsa, N. O. Nagybányai, and A. Oláh.A pszichológiai mérés alapjai: elmélet, mód- szer és gyakorlati alkalmazás. Bölcsész Konzorcium, 2006.

[17] C. Sagan.Dragons of Eden: speculations on the evolution of human intelligence. Ran- dom House Publishing Group, 2012.

[18] S. C. Seow. Information theoretic models of HCI: a comparison of the Hick-Hyman law and Fitts’ law.Human-Computer Interaction, 20(3):315–352, 2005.

[19] E. Toulouse and H. Piéron. Toulouse-Piéron-Revisado Prueba perceptiva y de aten- ción. TEA, 8 edition, 2013.http://www.web.teaediciones.com/Ejemplos/Extracto_

libro_TP-R.pdf(visited: 2018-09-09).

[20] H. A. Witkin, H. B. Lewis, M. Hertzman, K. Machover, P. Meissner, and S. Bret- nall Wapner.Personality through perception: an experimental and clinical study. Harper, Oxford, England, 1954.

[21] R. M. Yerkes and J. D. Dodson. The relation of strength of stimulus to rapidity of habit-formation.Journal of Comparative Neurology and Psychology, 18:459–482, 1908.

All authors are with the University of Debrecen, P.O. Box 400, H-4002, Debrecen, Hungary.

Email address: {batfai.norbert, besenczi.renato}@inf.unideb.hu