Learning practices in video games
One of the most important characteristics of video games is to always teach the player new mechanics and strategies. Although there are many different ways of achieving this, the most used one is to implement a tutorial. Tutorials let players learn from information transmitted in-game from text, pictures, videos and or sounds. In a more general context, learning from knowledge has been subject to many theories notably Benjamin Bloom’s Taxonomy. However, educational psychologist David Kolb argues experiential learning might be more efficient. In an attempt to compare both method, we put together a game with an optional tutorial to test on players. The results showed that in the case of a relatively simple game, both are valid methods to teach a player. However, more complex games could benefit of having more elaborated tutorial designs involving both experiential learning as well as learning from knowledge.
Chapter 1 –Benjamin Bloom and David Kolb
Kolb’s Learning Cycle
Learning cycles in games
Chapter 2 - Methodology
Chapter 3 - Results
Comparison with external sources
One of the most challenging tasks for a game developer is to make a game that is playable for the widest range of players. To achieve this, designers will most often create a tutorial explaining controls, mechanics and so on. Modern games will employ a wide variety of tutorials ranging from classic manuals to diluted version of it in the form of hints and tips given throughout the gameplay. However, tutorials are not the only way to teach a player how to play. Players also learn through experiencing situations and mechanics.
Learnability is one of the most important elements in the usability of a game and more generally of a computer software. Hence wise it has been subject to a lot of changes based on theories that had a lot of influence on the way we have structured education in the past 50 years. We will attempt to confront two learning taxonomies or cycles, Benjamin Bloom’s Taxonomy as well as David Kolb’s Experiential Learning.
Bloom’s Taxonomy revolves around the learner acquiring knowledge before practicing where Kolb’s Experiential Learning states that the learner should experience situations to draw knowledge from them. This work revolves around an experiment that aims to present players with a game that can be taught both by knowledge, through tutorials as well as by experiencing the game, through game design and more precisely level design.
Chapter 1 will go through both Bloom’s and Kolb’s taxonomies, explaining them as well as opposing them to their respective critics. The focus in this chapter will be to link their work and theories to games and explaining how it can be applied to game design.
Chapter 2 will explain the methodology of this research in more details. It will go through all the details concerning the experiment describing how the research is being conducted, how were the results obtained and how to interpret them.
Finally, chapter 3 will present the results in a complete and readable way. From this chapter, we will be able to draw conclusions in the context of the experiment and may let us draw more general theories about tutorials in games.
Chapter 1 –Benjamin Bloom and David Kolb
Every game is a new experience for the player and comes with a bunch of new concepts, ideas, and even sometimes language. For the player to get good or even master the game, he must learn and train the learned skills. As Jesper Juul says “To play a game is essentially a learning experience where the player acquires the skills needed to overcome the challenges of the game” (J. Juul, 2011) (1). In most cases, the game itself is the main learning factor for the player which makes it an essential element to design.
However, a game should not be entirely about learning, it should also be entertaining for the player. It should always push him to improve his skills by creating steps and challenges to overcome and eventually master the game. Usually, designers try to make a rhythm to balance learning, playing and challenging. That way players have the time to learn, have fun using the acquired skills and mechanics and eventually if they are good at it, show it off in a final challenge. However, the concept of creating steps for a student to learn was not born with game design. It is usually associated with educational psychology, which aims to study human learning from both cognitive and behavioural perspectives. Which brings us to Benjamin Bloom and David Kolb, two educational psychologist who elaborated theories on the steps of learning.
Between 1949 and 1953, a series of conferences designed to improve communication between educators on student skills and examination later led to the publication of “Taxonomy of Educational Objectives: The Classification of Educational Goals” (2) in 1956 by Benjamin Bloom. This book included the first version of Bloom’s cognitive taxonomy. It remains today, 60 years after its original release, a standard reference in terms of educational teaching and skill evaluation.
The Taxonomy is divided in three hierarchical models: Knowledge-based, Emotive-based and Action-based. Although they share some similarities, they are different in the way they are structured. We are mainly going to focus on the Knowledge-based one, also called the Cognitive Taxonomy since it is the most relevant for our case study. This taxonomy is structured in six levels of learning each one requiring the previous to be acquired before progressing to the next one.
The first level consists in recognizing et remembering terms and concepts. This level is one of the most important since it is the first contact the learner has with the concerned subject and will therefore influence almost all the following steps. Moreover, it is often divided in three more detailed steps:
- Knowledge of specifics: Terminology and specific facts.
- Knowledge of ways of dealing with specifics: Conventions and trends.
- Knowledge of universal and abstract facts: Principles and theories.
Comprehension can be done in several ways, but usually it will consist in organising, comparing and being able to describe in different ways the knowledge. These steps can also be called translation (encoding information in some other form) and interpreting (pre-ordering the knowledge in different ways).
This step is all about using the now acquired knowledge and apply it to solve problems. It makes the learner be used to use his knowledge in different ways and situations.
Analysing works hands in hands with Application, it is all about understanding and identifying the relationship between problems and the answer elements.
Also called “Create” in the revised version of 2001, synthesis consists in organising the knowledge into re-usable abstract patterns not clearly present before.
Finally, this last step involves making judgements about the acquired knowledge. It makes certain elements of knowledge become more important because more used or more understood.
Figure 1 Benjamin Bloom's Taxonomy
According to Benjamin Bloom, these six steps of educational practice are enough to learn and master skills. Although it has been used for a long time, it has also received a lot of criticism on different aspects. One of the most common one is related to its simplicity. Bloom’s Taxonomy’s structure is often said to represent the learning process to simply. “It is assuming that cognitive process is ordered on a single dimension of simple to complex behaviours” – Furst (1994) (3). In other words, the fact that the taxonomy is structured by steps that follow each other is not representative of the learning process. Even though it is divided into 3 sub-hierarchies (Cognitive, Emotive and Active), it still cannot be applied to everyone with a constant success rate.
Another common critic is that certain levels of the taxonomy will either overlap each other or will occur in different orders. Per example, some demands for Knowledge can be more difficult than other for Analysis and/or Evaluation. (Ormell (1974)) (4). This has to do with the fact that the taxonomy is based on a level of difficulty which is not a constant variable in every situation.
Marzano (2006) (5), proposes that the difficulty of a process consists of two elements:
The inherent complexity of a task. (how many steps to do it)
The familiarity of the person with that task. (which can be worked on overtime to improve efficiency).
According to him, because of the taxonomy being based on a difficulty scale, it is never applied the same way and most of the time it would be reduced to fewer steps: Learning, Applying and Evaluating. It is too dependent on both the teacher’s and learner’s ability to respectively teach and learn.
Kolb’s Learning Cycle
David A. Kolb in 1984, published another taxonomy of learning but this time more focused on the learner rather than on the teacher. Kolb’s Experiential Learning Theory (ELT) (6) has been and still is widely used in adult learning. It was notably used in learner-centred pedagogy, especially in management and business but is still relevant to other domains.
Kolb’s Learning cycle involves four stages of learning. Each level can be the entry point for a new learner. Depending on what stage the learner has entered he will be classified as a certain type of learner.
This first step consists in the learner experiencing a new situation or a new instance of an already known situation. For experiential learning to happen the experience needs to bring new elements unknown to the learner.
The learner then reflects on what he experienced. Similarly, to the Comprehending step of Bloom’s taxonomy, it consists on organising and comparing the memories.
During abstract conceptualisation the learner makes sense of what happened during the experience, an interprets the events to draw some theories. The theories the learner draws are the core element of the knowledge acquired through experiential learning. This step could be compared to “Application”, “Analysing” and “Synthesis” from Bloom’s learning cycle.
Finally, active experimentation is when the learner puts into practice the knowledge he acquired in the previous steps.
In addition to his experiential learning theory, Kolb also designed a Learning Style Inventory (LSI). It gives a description of different types of learner depending on what step of the ELT the learner started on. It consists of four types of learner.
Diverging (entry point between step 1 and 2)
Look at things in a different perspective. Prefer watching than doing. Prefer to work in groups, open-minded to take criticism.
Assimilating (entry point between step 2 and 3)
Prefer good and clear information, more interested in concepts and abstracts than people.
Converging (entry point between step 3 and 4)
Put their learning into practical issues. Prefer technical tasks an experimenting with new ideas.
Accommodating (entry point between step 4 and 1)
Prefer do things practically. Attracted to new challenges and to intuitive problem-solving.
Figure 2 David Kolb's Learning Cycle
Compared to Bloom’s taxonomy, Kolb’s learning cycle does include different type of learners. Even though it consists of a different learning process, it fixes Bloom’s Taxonomy’s main flaw. Furthermore, it doesn’t rely on difficulty but rather on the action the learner does. However, it is not perfect either and has also received criticism.
One of the most obvious ones is that experiential learning has very apparent limits. Although a lot can be learned from it, the theories the learner draws from his actions are totally up to him and rely entirely on his understanding. (Beard and Wilson (2002)) (7). This can lead to false conclusions and then to learning false theories.
Moreover, Jarvis P. (1987) (8), states that it should consider cultural and inherited experience/conditions of the learner. And that the concept of having steps/Stages does not fit well with everyone’s reality.
Learning cycles in games
In games, Bloom’s taxonomy translates to instruction manuals and informative tutorials. It consists of teaching the player by informing the player about what is going to happen before letting him experience it. Similarly, to Bloom’s Taxonomy, game designers will try to make steps for the player to learn gradually.
Koichi Hayashida explains in an interview at GDC 2012(9), on how Mario games and especially Super Mario 3D Land uses a 4-step process to teach the player every mechanics. “First, you have to learn how to use that gameplay mechanic, and then the stage will offer you a slightly more complicated scenario in which you have to use it. And then the next step is something crazy happens that makes you think about it in a way you weren't expecting. And then you get to demonstrate, finally, what sort of mastery you've gained over it.”. In this case, the first step corresponds to Bloom’s Taxonomy’s first 2 levels, remembering and understanding mechanics in a safe space. Increasing the difficulty then reinforce the skill acquired in the first step by applying them. Rearranging the mechanic in the way the player didn’t expect will make him think about how will he have to use the mechanic in the future (analysing and evaluating). Lastly, the player must use that knowledge to its full potential which is what happens when a student “creates” something. It is difficult to solve problems that require theory that has not been learnt or practiced yet. Here, we can see how Bloom’s taxonomy has influenced not only educational practice but also game design.
In tutorials, the game creates a “safe” environment (that usually does not directly influence the rest of the game) where the player can try out the mechanics. James Paul Gee in (J.P. Gee, 2005) (10) compares tutorials to fish tanks where “we create simplified systems, stressing a few key variables and their interactions, learners who would otherwise be overwhelmed by a complex system get to see some basic relationship at work and take their first steps towards their eventual mastery of the real system.”. It is here important to note, that the fish tank is a closed system, meaning that whatever happens inside, there a no consequences outside it. This gives a degree of freedom to players to try and test. However, it might also be the reason why some players find tutorials annoying, because they are not experiencing a real situation making it feel as if they were wasting time.
Tutorials in general make the player go through all 6 stages of Bloom’s Taxonomy to make him ready to confront future challenges. Most tutorials will start with information that the player has to acknowledge/remember (Step 1 and 2). The player will then try it out in different ways (Step 3 and 4) and the player should then ready for future challenges (Step 5 and 6).
Regarding Kolb’s learning cycle, it can be observed in games when the player uses any previous experience inside video games. Players and especially experienced players usually know how to differentiate bad and good things even in new situations. In these cases, the players experience something new, reflect upon it, and theorise accordingly. Concerning non-experienced players who do not understand these things straight away, they will learn from their failures and may be able to use that experience in future situations.
However, this heavily relies on the game’s design. Every game is not made for being picked up by anyone and mastered in a short amount of time. Designers must think about how they can make their game conventional and following real-life principles while keeping the idea’s originality. Per example, coins in Mario could have been any other objects, but designers chose the coin because it is an element “that anybody would look at and go “I definitely want that!””. (Shigeru Miyamoto) (11). This type of design choices will define how complex a game is.
Experiential learning in games relies entirely on the player’s understanding of the game. Games will use what is acquired in the early gameplay to introduce new elements as the player goes. One important thing to note, however, is that because games like that rely on the player’s interpretation, which is not constant, they might need some sort of security that will help confused players. Often, this “security” consists in giving different ways for the player to reach the end of a level an easy and intuitive one and a harder one. Hence wise, just like there are different Learning Styles, there are different Play Styles.
One of the biggest flaws for Kolb’s Experiential Learning is that it might lead to draw false conclusion, leading to misunderstandings about the subject. This however is fixed by the format that video games are. Players can have an infinite amount of tries with real life consequences (assuming the player has the time and energy for it). In these situations, drawing false conclusion is not that important since even if several tries are not enough, the player might be able to theorise about this experience later. For this reason, video games are more and more used for professional learning. In those cases, game designers act as facilitators who must prepare the space for the learners to use and try out new things. Having virtual space where actions have no real consequences is very important for learners to understand, draw conclusions about what they are doing before experiencing it in real life.
However, a lot of games nowadays are constrained to apply conventions put in place by the most popular one. These conventions can not only be considered as key to success, but more importantly they greatly increase the chances that the audience already experienced similar mechanics facilitating the hands-on experience. Therefore, even though most games still have tutorials they are often neglected because players most often assume that they know how to play the game and will not pay attention to it. For that reason, it is difficult to evaluate the importance of a tutorial over what we could call experiential gaming in real world situations.
Chapter 2 - Methodology
Given that there are different ways of teaching a player how to play, an interesting question comes to mind. Which one teaches more efficiently? We are not going to give a definite answer but instead give an idea of which one might be better in a specific situation. More specifically, it aims to determine the importance of a tutorial in video games, and if it is required to teach the player how to play. The results could eventually give us an insight on good uses of tutorials in games.
To conduct this research, a game was made using Unreal Engine 4 making it impossible for any players to have had experience on it prior to the experiment. It was made to be playable with a controller to facilitate the hands-on experience. The game consists of a limited amount of mechanics so that the players should be able to discover all of them by himself. The game will make use of these mechanics to make the player solve puzzles.
Figure 3 Game’s mechanics
The game has two core mechanics, absorbing elements and shooting the absorbed elements to affect the environment (see Figure 3). There are two elements to use, fire and ice. Naturally, fire melts ice, and ice extinguishes fire. The player will have to use these two core mechanics to reach the end of each level. Additionally, gates that can be open or closed via triggers hidden in the level will block the player’s path. The levels are sorted in order from the easiest to hardest. The first two level introduces movements, aiming, shooting and absorbing elements. Playing them should be sufficient to learn how to play the game. The third level in addition of being a final challenge for the first levels shows a new way of absorbing elements but is optional to use and experience. Instead of using element sources (visible on the left of the player in Figure 3), elements can be absorbed from walls. The last two levels act more as a challenge for the players to confirm that they assimilated these mechanics.
Figure 4 Example of the “non-intrusive” tutorial
The game is playable in two modes. A Tutorial mode, where the player is taught the mechanics before being able to use them. The tutorial is implemented in a totally non-intrusive way (see Figure 4) and does not stop the player from progressing. This mode follows Bloom’s Taxonomy’s way of teaching, giving the player information before him having to experience them. Another mode, the “Experiential learning” one consists in the same levels without the tutorial. It is up to the player to figure out and apply the mechanics before assimilating them. Both modes represent the two type of players we are comparing here. However, we noticed that some players are stuck very earlier on because of the controls. On the one hand, the tutorial gives the controls and players can deduce that there are no other buttons to acknowledge, on the contrary playing without the tutorial make the players think about any possible control scheme making them take longer to complete a level. To adjust that, we decided to add a third category of player to look at. These players should play without a tutorial but are briefed on the controls before starting the play session. That way we can also observe how important the knowledge of the controls is compared to mechanics.
In terms of level design, all the levels have a clear goal that the player can see straight from where he starts. That way he should always know where to go, helping him to solve the puzzles. The two elements (fire and ice) have distinctive colours that player should be able to recognise. They also are indicated by environmental elements that have the corresponding colour. The goal here is to limit every external factor that could influence the gameplay experience in any ways.
In terms of results, the game displays the time taken to accomplish the levels at the end of the five levels (see Figure 5). Additionally, every player’s gameplay will be recorded to confirm the time taken, as well as determining elements that influenced the players’ time. Per example, did they take more time because they did not understand the mechanics, or because the controls were not assimilated correctly? While conducting a play session, we will have no interaction with the players to avoid any flawed results. However, that will let us get additional information concerning the player’s emotion and reaction to certain elements. Lastly, we will ask the players if they feel like they know the game well enough to carry on other levels. Although this is entirely relying on the player himself, it will give us an insight on the level of comprehension that players had at the end of the five levels. Results will be presented in the form of a report and a complementary video showing evidence of the results.
Figure 5 Game’s result page
The research should be made on experienced players only. Because it is based on educational practice, it should be applicable to most people, however, the difference in cultural background is too big to be able to compare both types of population. Keeping it to one type will facilitate the result interpretation and help draw the correct conclusions. Players won’t be told anything about the game prior to the experiment and will be randomly assigned a mode to play to avoid any bias.
We should find that players who play with the tutorials have better times in the learning levels (where the tutorial is here to guide them). Especially when it comes to the first level, and the learning of the control where we should see players without tutorial or brief on the controls taking more time getting themselves familiar with the game. However, we should see results that are more similar when it comes to the last two levels simply because at this point all players would have experienced all the mechanics.
Chapter 3 - Results
After testing the “experiment” game on 18 people, we collected the results in the following figure. We had six different players for each of the modes. Although it is a very small number of players, it will already give several possible answers. The graph below presents the results we got per player and per levels. It also shows the total time taken by a player to finish the five levels. They are sorted by the type of test they played, Bloom, Kolb and Mix who are the players who played without tutorials but with knowledge of the controls.
Figure 6 Experiment's results
The first observation we can make is that all players at the end of the play session felt like they knew the game enough to carry on playing. This reinforce the idea that the game was designed well enough for all players to understand the mechanics regardless of if they played a tutorial or not. However, it is difficult to draw conclusion from the raw data. After a bit of data reorganisation, we ended with the following graphs. It greatly facilitates the readability and the search for answers.
Figure 7 Experiment's results: level averages
Figure 4 shows the time on average per type of player and per level to complete the concerned level. We can see that in level one, two and three Bloom players and Mix players have very similar times compared to Kolb players. It is worth noting that it is in these levels the players learn the most in terms of mechanics. The last tutorial tip Bloom players have is displayed at the beginning of level three.
In level four however, Kolb and especially Mix players both have longer times compared to Bloom players. Level 4 can only be completed by the use of a mechanic that is taught through tutorials only in level 3 (and is optional to use in this context giving Bloom players the advantage). However, we can also notice that the longest time taken for completion was obtained by a Bloom player.
Finally, all players had similar times in the last level proving again that all player regardless of the mode they played learned all the basics of the game and were able to apply it.
Figure 8 Experiment's results: Total time averages
Figure 6 shows the total time on average and per type of player taken to complete all five levels. The average time per player type is written in white inside each box plot. The white bar accurately represents the corresponding value.
We can notice that Bloom players who played with tutorials have on average completed the test faster than any other type of players, with an average of 3mins 7secs. Kolb players (without tutorials) have an average the longest completion time with 3mins 49secs. Finally, Mix players have on average better times than Kolb players, but were not as fast as Kolb ones with 3mins and 35secs.
In terms of upper and lower times limits, Bloom players have the shortest time out of all players with 2mins 22secs. However, none of the mix players had a quicker time than Kolb players’ shortest time. The upper limit follows the same ranking as the average with Bloom players, Mix players and Kolb players having the longest time.
We wanted to see the importance of a tutorial during the first hands on experience of a game. We can confirm our first hypothesis that Bloom and Mix players would understand the mechanics quicker and get through the first three levels in a shorter amount of time. This shows that besides having a full tutorial that explains all mechanics, a simple brief on the controls is enough for players to get familiar with the game without prior experience. In the context of a game as simple as the one used for the test, we can say that a full tutorial is not the most efficient solution to teach the player although the way it was implemented here did not hamper the player’s progress.
In the fourth level, we also noticed that most Bloom players benefited from having a tutorial explaining them a mechanic. It is worth noting here that the mechanic was always available and not hidden away from the player to experience it. However, it does not apply to everyone as the player who obtained the longest time for this level, all player type together, was playing with tutorials. In that regards in a game like this one, tutorials can be more efficient to teach small tricks and tips depending on the player. Nonetheless, tutorials are subject to interpretation in the sense that games cannot force player to read the tutorials instead of skipping them. Some players will not pay attention to tutorials by mistake or purposely which will affect their progress later in the gameplay.
Another observation we can do on level 4, Is that Kolb players have on average better times than Mix players. The explanation for this is that some Kolb players experienced the “hidden” mechanic in level 1, as they were figuring out he control.”. So, they already knew how to react when being confronted with level 4. These “Happy Accident” could easily give experiential learners the advantage over the others since in addition of acquiring the intended skills for the level, they also learned a skill that has not been shown in tutorials yet.
Finally, all players had similar times in the last level confirming that regardless of how the players experienced the game they all learned how to play the game to a certain level. It proves that both playing with tutorials and playing without them are valid ways of learning how to play a game. In other words, learning through experience (Kolb’s Learning Cycle) can be as efficient as learning through knowledge (Bloom’s taxonomy) in terms of learning results. Learning might be longer by experience but gives similar results in the end.
This test could have been improved in several ways. First, we did not make the players aware of what data we were collecting on the gameplay prior the test. On the one side, it did not influence the way of play and let them play “naturally”. On the other hand, some players stopped or took some time to explore the levels rather than completing each level as fast as possible. Luckily, because of short longevity of the test it was a minor problem. Secondly, the amount of people tested was too small, although it was just enough to notice a difference between different types of people. Finally, the longevity of the test was too short, resulting in having an order of magnitude in seconds, which is excessively small to draw concrete conclusions.
Comparison with external sources
An experiment conducted by Erik Andersen (12) and his associates investigated the impact of tutorials in video games. They tested the efficiency of tutorials on learnability in video games of varying complexity. The experiment was tested on over 45 000 players and divided into three games, Refraction and Hello World being described as “casual” games and Foldit being more complex and uncommon.
They found that the efficiency of tutorials depends greatly on the complexity of the game in question. Within the three games, tutorials had the greatest value in the most complex and unconventional game where it increased playtime by as much as 29% on Foldit, meaning that players understood the game better and enjoyed it more. On the other hand, tutorials had “surprisingly little impact” on less complex games like the two others. Those results reinforced that in less complex games, the use of a full tutorial is not always necessary, and a simple presentation of the controls could be sufficient for the players to play and learn only from experience.
These experiments being limited to the games mentioned above, it is difficult to clearly draw a universal rule about tutorials. More research would be required on different genres to theorise about the subject. Another element that would require further work is the tutorial type. Depending on the way the tutorial is presented and on how the player reacts to it, learnability can greatly vary. Intrusive and/or non-skippable tutorials can have negative effect despite the intended effect of making the player focused on learning before experiencing. One last element that can be studied regarding tutorials, is player investment. Tutorial greatly affect the way players understand the game and therefor the quality of their playtime.
After confronting Benjamin Bloom’s taxonomy and David Kolb’s learning cycle, we explained how they are strongly linked to video games. Both theories describe different ways of teaching a student how to learn and a player how to play. We then elaborated an experiment to give us an insight on the benefits of using each of these theories.
In the search for answers, we made experienced players try a game either by following Bloom’s taxonomy by making them learn facts, theorise and then act or Kolb’s Learning Cycle by letting them try, theorise and apply. Although the experiment conducted here was very limited it was successful enough to confirm the hypotheses we had and to give an idea of what type of tutorial is necessary in certain situations.
First, we can say that people learning through knowledge will always be better prepared to understand new situations simply by the fact that they can mentally prepare themselves to what they are going to experience. However, people learning through experience might not be prepared to understand everything but will know how to react quicker when it happens.
Additionally, experiential learning can take longer to reach the same level of skill than a knowledge-based learning. Overall making it a bit less efficient in the context of our experiment unless they have “Happy accidents”. In which case, they can have a massive advantage over other learners.
However, if done correctly, a mix between both Bloom’s Taxonomy and Kolb’s Learning Cycle can be as efficient but require less attention to learn. In the context of our game, tutorials might have been perceived as a negative element (resulting in some players not reading them). Having a non-intrusive briefed on the controls had a massive impact on players’ performance.
The key part of this text is to realise that tutorials are necessary depending on several factors. These factors are mostly about the game itself: complexity, amount of interactions possible, amount of interactions to other players if multiplayer. Types of tutorials are also chosen depending on other factors: what information must be transmitted to the player, how much of it, how complex are the interactions. Game designers must determine if they require a tutorial or not early in a game’s development as it would gain a lot from playtesting.
(7) Beard.M., Wilson.J., 2006. Experiential Learning: A Best Practice Handbook for Educators and Trainers. Kogan Page Publishers.
(2) BLOOM, B. S. (1956). Taxonomy of educational objectives; the classification of educational goals. New York, Longmans, Green.
(3) Furst, E. (1994). “Bloom’s Taxonomy: Philosophical and Educational Issues.” In Anderson,
L. and Sosniak, L. (Eds.) Bloom’s Taxonomy: A Forty-Year Retrospective (pp. 28-40)
(8) Jarvis, P. (1987) Adult Learning in the Social Context, London: Croom Helm.
(1) Juul, J., 2011. Half-real: Video games between real rules and fictional worlds. MIT press, p95.
(6) Kolb, D., 1984. Experiential Learning. Prentice-Hall.
(12) Andersen, E., O'Rourke, E., Liu, Y., Snider, R., Lowdermilk, J., Truong, D., Cooper, S. and Popovic, Z. (2013). The Impact of Tutorials on Games of Varying Complexity. University of Washington.
(10) Gee, J.P., 2005. Learning by design: Good video games as learning machines. E-Learning and Digital Media, p120.
(5) Marzano, Robert & S Kendall, John. (2006). The need for a revision of Bloom’s taxonomy.
(4) Ormell, C.P. (1974-1975). Bloom’s Taxonomy and the Objectives of Education. Educational
Research, 17, 3-18.
(9) Hayashida, K., 2012, The Structure of Fun: Learning from Super Mario Land’s Director, GDC 2012.
(11) Miyamoto, S., 2010, Iwata Asks - Super Mario Bros 25th Anniversary.
Adams, E., 2011, The designer’s notebook: Eight ways to make a bad tutorial.
Halina, M., Steps in conducting a research project or experiment. Retrieved from http://www2.hawaii.edu/~halina/ExpDesWk/ResSteps.pdf (15/12/17).
Shuttleworth, M., 2008, Conducting an Experiment. Retrieved from https://explorable.com/conducting-an-experiment (15/12/17).
Goulet, C., 2003, Interpretation of a social experiment’s results. Retrieved from http://pagesped.cahuntsic.ca/sc_sociales/psy/methosite/consignes/entonnoiretape4.htm (15/12/17).
Figure 1 – Bloom’s Taxonomy – Patricia Armstrong – Bloom’s Taxonomy.
Retrieved from https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/ (15/12/17)
Figure 2 - Kolb’s Learning Cycle - McLeod, S. A. (2013). Kolb - learning styles.
Retrieved from www.simplypsychology.org/learning-kolb.html (15/12/17)
Figure 3 – Game’s mechanics (Screenshot)
Figure 4 – Example of the “non-intrusive” tutorial (Screenshot)
Figure 5 – Game’s result page (Screenshot)
Figure 6 – Experiment’s results
Figure 7 – Experiment’s results: Level averages
Figure 8 – Experiment’s results: Total time averages