Experience with entrepreneurship learning using serious games

The teaching of entrepreneurship is currently a fundamental pillar in the construction of social responsibility with strong impact on the future economic development of society. The use of serious games in the entrepreneurship field is a way of encouraging students' motivation to become entrepreneurs and develop their skills in the field. In this paper, we describe the experience of using a serious game in the entrepreneurship field in the context of the classroom. For that, we adopt a quantitative research technique based on a survey research to measure the different characteristics experienced by higher education students when using serious games in the classroom to learn entrepreneurship. The students have multidisciplinary competences, coming from courses with strong emphasis in the area of management and technology. The use of the serious game allowed students to develop skills mainly in terms of innovation, leadership, strategic thinking, problem solving, business launch and risk management. On the other side, there were no significant benefits in terms of accounting skills, compliance with regulations, interpersonal, communication, and self-control and discipline.


Introduction
The technological advances that have occurred in recent years have allowed schools to be better equipped for this new digital era. Likewise, the production and, mainly, the consumption of digital games have expanded, reaching a very high popularity level, and are increasingly present in our daily life. The gamification concept, as a strategy of teaching in higher education, is based on the idea that learning through play favors students' motivation and autonomy. The opportunities created by gamification can originate rich and motivated experiences and, therefore, results in a learning process more successful, since students are more involved in the action.
Gamification is gaining significant importance in environments such as business, marketing, or corporate management. However, its application in education is still an emerging trend (Dicheva, Dichev, Agre & Angelova, 2015). Within the education domain we can realize that the majority of approaches are dedicated to the children instruction, while its adoption in higher education is still quite low (Hauge, Bellotti, Berta, Carvalho, de Gloria, Lavagnino, Nadolski & Ott, 2013;Ferreira, Gouin-Vallerand & Hotte, 2016). Some authors such as Yunyongying (2014) and Catalano, Luccini and Mortara (2014) advocate that the integration of gamification tools in the classroom needs an adaptation of curricular design to make it effective and maximize the learning impact. In any circumstance, the current impact of gamification on the educational system cannot be underestimated. Boyle, Connoly and Hainey (2011) confirm that gamification can help in learning, skill acquisition and behavior change. Poondej and Lerdpornkulrat (2016) provide empirical support that students clearly consider that gamified learning activities help them to engage in learning. In the same direction, Buckley and Doyle (2016) state that gamified learning interventions may increase student engagement and enhance learning. They also state that the students' participation in the gamified learning process is dependent on whether the student is motivated intrinsically or extrinsically. Noemi and Máximo (2014) show various cases of successful serious games and study their influence in the learning process. Wilson, Calongne and Henderson (2015) present a gamification case study and introduce two design models that employ game mechanics and incentives to encourage positive outcomes. Cain and Piascik (2015) adopts serious games in pharmacy education; Pourabdollahian, Taisch and Kerga (2012) uses serious games in manufacturing education; and Le Compte, Elizondo and Watson (2015) proposes the use of serious games for cyber security.
Serious Games (SGs) are targeted at learning and education, despite the challenges in terms of game design and their adoption in different learning, academic and interdisciplinary contexts ( A l m e i d a , B o l a e r t , D o w d a l l , L o u r e n ç o & M i l c z a r s k i , 2 0 1 5 ) . B r e u e r a n d B e n t e ( 2 0 1 0 ) p r e s e n t a classification regarding the topology of serious games and design a set of strategies to allow the integration of commercial off-the-shelf entertainment games in embedded learning settings. However, the inclusion of serious games in a traditional classroom environment presents some risks and challenges. McDaniel and Telep (2009) propose a set of best practices for integrating game-based learning into online teaching. Among them, we highlight the use of collaborative technologies and virtual worlds, embracing interdisciplinary, and asking students to be producers instead of just consumers. Arnab, Berta, Earp, de Freitas, Popescu, Romero, Stanescu and Usart (2012) propose specific implementation strategies, collaboration and game building techniques that could facilitate the use of serious games in formal education systems.
SGs can be used as simulations that intend to provide real and immersive games which offer an enriched educational experience. They can be used in several areas, such as military, health, training or education. Botte, Matera and Sponsiello (2009) propose a taxonomy that intends to make a systematic schematization and classification of the different spheres and aims of the simulations. Barjis, Sharda, Lee, Gupta, Bouzdine-Chameeva and Verbraeck (2012) describe the experience of using simulation and virtual environments, such as Second Life and Innov8, to provide a more innovative teaching service. The results of this study demonstrated that this approach offers positive implications for the quality of education in the field of management science and systems engineering. Ahmed and Sutton (2017) state that simulations have the power to push the participant into experiencing an immersive environment.
Entrepreneurship education is a relatively new area in all levels of education, including higher education. However, in recent years, we have assisted to an exponential increase of the most diverse isolated courses and curricular units within higher education institutions. Moustaghfir and Sirca (2010) perform a scientific review in entrepreneurship and refer the high importance of entrepreneurialism in universities and the accommodation of it in interdisciplinary learning modes. Another scientific review was made by Neck and Greene (2011) that explores how entrepreneurship is taught in classrooms and proposes a new approach to teach it as a method. Moses and Mosunmola (2014) examine the relationship between entrepreneurship curriculum and pedagogical challenges in promoting student motivation. Esmi, Marzoughi & Torkzadeh (2015) discuss the use of teaching-learning methods in the entrepreneurship curriculum. The study proposes a set of effective methods of teaching entrepreneurship and presents its validation. Nevertheless, the teaching of entrepreneurship does not necessarily have to be integrated into university curriculum. As referred by Pittaway, Gazzard, Shore and Williamson (2015), student clubs that intend to enhance entrepreneurial learning can be found in many universities. La Guardia, Gentile, Dal Grande, Ottaviano and Allegra (2013) developed a blended model based on the use of a serious game within the framework of the PNPV * EU-funded project. The proposed model proves that it is possible to build a learning space that fosters a learner's entrepreneurial mindset through experiential learning. A similar experience was carried out by Bellotti, Berta, de Gloria, Lavagnino, Antonaci, Dagnino, Ott, Romero, Usart and Mayer (2014) that discusses the adoption of SGs for supporting the development of an entrepreneurial mindset in the context of eSG † project using studies from three European countries (i.e., Italy, Spain and Netherlands). Therefore, it is possible to conclude that the use of entrepreneurship SGs can also have an impact on the entrepreneurial intent and the mindset of the students. Katharina (2015) also confirms it using an exploratory study based on 73 items that provided evidence that the use of a SG in a Digital Game-Based Learning (DGBL) environment influences the entrepreneurship behavior and intentions. Somewhat surprisingly, at first glance, the study conducted by Newbery, Lean and Moizer (2016) concluded that the use of serious games by undergraduate students has a significant negative impact on entrepreneurial intent, which is mainly due to the experiences faced by students while playing the game that showed them the difficulties and uncertainties that an entrepreneur feels when he intends to start and develop a new business.

Method
Quantitative research techniques were adopted in order to study the performance of Oracle relational databases. Quantitative research method is characterized as a systematic approach of investigation during which numerical data are observed, collected, transformed and analyzed by the researcher (Creswell, 2013;Martin & Bridgmon, 2012). This approach tries to find evidence that could support or contradict an idea or hypothesis and, therefore, confirm or validate relationships to develop generalizations that contribute to theory (Williams, 2007). According to Sukamolson (2010) there are four types of quantitative research techniques; (i) survey research, (ii) correlational research, (iii) experimental research and (iv) causal-comparative research. In our work we adopted the survey research and we used scientific sampling and questionnaire design to measure the different characteristics experienced by higher education students when using serious games in the classroom to learn entrepreneurship. Additionally, the adoption of structured questionnaires are efficient tools that easier data analysis and, at the same, maintaining the anonymity of respondents.

Sample
Our sample consists of students who attended the curricular unit of entrepreneurship during three consecutive school years from 2014 to 2017. The students attending this curricular unit come from four undergraduate courses; (i) Computer Science Engineering (EI), (ii) Computer Science for Management (IG), (iii) Management (GE) and (iv) Engineering and Industrial Management (EGI).
The questionnaire is composed of 26 questions divided into three sections. The purpose of each section is mentioned in table 1. In all sections of the questionnaire, we use a multiple choice grid with the following scale; 1. not important, 2. slightly important, 3. important, 4. very important and 5. crucial. This approach guarantees that respondents can easily and quickly answer the questionnaire and they are restricted to a finite set of responses. Additionally, this approach allows the inclusion of more variables in the study, because the format enables the respondents to answer more questions at the same time required to answer few open-ended questions.

Platform
ENTRExplorer * game was developed in the context of a European project coordinated by the Economic Policies Research Unit from the University of Minho (Portugal). The game is a cloud-based application where the users can simulate the process of creating a new start-up company and the daily basis of a new business. The gameplay is performed in different scenarios that are mapped by different floors or a building. On each floor/level the player has access to the most relevant knowledge about the phases of creating a new business. The information provided in the game guide the player trough the different phases in the process of writing a business plan, he can learn and apply the lessons learned, being able to complete its own business plan, a tool provided in the game (Almeida, 2017).
The game is composed of eight levels with the following goals: o Level 1: Market and ideas -identification and classification the economic activities by sector (primary sector, secondary sector and tertiary sector); o Level 2: Strategic positioning -understand and adoption of strategic positioning, attending to the customer's expectation, competitors' position and the benefits of the new product/service; o Level 3: Product strategy -classification of products according to their scope and levels. In addition, understand the decisions that need to be taken related to the product mix; o Level 4: Price strategy -understand the importance of the price in a product/service. In fact, price is the only element of marketing that can generate revenue. It is also one of the most flexible elements that can be changed quickly adapting itself according to the needs; o Level 5: Distribution strategy -perceive the types of distribution channels available (e.g., intensive, selective and exclusive) and the functions of the distribution channels; o Level 6: Communication strategy -explain how the communication process works and how we can formulate a communication strategy attending to the audience, message and objectives; o Level 7: SWOT analysis -present the SWOT technique, which is an important tool used for strategic management. It consists of collection important data elements from the internal environment (strengths and weaknesses) and external (opportunities and threats) of the company; o Level 8: Financial viability -present the main financial maps, such as balance sheet and income statement. Besides that, it presents key financial analysis indicators (e.g., net present value, internal rate of return and payback) to realize the viability of a new business.
Finally, the player is able to experiment a simulation on the virtual market, which is linked to the previous learning levels, so a good score and performance throughout the learning levels, gives more chance of being successful. A typical badly prepared entrepreneur will have a lower probability of being successful, in the contrary, a well prepared and informed entrepreneur will have a better chance of being successful in the business. Besides that, the virtual market is multiplayer and allows several online players to interact between them, which will turn this experience more immersive and enriched for the players.

Data Collection Tools and Data Analysis
The questionnaire was created using the Google Drive and the responses were recorded in a spreadsheet. It was available for the students between 10th of April 2017 and 30th of April 2017. For the analysis of the data the Stata software v13.0 was used. Stata is a complete, integrated statistical software package that provides a high level of statistical methods for descriptive and inferential statistics. In order to have an idea of the data distribution, we used elementary techniques of descriptive statistics, such as the calculation of the mean, standard deviation, minimum and maximum values. Besides that, a hypothesis test was conducted to determine whether there is enough evidence that the impact of serious games in students' learning is different for each course. For that, we compute the p-value, which is the probability that a test statistic at least as significant as the one observed would be obtained assuming that the null hypothesis were true. The smaller the p-value, the stronger the evidence against the null hypothesis. Finally, a α value equal to 0,05 was used.

Results
We obtained a total of 83 valid answers, respectively: 30 answers from students of the Management course (GE), 26 from Computer Science Engineering (EI), 14 from Engineering and Industrial Management (EGI) and 13 from Computer Science for Management (IG). Around 67% of our respondents come from the courses of GE and EI. Looking to the technical competency dimension (Table 2), the students consider the most important benefits of the adoption of ENTRExplorer in the classroom is in terms of problem solving, followed by the environment monitoring. The amplitude is highest for the organizational skills and lowest for problem solving, where all students consider that the impact in terms of problem solving is at least equal to 3 (important). The standard deviation is highest for the technology implementation. Considering the business management dimension (Table 3), the students state that there are clearly two most important variables; (i) business launch and (ii) decision making. The standard deviation has its highest value for "compliance with regulation skills" and the lowest for "business launch". The amplitude is lower for "business launch" and "growth management" variables. The dimension "personal entrepreneurship" is composed of eight questions (Table 4). The students state that the top three most important benefits of serious games in this dimension are in terms of innovation, leadership and strategic thinking. The standard deviation is similar across all variables, but it is possible to realize that the "innovation" variable has the lowest value. Furthermore, the amplitude is lower for the "leadership" and "network building" variables. After that, we analyze in more detail the behavior of the most important benefits considered by students. We start by drawing the boxplot for all variables that have a mean higher than 4 (very important) like it is shown in Figure 3. Three variables "innovation", "strategic thinking" and "risk management" present outliers. A curious situation happens to the "risk management" variable where quartile 1, median and quartile 3 are equal.

Figure 3. Representation of most relevant benefits in a boxplot
Additionally, we consider a hypothesis test (one-tailed t-test, α = 0,05) for the most important group of variables. We intend to test if the benefits brought by the introduction of serious games in the area of entrepreneurship are higher for some of the courses attended by students. Looking to table 5 it is possible to conclude that students from the courses of GE, EI and IG don't consider that the benefits brought by the introduction of the game in the classroom are above the average. In fact, the students from IG course consider that the benefits are inferior to the average in all variables considered. Statistical significance can only be found in the students from the EGI course in the following variables; (i) problem solving; and (ii) risk management.

Discussion
The impact of the adoption of an entrepreneurship game in the classroom context is not uniform for all analysis dimensions. Looking to the average of all questions involved in each section, it is possible to realize that the impact is higher for the personal entrepreneurial. This finding is aligned with the current literature in the field, where Kuratko (2016) states that fundamental entrepreneurial characteristics are inner control, planning and goal setting, risk taking and problem-solver. On the other side, serious games in the field of entrepreneurship don't appear to have so much impact on the development of technical competencies and business management knowledge. Exceptions to this conclusion are in terms of problem solving, decision making and business launch features.
The most important benefits of the adoption of serious games are; (i) innovation, (ii) leadership, (iii) strategic thinking, (iv) problem solving, (v) business launch and (vi) risk management. These elements are considered as having very important benefits by our students' sample. The use of serious games makes it possible for students to access an innovative and different environment that they typically don't find in a traditional classroom, where the acquisition of knowledge follows a more expository approach. Leadership is another competence that can be developed through the use of serious games, as demonstrated in the study developed by Buzady (2017) where the concepts of flow, leadership and serious games are analyzed together. Strategic thinking and problem solving are two concepts that are interconnected and in which the real experience of playing a game helps to be developed. The students also mentioned that the use of the ENTRExplorer helped them to have a more precise notion of the difficulties that they may feel in the market, particularly when they played the business simulation level. Apparently, this situation could result in an inhibition, since the students felt difficulties in reaching a high score in the game. However, students feel that having a real sense of these difficulties before launching their own business is clearly preferable to embarking on an entrepreneurship project in which the unknown is seen as a threat.
On the other side, the least important elements can be divided into two groups; (i) associated with the characteristics of a serious game, (ii) associated with the features of the ENTRExplorer. In the first group we may find the following elements; (i) communication, (ii) self-control and discipline and (iii) interpersonal. The students considered that the use of the game does not promote debate and oral communication, which decreases their perception about the importance of communication. In order to mitigate this situation, it was proposed the organization in groups of 2 students. Likewise, some difficulties were also experienced in the articulation of the syllabus contents of the curricular unit with the themes addressed in the game. These difficulties were different according to the students' background. As a consequence, students felt more difficulties to get well prepared for a written evaluation exam. This discomfort was lower when the knowledge obtained by playing the game is applied in the development of their own project of entrepreneurship. In the second group appear the following elements; (i) accounting and (ii) compliance with regulation skills. The area of accounting is little discussed in the game, because it is essentially seen as an activity that appears in the daily management of a business. Knowledge of legislation is also another aspect that is not addressed in detail in the game, since fundamentally the norms, rules and regulations are different according to each country. In addition, the fact that regulatory standards change frequently makes it difficult to include them in the game.
Finally, the benefits faced by the students are not totally uniform for all courses. The hypothesis test formulated reveals that students of EGI course feel that the use of ENTRExplorer helped them in the acquisition of more competencies in terms of problem solving and risk management. A fundamental competency of an EGI student is identifying and analyzing problems in professional practice and developing effective solutions to solve them, in order to optimize and continually improve the various systems in the engineering field. This is precisely one of the most important benefits brought by the use of serious games in the entrepreneurship field, so that its adoption allows these students to improve this competence. One consequence of this situation is that EGI students, through practice feel more receptive to taking risks, as evidenced in this study.

Conclusion
Fun and entertainment are not the only benefits of online games. There is a new class of games, called serious games, that arise with specific purpose and contents that allows to present new situations, discuss solutions, built knowledge and practice particular activities. Serious games can be applied to the area of creating new businesses in the context of a curricular unit of entrepreneurship, which will enable students to experience the typical decision and the difficulties faced by future entrepreneurs in the business market. Through this study it was possible to identify the most important and the least relevant benefits of an entrepreneurship serious game. The most important benefits, ordered by its priority are; (i) innovation, (ii) leadership, (iii) strategic thinking, (iv) problem solving, (v) business launch, and (vi) risk management. On the other side, the least relevant elements are; (i) communication, (ii) self-control and discipline, (iii) interpersonal, (iv) accounting and (v) compliance with regulation skills.
Finally, it should be noted that the importance given to each of these benefits is slightly changed according to the background of students. The students coming from the EGI course appear to have more ability and motivation to develop their competencies in terms of problem solving and risk management through the use of the ENTRExplorer game.

Recommendations
The adoption of serious games in the teaching of entrepreneurship must be accompanied by its respective curricular framework. This implies that since the beginning there should be a concern to adjust the serious game to the content taught in the curricular unit. Furthermore, since entrepreneurship is a multidisciplinary topic, it is important to ensure that all students have a minimum set of competencies and knowledge to follow the evolution of the game, to avoid eventual phenomena of demotivation. The benefits brought by the introduction of the serious game may be slightly different according to the students' background.
As future work it would be interesting to study if the introduction of an entrepreneurship serious game will have an impact on the academic success of students in the curricular unit of entrepreneurship. Finally, it would also be relevant to analyze its impact on the appearance of new university spin-offs, which can be founded by its students.