Assessment of readiness for learning and academic success on computer assisted learning : A study on computer integrated manufacturing with lathe

With the improvements in technology, computers are being used for assistance in learning and teaching in different fields. Therefore, Computer Assisted Learning is used for our study to teach operating computer numerical controlled lathe machine in computer integrated manufacturing lab. Our education is planned in major phases: traditional teaching and computer assisted learning. Before the education students took a questionnaire with different subscales to determine the level of readiness for learning. And after the education they took a test on operating the lathe machine to measure the success of the education. Then the students were asked to design a project. This education is a comprehensive one and it is not always possible to make all students use the lathe machine since there can be some issues on safety, time management and waste of products. So with this study it is intended to determine the effect of readiness for learning and academic success on the students and give this education accordingly. A dichotomous classification problem is constructed with the subscales of readiness for learning questionnaire and academic success as features and lathe test result as class labels. Consequently, a model that can determine whether a student would be successful in learning to operate the lathe machine with the subscales of the readiness for learning questionnaire (motivation, health – nutrition, learning, planned working, efficient reading, writing, listening, note taking, attending a course, preparation and attending an exam) and GPA is constructed with Support Vector Machines and leaveone-out cross validation.


Introduction
With the increasing improvements in technology, teaching and learning methods are changing.As people become more prone to the technology, the interest to bring technology to classroom also rises [29].Additionally, the availability of computers and related technological equipment such as smartboards and mobile devices increases and Computer Assisted Learning (CAL) becomes an important part of many curricula [16,21].CAL is defined as learning or teaching different subjects through computers with subject wise learning packages/materials [13].Another definition can be a computer program or file developed specifically for educational purposes [28].The impact of using CAL for assisting teaching and learning has been a popular subject during the past four decades.The overall findings of these researches show that CAL in many ways is more effective than traditional instructional practices like lecturebased, textbook -based and/or physical hands on [30,17,12].Students attending a course supported with CAL have more knowledge and are better at developing process skills.
Using CAL for education has the potential to produce higher learning outcomes.CAL provides the students with the opportunities like systematically exploring hypothetical situations, interacting with a simplified version of a process or system, changing the time-scale of events, and practicing tasks and solving problems in a realistic environment without stress compared to the traditional learning environment with textbooks and lectures [34].The saving of time, allowing teacher's to devote more time to the students instead of to the set-up and supervision of experimental equipment; the ease with which experimental variables can be manipulated, allowing for stating and testing hypotheses; and provision of ways to support understanding with varying representations, such as diagrams and graphs are the possible reasons to use CAL in education [8].CAL can support authentic inquiry practices that include formulating questions, hypothesis development, data collection, and theory revision.By actively involving learners in exploring and discovering, CAL can be a powerful learning tool, as learning via doing and seeing is retained longer than learning via listening or reading [24].That is why CAL is used in teaching many areas [23] like language [1,33] medical [25] mobile [32] and mathematics [22] learning.These reasons also make the use of CAL in science and engineering education a must since teaching these disciplines require many experiments.Even though the tendency toward more learner-centered instead of teacher-centered education has caused CAL approach to be popular [36] the extent to which control can be turned over from the teacher to the learner does have its limits.Learners might have difficulties in generating and adapting hypotheses, designing experiments, interpreting data and regulating learning if there is insufficient support for the processes of discovery learning within CAL [20].In other words, if there is a lack of guidance, learning can fail.The effectiveness of CAL is dependent upon the way which it is used [10].CAL is most effective when it is used as a supplement and incorporates high quality lecture based course [20].Accordingly in many teaching and learning scenarios including also ours, CAL can be a supporting way of traditional teaching [37].However, this does not mean that CAL does not have an improving effect of on traditional education.Strictly speaking, traditional education can be enhanced by the application of CAL [37].And for the engineering case, when used appropriately, CAL enables the students to have an inquiry based and genuine study.CAL system also brings brand new aspects to flexibility, safety and efficiency which are very important in engineering.
Thus, an educational methodology including a CAL tool along with instructor guidance for designing simple mechanical parts using Computer Integrated Manufacturing (CIM) with lathe is used in this study.CIM is the application of computer science technology to the enterprise of manufacturing in order to provide the right information to the right place at the right time, which enables the achievement of its process and business goals [4].Computer Aided Design (CAD) and Computer Aided Manufacturing (CAM) parts of CIM are integrated in the CAL tool which not only helps the students to visualize their designs on turning, but also provides flexibility, safety and efficiency.These properties of CAL and CIM are very important for engineering study [11].
Before starting this CAL session for CIM, students took a questionnaire to determine their dimension of readiness for learning.Dimension of readiness concept is very important factor and pre-requisite for learning and teaching processes [9,18,15] and is used in teaching many different subjects [3].The students' being ready for learning can change the learning outcomes totally [6] and make students happy [31].If a student is not ready for learning, this can cause emotional problems like unhappiness, stress and rage.A questionnaire for readiness can include different subscales like the abilities of the students, their motivation, and even their health conditions.Different aspects can be included to measure if the students are ready to learn.Educators continually complain that students are not ready for learning.They show up for school underfed or malnourished, angry or apathetic, stressed, threatened, sleepy and unmotivated.These emotional problems decrease their eager to learn study, read, write, take notes, listen to a lesson and attend a lesson or an exam.Therefore our questionnaire includes 10 subscales namely motivation, health -nutrition, learning, planned working, efficient reading, writing, listening, note taking, attending a course, preparation and attending an exam which are measured via 72 items.In addition to these subscales measured via the questionnaire, grade point averages (GPA) of the students are also taken into consideration in this study since GPA is also an indicator of readiness for learning [26].GPA shows the previous academic success of the students that are taken into the lathe study on CIM with CAL.
In our previous works, undergraduate engineering students were divided into two groups.While the first group's educational environment was a class with computers to learn NC programming, and the second group studied in the CIM laboratory a small manufacturing environment [39,14].The results of these studies showed that using CAL has a positive effect on teaching how to use technical equipment as lathe or milling.Based upon these findings, this study is planned to find out if readiness for learning and academic success influence the lathe study on CIM with CAL. a readiness for learning questionnaire that has separate scores to evaluate the effects of motivation, health -nutrition, learning, planned working, efficient reading, writing, listening, note taking, attending a course, preparation and attending an exam was applied to the participant students before taking the course and their scores on each subscale along with their GPAs are used as features in a binary classification problem.The aim of the classification model is to predict whether a student has the ability to be successful on the final lathe test, which is applied at the end of the whole education phase.In other words, purpose of this study is to determine whether readiness for learning and academic success represented with GPAs affect the undergraduate students' learning process in our educational methodology to operate a lathe.
As classification algorithms, k-Nearest Neighbor (kNN) and Support Vector Machines (SVM) are used and the success of the models in discriminating the successful and unsuccessful students in this education is evaluated according to model accuracy, specificity and sensitivity scores.
This paper is organized as follows: In Section 2, the educational methodology including the lecture based course with CAL is explained.Brief explanation of the data acquisition technique and the data is given in Section 3. Section 4 summarizes the used classification and cross validation methods in general.Section 5 presents the experimental results.The conclusions and discussions are shown in Section 6.

Educational Methodology
Computer Assisted Learning composes an educational environment that assists the students in learning a particular subject by using a computer program or an application.Here, the crucial word is assist which means that the program is not alone in learning the process, there are other methods involved.The word -"assist" -points on the significance of the other methods involved into learning process.In our teaching methodology, CAL is a part of the entire approach of instructional methods, in other words it is not the only way but one of the ways to deliver the content of the course.For this context our educational methodology emphasizes an integrated approach to teaching a subject.Particularly computers or programs in CAL system lend assistance to the overall learning strategy, which is composed other methods of instruction (e.g. the lecture, presentation, learn mates, text etc.).
In this research, participants were selected from undergraduate Industrial and Mechatronics Engineering students of Bahçeşehir University.Before starting the education, the questionnaire for readiness with different subscales was taken by all the students.The education phase includes two sub phases: traditional teaching and CAL.In the first phase, the instructor gives a brief history of the computer numerical control (CNC) lathe machine, the evolution of machine in the industry, the variety of conveniences that the machine accommodate in production, the structural features of CNC lathe machine, the sort of materials that can be engraved, the range of cutting tools that are commonly used, the working principles of the coordinate axis and NC codes are introduced theoretically.
The second phase consists of CAL via a CIM environment.In this CIM environment -which is actually a laboratory -there is an automated storage and retrieval system (ASRS) with racks contains raw materials and final goods.Additionally there are four robots, one to take raw material from and load final good to storage, one for interacting with the milling machine and one for the lathe machine, last one for taking products from conveyor to quality control unit and loading controlled good to conveyor.As mentioned, the laboratory also includes CNC lathe and milling machines.And there is a unit for quality control.In this part of the education phase which takes place in the CIM laboratory there is an opportunity of using CAL for machine operations.All students completed the courses given in these two phases.
In the CAL phase, mainly operating the CNC lathe machine is described.CNC Lathe machine can be commanded with software shown in Figure 2 that was introduced to the students both by the instructor and CAL tool.In a part of the course, the students themselves used the CAL tool to learn how to operate the CNC lathe machine.The CAL tool also includes an interface to control the machine.This interface is composed of various parts.The middle part of the screen displays the 3D working of CNC Lathe Machine.In this area, students can zoom into/out of the machines.Also students can observe the machine movements during its operation.They can select various tools for experimentation and modeling from the tool bars.Different cutting tools that are available in tool holders of the machine are also usable in this CAL tool.The content displayed in this part of the screen depends on the machine and the tools used.The NC Code edit window provides participants to edit and follow up their codes.The below-hand side of the screen houses machine information panel as well as the verification window that enable the learners to see the tool and the machine movements.The right-hand side contains jog control and operator panel.This computer software makes controlling the machine easier.During education the software is integrated to form a part of a multi instructional learning program using the CAL basics.Our study investigates the effect of readiness for learning and academic success on using lathe via CIM.Each part of instruction -traditional or CAL -plays a different role within the strategy of education.The aim of using CAL is to increase the knowledge gain whereas traditional teaching compensates the parts where CAL fails with guidance.On the contrary, this strategy also helps us to find out the parts or objectives of the course where the traditional teaching methods are failing and where the computer or the application software can help.In addition to these, this strategy determines the level at which the CAL tool is useful showing the educational objectives which the computer program alone cannot teach and thus the level of required support from the instructor based method.Thus this strategy results in an educational program shown in Figure 2 within which each instructional method compliments and supports the other to ensure that no part of the course is left unclear.After delivering the course content to the participants using the educational strategy explained above, the participants first took a test on CNC lathe machine and then they were asked to design a project if their scores from the test are adequate since students are more motivated during project based phases of any course [5].Projects included a sample design that would be produced on the CNC Lathe.To design the sample, the students sketched a scaled design of their own samples with paper and pencils for machine production.Subsequently, they used the CAD/CAM programs to computerize their sketches and produce NC.Consequently, they verified their designs in the software to prevent potential machine hazards.If a mistake has been occurred during verification, they would return back to the design stage and followed the same steps until the verification stage accomplished.The product has been carved out after operating the CNC lathe machine.The students who failed in CNC lathe test had many problems in designing sample stage.Deciding whether a student can design a sample is an important issue.To make sure that a student would be successful on this design phase can save time, avoid waste of materials and potential hazards.Additionally, giving this education is both inconvenient and time consuming since it requires two phases.Accordingly, the aim of this study is to determine the effect of readiness and academic success on learning how to operate the lathe machine.Thus if a student is not ready for learning, he/she will not be included in the whole education phase.

Data Acquisition
The data collected in the context of this study belongs to 51 Industrial and Mechatronics Engineering undergraduate students (9 female, 42 male) who take a special course on Computer Integrated Manufacturing (CIM) in a laboratory designed for this purpose in Bahçeşehir University.The mean of students' ages is 20.5 with standard deviation 1.62.All students attend a two phase course as explained above.Before taking the course a questionnaire having different subscales like motivation, health -nutrition, learning, planned working, efficient reading, writing, listening, note taking, attending a course, preparation and attending an exam was applied to each student to determine their level of readiness for the learning phase.A sample question of this questionnaire is shown in Figure 3. Also the GPA of the students was taken into consideration to find out whether general success of a student affects learning or vice versa.Afterwards, students were given a test based on turning (sample question is given in Figure 4), in other words operating the CNC lathe machine, and the students were categorized according their test scores: if a student gets a grade more than 50 over 100, he/she is considered as successful and otherwise unsuccessful.

Classification Methods
The dataset used in this study consists of the motivation, health -nutrition, learning, planned working, efficient reading, writing, listening, note taking, attending a course, preparation and attending an exam subscale scores of readiness for learning questionnaire and GPA of the students.Using these as features, it is tried to be determined whether the readiness and academic success affects the success of CAL on lathe via a binary classification problem.Two different classification algorithms, namely k-Nearest Neighbor (kNN) and Support Vector Machines (SVM) were used.As known kNN is a nonparametric classifier that labels an unknown sample by taking the nearest neighbors into consideration [2].The distance of each sample to unknown sample is calculated to determine the neighbors.As for SVM, it is a very popular parametric classifier which aims to find the optimally placed hyperplanes to discriminate the classes from each other [35].The closest samples to these hyperplanes are called support vectors, and the solution is defined in terms of this subset of samples which limits the complexity of the problem.Because the optimization problem has a unique solution, any iterative optimization procedure is not needed for convergence [2].In order to build a robust model, it is important to use as much of the available dataset for classifier training.However in many applications and also in our case usually the dataset is limited.Though, initially conventional 10-fold cross validation is used here.The dataset is divided into 10-folds, one fold is left for validation and the remaining nine are for training the model [7].The average accuracy of the folds is given as the very last accuracy rate.
With our limited number of samples, the accuracies, that are obtained using 10-fold cross validation with different classification methods, are not satisfactory and varies from fold to fold.Actually n-fold cross validation method has a special case called Leave-One-Out cross validation [7].With this cross validation method not the 1/n of the samples are left for validation but only one [2].In other words, the classification algorithm is run for the number of samples and in each run n-1 samples are used for training and the remaining one sample for validation.Once again the average accuracy of the folds is given as the very last accuracy rate.

Experimental Results
To determine whether these subscale scores (motivation, health -nutrition, learning, planned working, efficient reading, writing, listening, note taking, attending a course, preparation and attending an exam) that we use as features are associated with learning how to operate the lathe machine via our educational methodology, first Pearson Correlation Coefficients (PCC) of these features with the lathe test score is calculated using linear correlation.As seen from Table 1, the most correlated feature is GPA, followed by efficient reading.The correlation rate of these features with GPA is also close to the correlation rates with lathe scores.A linear relationship is demonstrated with PCC.However, sometimes features might have a non-linear relationship with the class information.Thus, mutual information is applied to find out if there is any non-linear relationship between the feature and the learning lathe.As known to apply mutual information, the values must be discrete.For discretization, for each feature, we use its mean l and its standard deviation r as in [27].The feature values between l -r/2 and l + r/2 are converted to 0. The four intervals of size r to the right of l + r/2 are converted to discrete levels from 1 to 4 and the four intervals of size r to the left of l -r/2 are mapped to discrete levels from -1 to -4.Very large positive or negative feature values are truncated and discretized to ±4 appropriately.As seen from Table 2, even though note taking and succeeding in learning lathe has a very low linear correlation (0.1134), it is most non-linearly correlated feature according to mutual information after GPA followed by learning.Even though the mutual information correlation values might seem very low, one should keep in mind that mutual information correlation coefficients do not have an upper boundary like 1 as in PCC.In other words, the important thing in mutual information that counts is not the value of the correlation but the order.The lathe test is applied to the students after defined educational phases explained in our methodology.The performance of all the students on this test according their readiness for learning questionnaire score and GPA is shown graphically in Figure 5.
As seen from the figure apart from some outliers (3 or 4 for each class) a linear discriminant function can easily be obtained to separate two classes from each other.Students that do not pay enough attention to complete the readiness for learning questionnaire might cause the outliers and their effect can be reduced by gathering a larger dataset.
Then, the test scores and GPA are fed into kNN and SVM classifiers with 10-fold cross validation method.For the kNN classifier Euclidean distance metric is used for 1, 3, 5 and 7 neighbors.Shown in Table 3, best classification accuracy is obtained with 1NN which is again a consequence of the small number of samples in our dataset.Even though the result might seem to be close to a random guess with kNN, it is obvious that with a SVM model with linear kernel they are more promising.By interpreting Figure 5, it is seen that a linear model is more suitable for our dataset.In addition to SVMs' being more robust, this is also an important criterion to explain our results with 10-fold cross validation.For SVM LIBSVM package [19] is used and the cost parameter (c) for linear kernel is selected as the default value, ie 1.For the RBF kernel again the defaults values are used, the value of the parameter c is 1 and the kernel width (g) is 1 / number of features which makes 0.091 in our case.To analyze the results of 10-fold cross validation in detail, accuracy of each fold is recorded and displayed in Figure 6 and 7.As is seen, the accuracies obtained by applying SVM with the linear kernel are the most stable ones.For SVM with the radial basis function (RBF) kernel and kNN with different number of neighbors, accuracy varies considerably from fold to fold.This is a consequence of small number of samples in folds and the outliers.On this wise, we decide to apply LOO cross validation to obtain larger training sets and to avoid the effect of outliers.Using kNN with LOO cross validation increases the accuracy values for each neighbor value (k = 1, 3, 5 and 7) since with larger training sets, more relevant neighbors can be selected for the algorithm.The highest accuracy is obtained with three neighbors which is sensible for a dataset of this size.Along with the accuracy ratios, also sensitivity and specificity values, which are correctly identified positive and negative samples respectively, are given.As shown in Table 4, the specificity of classification with k = 3 neighbors is 0.7391 which proves that this classification algorithm can be used to classify the negative samples in an ensemble model where each algorithm is responsible for the classification of the instances that the others fail.
As for SVMs, the accuracy ratios for all kernel types also increase.Recurrently, the highest accuracy is obtained with the linear kernel.Whereas the specificity value for this classifier is same as the kNN with three neighbors, the sensitivity value is much higher.Besides, the accuracy produced with linear kernel is close to the RBF one and the specificity is a little bit lower.

Conclusions and Discussion
Due to the recent interest in using computers for learning, a two phase education with traditional teaching and CAL of CIM lathe operation was planned in our study.The education was followed with a test to measure the success of the education and then students were asked to design a project.As explained this whole phase is very exhaustive.Additionally giving the opportunity to any student to use lathe machine can cause safety, time management and waste of products.The aim of this project was to use subscales of the readiness for learning questionnaire and academic success to determine whether it is appropriate to give a student this education and subsequently the opportunity to use the device.The students were classified as successful or unsuccessful via the lathe test and the problem was considered as a two class classification algorithm.As a result of the analysis of our dataset, with a parametric classifier -in our case SVM -it was possible to determine whether a student would be successful in learning to operate the lathe machine with the subscales of the readiness for learning questionnaire (motivation, health -nutrition, learning, planned working, efficient reading, writing, listening, note taking, attending a course, preparation and attending an exam) and GPA.Since dataset used in this study is limited, we have also tried different cross validation techniques and achieved higher accuracy rates with LOO cross validation.Apart from the findings presented in our study, it presents the opportunity to explore whether a more robust model can be constructed with more samples.Moreover, different classification methods along with various cross validation techniques can be tried to achieve higher accuracy rates.Last but not least, taking into the correlation and mutual information results into the consideration, a brand new well designed questionnaire with appropriate subscales can be developed.

Figure 1
Figure 1 CNC Lathe software

Figure 2
Figure 2 Educational methodology steps

Figure 3 A
Figure 3 A question from readiness for learning questionnaire

Figure 4 A
Figure 4 A question from the CNC lathe test

Figure 5
Figure 5 Student dispersion according to readiness for learning questionnaire score and GPA

Figure 6 Figure 7
Figure 6 Accuracy per fold for kNN with 10-fold cross validation

Table 1
Pearson correlation coefficients of subscales and GPA Lathe

Table 2
Mutual information of subscales and GPA

Table 3
Accuracy for kNN and SVM with 10-fold cross validation

Table 4
Accuracy for kNN and SVM with leave-one-out cross validation