Findings from the assessment of students ’ learning outcomes in engineering mathematics

Engineering Mathematics which is one of the core subjects in civil engineering education is designed to equip civil engineering students with a strong mathematical knowledge for solving engineering problems in other courses such as fluid mechanics, soil mechanics, structural mechanics, transportation and highway engineering. The purposes of the present study are to report the findings from the assessment of students' learning outcomes in a mathematics test and to explore the issues related to the misconception and difficulties in the manipulation of the solutions. A total of 136 third year degree students were given a set of four questions related to probability and statistics in a test during the first semester of the academic year 2015-16. The answers provided by students were analysed with respect to their being correct, partial, wrong and void which yielded a general description of students' performance. It was found that only 35% of students could successfully use the integration by parts for evaluating the double integrals in probability related questions. 33% of students had a misconception about the complement of an event when they worked out the probability. In addition, only 38% of students could correctly use the logarithmic function and determine the maximum likelihood estimator from random samples. It was concluded that 60% of students faced with various levels of difficulty in the manipulation of the solutions.


Introduction
In outcome-based education, assessment at a subject level is used for assessing students' learning outcomes and performance in a subject.The focus is on the students and the subject.During the learning stage, students are able to demonstrate various levels of understanding such as prestructural, uni-structural, multi-structural, relational, and extended abstract which can be distinguished by Structure of Observed Learning Outcomes (SOLO) taxonomy (Biggs & Collis, 1982).The uni-structural is to represent the lowest level of understanding, while the extended abstract can be regarded as the highest level of understanding.This general model has been adopted for particular tasks in different disciplines (Colley, 2003;Knight, Meyer, Baldock, Callaghan & McCredden, 2014).Many assessment methods such as assignments, tests, presentations, projects, laboratory works, field exercises and examinations are intended to address what students are supposed to learn.As assessment can serve as feedback to students, they are informed of their performance in the assessment and pay more attention to their progress and attainment.It can also be regarded as formative purposes to let students know how well they are on track during learning and how they may improve for students' learning.On the other hand, instructors can ascertain how well students have learned what instructors intend them to learn for improvement in teaching.
Delivery of teaching and learning should be made in constructive alignment with assessment of a subject (Biggs, 2003).In the subject curriculum, teaching and learning activities together with assessment tasks are designed based on a set of the intended learning outcomes which describes the qualities students are expected to develop through their learning experience and enumerate how instructors expect them to demonstrate their understanding of these topics.A student who has the best understanding will receive an excellent grade.Conversely, a student who attains the minimal understanding of the topics will receive a low grade.Teaching and learning activities are thus arranged for students to construct their knowledge and to facilitate achievement of intended learning outcomes.Appropriate assessment criteria and methods are also devised to align with the intended learning outcomes of the subject.
Engineering Mathematics which is one of the core subjects in civil engineering education is designed to equip civil engineering undergraduates with a strong mathematical knowledge for solving engineering problems in other courses such as fluid mechanics, soil mechanics, structural mechanics, construction management, transportation and highway engineering at the Hong Kong Polytechnic University.Students' performance in Engineering Mathematics is assessed by both continuous assessment and a final examination.Continuous assessment involves assessment at different points of the learning process, and is carried out on an on-going basis while students are processing through the subject of study.It includes tests, assignments, mini-projects and other forms of classroom participation.Engineering Mathematics aims at enabling students to acquire fundamental concepts, to apply mathematical knowledge and tools for feasible solutions of practical problems in civil engineering.By the time of graduation students are expected to have an ability to apply the fundamental of applied science, mathematics, and statistical methods to formulate effective solutions across a wide range of civil engineering domains as one of the graduate attributes in the degree programme and one of accreditation criteria for engineering degrees (Hong Kong Institution of Engineers, 2013).

Context of the study
A sound knowledge of mathematics and statistics is a prerequisite for thorough understanding of engineering subjects.However, it is found that many students have learning difficulties with the fundamental subjects at the beginning of their university studies in recent years.In this regard, Mathematics Learning Support Centre (MLSC) of the Hong Kong Polytechnic University is a place where students from all disciplines can get additional assistance in the mathematical and statistical content of their courses.Strategies are also being specifically developed to support civil engineering students with their mathematics simply because professional engineer must acquire not only empirical but also abstract understanding of mathematics (Pyle, 2001;Sazhin, 1998;Sutherland & Pozzi, 1995).It is necessary to strike a balance between practical applications of mathematical equations and in-depth understanding while teaching mathematics to engineering students.
The subject curriculum for Engineering Mathematics is devised to enable civil engineering students to master the basic concepts in probability theory and statistical analysis, and is of particular relevance to applications in civil engineering.Specifically, students are expected to learn the fundamental concepts about probability, the probability distributions of discrete and continuous random variables, expectation and variance of random variables, jointly distributed random variables, marginal probability mass function, marginal probability density function, independence, covariance, correlation, sampling distribution, the Central Limit theorem, estimation of parameters from samples, interval estimation, determination of sample size required, probability plots, goodness-of-fit test for distribution, linear regression and correlation analyses.Students will be able to achieve the following intended learning outcomes upon completion of the subject,  summarize and present information effectively from data;  apply mathematical reasoning for analysis of essential features of different problems;  master descriptive and inferential statistics and make their applications in real-life problems;  apply the fundamentals of mathematics and science to formulate problems and obtain solutions in civil engineering;  critically analyze and interpret the models formulated and solutions obtained to support the synthesis of logical and cost-effective solutions;  integrate knowledge across different subject domains, including construction management, structures, geotechnics, hydraulics, environmental and transportation engineering;  communicate solutions logically and lucidly through calculation, sketch, drawing and in writing.
In the present study, the findings from the assessment of students' performance and learning outcomes in a mathematics test are reported and the issues related to the misconception and difficulties in the manipulation of the solutions are further explored.

Method
A total of 136 third year degree students were given a set of four questions related to probability and statistics in a mini test for both formative and summative assessments during the first semester of the academic year 2015-16.These questions are devised to assess their conceptual understanding of mathematics and indentifying their weakness of mathematical foundation.The focus of the study are to observe students' solution processes in solving problems related to probability and statistics, to identify where they have difficulties in the manipulation of the solutions and have misconception.The students were asked to answer the following questions in the current study.

Questions
1a) The delay (in months) in the completion of two civil infrastructural projects A and B is respectively denoted by two random variables X and Y with the following joint probability density function: f(x,y)=e -(x+xy) ,x>0 and y>0.Determine the marginal probability density functions of X and Y.
1b) Determine the probability that both X and Y are delayed for more than 1 months.That is, P(X>1 and Y>1).
2) The 28-day compressive strength of concrete cube is being examined.Suppose observations on 60 concrete cubes yields a sample mean strength of 37.75MPa.The sample standard deviation of concrete cubes is known to be 4.54MPa.Determine the two-sided 95% confidence interval of the population mean strength.
3) The distribution of fatal accident in a year, X, may be modelled with the Poisson distribution having the probability mass function as p(x)=e -  x ⁄x! x=0,1,2,3… in which  is the parameter of the distribution.Suppose that the following measurements of the fatal accident were observed in the past ten years: 150,280,250,320,190,410,360,260,290,230.Estimate the parameter by the method of maximum likelihood.
Compute the least-squares regression line for predicting elongation from the applied load, and the coefficient of determination.

Results
The answers provided by students were analyzed with respect to their being correct, partial, wrong and void which yielded a general description of students' performance in the mini test.The findings which are related to the student's performance of the subject in the test are reported and shown in Table 1.Based on the collected data, the lowest and highest percentages of 35% and 45% students getting the correct answer are in questions 1a and 2 respectively.On the other hand, the lowest percentage of students getting partial answer is in question 1b and its percentage is 28%.The highest percentage of students getting partial answer is in question 1a and its percentage is 59%.Besides, the lowest and highest percentages of students calculating the wrong answer are in questions 2 and 1b respectively.Only 1% of students worked out the wrong answer in the question 2 but 33% of students evaluated the wrong answer in the question 1b.The unanswered rate of 5% is in question 3, while the other questions were answered by students.In the question 1a, it was found that only 35% of students could successfully use the integration by parts for evaluating the double integrals in probability related questions.Many students had great difficulty in applying the mathematical technique of the integration by parts for the determination of the marginal probability density functions of the random variables X and Y.The integration by parts should have been learnt in other elementary mathematics courses.
In question 1b, 39% of students could completely obtain the correct answer when they evaluated the double integral in order to determine the probability P(X>1 and Y>1).However, it was found that 33% of students had a misconception about the complement of an event when they worked out the probability.These students wrote the following incorrect formula.That is, P(X>1 and Y>1) = 1 -P(0<X<1 and 0<Y<1).Consequently, a wrong answer was yielded.The event or domain which has to be taken into consideration for evaluating the double integral is depicted in Figure 1.The complement of the event or domain for the correct evaluation of the double integral is also shown in Figure 2. In the 2nd question, 45% of students reached the correct answer when they determined the twosided 95% confidence interval of the population mean strength.However, many students could not correctly use the student's t distribution to determine the confidence interval.A few students wrongly used the normal distribution to determine the confidence interval even if the population variance of concrete cubes is unknown.
In the 3rd question, it was found that only 38% of students could correctly use the logarithmic function and determine the maximum likelihood estimator from random samples.A portion of students did not master the mathematical knowledge of logarithmic function which should have been learnt in basic mathematics courses.On the other hand, a few students did not know about the method of maximum likelihood.This is the reason why 5 % of students chose not to answer this question.
In the 4th question, 43% of students knew about the formulae to correctly compute the leastsquares regression line and the coefficient of determination.However, many students reached the partial answer because they got confused about the symbols such as sample variances and sample coefficient of correlation, and incurred mistakes in the process of the solution.A few students did not grasp the relationship between the coefficient of determination and the coefficient of correlation as well.Overall, it was found 60% of students faced with various levels of difficulty in the manipulation of the solutions because the result revealed that about 40% of students were able to reach a correct answer in each question as shown in Table 1.Many students did not yet master the use of integration by parts in dealing with the double integration of the joint probability density function in the 1st question.In particular, one-third of students had a misconception about the complement of an event in the manipulation of the probability in question 1b.Some students incorrectly used the logarithmic function to convert the likelihood function in the 3th question because of their weakness of mathematical foundation.Many students made mistakes in the process of solutions in the 4th question.

Discussion and Conclusion
The mini test was designed to assess students' performance as well as learning outcomes in the present study.Overall, students have attained the intended learning outcomes satisfactorily based on the collected data and findings.Students could summarize and present information from data, and were capable of applying mathematical reasoning to describe and analyze data sets.In addition, students could use statistical inference to know more about the population from data sets.Students also benefited from integrating mathematical concept and theories into practical applications in different subject domains such as construction management, structural mechanics, transportation and highway engineering.They were able to formulate problems and present solutions logically and lucidly through calculation and in writing.
The findings from the assessment of students' performance and learning outcomes revealed that many students were not able to construct their own knowledge of fundamental mathematics satisfactorily.A portion of students could still not master the use of integration by parts in solving the single and double integrals based on the observation of students' solution processes even if they have studied this topic in other elementary mathematics courses.In addition, many students faced various difficulties in the manipulation of the solutions because they did not grasp the basic knowledge such as the logarithmic function, and got confused about the statistical symbols such as sample variances and sample coefficient of correlation.Besides, some students had a misconception about the complement of an event because they could not think about the problem of the complement of an event in the Cartesian coordinate system graphically.The misconception indicated that students might not construct their own knowledge satisfactorily.
It is concluded that only 35% of students could successfully use the integration by parts for evaluating the double integrals in probability related questions in the study.One-third of students had a misconception about the complement of an event when they worked out the probability.Moreover, only 38% of students could correctly use the logarithmic function and determine the maximum likelihood estimator from random samples.Overall, 60% of students faced with various levels of difficulty in the manipulation of the solutions.

Figure 1 .
Figure 1.Event or domain of consideration Figure 2. Complement of the event

Table 1 .
Percentage Values for Students' Performance in the Test