Examining school variables affecting PISA 2012 math achievement in Turkey and Shanghai-China

The main purpose of the research is to examine school variables that have effect on Programme for International Student Assessment (PISA) 2012 math achievement in Turkey and Shanghai-China. The research was designed in casual comparison model. Research population was constituted by student in the age group of 15 in Turkey and Shanghai-China in 2012. The sample consists of 4,848 students and 170 schools in Turkey and 5,177 students and 155 schools in Shanghai-China that participated in PISA 2012. Two-levelled hierarchical linear modelling was used to analyse data because the data collected in PISA 2012 had a hierarchical data structure. As a result of the analysis, variability in math scores, 63% in Turkey and 47% in Shanghai-China, was found due to the difference between the mean math scores of schools. It was determined that MACTIV, SCMATEDU and TCMORALE in Turkey and MACTIV, in Shanghai-China statistically affect on math achievement.


Introduction
The basic goal of education is to get individuals to acquire qualifications that they would need throughout their lives.For this purpose, education systems and sub-systems are founded.Managers, teachers and other trainers, who are involved in education system, struggle for the success of their schools.Student success is one of the most basic indicators of whether schools function in accordance with their goals.There exist many studies on variables affecting student success (Akyuz, 2006;Goddart, Sweetland & Hoy, 2000).Student success related characteristics may be family or personality traits, it may be school or class in which they study, as well (Hox, 1995).Thus, these characteristics, which might be related to success, should also be addressed (Raudenbush & Bryk, 2002).
Educationalists and researchers have been debating on school variables affecting student success for decades (Darling-Hammond, 2000).This study focuses on school variables that have impact on student success.There are many variables related to the school; however, some variables were selected to be investigated since it was not possible to investigate all of them within the scope of this study.The first variable is school size.While many studies conducted on the impact of school size (SCHSIZE) on student success show that there is not a difference between small and large schools (Caldas, 1987;Haller, Monk & Tien, 1993), some studies show significant differences (Bates, 1993;Celebi, 2010).The second is the levels of morale teachers have (TCMORALE).If schools involve teachers whose morale is high, students may have high morale.This situation influences student success directly (Whitaker, Whitaker & Lumpa, 2000).The third one is math extracurricular activities (MACTIV) at school.Such activities as competitions, computer clubs and math clubs at schools are activities evaluated in this framework.The last one is teaching resources of schools (SCMATEDU).It seems that science laboratories, teaching materials, computers, Internet connections, computer softwares, inadequacy or absence of library materials at schools have the potential of affecting student success.

Purpose of the study
Determining variables that affect student success is extremely important in terms of enabling education and training activities to be effective.This study attempts to unfold school variables affecting student success by comparing Turkey's math achievement in Programme for International Student Assessment (PISA) 2012 exam with Shanghai-China that ranked first in the same exam.In this way, similarities and differences among variables affecting Turkey and Shanghai-China can be revealed more explicitly via an international comparison.
1. Are there differences among schools which participated in PISA 2012 study in Turkey and Shanghai-China in terms of math achievement?2. If there are any, what are school level variables that clarify differences among schools which participated in PISA 2012 study in Turkey and Shanghai-China in terms of math achievement?

Method
As this research analysed the effects of student and school variables on PISA 2012 math achievement, it was based on causal-comparative model.
Population was composed of 15-year-old students in Turkey and Shanghai-China in 2012.Sampling was composed of 4,848 students and 170 schools from Turkey, 5,110 students and 155 schools from Shanghai-China that participated in PISA 2012 (OECD, 2012).Schools having participated in PISA 2012 treatment were selected through stratified sampling method by international statistical company-Westat according to 12 statistical territory, programme type and ratio of students' representativeness among all schools which had or could have 15-year-old students.
Hierarchical linear models (HLMs) are generalisations of regression methods used with the aim of causal comments, various estimations and data reduction (Raudenbush & Bryk, 2002).In exhaustive sampling studies, because, the fact that students who study in the same school show greater similarity than other students who violate independence of observations; also some schools' being more homogeneous in terms of specified feature that violates equation of variances hypothesis, multiple regression produces non-objective results (Raudenbush & Bryk, 2002).
In order to avoid non-objective results of multiple regressions, HLM was used.

One way ANOVA model with random effects
To answer the first research question, one-way analysis of variance (ANOVA) model with random effects, also known as null model, was used.

 =+
Level 2: 2) It is assumed that each level-1 error, rij, is normally distributed with a mean of zero and a constant level-1 variance, σ2.Notice that this model predicts the outcome within each level-1 unit with just one level-2 parameter, the intercept, β0j.In this case, β0j is just the mean outcome for the jth unit.That is, β0j = μγj (Raudenbush & Bryk, 2002, p. 23).
The level-2 model for one-way ANOVA with random effects is: with γ01 set to zero, γ00 which represents the grand-mean outcome in the population and u0j is the random effect associated with unit j and is assumed to have a mean of zero and variance τ00 (Raudenbush & Bryk, 2002, pp. 23 and 24).

Means as outcome model
With the aim of answering the second research question, means as outcome model was used in the analysis of HLM.

Are there differences among schools which participated in PISA 2012 study in Turkey and Shanghai-China in terms of math achievement?
According to the results of one-way ANOVA model with random effects which was constructed to find out whether there was a difference among schools which participated in PISA 2012 study in terms of math achievement; to be able to determine if there was a significant difference between average math achievement of schools, total variance of student scores were divided into two as level-1 (students' level) and level-2 (schools' level).In Turkey, while student level variance was estimated as (σ 2 ) 3,156.30 and school level as (τ00) 5,476.10,In Shanghai-China, student level variance was estimated as (σ 2 ) 5,490.15 and school level variance as (τ00) 4,801.37.Moreover, the fact that variation among schools at both countries is statistically significant (p < 0.001) indicates that there is a difference among math achievement of schools having participated in PISA 2012 (Table 1).Shared part of total variance between student and school level is determined through interclass correlation (ICC) (Raudenbush & Bryk, 2002).Through proportion of school mean to total variance, almost 63% of total variance in Turkey and 47% in Shanghai-China may be proposed to stem from discrepancy among achievement mean of schools having participated in PISA 2012.Of the total variance, 37% in Turkey and 53% in Shanghai-China stem from differences among students.It is seen that variance proportion explained at school level supports two level hierarchical model usage in the analysis of data (Turkey = 0.63 > 0.01; Shanghai-China = 0.47> 0.01).
As a result of one-way ANOVA model with random effects, math achievement mean (β0j) reliability estimation was predicted as r = 0.97 in Turkey and Shanghai-China.High reliability indicates that sampling size was enough and data distribution was well-balanced (Raudenbush & Bryk, 2002).

What are school level variables that clarify differences among schools which participated in PISA 2012 study in Turkey and Shanghai-China in terms of math achievement?
In order to find out school level variables that clarify differences among schools which participated in PISA 2012 in terms of math achievement, means as outcome model was constructed and analysis results of the model is given in Table 2.When Table 2 is examined, it is seen that MACTIV (γ01 = −13.65,p = 0.001), SCHSIZE (γ01 = −0.02,p = 0.002), SCMATEDU (γ03 = 22.39, p < 0.001) and TM (γ04 = 11.21,p < 0.05) in Turkey, and MACTIV (γ01 = 26.50,p < 0.001) in Shanghai-China are statistically significant in terms of math achievement.
Effect size was calculated in order to evaluate how important were statistically meaningful impacts of independent variables on dependent variables in practice (Atar, 2014, p. 130).Considering the effect size, MACTIV (18%), SCMATEDU (30%) and TM (15%) in Turkey, while MACTIV (38%) in Shanghai-China, were found influential on student success.Though SCHSIZE was found statistically significant in Turkey, it became evident that it was not significant in practice when effect size was measured.
While in one-way ANOVA model with random effects model-school level variance was estimated as 5,476.10 in Turkey and 4,801.37 in Shanghai-China, and as a result of the means as outcome modelschool level variance was estimated as 4,013.94 in Turkey and 3,121.80 in Shanghai-China with the addition of school level related variables.In proportion of differences among variances estimated in these two models to the total variance at school level, it was found that the variance among school means of the variables added to school level model explained 0.27 of them in Turkey and 0.35 in Shanghai-China.

Discussion
Considering explanatory level by student level variables of total variance explaining students' math achievement, it can be suggested that student variables have less impact in Turkey than in China.When examining total variance explanatory level of school level variables, school level variables in Turkey are much more influential than in Shanghai-China.While 27% of total variance at school level in Turkey is explained by MACTIV, SCHMATEDU and TCMORALE, 35% of total variance in Shanghai-China is clarified by MACTIV.The sole common variable that is influential on mathematics success in both countries is MACTIV.It can be suggested that the school's extracurricular math activities are rather influential on student success.With regard to that variable, it can be told that Shanghai-China is in a much better condition than Turkey (26.50 > 13.65) and the country should be taken into consideration in terms of math achievement level it got as well.In Turkey, the most effective variable is school sources at school level.As school sources are in good condition, so success levels of schools improve considerably.Focusing of schools that lack adequate sources and seeking solutions may be effective on developing success level as a country.