Evaluation of the discriminating ability of the Greek state foreign language exam ( KPG ) for English by means of the factorial analysis of correspondences

The aim of this paper is to investigate the discrimination ability of the KPG English Language Examination Battery of May 2012 -A Level (A1 – A2). The present study proposes a new methodology for evaluating the discriminating ability of the test items. More specifically, the paper proposes the assessment of the test to be carried out by means of the Factorial Analysis of Correspondences (Analyse Factorielle des Correspondences AFC). This specific analysis can differentiate the sub-level A1 from that of A2. The more essential target of the study is the identification of the items that differentiate Greek speaking students’ level of English Language Competence. The survey sample consisted of 101 students of Greek state schools from different areas of Greece, who completed the KPG English Language Examination Battery of May 2012 -A Level (A1 – A2). The participants of the research were 6 th grade primary school and 1 st grade secondary school students. The results confirmed the suitability of the above mentioned methodology.


Introduction
The Greek state certificate for foreign language proficiency known as KPG offers exams in European languages of 'value' to Greek society.The specific exam battery is addressed to people who live, study or work in Greece.The only condition is that they understand the Greek language, as it is used to a limited extent in the exams.This is an instance of its local character, which is combined with its global aspect ('glocal' trait), as it refers to various sociocultural realities and is acknowledged nationally and internationally (Anastasiadou & Tiliakou, 2015).
As it is aligned with the Common European Framework of Reference for Languages it provides for the certification of three levels of language ability.

Theoretical framework
As with any language test, for reasons of fairness, the quality of the items must be investigated in order for poor items to be reviewed and possibly extracted from the test.Specifically, in this paper the discriminating ability of items is being checked.Discrimination is a fundamental property of language tests in their attempt to capture the range of individual abilities.On that basis, the more widely discriminating the test, the better it is (Davies, 1999).The quality of individual items is also imperative

131
for the validity of a test.Therefore, it is very important to determine to what extent each item contributed at discriminating against candidates based on their mastery of reading skills in this particular case, or the general linguistic knowledge which these activities assess (Tsopanoglou, 2010).
If high achieving students correctly answer the more demanding items, as expected, then their linguistic aptitude is properly assessed.If they give the wrong answer to an item that low achieving test takers answer correctly, then this item performs poorly in assessing levels of linguistic competence.This is a problem, as each item that does not contribute effectively to the test purpose lowers its validity (Lado, 1961: 342,350).

Aims of the Research
The aim of this paper is to investigate the discrimination ability of the KPG English Language Examination Battery of May 2012 -A Level (A1 -A2 according to the Common European Frame of Reference).The overriding target is to differentiate level A1 from level A2.The underlying goal of the paper is to locate those items that differentiate the level of language ability of the students.The present study proposes a new methodology for evaluating the discriminating ability of the test items.More specifically, the paper proposes that the assessment of the test is carried out by means of the Factorial Analysis of Correspondences (Analyse Factorielle des Correspondences -AFC).This specific analysis can differentiate the sub-level A1 from A2.

Sample
The sample of this research consisted of 101 students, who completed the A level English Language Greek State Certificate (KPG) of May 2012.The candidates were 12-13 years old, 6 th grade primary school and 1 st grade secondary school students.The students were from northern as well as southern regions of Greece, and came from different backgrounds.

The A level KPG test of May 2012
This test included 20 activities in total.The oral exam was not examined as it touches on different issues compared to the rest of the exam battery, which is more explicit and measurable.Out of the 20 activities, 11 are reading comprehension tasks, 4 activities assess writing production and the last 5 are listening comprehension exercises.Specifically, the eighth and ninth task of the Reading Comprehension will be investigated in the present paper.
Activity 8: This is a closed type task, where the candidate must match the questions of the interview with the answers.The interviewee is a Robotics Professor talking about Asimo, a very special robot (Figure 1).
Concerning the coding of the items for the statistical process of data, item 33, which refers to what Asimo is, is coded as RC8.1.Item 34, which refers to how special Asimo is, corresponds to RC8.2.Item 35, which asks how easy the Asimo project was, corresponds to RC9.3, and item 36, which asks whether Asimo is cheap corresponds to RC8.4.

132
Figure 1 133 Activity 9: This is a multiple choice task of converging production, with 3 options (Figure 2).The test takers are requested to choose the right option in relation to four small texts of informal and informative discourse.Text A is an advertisement about a weather update to be sent to people's emails.Text B is a dictionary entry for the word clown.Text C is a funny story about a boy on a bike and text D is from the news about a popular singer.Out of the 5 items one refers to text A, one to text B, two to text C and one to text D.
As regards the coding of the items for the statistical process of data, item 37, which refers to what Text A talks about is coded as RC9.1.Item 38, which refers to what type of book text B was extracted from corresponds to RC9.2.Item 39, which characterizes text C corresponds to RC9.3.Item 40, which asks how the old lady felt corresponds to RC9.4, and item 41, which asks who text D is about is coded as RC 9.5.

Research Methodology
The research data were analysed via Classical Item Analysis and Factorial Analysis of Correspondences (Analyse Factorielle des Correspondences -AFC).By applying the Factorial Analysis of Correspondences, we achieve an almost global description of the situation aided by a lower number of new composite independent variables, the so-called factors.The factors, which take the 134 form of axes, vertical in two, are the factorial axes and are created from the composition of groups of initial variables, resulting in a more revealing search of relationships among variables-items in our case.The interpretation of the findings of the Factorial Analysis of Correspondences takes place in the first factorial planes, those namely, which are created by the factorial axes based on the rates of characteristic values   , where . for every axis, with values between 0 and 1.The results of Factorial Analysis of Correspondences, extracted with MAD software (Karapistolis, 2014), are interpreted via inertia, which every factorial axis (criterion 1) explains, and finally, via correlation (Cor) and contribution (Ctr).These indices allow us to immediately discern the most significant and determining variables or objects, contributing to the creation of factorial axes.Values

Results of the Factorial Analysis of Correspondences (Analyse Factorielle des Correspondences -AFC).
Activity 8: The analysis of the table of data via AFC gives initially table 1, which shows the characteristic values of Burt table, as well as the inertia percentages of each factorial axis.Table 2 allows us to distinguish the amount of the principal factorial axes, which are the most suitable in interpreting the results.The inertia percentage of each factorial axis offers the ability to know the percentage of importance each axis conveys (criterion 1).According to the rates accompanied by the histogram (Table 2) the percentage of importance of the first factorial axis is 77.98%; of the second it is 10.18%; of the third it is 5.58%; and of the fourth it is 4.57%, and 1.69% is the percentage of the fifth factorial axis.The totality of information provided by the first five factorial axes, amounts to 100%.In the following table, we can see the sum of information offered by the first five factorial axes.Based on the cumulative frequency, the first two factorial axes interpret 86.98% of the total variance of data (Table 1).This percentage is considered satisfactory for data interpretation.Then, from the table of results of the factorial correspondence analysis, and according to the above mentioned selected criteria (inertia, correlation and contribution), we detect the variables contributing to the formation of the first two factorial axes.
The variables, which were deemed most significant for the extracted factorial axes according to the two criteria: correlation (

 Cor
, criterion 2) and contribution (Ctr≥1000/numbers of variables, criterion 3) are presented in detail as follows (Table 2).Interpretation of the first factorial axis e 1: More specifically, based on the answers of the test takers, and as it derived from the factorial analysis, the first axis -factor e1, with eigenvalue 0.2455826 explaining 77.98% of total variance, is constructed from classes RC8.10, RC8.11, RC8.20, RC8.21, RC8.30, RC8.31, RC8.40 and RC8.42.In fact, the first factorial axis e 1 is formed by those variable classes projecting the ability of the examinees as they appear in Activity 8 (Figure 3).
In greater detail, on the left of the first factorial axis we perceive the classes establishing the quite unsuccessful response of the test takers concerning the matching of questions with their respective answers.On the other hand, on the right of the first factorial axis are the classes determining the quite successful response of the examinees in Activity 8, and especially items RC8.1, RC8.2, RC8.3 and RC8.4.Thus, we can assume that the first axis expresses a new composite variable, which defines the ability (high-low) of the testees in Activity 8. Accordingly, we can conclude that the first axis expresses a new composite variable, which determines the differentiation of students of level A1 and A2 regarding Activity 8 and items RC8.1, RC8.2, RC8.3 and RC8.4.Moving further on the axis from left to right we observe the gradation of the testees' performance.First we notice their wrong answer about what exactly is Asimo (RC8.10)(Cor=164, Ctr=150), how special Asimo was (RC8.20)(Cor=98, Ctr=101), and their correct answer to how easy the Asimo project was (RC8.31)(Cor=52, Ctr=40), or whether Asimo is cheap (RC8.41)(Cor=174, Ctr=65).On the right of the second factorial axis the examinees who answered incorrectly how easy Asimo project was, are positioned (RC8.30)(Cor=52, Ctr=64), while we find the testees, who answered correctly how special Asimo was (RC8.21)(Cor=98, Ctr=98), what exactly is Asimo (RC8.11), and incorrectly whether Asimo is cheap (RC8.40)(Cor=174, Ctr=65).These are level A2 students.Activity 9: The analysis of the table of data with AFC yields initially table 3, which presents the characteristic values of Burt table as well as the inertia percentages of each factorial axis.Table 3 allows us to discern the number of the principal factorial axes, which are the most suitable for the interpretation of the results.The inertia percentage of the factorial axis permits us to know the percentage of importance each axis conveys (criterion 1).

First factorial plane
According to the values, which are accompanied by the histogram (Table 4), the percentage of importance of the first factorial axis is 60.05%, of the second it is 15.54%, of the third it is 9.49%, of the fourth it is 7.60%, 5.49% is the percentage of the fifth and 1.83% of the sixth factorial axis.The totality of information, provided by the first 6 factorial axes, amounts to 100%.This information appears in the following table.Based on accumulative frequency the two first factorial axes read 86.98% of total variance of the data (Table 3).This percentage is considered adequate to interpret the data.Afterwards, from table 4 of the results of the factorial analysis of correspondences, and according to the above mentioned selected criteria (inertia, correlation and contribution), we detect the variables contributing to the formation of the first two factorial axes.
The variables of highest importance for the extracted factorial axes according to the criteria correlation (  4).
More specifically, the first factorial axis e 1 is constructed by those classes of variables that project the language ability of examined students, and which refer to Activity 9.At the beginning we observe the failure of the test takers who were classified in level A1.In particular, on the left side of the first factorial axis we note the candidates classified as A1 (Cor=789, Ctr=69), and answer wrongly regarding the feelings of the old lady in text C (RC9.Interpretation of the second factorial axis e 2 : The variables, which are most important for the second factorial axis referring to Activity 9, according to the criteria inertia and contribution, are thoroughly presented in the following table (Table 4).In greater detail, the examinees' responses together with the factorial analysis yielded the second axis-factor e 2, with eigenvalue 0.0469329, 139 which explains 21.11% of total variance and is composed of classes RC3.20, RC3.21, RC3.30 and RC3.31 (Figure 7).The above mentioned level interprets (Figure 6) 77.33% of total inertia -information, a quite satisfactory percentage.The first factorial axis presents the extreme cases, while the second the middle ones.On the first factorial axis on the left, and between the second  

Conclusions
The validity and control of discriminating ability of the Greek State Foreign Language Certificate, was instituted in 1999 and is known by the acronym KPG.This aims to ensure that this qualification is a suitably objective tool that can measure the level of language awareness, and that the difficulty of the exam does not differ from one administration to the next.Factorial Correspondence Analysis is a very important method of evaluating the validity, fairness and discriminating ability of the language certificates.
The present research investigated the above mentioned properties of the May 2012 KPG in English at levels A1/2, with particular reference to Reading Comprehension by means of the Factorial Analysis of Correspondences.In total, 101 students of a 6 th grade primary school and a 1 st grade secondary school took part in the research.In the present paper, two activities (activities 8 and 9) were thoroughly studied out of the eleven activities of the Reading Comprehension section.
In particular, the results of the research showed that items RC8.1, RC8.2, RC8.3 and RC8.4 of Activity 8 could differentiate between 'high' and 'low' achieving candidates and consequently they do not need revision.Activity 9, however, is relatively problematic concerning the discrimination ability of all its items.Items RC9.5 and RC9.2 could not differentiate 'good' from 'bad' test takers and needs to be reviewed.
In conclusion, there are items that need to be reviewed so that difficulty and discriminating ability are ensured.It is very important to undertake further research into new and larger samples and other minority populations, and to study the impact of demographics on student achievement.
the exam is almost identical with the other levels, namely: Module 1: Reading comprehension, Module 2: Writing, Module 3: Listening comprehension, Module 4: Speaking Module 1 aims at evaluating the test takers' competence in understanding written and multimodal texts.Two Reading Comprehension activities from this module are examined in this research.Nomenclature Discriminating ability: to what extent the item captures the range of individual

Figure 3 :
Figure 3: Factorial axis e 1 of Activity 8Interpretation of the second factorial axis e 2 : The variables, which are more important for the second factorial axis as regards Activity 8 according to the two criteria: inertia and contribution, are thoroughly exhibited on the following Figure (Figure4).In fact, based on the examinees' responses, and as it stems from the factorial analysis, the second axis-factor e 2 with eigenvalue 0.0320603 explaining 10.18% of total variance is composed from classes RC8.10,RC8.20,RC8.30, RC8.40, RC8.11,  RC8.21, RC8.31, RC8.41, Α1 and Α2.

Figure
Figure 5: First factorial plane are displayed in detail below (Table 40) (Cor=811, Ctr=144), or what kind of text is text C (RC9.30) (Cor=718, Ctr=81), and what text A talks about (RC9.10)(Cor=626, Ctr=105).On the right of the first factorial axis, the item classes of the first factorial axis are gathered, which denote candidates' success in the items of Activity 9.In fact, on the right side of the first factorial axis we note the high linguistic ability of candidates since they are categorized as level A2 (Cor=789, Ctr=152), regarding their correct answers, not only in the question on what text A says (RC9.11) (Cor=626, Ctr=63), but also what kind of text is C (RC9.31) (Cor=718, Ctr=81), and finally what feelings was the old lady experiencing in text C (RC9.41) (Cor=811, Ctr=121).

Figure 7 :
Figure 7: Factorial axis e 2 of Activity 9 More specifically, on the left side of the first factorial axis we find the classes defining the failed answer of the test takers regarding reading comprehension.In contrast, on the right of the first factorial axis are the classes designating the successful responses of candidates to the particular reading comprehension activity and items RC9.1, RC9.2, RC9.3, RC9.4 and RC9.5.First factorial level 2 1 e e  .The above mentioned level interprets (Figure 8) 77.33% of total inertia - information, a quite satisfactory percentage.

Table 2 :
Table of coordinates FA of the 1 st and 2 nd factorial axis of Activity 8

Table 4 :
Table of FA coordinates of the 1 st and 2 nd factorial axis of Activity 9