Investigation of the discriminating ability of the Greek State certificate of language proficiency by means of vacor method : The case of Greek-speaking and Turkish-speaking bilingual students from the muslim minority of thrace

The present research aims to explore the discriminating ability of the Greek State Language Certificate known as KPG, administered in May 2012, which examined the English language at A1/A2 Levels according to the scale of the Common European Framework of Reference for Languages. The proposition of this paper pertains to the evaluation of the examination via the Vacor technique. The main target is the accuracy of differentiation of level A1 from level A2. More specifically, the goal is to identify items that differentiate the two levels for the Greek-speaking students as well as the Turkish-speaking students of the Muslim minority of Thrace. Accordingly, the suitability of items leading to the differentiation and classification into levels of the students’ linguistic ability in English is investigated. The Vacor technique is proposed, whereby the suitability of each item and its contribution to the candidates’ ranking into A1/A2 levels is checked. Therefore, the method is applied to two different groups, as mentioned above, and the results are compared. It was concluded that several items did not discriminate A1 from A2 levels in the English language in both groups. Thus, the present study confirms the choice of the Vacor technique for reviewing the suitability of the KPG qualification for English A1/A2 of May 2012. It is recommended that test items are reviewed in order to discriminate between levels for all groups of candidates.


Introduction
The Greek state certificate for foreign language proficiency, known as KPG, administers exams in European languages to people in Greece.This is done at no material or symbolic profit.
The KPG exams are scaled according to the Common European Framework of Reference for Languages.They comprise the following levels: The A level exam in English, which will be investigated in this paper, was designed to accredit the A1 Basic knowledge or A2 Elementary knowledge of students aged 10-15 years old, that is, to assess the language competence acquired both in and outside school.
The format of the exam is almost identical with the other levels: Module 1: Reading comprehension, Module 2: Writing, Module 3: Listening comprehension, Module 4: Speaking.
To elaborate further, Module 1 aims to measure a) the individual's competence in understanding written and multimodal texts and b) his/her grasp of language concerning lexical and grammatical elements as used in statements and short texts.Module 2 evaluates the candidate's ability to compose brief texts in writing, on the basis of given prompts.Module 3 seeks to measure the examinee's skill at understanding spoken language in simple forms.Module 4 aims to assess the candidate's aptitude at uttering a message orally.
In Module 1 the test taker receives information and responds to it.More specifically, he/she is asked to understand written and mostly multimodal texts.Seven out of the 11 activities of the first module have images accompanying the utterances.Authenticity is favoured in this way since multimodality has been enhanced in our technology age.Moreover, as mentioned earlier, the texts are designed for young learners and pictures are expected in reading material targeted at this age group.
Language Proficiency and its measurement is no simple matter.The format of the exam is crucial, as it may help a test taker respond correctly, or totally disorientate him/her (Laborda, 2007).Therefore, it should be considered whether items in an activity measure what is intended to be measured or have flaws that prevent them from measuring candidates' abilities correctly.

KPG
Greek State exam battery for certification of foreign language competence A1/A2 Beginner / Elementary level of language

Item of a Reading Comprehension Activity
Item discrimination When an item discriminates high-achieving from low-achieving students Sofia D. Anastasiadou & Chrysanthi S. Tiliakou (2016).Investigation of the discriminating ability of the Greek State certificate of language proficiency by means of vacor method: The case of Greek-speaking and Turkish-speaking bilingual students from the muslim minority of thrace.International Journal of Learning and Teaching. 8(2), 109-118. 111

Aim of the research
The main aim of the research is to investigate the accuracy of the differentiation between levels A1 and A2 of the English Language Certificate of May 2012.In particular, the objective of the study focuses on detecting the specific items on the basis of which the level of students is differentiated.This is of utmost importance, as Differential Item Functioning should be avoided and the suitability as well as the fairness of the items leading to the classification of candidates into levels A1 and A2 should be ensured.To this end, there is an attempt to apply the Ascending Hierarchic Classification using the Vacor method in two distinct samples of students, one of which consists of Greek-speaking students and the other of Turkish-speaking students from the Muslim minority of Thrace.

Sample
The first sample of this research consisted of 101 students, who completed the A level English Language Greek State Certificate (KPG) of May 2012.The candidates were 12-13 year old students of the 6 th grade of Primary School and the 1 st grade of Secondary School.The students were from northern as well as southern regions of Greece, coming from different backgrounds.The second sample was formed of 40 Turkish-speaking students of the Muslim minority of Thrace.

The A level KPG test of May 2012
This test included 20 activities in total.The oral exam was not included as it touches on completely different issues compared with the rest of the exam battery, which is more explicit and measurable.The evaluation of oral competency is the object of another research.Out of the 20 activities, 11 are reading comprehension tasks, four activities assess writing production and the last five are listening comprehension exercises.Only the eighth and ninth tasks of the Reading Comprehension will be investigated in the present paper.The two activities are objective, as there is only one correct answer.Moreover, they are types of indirect testing as the candidate does not produce language in context but expresses his/her language competence indirectly by choosing the correct option or matching utterances appropriately.Accordingly, the activities consist of discrete items and not of integrative tasks, which are formed by more lifelike components (Council of Europe, 2001).

Activity 8
This is a closed-type task, where the candidate must match the four questions of the interview with the answers.There is an extra answer to distract the candidate.The interviewee is a Robotics Professor talking about Asimo, a very special robot.Concerning the coding of the items for the statistical process of data, item 33, which refers to how Asimo is coded, corresponds to RC8.1; item 34, which refers to how special Asimo is, corresponds to RC8.2; item 35, which asks how easy the Asimo project was, corresponds to RC9.3; and item 36, which asks whether Asimo is cheap, corresponds to RC8.4.proficiency by means of vacor method: The case of Greek-speaking and Turkish-speaking bilingual students from the muslim minority of thrace.International Journal of Learning and Teaching. 8(2), 109-118.112

Activity 9
This is a task of converging Production and is multiple choice with three options.The test takers are requested to choose the right option in relation to four small texts of informal and informative discourse.Text A is an advertisement about a weather update to be sent to people via e-mail.Text B is a dictionary entry for the word clown.Text C is a funny story about a boy on a bike and text D is from the news about a popular singer.Out of the five items, one refers to text A, one to text B, two to text C and one to text D.
As regards the coding of the items for the statistical process of data, item 37, which refers to what Text A talks about, corresponds to RC9.1; item 38, which refers to what type of book text B was extracted from, corresponds to RC9.2; item 39, which characterizes text C, corresponds to RC9.3; item 40, which asks how the old lady felt, corresponds to RC9.4; and item 41, which asks who text D is about, corresponds to RC 9.5.

Research methodology 2. Research Methodology
The research data were analysed via Ascending Hierarchical Classification with Vacor Methodology.The Vacor method is used to study the properties of statistical units.The method provides the tree diagram of classification, which offers the opportunity to determine the percentage of contribution of each variable to the characterisation of each class, as well as the variables contributing to the node splitting (Karapistolis, 2010).The latter is a process proposed by Bezencri in 1980(Bezencri, 1980) with the algorithm named Aide CAH Vacor (Papadimitriou, 2007).In addition, this particular methodology is the connecting link between Automated Classification (CAH) and Factorial Analysis of Correspondences (Papadimitriou, 2007).The results of Ascending Hierarchical Classification with the Vacor method, which were extracted with MAD software (Karapistolis, 2014) Moreover, the index defining the variables that contribute the most to the disruption of each node is   i COD j , which is the squared cosine of the angle formed by the line segment of the   is investigated, which is the percentage of inertia of the dipole divided by the total inertia of the cloud on the variable axis (Drosos, 2004).

115
Group B, which is considered a node, is divided into subgroup B1, containing properties RC8.30 and RC8.40, which are superior relative to RC8.10 and RC8.20 in class B2.
Turkish-speaking students of Muslim minority: The application of CAH, which differentiates the variables as to their properties, yields this node typology (Figure 2): Node O, which constitutes the total group, is divided into group A, which comprises properties RC8.21, RC8.31 and RC8.41.

2 Results of the Hierarchical Classification with Vacor Methodology for Activity 9
Greek-speaking students: The application of CAH, which differentiates the variables as to their properties, results in the following node typology (Figure 3): Group B, which is considered a node, is divided into subgroup B1, containing properties RC9.20, RC9.30, RC9.50 which are superior relative to properties RC9.10, RC9.40 subgroup B2.
Turkish-speaking students of Muslim minority: The application of CAH, which differentiates the variables regarding their properties, gives this node typology Figure 4):

Conclusions
The validity and discriminating ability control of the Greek State Foreign Language Certificate, known as KPG, which was established in 1999, aims to ensure that the test is a suitable and objective tool to measure the language awareness level and that its difficulty is not different from one administration to the next.The Ascending Hierarchical Classification with Vacor Methodology determines the suitability of each item and contributes to the classification of students into A1 and A2 levels of language competence.The present research investigated item discrimination, which ensures validity and therefore fairness of an exam, in this case the KPG Certificate.It is a widely accepted fact that item discrimination should be invariant.The relative abilities of candidates that a test is designed to measure should not depend on the items chosen (McNamara, 1996).Accordingly, if one item discriminates 'low-' from 'high-' achieving individuals, whereas another item does not, then the inferences made about the examinees based on their scores are heavily blurred.
In the case of minorities, of course, possibility for bias must be investigated.Besides, bias may be claimed if differences between the two groups (minority, majority) are attributed to reasons irrelevant to performance (Davies, 1999).In other words there is construct -irrelevant variance, which distorts tests results and makes a unidimensional test multidimensional: The test measures -reveals something more than what it is intended to measure.(McNamara & Roever, 2006).
A first step to overcome such dangers is to establish Differential Item Functioning, by identifying the items that function differently for two groups of test takers (McNamara & Roever, 2006).
In the research, 101 students of the 6 th grade of Primary school and the 1 st grade of Secondary School took part, as well as 40 Turkish-speaking students from the Muslim minority of Thrace.In the present paper, two activities were thoroughly studied (Activities 8 and 9) out of the 11 activities of the Reading Comprehension section.
In particular, the results of the research showcased that, for Activity 8 for the Greek-speaking students, those categorized in level A2 answered items RC8.2 RC8.1 correctly, whereas the students classified in level A1 failed to answer items RC8.3 and RC8.4 correctly.Thus, only item RC8.4 differentiates 'high-' from 'low-' achieving candidates.
Furthermore, the research indicated that the Turkish-speaking students were classified in level A2 for Activity 8, as they answered RC8.4 correctly, while no item seemed to be a determining factor concerning level A1 differentiation.
As for Activity 9, the results showed that the Greek-speaking candidates classified as A2 level answered items RC9.1 and RC9.4 correctly, while the students classified as A1 did not answer these items correctly.Thus, only items RC9.1 and RC9.4 differentiate the test takers into the two levels.
As regards the Turkish-speaking students, who were classified in level A2 for Activity 9, they answered only items RC9.2, RC9.3 and RC9.4 correctly, while no item seemed to play a crucial role in A1 level classification.
From the above findings it is clear that item RC8.4 of Activity 8 and item RC9.4 of Activity 9 determine A2 level of both the Greek-speaking and Turkish-speaking candidates, while it is unclear whether they affect the differentiation of the KPG exam in levels A1 A2.In more detail, whereas only three items differentiated levels A1-A2 for the Greek-speaking group (RC8.4,RC9.1 and RC9.4), no items were found to differentiate A1 from A2 level students in the Turkish-speaking group.Moreover, one item was the most problematic as it did not even fall under A1 or A2 classification, namely RC9.5.
In conclusion, there are items that need to be revised so that consistency of the criteria of difficulty and discriminating ability is ensured.Further research in larger samples and other minority populations should be carried out, to establish the validity of the Greek State Language Certificate.
In addition, the use of the Ascending Hierarchic Classification using the Vacor method is a suitable technique, allowing the investigation of the suitability of each item and its contribution to its classification into levels A1 or A2 of language competence.
, are interpreted by means of correlation   i COR j , which is the squared cosine of the angle formed by the line connecting the centre of gravity of the initial node O C with the centre of class i C with the axis of a essentially the relative contribution -percentage of inertia -of i C node in relation to the total inertia of the cloud the axis of the variable (Drosos, 2004).
from the disruption of the node i C , from the axis of a variable.It establishes the extent of the contribution of the variable in the interpretation of the disruption of node i C .To determine the causes contrasting groups   i A and   i B of the dipole, the centres of gravity are examined in the variable area(Papadimitriou, 2004).Finally, the index

Figure 1 .
Figure 1.Node typology of Greek-speaking students for Activity 8

Figure 2 :
Figure 2: Node typology of Turkish-speaking students of Muslim minority for Activity 8

Figure 3 :
Figure 3: Node typology of Greek students for Activity 9

Figure 4 :
Figure 4: Node typology of Turkish-speaking students of Muslim minority for Activity 9