A study of value judgements in evaluations of undergraduate programs in Argentina

The evaluation of the quality of higher education is a key issue involving a diverse array of factors, a fact which necessitates a revision of the accreditation policies and systems already systematized and embedded in Argentina. Accreditation of undergraduate programs consists of recognizing that an academic program meets certain basic quality standards and criteria previously established. This research analyzes the makeup and creation of accreditation standards for Biochemistry programs in Argentina. The present research is exploratory, qualitative and quantitative, and it is of a descriptive-interpretative nature. A documentary analysis of the regulation that approves the accreditation standards for this program was carried out.The results demonstrate differences in the makeup of said standards in terms of the number of objects or components of evaluation and the number of variables associated with each one of these. As regards their creation, regulation standards comply with construct, measurement and formal requirements to a lesser extent than in their disaggregate form as objects and variables of evaluation.


Quality, evaluation and accreditation in higher education
In the field of education, quality includes the evaluation and accreditation of an academic program, among other aspects.As such, quality can be defined as the recognition that an academic program receives in relation to basic standards and criteria previously established.Therefore, procedures for ensuring quality are mechanisms which guarantee that an institution, an academic program or the like fulfills the minimum markers established (Salas-Perea, 2000).
The evaluation of the quality of higher education is a key issue involving political, social and economic factors, a fact which necessitates a revision of the accreditation policies and systems already systematized and embedded in Argentina (Corengia, Del Bello, Durand & Pita, 2013).
It has been shown that an analysis of accreditation decisions for academic programs put up for accreditation, focusing on each program's fulfillment of its commitments, its actions carried out to correct deficits identified and its substantial improvements, has had a positive impact on quality improvement (Filippa, Lopez & Reznik, 2015).
Nevertheless, it has also been observed that the job of those who carry out processes of evaluation and accreditation based on standards is a difficult one, given their complicated management and systematization (Sanchez-Quintero, 2014).
A study completed by Atario et al. (2014) analyzed evidence of quality improvement at Argentine universities by studying evaluation reports.Their work demonstrated inconsistencies related to the observations of different evaluators, repetitions of judgements issued and the inclusion of new criteria in accreditation decisions (Atairo et al., 2014).
The revision of accreditation procedures is being implemented by several institutions.These institutions have come to the conclusion that these procedures must be universal and uniform so as to prevent inequalities in the evaluations of professional education.For this reason, correcting possible deficiencies in accreditation processes is very relevant (Rosselot, Mateluna, Estevez & Radrigan, 2013).
At the same time, evaluation and accreditation systems imply the existence of: systems used to search markers which are systematic and pre-planned; mechanisms for registering and analyzing data which are safe and reliable; a series of criteria or elements upon which the evaluator is able to compare the markers of the object evaluated; value judgements which are constructs of an evaluation, different from descriptions or proposals; and decision-making which leads to the improvement of the object evaluated and which is not always given the importance it deserves or taken into account after an evaluation procedure (Toranzos, 2010).

Value judgements
The value judgement issued in an accreditation decision is a declaration of the quality of an academic program and is the result of a reflective analysis of the information and history provided by the program.What is most important is that the decision be dutifully founded on clear and precise information and markers so that another evaluator may subsequently confirm its validity (Manual of accreditation rules and procedures, 2008).
The value judgements must be objective, given that they are generally held to be true, a guarantee of objectivity.If, therefore, these value judgements tend towards the subjective, they will be considered neither true nor false, and will consequently be unfair (Ortiz-Millan, 2013).
A judgement of the quality of an academic program up for accreditation should be considered a fundamentally qualitative process, based on an integral analysis of its dimensions and components by way of quality markers and criteria.Likewise, results of the evaluation and value judgements issued should always be discussed with both the authorities and others related to the academic program so as to foster debates which will allow for strengths and weaknesses to be identified, quality to be verified and personal and institutional commitments to be established, all seeking to continually improve the quality of the academic program (National Agency for the Evaluation and Accreditation of Higher Education, 2007).

Methodology
In accordance with its depth and objective, our research is exploratory, qualitative and quantitative, and is of a descriptive-interpretative nature (Hernández-Sampieri, 2010).
In its first phase, an analysis of the distribution of value judgements for the eight undergraduate accreditation decisions included in the study was carried out.The number of value judgements and their correspondence to the number of evaluation objects and variables to which they made reference were also determined.In the second phase, we studied the judgements' form related to their linguistic and pragmatic features.Of the eight accreditation decisions, four belonged to state universities (su) and four to private universities (pu).
To study the distribution of value judgements in these decisions, we used the Atlas ti program.For each judgement, we identified its belonging to each of five defined dimensions in function of the object of evaluation and variable to which it made reference.
We coded the judgements by placing a capital "J" in first position; followed by a correlating number, beginning with the first judgement found in the decision as number one "1"; after this number, an acronym denoting the dimension Institutional Context "CI", Program of Study "PE", Academic Body "CA", Students and Graduates "AG" and Infrastructure and Equipment "IE"; and finally the letter of the decision in which the judgement had been performed (A; B; C; D; F; G; H or I).
After coding the judgements, an analysis of the distribution of the total number of judgements in decisions belonging to private universities (pu) or state universities (su) was carried out.
Additionally, we determined the total number of objects of evaluation (oe) and nominal variables (nv) presented in the accreditation standards which were evaluated or not evaluated using value judgements.
The characterization of value judgements implied recognition of certain discursive linguistic features and pragmatic features.The former include the analysis of the discursive plot, value taxonomy, value markers and syntactic structure.Pragmatic features were analyzed using the markers entitled observation and brevity (Gonzalez-Rodríguez, 2011).

Analysis of value judgement distribution in accreditation decisions
According to our analysis of accreditation decisions for Biochemistry programs, both from su and pu, three hundred ninety four (394) value judgements were observed, with one hundred sixty eight (168) belonging to pu decisions and two hundred twenty six (226) belong to su decisions, for an average of 43% and 57% respectively.
Of a total of four hundred thirty two (432) oe to be considered in the eight accreditation decisions, two hundred twenty five (225) were evaluated using value judgements, a number which corresponds to fifty two percent (52%).As relates to nv, of a total of one thousand four hundred forty (1440), only four hundred were evaluated, representing twenty eight percent (28%).

Discursive plots
With respect to the discursive plot indicator, of the total value judgements, the most frequent according to the predominant plot were those characterized as descriptive (expressing happenings and changes in space) (44%), the second most frequent being argumentation (textual expressions creating relationships between concepts or statements) (39%) and a lesser number corresponding to explicative discourse (textual expressions regarding the arrangement or breakdown of conceptual representations) (17%).

Linguistic markers of value
The analysis of this dimension and in relation to linguistic markers of value demonstrated that the majority of judgements were characterized as being attributes (69%) rather than subjectives (31%).

Value taxonomy
Results related to this marker demonstrated that the predominant characteristic in these judgements was their being binary (77%).That is to say, the judgements' lexicon expresses a positive or negative valuation, or more absolute semantic oppositions, as for example adequate/inadequate or not adequate.The second most prevalent characteristic was that of polar judgement (22%), when the judgement allows for a continuous spectrum between two poles or extremes, a spectrum which shows that an intermediate zone can indeed exist, varying according to the object being classified.
Finally, only 1% of the judgements presented the linguistic marker of relative valuation, those that imply a different direction in relation to a certain term (far/near; before/after).

Syntactic structure
An analysis of the total judgements showed that most were complex (with simple structures coordinated or subordinated to incorporate new information for diverse purposes, to clarify, justify, argue, etc.) (59%), whereas fewer were simple (with only one prototypical structure: subject, verb and object) (41%).

Observation
The results indicated that the majority of judgements gave partially observable information (55%), whereas fewer gave observable information (45%).

Conclusions
After analyzing all of the decisions, we were able to determine that some objects of evaluation and nominal variables were evaluated on more than one occasion by different value judgements, both in the same decision and among various decisions, while others were not evaluated at all in the decisions analyzed.
From data obtained, we infer that value judgements make reference to very few of the different categories that make up the standards.In the case of oe, only forty seven percent (47%) were evaluated, with more being evaluated in su decisions than in pu decisions.In terms of nv, only twenty four percent (24%) were evaluated, again with more being evaluated in su decisions than in pu decisions.
As regards the characterization of value judgements, the dimension of discursive linguistic features considered value judgements as textual units of analysis, focusing on elements of meaning (lexicon, syntax, phonetics, pragmatics) that constitute a semantic whole.This dimension includes classification according to plot, demonstrating that the majority of judgements analyzed were descriptive, as the evaluator limited himself to simply characterizing the situation presented in the parameters.The description produced consists of a mere repetition of the parameter with the addition of valuation, for example "low."In this case, the construction presents the state verb 'be' which indicates a static, fixed and predetermined situation, without the possibility for change.
The low frequency of argumentative and explicative plots highlighted the evaluators' lack of specific positioning supported with arguments and/or explanations that give way to possibilities for the transformation of the reality evaluated.
The analysis of linguistic markers of value demonstrated that the majority of judgements were of an attributive type as opposed to being subjectives.This means that most judgements were more prototypical as they presented this feature of attribute, referring to the selected object's own characteristics.They differ from the subjectives which refer to personal and subjective assessment of judgements.It is important to highlight the need for objectivity in the issuing of value judgements as the results showed a marked personal assessment, whereas on the contrary, judgements should focus more on the object evaluated than on the evaluator's partial valuation.
As regards this marker, the judgements tended to use binary taxonomies that were dichotomous and exclusive.
Although judgement in itself presents a range of values, most being subjectives, most were binary: sure/unsure or reliable/non-reliable, for example.There were fewer cases where valuation was able to be placed on a polar continuum as part of a scale that allowed for clearer placement of the value, as with the example: "low." In this case, we see that the term "low" is part of the high-medium-low scale which is stronger semantically and more closely approaches the idea to be expressed.
The values are open to different possible interpretations depending on taxonomy type.The more binary a relationship, the more decisive and closed it becomes.This causes a sense of statism difficult to modify without alternatives.On the contrary, polar taxonomy tends to open the spectrum evaluated to change.
As regards the judgements' syntactic structure, simple forms were those in which only one prototypical structure can be identified: subject, verb and object.However, complex forms in which simple structures coordinate or subordinate incorporating new information for diverse purposes, to clarify, justify, argue, etc, were the judgements' predominant syntactic structures.
Upon analyzing the formal requirements, those markers referring to brevity (judgement must be expressed with semantic clarity and without ambiguities, briefly and precisely) and observation (data backing the judgement are more or less observable), the results portrayed that the majority of judgements issued were brief and imprecise, with a minority being extensive and precise.
In conclusion, we infer that if the evaluation and accreditation systems were more uniform and homogenous, value judgements could be more argumentative, simple, brief and precise, and less descriptive, complex, imprecise and subjective.