Positive , negative or neutral ? The “ appraisal ” variable in content analyses studies of the media

This article interrogates the assumption that media content analyses based on mixed-method research are free of contradictions. We argue that when qualitative and quantitative methods are used simultaneously, the different research paradigms applied in the process cannot be reconciled in a bid to produce consistency. In order to do this, we review in this paper the findings and methodology of a project funded by the EU Fundamental Rights Agency on the coverage of migrants and minorities in the UK press. Focusing more specifically on the codification and interpretation of the Appraisal variable used in the project, we investigate how its subjective definition and challenging quantification are indicative of some of the problems of consistency accompanying the intersection of qualitative and quantitative approaches to content analysis. In this sense, we do not dismiss the potential of mixedmethod research to deliver insightful findings, but warn against the mechanical application of cross-paradigmatic approaches, and argue that the gaps and inconsistencies exposed by different paradigms can reveal more about the ambivalence of media representation than their uncritical synchronisation can. Key-words Mixed methods. Content analysis. Journalism. Human rights.

four main reasons: they define philosophical assumptions and stances, frame inquiry logics, work as guidelines for practice, and limit sociopolitical commitments (JOHNSON;ONWUEGBUZIE;TURNER, 2007).Discussing the importance of mixed methods and research paradigms in political communication studies is not the same as considering mixed methods as a distinctive methodology inquiry, because they are not (GREENE, 2008).Nonetheless, understanding how qualitative and quantitative methods are combined can be crucial to answer simple questions of the field of political communication, such as: is media coverage overall in favour or against a minority groups?
Or more specifically, how are migrants and minorities appraised in the press?
This article starts and ends with a simple but still puzzling epistemological question: How do we ascertain the validity of the findings of a mixed-method content analysis of media representations?How reliable is such a crossparadigmatic approach, and is there a way in which the limitations of post-positivism (quantitative analysis) and constructivism (qualitative analysis) can be overcome or at least intersected engendering a productive, even if not comprehensive or absolutely flawless, dialogue?In this sense, our argument addresses questions raised by Husband and Downing (2005, p. 20) about the need to interrogate the assumed compatibility between research method and question: We must remain self-consciously aware of how specific research questions attract particular theoretical paradigms and tend to pre-empt the choice of research method.Consequently, we must question the adequacy of the method for the research question.We must look beyond the pragmatic focus of the specific research project.And we must seek an inter-disciplinary sophistication in learning how to appropriately articulate different research inputs into a coherent and legitimate synthesis.
Whilst the quest for alternative theoretical paradigms and interdisciplinary methodologies extends the scope of research outputs beyond the purely empirical findings, it also draws attention to the inherent problems of mixed-method research.
If the "legitimacy" of a research methodology is to be ascertained against its openness to varied epistemologies, it is equally important to be alert to the susceptibility of such methodological integration to incoherence and misinterpretation.
In our discussion of mixed-method research, we have focused on one particular variable, that of Appraisal, which has been a major methodological preoccupation in content analysis research (POOLE, 2002;RICHARDSON, 2004).
Our argument is that the Appraisal variable can be a productive way of intersecting quantitative and qualitative analytical approaches.However, the inherent subjectivism of the term, as well as the semantic and pragmatic difficulties its quantification poses, makes this initial felicitous interpretation of the variable somewhat optimistic.This article, therefore, outlines the pitfalls of the Appraisal variable, in view of its potential to reduce complex issues of representation to simplistic binaries when codified without sufficient discursive analysis.Finally, we propose some solutions to these problems, arguing not so much for the dismissal of the Appraisal variable but for its judicious application.Likewise, we do not dismiss the potential of mixed-method research to deliver insightful findings, but warn against the mechanical application of crossparadigmatic approaches, and argue that the

3/21
There are some pre-existing limitations in the FRA pilot per se that we will not address here.Firstly, it is well-known that content analysis as a method has its own limitations.It ignores aspects of the meaning making process such as social institutions of discourse production and consumption, focusing instead on the analysis of the message itself.In other words, the pilot methodology circumscribes the fact that 'in the media commonality is not only imagined, but also negotiated' (GEORGIOU et al., 2007).Secondly, the project is restricted to the analysis of national media, ignoring the presence of niche media and regional media, where minority media are usually consumed and produced (ALIA; BULL, 2005).Thirdly, the analysis of content as an

Mixed-method research
As Cottle aptly argues, a considerable challenge in minority studies and their relation to the media is understanding "what methods best serve to analyse media representations and uncover how they circulate messages and generate meanings" (COTTLE, 2006, p. 7).This means that understanding 'the complex ways in which media are often implicated within conflicts while disseminating ideas and images about them' (8) depends on the method used to frame the analysis.The FRA pilot project used a mixed-method approach, combining quantitative analysis (i.e., coding all articles about minorities or migrants in terms of a set of variables such as minority group, religion, ethnicity, minority issue, headline, newspaper section, thematic area, etc.) with qualitative analysis (i.e., coding the range of attributes and terms used to refer to minority groups).The limitations of both methods are mostly found in their comparison to one another: quantitative variables (e.g., 'number of mentions of a minority group') cannot be used to qualify the context in which a specific minority group is referred to; and the qualitative list of terms used to refer to a specific minority group cannot be counted in any representative way to indicate with precision the frequency with which the group is represented in the entire sample.
Ideally, the two databases should complement each other.The problem, however, is that the mixed-method approach does not consist of merely adding up the two datasets, but rather invites the scrutiny and reconciliation of the discontinuities between the two research paradigms, which roughly correspond to positivism / postpositivism and constructivism / interpretivism (MORGAN, 2007).Thus, while the number of mentions of a specific minority group refers in a nutshell to "the positivist notion of a singular reality, the one and only truth that is out there waiting to be discovered by objective and value-free inquiry", the qualitative analysis of a list of terms and attributes used to describe a minority group refers to 'the idea that there is no such thing as a single objective reality and that "subjective inquiry is the only kind possible to do" (ERLANDSON et al., 1993, p. xi apud FEILZER, 2010).Mixed-method analysis attempts to bridge the gap between these two opposite research conceptualisations (e.g. the two datasets) by taking account of the different premises from which they operate.This should allow for a research method that not only focuses on the problem to be researched and the consequences of the research, but also Whilst registering and quantifying various elements that construct the semantic macrostructures (DIJK, 1988, p. 17  as well as some rhetorical operations (irony, puns, similes, etc.).Furthermore, the Appraisal variable, as applied to the quantitative and qualitative datasets, could not be useful for the analysis of what van Dijk has called "global structures" (DIJK, 1988, p. 17) -in particular, news schemata -which affect and effect the cognitive scripts and models at the core of audiences' reception of media messages.
As already mentioned, the extratextual analysis -of cognitive (of the participants in the communicative event), pragmatic (of the speech acts) and social (of media production and consumption) factorswas tangential to the FRA project; however, textual elements which are functional in the cognition and reception processes should not be ignored.As van Dijk points out, the 'rather subjective analysis' of media opinions (van Dijk's case study is specifically editorials, which by generic conventions are almost never value-free) needs "to be complemented with more subtle analytical methods, involving the description of argumentative structures, the explication of presupposed (tacit) assumptions, norms and values, and an analysis of style and rhetorical devices."(DIJK, 1988, p. 126-7).
It is in the 'description', 'explication' and 'subtle analysis' of media opinions that the mere quantification of the Appraisal variable we have performed does not succeed as intended.

Appraisal in quantitative analysis
The FRA project monitored for four months the main newspapers in six EU countries: the UK, that religious minorities are a central issue in the UK press coverage (Table 1).Of the total number of articles mentioning any Actor/Speaker with a MMG background (543), more than half (52%) identify at least one Actor/Speaker by their religion.This is valid for all the newspapers analysed (ranging from 44% in The Sun to 61% in The Mirror), as well as when analysed by paper type and by political orientation (which in all cases is just over half of the articles).Considering the ratio between articles identifying confessional identities, articles 2 Index A (column 6) is calculated by the "% of MMG articles with No religion" (column 5) divided by "The total % of articles with MMG" (column 2).Index B (column 7) is calculated by dividing the "% of MMG articles with Muslim background" (column 3) by the "% of articles with Non-Muslim background" (column 4), subtracted by 1.

Appraisal in qualitative analysis
The qualitative dataset of the FRA project recorded the attributes and terms used to refer to MMGs in the newspapers covered by this study.Given the qualitative aspect of this part of the project, the team did not count the number of occurrences of a specific term used with respect to a MMG in the same article.
Furthermore, terms that were significantly similar were coded only once, e.g., "Eastern European" and "Eastern Europeans".This has enabled the team to record the range of vocabulary used to represent minorities and migrants, compare across different newspaper editions and historical periods.
A distinction was also made between "Implicit" and "Explicit" terminology, which has provided    for the FRA project failed to take into account.
Focusing on one media event only will enable us to trace sequential semantic elements, which can help one deconstruct the narratives media In both The Guardian and The Telegraph entries the exclusion of the Muslims from the British national identity is obvious through the use of personal pronouns that identify "belonging": referred to "the very people whose loyalty to Britain we most need to win" and "some people in this country who are British subjects" (Guardian,

Conclusion
The aim of this article is to offer some critical
offer a wide range of methods, measures and analytical techniques to generate and evaluate findings in the field (BUCY; HOLBERT, 2011; CHADWICK; www.e-compos.org.br| E-ISSN 1808-2599 | 2/21 gaps and inconsistencies exposed by different paradigms can reveal more about the ambivalence of media representation than their uncritical synchronisation can.To this end, the article discusses the findings of a pilot research project commissioned by the EU Fundamental Rights Agency on the press coverage of minorities and migrants in the UK and five other EU countries for the period February-June 2008.The research output included two extensive datasets (quantitative and qualitative), and it is the combination of these databases (and the findings) that we focus on.The sample used in this article refers to the Muslim minority and migrant group in the UK.We have little to add to the debate on what constitutes 'good' or 'bad' media practices in the representation of minorities and migrants (which is related to the normative direction in policy analysis); instead, our objective is much simpler: to discuss the validity of the findings of the FRA project and the viability of its methodology in view of the complexity of the Appraisal variable.
explicit and rational message is likely to overlook the 'symptomatic reading' of content (HUSBAND; DOWNING, 2005), obfuscating ironic and figurative representations of migrants and minorities.At the same time, it is important to acknowledge that the FRA pilot has systematically provided empirical data about the coverage of minorities in the European press.It has equipped us with valuable cross-temporal and cross-country indexes and sources of research that will advance the comparative study of media and policy systems (HALLIN; MANCINI, 2004).Its design has been left deliberately open for input from different country teams so as to accommodate the inevitable subjective element of the media research process, reflecting the cognitive maps and public imaginaries of the participating research teams.In this sense, we wholeheartedly and unreservedly support the intent and purposes of the pilot.Monitoring the role of minorities and migrants in the European Union media is for certain a necessary task, which we think should become a permanent project.Our contribution aims to interpret the findings of this project as much as to contribute productively to future discussions about the applicability and reliability of mixed-method research in the analysis of media representations of migrants and minorities.
www.e-compos.org.br| E-ISSN 1808-2599 | demonstrates that by "looking at phenomena from different perspectives [...] [it can] provid[e] an enriched understanding" ( JUCK, 1979 apud FEILZER, 2010, p. 9).The FRA decision to code in nine different ways the Appraisal variable is, in our opinion, such an attempt to bridge the conceptual gap between these two paradigms. 1Such an elaborate codification has several advantages.Firstly, it quantifies the overall perspective of the particular newspaper with respect to specific minority groups or related issues, through variables such as Overall Appraisal from the quantitative dataset and Article Connotation from the qualitative dataset.Secondly, it quantifies the perspectives of different participants in the media event through variables such as Appraisal of Actors (coding how Actors have been appraised) and Appraisal of Speakers (coding how Speakers appraise minority related issue/actor).Thirdly, it enables the breaking down of the Overall Appraisal of the article into its component parts and the quantification of the appraisal of these microelements through variables such as Appraisal of Actors, Appraisal of Speakers and Appraisal of Issues.Fourthly, it quantifies the appraisal of minority groups and minority issues present in the headline and the leads, and how this relates to the tenor of the entire article through the variable Headline Appraisal and Overall Appraisal.In addition to these processes of quantification, the use of the Appraisal variable in the qualitative dataset enables the registering and partial analysis of appraisal by coding the connotation of the individual words used to refer to a minority (General Connotation) and the overall connotation of the representation of the same minority in the article (Article Connotation).In this way, the Appraisal variable in the FRA methodology tries to bridge the gap between the seemingly 'value-free' quantification of one dataset and the subjective interpretation of the other not only by quantifying different appraisals at the lexical, syntactic and textual level, but also by deconstructing the facile uniformity of the notion of 'Appraisal' and taking into account diversity of perspectives and levels of agency, as well as visibility in the media and complexity of discourse analysis.

1
) of the analysed articles, the different levels of the Appraisal variable of the FRA methodology have not been that useful in effectively identifying and critically analysing some sequential elements of news microstructures, e.g., syntax (word order and sentence structure) and text grammar (coherence and cohesion), referential and predication strategies (REISIGL; WODAK, 2001), Out of these nine, seven Appraisal variables are used in the quantitative dataset (Overall Appraisal, Headline Appraisal, Appraisal of Speakers, Appraisal of Actors, Appraisal of Issues, Image Connotation and Caption Connotation) and two in the qualitative one (Article Connotation and General Connotation).
France, Germany, Poland, Spain and Hungary.From February to June 2008, three days of coverage were selected for coding, each one including four newspapers in each country.All articles of each edition were coded based on the presence of any mention of a Migrant or Minority Group (MMG) in the headline or body of the text.After the end of the coding period, the six countries coded a total of 42,951 articles, a volume that decreased to 31,155 once the decision was taken to drop the data for the first month of the project (February 2008) in order to improve the compatability and reliability of the findings.Of the 31,155 articles coded in all countries, 6.9% (2,164) mention a MMG.The UK is the country with most articles coded in the project (543 out of 7,092 articles coded mention a MMG, or 7.7% of the total sample).Analysing the dataset by type of MMG religion, the FRA sample confirms what other authors (COTTLE, 2006; HUSBAND; DOWNING, 2005; RICHARDSON, 2004) have previously identified: www.e-compos.org.br| E-ISSN 1808-2599 | with Muslim MMGs appear 70% more frequently in the sample than articles identifying other, Non-Muslim religious background.Although in some newspapers the ratio is negligible (e.g., 8% in The Guardian) suggesting different religious identities receive comparable exposure, in Tabloids the trend is reversed (240% in The Sun, and 300% in The Mirror) with Muslim MMGs being explicitly identified up to four times more frequently than other confessional groups.Two conclusions can be reached at this point: Firstly, religion is a central issue in the UK press; and secondly, when religion is mentioned, Islam is by far the most important religious background referred to.But how does this high level of visibility of Muslim groups in the media translate in the appraisal of the Islamic religion?We address the question in this section by assessing the advantages and disadvantages of a contentanalysis project which has adopted quantitative variables to reflect the appraisal of MMGs.The FRA project breaks down the task of measuring appraisal into five different quantitative variables, each one referring to a different level 7/21

5
clues about the frames of reference available to the newspaper readers and to what extent newspapers rely on such forms of received knowledge in order to convey their message.Finally, we distinguished between the connotation of the individual word used to refer to a MMG (General Connotation) and the connotation of the overall representation of this MMG in the article (Article Connotation).This distinction in what is effectively two kinds of Appraisal has made possible the comparison between the meanings conveyed by non-sequential structures (the semantics of individual words and nominal phrases), and representations complemented by the sequential and "global structures" (DIJK, 1988, p. 17) of the article (syntax and text grammar, style and rhetoric, argumentation and news schemata).The discursive element in Article Connotation 5 , perhaps not as reliable or quantifiable as a numerical representation, breaks down the multi-layered nature of media appraisal.Moreover, the Article Connotation variable from the qualitative analysis is more specific than the Overall Appraisal variable from the quantitative analysis, as it conveys the media appraisal of one specific MMG, whereas the Overall Appraisal variable from the quantitative dataset addresses the appraisal (by all parties) of all MMGs and MMG-related issues referred to in a given article.In this way a degree of specificity is achieved in the qualitative analysis, whereby researchers can compare media attitudes towards different MMGs within the same article or towards a given MMG across different articles, newspapers and countries.The shortcoming of this qualitative methodology is that similarly to the quantitative analysis, the discrepancy between different levels of appraisal (General Connotation and Article Connotation) was merely observed and noted down.The methodology of collecting terms in lists does not necessarily give a clear idea of how -linguistically, stylistically and rhetorically -a newspaper redefines the semantics of an individual term into something different at the level of the text.Possible avenues for further exploration include cultural and other narratives used to frame MMGs, Article Connotation was coded not simply as a numerical value but had an explanatory discursive side to it, which summed up in a sentence how an MMG was represented in the article overall.

21 6
logical structure and argumentation of news discourse(EEMEREN et al., 1997); and syntax, style and rhetorical devices(CAMERON, 1996;   JUCKER, 1992).Although, the Ambiguous value of the Appraisal variables is meant to accommodate non-literal meanings (especially in puns, irony, sarcasm, etc.), its vagueness allows for the conflation of too many different strategies.TheMuslim MMG, as mentioned in the quantitative analysis earlier, was significantly more visible than other religious identities in the UK press coverage throughout the duration of the FRA project; the MMG was mentioned in ALL issues of ALL papers for the period under consideration.The total number of terms used with reference to the Muslim MMG 6 is the second highest with 570 entries (with terminology about the Majority group coming first).At the individual word level (General Connotation), the terms used to refer to Muslims tend to be neutral or negative.The latter usually invoke terrorism, religious fanaticism and militancy, and in fewer cases moral depravity.With positive connotations, the MMG is framed within the discourse of extremism again, but this time as a direct or indirect victim of terrorism.At the level of the text (Article Connotation), there is more ambivalence.Positive connotations tend to represent Muslims as a community targeted by Western military interventions (the conflicts in Iraq and Afghanistan) or Western racial and religious prejudice (especially in the UK context), as well as a community torn by internal disputes (between Sunni and Shia, progressive and conservative, moderate and extremists).Outside the 'war on terror' discourse, Muslims are also represented as a religious community of peace and multicultural tolerance, as a group suffering the oppression of regimes in China, Libya, and the Middle East, and within the UK context as a community suffering from economic deprivation and targeted by institutional and communal discrimination.This image of victimisation, however, is outweighed by the negative representation of Muslims as dangerous extremists and terrorists, as a selfsegregated community unwilling to integrate in the host society (UK context), as a threat to Western values and other religious communities (mainly Christians and Jews), as a conservative and misogynist ethnic group.Similarly, Islam is discussed as a faith potentially hostile and violent to unbelievers, but also as a system that can be taken advantage of both by fundamentalist preachers/terrorists and by Western political leaders (for election campaigns, for example).The analysis of Appraisal of Issues in the quantitative sample supports the findings above.Among the top ten issues from the 65 minority 12/In the qualitative dataset on the Muslim MMG our research team included not only terms that explicitly designated an individual or a group as Muslim, but also references to Iraqis, Palestinians, Pakistanis and Arabs.In this sense, our approach differs from the one used by ElizabethPoole (2002), insofar as media operate by suggestion and association, and even when an explicit designation of a MMG is absent, readers are invited to make the cognitive leap and identify an individual as minority by certain visual or verbal clues present in the public imaginary.

21 7
issues coded, "Terrorism with minorities as (presumed) threat or perpetrators" is an issue almost exclusively associated with the Muslim MMG (N=96 out of 104) and tended to be appraised negatively (index of -0.71).Muslims are positively appraised in relation to "Discrimination by public bodies & institutions" (0.44), which is an index lower than that for the Appraisal of Issues related to Non-Muslim MMGs (0.81), but still higher than the Appraisal of Issues related to Nonreligious MMGs (0.32).The findings from the quantitative and qualitative datasets largely reiterate the conclusions drawn by earlier studies of media representations of Muslims in the western press(KARIM, 2003;   POOLE, 2002; POOLE; RICHARDSON, 2006;   RICHARDSON, 2004).Further to that, however, we argue that the qualitative methodology of the FRA project highlights the discrepancy between appraisals at a micro (individual word) level and those conveyed at the macro level of the text, thus corroborating our earlier conclusions about the incompatibility of the different Appraisals in the quantitative analysis.In the rest of this paper, therefore, we will both narrow down the scope of the FRA qualitative analysis by discussing the coverage of a single media event that involved the Muslim MMG, and expand on the analysis of elements that the qualitative dataset designed 13/Médias com a notação * se referem a células nas quais N < 10 artigos.

21 8
build around Muslim MMGs (primarily through thematic clusters), as well as the argumentation and referential structures used to affect readers' cognitive maps.This can be seen as one of the possible solutions to the methodological challenges facing the Appraisal category in content analysis -the need not only to quantify it at the micro level of Appraisal of Speakers, Appraisal of Actors and Appraisal of Issues, but also to explicate and analyse it in a contextualised and diachronic manner.5 Unpacking 'Appraisal' in contextFor the purposes of this article, our analysis addresses a major controversy in UK political life in 2008: the debate surrounding the 42-day pre-charge detention bill.8Whilst most of the print media coverage of the debate focused on Gordon Brown's political manoeuvres, the Muslim MMG were not left completely out of the limelight.Just about.For what the FRA qualitative tables did not capture is the cameo presence that the minority group had on pages and pages of analysis of the erosion of civil liberties.If quality newspapers mentioned the Muslim MMG in the context of the perceived violation of civil liberties by the Labour government, the tabloid press tapped into public fears of terrorism.And even though terrorism is not explicitly linked to the Muslim MMG in the UK, the fact that the newspapers (with the exception of The Mirror) elsewhere discuss the bill with respect to 'disaffected' British Muslims help their readers perform the cognitive leap of associating terrorism with Islam.This demonstrates the need to include in the qualitative dataset not only explicit references to the Muslim MMG, but also less direct ones (in contrast to POOLE, 2002), given that media work by suggestion and mould their readers' cognitive maps through associative links.If we look at the terminology in articles that refer explicitly to the Muslim MMG, the sample confirms the findings of the project as a whole.At the level of the individual word (General Connotation), most of the terms are neutral, with explicit positive and negative terms used rather infrequently.The positive attributes define the Muslim MMG as successful in their host society ("top Muslim"), whereas the negative ones allude to possible terrorist affiliations ("terrorists", "Islamic nutters").The picture at the textual level (Article Connotation) is more varied.When it comes to positive Article Connotation, the group is seen as a victim of past and new counter-terrorism legislation: "Britain has become for these men not a refuge but Kafka country" (Guardian 10/06).The right-wing press, on 14/After a heated debate, the House of Commons voted in favour.This slim victory of the Labour Party Anti-Terrorist legislation (of mere 9 votes) was short-lived and the House of Lords subsequently returned the 42-day bill.www.e-compos.org.br| E-ISSN 1808-2599 | the other hand, acknowledges the place of the Muslim community within British society as hard-working and respectful of liberal values: "The Muslim community will do all it can to protect our society and our values" (Sun 10/06); "The British Muslim Forum said they were reassured after speaking to ministers that safeguards in the new legislation would protect civil liberties" (Telegraph 10/06).This, somewhat surprisingly, has led to the appeal of The Sun to its readership to listen to the Muslim voice for the endorsement of a rather unliberal bill: "In the furore over 42-days detention for terror suspects, we had yet to hear the authentic voice of the Muslim community" (Sun 10/06).Similar to the left-wing press sample, Muslim victimisation is present; however, the minority are discussed as victims of terrorist attacks, not of discriminatory legislation: "He says London's 7/7 bombing outrage taught Muslims they are as likely to be victims of extremism as anyone else" (Sun 10/06); "As we saw in the July 7th attacks, Muslims are just as likely to be victims of these plots as any other British people" (Telegraph 10/06).What we need to keep in mind when analysing the sample is that despite the positive Article Connotations of Muslim MMGs, the appraisal is more ambivalent than that.Firstly, this is due to the limited space Muslims are given in these articles; as already mentioned, most of the accounts focus on intra-and inter-party squabbles over the erosion of civil liberties, political manoeuvres of back-benchers, and the balance of 15/21 legislative and executive powers in the UK.Except for one article dealing with Muslim detainees' art and one letter to the editor, The Guardian -the staunchest left-wing supporter of liberal values and civil liberties -did not merit the minority with anything more than a one-sentence paragraph in its otherwise comprehensive discussions.At the same time, the extended analyses of Muslim MMGs in articles in the right-wing press (and the positive spin present in some of them) should not be misleading: representatives and even speakers from the minority group are used to drive the point home that the UK needs curtailment of civil liberties.Secondly, if one considers the content of the positive appraisals in the press, they raise some interesting questions too.Victimisation of the Muslim MMG seems to be a recurrent theme, discussing the group as either victims of unfair legislative acts (left-wing press) or as victims of Islamic terrorism (right-wing press).This doesn't necessarily represent the group as an agent empowered to do anything other than turning to illegitimate methods of resistance.Furthermore, the persistent designation of the Muslim MMG as a distinct group within British society implies incomplete integration; The Sun in particular spells out the need to listen to an "authentic" Muslim voice, even though its headline ("UK's top Muslim") might suggest a high degree of social cohesion.When it comes to the negative Article Connotation, the situation reiterates the findings of the project as a whole.Muslims feature as (potential) terrorists in both Broadsheets and Tabloids, and in the press of the entire political spectrum: "Our liberties are under threat from two sides.They are threatened by terrorists, especially takfiri jihadist ones, exploiting new technologies and an open society in order to kill, maim and terrify the innocent."(Guardian 12/06); "Terrorists plotting a suicide attack against the British Embassy in Denmark were arrested in September as they put together a huge bomb.The men were believed to be the remnants of a group co-ordinated by London-based Islamic militants using a series of secret internet sites" (Sun 10/06).The left-wing press is clearly more concerned with civil liberties at large, with The Guardian commenting on civil rights violations at home and abroad (but mainly Muslim countries): "deportations to countries that practise torture -Algeria, Libya and Jordan" (Guardian 10/06/08).Thus, even though Muslims tend to be seen in Guardian analyses as the victims of civil rights violations, they are also the litmus paper with which to gauge the extremity of the violation ("takfiri jihadist terrorists" and "countries practising torture" being the extreme).Being much more explicit and sensationalist, The Sun relies on instilling paranoia among its readers; featuring only one entry in the qualitative dataset ("terrorists"), the newspaper dedicates a double spread to Islamic terrorism, conflating it with all kinds of international terrorist acts in order to sway the public in favour of the counter-terrorism bill -"42 Reasons Why Britain Must Have 42 Days to Detain Terror Suspects" (Sun, 10/06).The dearth of synonyms used to designate the Muslim MMG is counterbalanced by the rhetorical strategy of the paper -listing in a seemingly factual manner numerous examples of violence from modern history -which overwhelms the reader in a bid to convince them of the presence of "terrorists" among us.Working in conjunction with the enumeration device that universalises various historical phenomena is the rhetorical strategy of differentiation.Whilst the British Muslim MMG is represented as alienated and divisive, the authors of these negative representations deploy lexical and syntactical structures that confer the right to belong to some and deny it to others: "Could it have anything to do with the fact that we have some people in this country who are British subjects, but who are spending a lot of time plotting the overthrow of our government and institutions by violent means?" (Telegraph 14/06).In its emphasis on Muslim exclusion from the national "we", this comment resounds with one of the few entries in the qualitative table that has ambiguous Article Connotation of the Muslim MMG: "And the home secretary says this won't contribute to the radicalisation of already deeply disaffected young Muslims" (Guardian 12/06).
), Muslims are not part of the British "we."Being the Other within, if not an absolute threat Muslims are definitely seen as a weakness to "our" "liberal" social fabric.Thus, for all their political differences both newspapers seem to tap into the same preferred national narrative of Englishness: white, Christian, European.The only exception in this exclusionary treatment of the Muslim MMG in this sample is in the "Letters and Emails" section of The Guardian from 10/06.The author discusses Muslims on a par with other dissenting groups such as environmentalists, thus avoiding the essentialist trap of seeing the minority group in terms of their ethnicity or religion only.He says: "The current Terrorism Act has already been used against Muslims, environmental protesters and elderly Labour Party conference delegates."But then proceeds to include Muslims in the "we" of those disenfranchised, or potentially disenfranchised by UK legislation: "If supporters of the bill feel confident of not being mistaken for these people [Muslims, environmental protesters and elderly Labour Party conference delegates], then perhaps they might reflect on how comfortable they feel that there are so many among us who, when afraid, are willing to give up others" liberty'(Guardian, 10/06).Drawing attention to the elusive boundaries between "us" and "them", and thus to the ambivalence of racial, ethnic and religious markers, the author invites sympathisers of the bill (and he is again careful not to define them in ethnic, gender, class or party terms) to empathise with the Other.He also highlights the potential complicity of everyone in "our society", which has been ridden by paranoia; through this argumentation strategy the author is invoking the ethical responsibility of each of us (members of this society, humanity as a whole) to be responsible for and protective of the Other.
insights into the methodological design the FRA pilot in particular and mixed-media research in general.The two databases of the project have not been designed to contradict or challenge each other, but to talk to each other.A primary example of the link between the two datasets was the attempt to codify quantitatively and qualitatively the highly subjective variable Appraisal.The attempt to quantify Appraisal has proven to be challenging, however.Linking the quantitative and qualitative data, the Appraisal variable requires a significant discursive component in addition to its quantification in a numerical value.This is corroborated by the incompatibility of the different levels of Appraisal codification in the quantitative analysis.Whilst each variable -Overall Appraisal, Headline Appraisal, Appraisal of Speakers, Appraisal of Issues, Appraisal of Actors -is meant to code a different level of media appraisal, thus reflecting the complexity of the category, the quantification does not provide clues about the project's findings on the macro level, i.e., that of the Overall Appraisal.Our comparative www.e-compos.org.br| E-ISSN 1808-2599 | quantitative tests have proven that the Overall Appraisal in an article is not a cumulative value of the appraisals at the micro level.Whilst generally the values have similar tendency -positive or negative -the numerical is not conducive to a very detailed analysis of the possible explanations behind a particular appraisal.When we look at more micro level variables (Headline Appraisal, Appraisal of Speakers, Appraisal of Actors and Appraisal of Issues), the data is more reliable, to a degree due to better contextualisation through additional parameters.Thus, we argue against the adoption of the Overall Appraisal variable in a content analysis project, and in favour of measuring appraisal at a micro level.The FRA qualitative dataset of terminological entries highlights the disjunction between semantic meaning of individual words and the connotations acquired at the textual level.What they could not demonstrate is how meanings change at the level of the sentence and the text (through syntax and text grammar); how specific rhetorical devices modify literal meanings and acquire new connotations (irony, metaphors, rhetorical questions, repetitions, etc.); or how the arrangement of the news story (news schemata and logic) affects cognition.With the case study in our article we argued for the need to further the qualitative analysis by focusing on particular media events in order to enable the detailed, comparative and diachronic analysis of media narratives about MMGs.We also complicated the picture by arguing for the need of further discursive explanation of the values of the Appraisal variables designed for the project (especially Article Connotation).This could be done by analysing uncoded aspects of the news stories such as thematic clusters, rhetorical and stylistic devices, and argumentation strategies.Such an analysis would demonstrate not only the internal ambivalence of the terminology used by media nowadays, but also the ambivalence that is at the heart of the variables and categories used to conduct mixed-method research.os estudos de valência da mídia enquanto método investigativo.Argumentamos que o uso de métodos quantitativo e qualitativo para análise de valência na mídia gera resultados inconsistentes quando analisados por paradigmas de pesquisa quantitativo e/ou qualitativos.Para fundamentar nosso argumento, analisamos os resultados e a metodologia utilizados em projeto financiado pela Agência de Direitos Humanos da União Europeia analisando a cobertura jornalística inglesa sobre imigrantes e minorias na região.Neste artigo selecionamos as variáveis quantitativas e qualitativas usadas no projeto para explicar a valência dos jornais analisados, e averiguamos a consistência de resultados entre estas.Ao fazer isso, aplicamos princípios da pesquisa de métodos mistos para analisar os resultados das variáveis de valência, e refletimos sobre como estas variáveis podem ser combinadas para responder aos objetivos gerais da Agência financiadora.Os resultados indicam que análise de valência em análise de conteúdo da mídia apresentam mais contradições do que soluções, e que os resultados precisam ser seriamente relativa dos antes de responder se a cobertura da mídia referente a determinado tema é positiva, negativa ou neutra.el supuesto de que el análisis de contenido de los medios de comunicación basado en métodos investigación mixtos está libre de contradicciones.Sostenemos que, cuando se utilizan simultáneamente métodos cualitativos y cuantitativos, los diferentes paradigmas de investigación aplicados al proceso no pueden conciliarse con el objetivo de producir consistencia.Para ello, se revisan en este trabajo los resultados y la metodología de un proyecto financiado por la Agencia de Derechos Fundamentales de la UE sobre la cobertura de los inmigrantes y las minorías en la prensa británica.Centrándonos más específicamente en la codificación e interpretación de la variable de evaluación utilizada en el proyecto, investigamos cómo su definición subjetiva y cuantificación desafiadora señalan algunos de los problemas de coherencia que conlleva la intersección de los enfoques cualitativos y cuantitativos para el análisis de contenido.En este sentido, no descartamos el potencial de los métodos de investigación mixtos para ofrecer resultados perspicaces, pero advertimos en contra de la aplicación mecánica de los enfoques cross-paradigmáticos, y sostenemos que los vacíos e incongruencias expuestos por diferentes paradigmas pueden revelar más sobre la ambivalencia de la representación de los medios que su sincronización acrítica.

Table 1 :
Percentage of articles with MMGs per paper type, political orientation and Actor/Speaker religious background 2 www.e-compos.org.br|E-ISSN1808-2599|8/21 of analysis.3Thefirstone(OverallAppraisal)referstothemacro-analysis of how the article as a whole appraises the specific MMG; the second appraise the MMG and how those passively referred to in the article (Actors) are appraised by Speakers and/or article, respectively.During the coding process, each Appraisal variable could take one out of five possible values: three of them refer to the presence of an explicit appraisal of a MMG ("Positive", "Negative" and "Ambivalent"), while two other values refer to the lack of or impossibility to code an explicit appraisal of a MMG ("No appraisal" and "Ambiguous").The analysis index presented in Table2sums up the average of and negative explicit appraisal are used by the article (ambivalent appraisal); and c) when explicit appraisal cannot be ascribed to a particular tone (e.g., ambiguous appraisal).However, when values of the Overall Appraisal variable are compared to other Appraisal variables, the results are not as conclusive.The Headline Appraisal variable, for example, presents a more negative Appraisal index for No religion MMGs than Overall Appraisal does (-0.14 vs. 0.12, respectively). in none of the cases the findings presented by the 3 There are four other variables coding Appraisal in each article.Two of these are used to code the pictures accompanying an article, which have not been addressed in this article; two more variables are used in the codification of the terminology used to refer to a MMG and are analysed in the following section.www.e-compos.org.br|E-ISSN1808-2599|OverallAppraisalvariable are sustained.We could conclude that Muslims feature in headlines slightly better than they do in the articles themselves.Appraisal of the article.Nevertheless, it should be to some extent in agreement with the latter.Both variables present a similar Appraisal index for No religion MMG articles (0.11 vs. 0.12, respectively); 9/21 4 Count for the tabled variables, in the following order: Total count (N), Muslims, Non-Muslims and No religion.Overall Appraisal and Headline Appraisal (N=543: 177, 104, 262); Appraisal of Issues (N=1,293: 463, 273, 557); Appraisal ofSpeakers (N=1,188: 464,  235, 489); and Appraisal ofActors (N=3,552: 1,442, 696, 1,412).An important point to keep in mind is that whilst there could be only one Overall Appraisal and Headline Appraisal value per article (hence their count corresponds to the overall count of MMG articles), there could be more than one Speaker, Actor or Minority Issue per article; hence the count of the corresponding Appraisal variables significantly exceeds the count of MMG articles.

Table 2 :
MMGs Appraisal indexes by religious group 4 www.e-compos.org.br|E-ISSN 1808-2599 | If we repeat the above procedures comparing Overall Appraisal to the Appraisal indexes for Appraisal of Speakers and Appraisal of Actors, a Overall Appraisal might give us clues as to the news schemata or political agenda a newspaper has, the variables Appraisal of Issues, Appraisal of Actors and Appraisal of Speakers all quantify appraisal on a micro level which does not add up to or correspond in a straightforward manner to the Overall Appraisal values coded in the sample.portrayMMGs in a generally positive way, but when they are described by others (they act as Actors), Muslims are mostly portrayed in a negative light.www.e-compos.org.br| E-ISSN 1808-2599 |

Table 3 :
Appraisal of Issues (top 7) by religious group 7