In approaching today’s topic, I spent the morning reading earlier entries to the Research Hub on related themes. I was amazed at the range of topics I have broached over the past three years. Someone really ought to advise me to get a life! 

My goal in re-reading the selected contributions was two-fold: first, to see if I have developed a consistent position on the topic of mixed methods and hybrids; and second, to see if there was any point in revisiting the topic. On the first point, I found that I have more-or-less developed my position in relation to some wonderful material I engaged over these years – thinking here of Primdahl et al. (2018),Roseneil (2011), St Pierre (2014), Mol (1999, 2002), Law (2004), Jackson and Mazzei (2012), and many others. Today I report on my current stance on these issues, leaving open the possibility that down the track I will read something that causes me to modify my position. That is what intellectual exchange is all about!

Many researchers propose blending WPR with other research approaches as an explicit strategy. In this entry I aim to look at one such proposal by Van Aswegan et al. (2019) because it helps me to reflect on the suggestion that useful research requires theoretical “hybrids” or “mixed methods”. In their exposition of CDPR (critical discourse problematization framework), the authors defend the need for a “good cop/bad cop” approach to research methods, with WPR characterized as the “bad cop” while a form of Critical Discourse Analysis (Fairclough 1995; Hyatt 2013a) serves as the “good cop”. I wish to explain and challenge the grounds for this characterization of WPR because I am concerned that others may find its simplicity attractive. However, first, I need to review some of the long-standing debates about method, methodological pluralism and “eclecticism”.

As a starting point, I find helpful Van Aswegan et al.’s distinction between, on the one hand, theoretical lenses, such as Critical Disability Studies, Post-Colonial Studies or feminist studies and, on the other hand, theoretical tools, such as Critical Discourse Analysis (CDA; Fairclough 1995) or critical higher education policy discourse analysis (CHEPDA; Hyatt 2013a). In my understanding “tools” provide analytic techniques whereas “lenses” can be said to refer to selected aspects of social relations (topic areas). I would add that “tools” reflect contrasting paradigms whereas “lenses” can and do cross paradigmatic lines. For example, both disability studies (Meekosha and Shuttleworth 2009) and feminist studies (Davis, 2008; Scott 2005) are characterized by intense internal debates about paradigmatic assumptions. Paradigms are understood to reflect competing worldviews due to contrasting ontological, epistemological, and methodological assumptions (Alvesson and Sandberg 2011: 255). In a paper on problematizations in health policy, I identify four dominant paradigms in health policy research: positivism, interpretivism, critical realism and poststructuralism (Bacchi 2016).

In terms of the heuristic distinction between “lenses” and “tools”, WPR provides a “tool” for critical analysis that can be applied using a variety of “lenses” – e.g., disability studies (see Apelmo 2021), post-colonial studies (see Gordon 2011), feminist studies (see O’Hagan 2020). As a “tool”, WPR reflects a particular theoretical stance and paradigm, Foucault-influenced poststructuralism (Bacchi and Goodwin 2016). Hence, it does not sit easily alongside theoretical interventions that dispute poststructural premises and posit quite different ways of seeing the world. Elsewhere, for example, I reflect on the epistemological and ontological tensions between a poststructuralist and a critical realist perspective (see Bacchi 2016; Research Hub entries 30 Nov. 2020; 28 Feb. 2021). Hence, I would question the ways in which Windle et al. (2018) and Holding et al. (2021) combine WPR with Critical Realism in their analyses. And for the same reason, I suggest that WPR cannot serve on Van Aswegan et al.’s research team as the “bad cop” to Fairclough’s “good cop”. Fairclough (2013: p. 185) declares himself committed to “(critical) realism” and a “moderate constructivism”, in which the point of critique is to ask “what the problems really are”. Given the paradigmatic tensions between poststructuralism and Fairclough’s CDA (see Research Hub entry 14 May 2018), therefore, WPR and CDA do not work “in harmony with each other” (Van Aswegan et al. 2019: 195) and the political implications of the paradigmatic tensions between them need to be carefully considered (see below). 

The mention of paradigms as important in this context is contentious. There are expressed fears about “line drawing” and claims to theoretical purity. Going further, to suggest that paradigms are significant in shaping research approaches can appear to be obstructionist, preventing constructive dialogue among researchers. I would, in fact, like to claim the opposite – that saying that paradigms matter (Bacchi 2016) produces the grounds for constructive dialogue. This dialogue would involve researchers in interrogating the deep-seated ontological and epistemological premises that shape their work and in considering how those premises matter to the arguments they craft and the propositions they advance. 

Readers will recognize here a theme that has appeared in previous entries (e.g., 10 Dec. 2017 on ontological politics) – the claim that methods are not innocent and that they shape realities (see Mol 1999, 2002; Law 2004). Approaches to political studies are not simply forms of analysis; rather, they are techniques of “truth” production and forms of political intervention. John Law provides this useful example: by deploying concepts such as “nation-state” unproblematically, analysts actually reinforce them. Hence, they produce a reality of nation-states. This “performativity of method” (Law 2004: 149-150) creates a responsibility to reflect on the “realities” one’s methods create. 

For example, as I argued with Malin Rönnblom some years ago (Bacchi and Rönnblom 2014), it is important to initiate a conversation that examines theoretical stances in terms of the politics they make possible. There we made the case that poststructural theoretical approaches make an important political contribution by pluralising the relations that constitute “the real” and refusing the temptation to freeze these relations. Borrowing from Michael Shapiro (1992: 12), the objective is to challenge what “is” as what “must be”, to loosen “the grip” of the assumed “facticity” of established institutions and power relations. This position leads to Fraser’s (2020) argument that what is needed is “ontopolitically-oriented research”, research that produces alternative socio-political relations. 

The issue of “mixed methods” in political studies (and in other fields) has a long history. A Sage Journal of Mixed Methods Research commenced in 2007, signalling a trend towards “methodological eclecticism”. This trend was prompted by impatience with “rigid paradigm adherence” and “endless disputation regarding qualitative and quantitative forms of inquiry” (Yanchar and Williams 1989: 3). Countering an “incompatibility thesis” which “suggests that methods based on contradictory theoretical assumptions cannot be coherently mixed in a single study or set of studies”, Howe (1988) put forward a “compatibility thesis” that held the opposite – that “methods are essentially disengaged from paradigms and able to be mixed any way that researchers desire without theoretical incompatibility, contradiction, or implication”. There were also links between the call for “methodological eclecticism” and American pragmatism, advising researchers to pursue “what works” in context (see Research Hub entry 30 April 2018). 

In reaction to this position, Yanchar and Williams (1989: 3) insisted on the need to “take seriously the inescapable assumptions and values that accompany the use of a method and the pursuit of practically useful results”. They opted, therefore, for a “softer version of the incompatibility thesis” that allows for “methodological flexibility”. They put forward five guidelines for method use to move beyond both “methodological eclecticism and paradigm rigidity”: contextual sensitivity, creativity, conceptual awareness, coherence and critical reflection. Under “conceptual awareness” they noted that “researchers and evaluators would appropriately emphasize the theoretical nature of their methods and their work by identifying assumptions, values and moral commitments that have practical and theoretical consequences”. The criteria of critical reflection would entail researchers acknowledging “the assumptive framework” they adopt as “fallible, alterable and, in need of critical examination” (Yanchar and Williams 1989: 9).

These suggestions signal a perspective close to the stance I take at the outset of this entry – that constructive dialogue among researchers would involve researchers in interrogating the deep-seated ontological and epistemological premises that shape their work and in considering how those premises matter to the arguments they craft and the propositions they advance. Yanchar and Williams (1989: 9) also usefully qualify the proposition that “method choices should be driven by questions”, insisting on the need for “continued reflection on the fundamental nature of research questions”. 

Two decades after Yanchar and Williams’ (1989) important contribution, Wolf (2010) examined the trend towards triangulation and mixing methods in comparative public policy research. Wolf’s contribution is made interesting by his admission in an opening footnote that his analysis reflects his position as “some kind of realist” alongside his claim that it is possible to 

“subscribe to a pluralist view that welcomes different basic assumptions as at least potentially enriching in certain research contexts (cf Moses & Knutsen 2007: 289) – notwithstanding the need to discuss possible incompatibilities within every concrete research design” (Wolf 2010: 160-161 note 1). 

It is this last proposition that is the most difficult to enact and unfortunately Wolf fails to test his own analytic assumptions in the way he recommends. For example, he starts his article with a mainstream, realist view of the topic for analysis, describing comparative public policy as “mainly concerned with ‘what governments do, why they do it, and what difference it makes’ (Dye, 1976)”. Moreover, his criteria for assessing method use include the need for robustness, rigour and “validity checking” (Wolf 2010: 146) – all concepts that fit comfortably within his self-declared realist premises. Paradigms are clearly at work in Wolf’s evaluation of research approaches despite his defence of “pluralism”.

Similarly, contrasting paradigmatic assumptions sharply distinguish WPR from CDA, challenging Van Aswegan et al.’s (2019: 186, 195) description of the approaches as “complementary”, “in harmony” and forming a “good cop/bad cop” research team. Applying the criteria of conceptual awareness and critical reflection endorsed by Yanchar and Williams (1989), I proceed to list areas and topics where paradigmatic distinctions between the approaches need to be acknowledged. 

1. At the most basic level the research task Van Aswegan et al. (2019) undertake is markedly different from that pursued in WPR. They (2019: 192) explain that their goal is “examining the ways in which the policy ideas are advanced and justified”. By contrast WPR directs attention to the conceptual logics embedded in policy and other proposals. Van Aswegan et al. (2019: 188; emphasis added) mistakenly describe WPR as targeting “the way in which an issue is represented or put forward by policy makers as the ‘problem’ to be addressed” – which might explain some of the confusion involved in their attempt to “marry” WPR and CDA (Van Aswegan et al. 2019: 186). WPR directs attention to problematizations located in governmental texts, not to the understandings or interpretations of policy makers (Bacchi 2015).
2. Given their expressed interest in “how the policy is legitimated”, unsurprisingly, Van Aswegan et al. target rhetorical devices (“warrants”) and the “power of language” (p. 187) as their primary concerns. On the latter they specify that CDA draws on a systemic functional linguistics approach to language. In WPR by contrast the interest is in discourses as knowledges, not as language use. The authors correctly note that the aim of WPR is “unearthing the conceptual logic operating behind the text” but then suggest that such logics are located in “linguistic paraphernalia” (Van Aswegan et al. 2019: 191-192). 
3. There is unevenness in the treatment of intentionality. Whereas the authors acknowledge that a problematization approach does not suggest “misrepresentation or malign intent” (Van Aswegan et al. 2019: 188), elsewhere Hyatt (2013b: 837) describes policies as “the outcomes of struggles ‘between contenders of competing objectives, where language – or, more specifically, discourse – is used tactically’ (Fulcher 1989:7), suggesting intentionality.
4. This stance on intentionality is linked to a form of “ideology critique” (see Hyatt 2013b: 839; Van Aswegan et al. 2019: 187) which sits uncomfortably with WPR’s poststructural premises (this topic will be pursued in a subsequent Research Hub entry).
5. Context is emphasized in both WPR and CDA. However, the latter ascribes to a realist perspective in which “context” is an uncontested domain which can simply be described. Linked to this view, CDPR (critical discourse problematization framework) describes genealogy as an exercise involving the “mapping” of CES (Comprehensive Employment Strategy) to “context” (Van Aswegan et al. 2019: 194). By contrast, genealogy in WPR involves tracing the detailed practices that produce specific governmental problematizations (Bacchi and Goodwin 2016: 22). In WPR, “contexts” are not uncontested spaces to be “filled in” since reflections on context are themselves interpretive (Bacchi 2009: 21).
6. While there is acknowledgement that in WPR, policy subjects “are not considered individuals with fixed identities formed through self-directed agency” (Van Aswegan et al. 2019: 187), it is unclear how this perspective is compatible with the focus on policy actors and their rhetoric. The topic of subjectification – how subjects are constituted in policies – which is a central theme in WPR, goes unmentioned. There is only a passing reference to “reflexive problematization” in Table 1 (p. 191) and reliance on the behavioural psychology theory of “conditioning” (Van Aswegan et al. 2019: 188). 

The overall argument in Van Aswegan et al. (2019: 190) is that WPR provides questions while CDPR, which they describe as a “structural” and “problem-oriented” approach” (pp. 187, 195), provides “evidence” for, or answers to, those questions. It is difficult to see how the two approaches can be “married” given the poststructural premises underpinning WPR and its explicit challenge to “problem-oriented” analyses. 

In WPR, there is no way to think or research outside epistemological and ontological premises. The best way to proceed, therefore, is to acknowledge the tensions among approaches that exist at the paradigmatic level and to discuss them. As signalled above, this discussion ought to focus on the political implications of the methods we adopt and on the “realities” those methods create. The challenge becomes developing arguments to convince researchers committed to alternative paradigms to reflect on the possible usefulness of this proposition, which may prove to be difficult given that it reflects the very poststructural premises those researchers may dispute. Pursuing the possibility of “ontopolitically-oriented research” (Fraser 2020) and illustrating its usefulness as a political intervention may offer a way forward.

References

Apelmo, E. 2021. What is the problem? Dis/ability in Swedish physical education teacher education syllabi. Sport, Education and Society. DOI: 10.1080/13573322.2021.1884062 

Alvesson, M., & Sandberg, J. 2011. Generating research ques- tions through problematization. Academy of Management Review, 36, 247-271.

Bacchi, C. 2009. Analysing Policy: What’s the Problem Represented to be? Frenchs Forest: Palgrave Macmillan.

Bacchi, C. 2015. The Turn to Problematization: Political implications of contrasting interpretive and poststructural adaptations. Open Journal of Political Science, 5: 1-12.  

Bacchi, C. 2016. Problematizations in Health Policy: Questioning How “Problems” Are Constituted in Policies. Sage Open, April-June, 1-16.

Bacchi, C. and Goodwin, S. 2016. Poststructural Policy Analysis: A Guide to Practice. NY: Palgrave Macmillan.

Bacchi, C. & Rönnblom, M. 2014. Feminist Discursive Institutionalism – A Poststructural Alternative. NORA – Nordic Journal of Feminist and Gender Research, 22(3): 170-186.

Davis, K. 2008. Intersectionality as buzzword: A sociology of science perspective on what makes a feminist theory successful. Feminist Theory, 9(1): 67–85.

Dye, T. R. 1976. Policy analysis: What governments do, why they do it, and what difference it makes. Tuscaloosa: The University of Alabama Press. 

Fairclough, N. 1995. Critical Discourse Analysis. Boston, MA: Addison-Wesley.

Fairclough, N. (2013). Critical Discourse Analysis and Critical Policy Studies. Critical Policy Studies, 7, 177-197. http://dx.doi.org/10.1080/19460171.2013.798239 

Fraser, S. 2020. Doing ontopolitically-oriented research: Synthesising concepts from the ontological turn for alcohol and other drug research and other social sciences. International Journal of Drug Policy, 82(2): 102610.

Fulcher, G. 1989. Disabling Policies? A Comparative Approach to Education Policy and Disability. London: The Falmer Press.

Gordon, Z. 2011. Deconstructing “Aboriginal Welfare Dependency”: Using postcolonial theory to reorientate Indigenous Affairs. Journal of Australian Indigenous Issues, 14(2-3): 14-29.

Holding, E., Fairbrother, H., Griffin, N., Wistow, J., Powell, K. and Summerbell, C. 2021. Exploring the local policy context for reducing health inequalities in children and young people: an in depth qualitative case study of one local authority in the North of England, UK. BMC Public Health, 21, 887.

 Hyatt, D. 2013a. “The critical higher education policy discourse analysis framework”. In Huisman, J. and Tight, M. (Eds), Theory and Method in Higher Education, Vol. 9, Emerald, London, pp. 41-59.

Hyatt, D. 2013b. The critical policy discourse analysis frame: helping doctoral students engage with the educational policy analysis, Teaching in Higher Education, 18(8): 833-845, DOI: 10.1080/13562517.2013.795935 

Howe, K. R. 1988. Against the quantitative-qualitative incompatibility thesis, or, Dogmas die hard. Educational Researcher, 17(8), 10–16. 

Jackson, A. Y. and Mazzei, L. A. 2012. Thinking With Theory: A New Analytic for Qualitative Inquiry. In L.A. Mazzei and A. Y. Jackson, Thinking with Theory in Qualitative Research: Viewing Data Across Multiple Perspectives. NY: Routledge

Law, J. 2004. After method: Mess in social science research. London and New York: Routledge.

Meekosha, H. and Shuttleworth, R. 2009. What’s so “critical” about critical disability studies? Australian Journal of Human Rights, 15(1): 47- 75.

Mol, A. 1999. Ontological politics: A word and some questions. In J. Law, & J. Hassard (Eds.), Actor network theory and after. Oxford: Blackwell. 

Mol, A. 2002. The body multiple: Ontology in medical practice. Durham and London: Duke University Press.

Moses, J., & Knutsen, T. L. 2007. Ways of knowing. Competing methodologies in social and political research. Basingstoke, UK: Palgrave Macmillan. 

O’Hagan, A. 2020. Introduction: “A critical practice of thinking otherwise”: Bacchi, gender and public policy analysis. En Feminismo/s, 35: 13-28. Monographic dossier / Dossier monográfico: A critical practice of thinking otherwise: Bacchi, Gender and Public Policy Analysis, coord. Angela O’Hagan, DOI: 10.14198/ fem.2020.35.08 

Primdahl, N. L., Reid, A. & Simovska, V. 2018. Shades of criticality in health and wellbeing education. Journal of Curriculum Studies, DOI: 10.1080/00220272.2018.1513568 

Roseneil, S. 2011. Criticality, Not Paranoia: A Generative Register for Feminist Social Research. NORA – Nordic Journal of Feminist and Gender Research, 19(2): 124-131.

Scott, J. W. 2005. Against Eclecticism. Differences: A Journal of Feminist Cultural Studies, 16(5): 114-137.

Shapiro, M. J. 1992. Reading the Postmodern Polity: Political Theory as Textual Practice. Minneapolis: University of Minnesota Press.

St Pierre, E. 2014. A Brief and Personal History of Post Qualitative Research: Toward “Post Inquiry”. Journal of Curriculum Theorizing, 30(2): 2-19.

Van Aswegen, J., Hyatt, D. and Goodley, D. 2019. A critical discourse problematization framework for (disability) policy analysis. “good cop/bad cop” strategy. Qualitative Research Journal, 19(2): 185-198.

Windle, A., Fisher, M., Freeman, T., Baum, F., Javanparast, S., Kay, A., and Kidd, M. 2018. Increased private health fund involvement in Australia’s primary health care: Implications for health equity. Australian Journal of Social Issues, 1-17; DOI: 10.1002/ajs4.45 

Wolf, F. 2010. Enlightened Eclecticism or Hazardous Hotchpotch? Mixed Methods and Triangulation Strategies in Comparative Public Policy Research.  Journal of Mixed Methods Research, 4(2): 144-167. Yanchar, S. C. and Williams, D. D. 1989. Reconsidering the Compatibility Thesis and Eclecticism: Five Proposed Guidelines for Method Use. Educational Researcher: 35(9): 3-12