MA-IS ePortfolio

In Unit 3, we were asked to choose one of the readings on a well-established research method to summarize, and reflect on its applicability to our research questions.  Each week we participated in discussions and reflected upon other research methods chosen by our classmates and added to our summary and reflections.

Summary

Content Analysis

Content analysis is one of the “tried and true” research methods that can be used to analyse the content of text, speech, and also images. Utilizing a quantitative approach, researchers can objectively measure the use of specific words, arguments, themes, or other categories in a sample of sources. Content analysis can be performed by simply counting words or phrases, or by coding content using pre-defined categories (and counting the number of occurrences of each category). This can be done manually by researchers or on a larger scale with the assistance of computer software. (Seale and Tonkiss 2018, pp. 403-405)

When content analysis is performed manually by researchers, “inter-rater reliability” can be tested to determine if researchers are likely to assign the same code to a given piece of content. The level of agreement between researchers/coders is measured using the kappa statistic (0 being “poor agreement” and 0.81-1 being “almost perfect agreement”). If a low level of agreement is achieved, code categories can be refined to improve agreement. (pp. 406-407)

One criticism of content analysis is that it functions to quantify “what is said” and is limited in establishing “how it is said” (p. 404). Keyword-in-context (KWIC) software can be used to display the words surrounding a specific keyword to help determine its context (p. 407); however, content analysis can also be combined with qualitative analysis and other research methods to allow for a more fulsome interpretation of context and meaning (p. 410).

The question of coding as an assertion of power and whether the act of coding can be truly objective or neutral was raised in the discussion forum during Week 4. Some forms of computer assisted-text analysis that rely upon word counts may be close to neutral; however, the decisions of researchers can and do impact findings.  For example, researchers have influence over what sources are included for analysis, how key words and phrases are grouped, how context is determined, how meaning is interpreted, and how themes are identified. Even “big data” analysis can be influenced by computer scientists who write the code which performs text mining and computer-assisted analysis.  Researchers should be mindful of the power and influence they have over seemingly “objective” methods.   

In Chapter 24 of Researching Society and Culture, Seal and Tonkiss (2018) describe several examples of how content analysis can be used in research. A particular example that stood out was the research project by Philo et al. that involved content analysis of television news reports in combination with focus groups and survey methods. They used these methods to investigate the interpretation and understanding of the Israeli/Palestinian conflict by study participants in comparison to the content of news media reports (p. 410).

In “A Critical Content Analysis of Media Reporting on Opioids”, Webster et al. (2020) used content analysis to examine themes present in Canadian newsprint media reporting on opioids, analysing 826 articles published by CBC News, The Globe and Mail, and the National Post from 2000 to 2016 using both quantitative and qualitative methods. The research team began by “counting the number of references to particular topics, issues, or persons to identify prominent issues, themes, and trends in [news media] coverage”, then developing a coding framework by examining “themes that emerged from… twenty randomly-selected articles,… [resolving] differences in interpretation… by consensus”, and ultimately performing an in-depth analysis of themes present in a sample of 128 articles (p. 2). They aimed to explore changes in news media reporting over time with regards to how opioid use was characterized including the nature of the problem, affected individuals, and potential solutions (pp. 2-4).

Reflection

My Research Question from Unit 1:

What are the differences between the general public’s perception of Satanism and the stated beliefs of The Satanic Temple, and how does public perception inform political responses to The Satanic Temple’s advocacy initiatives?

Content Analysis

Since content analysis lends itself to studying the use of language and to the analysis of news media, both in text/print format and in speech/radio/tv broadcasts, it would apply nicely to my research topic and question. Seal and Tonkiss (2018) also shared an example where computer software was used to count words used by Hillary Clinton and Donald Trump in their campaign speeches during the 2016 United States presidential election. The resulting counts were used to compare Clinton and Trump’s use of language and were also presented visually using word clouds (p. 408). It would be interesting to use this method initially to identify trends in the use of language by The Satanic Temple (TST) compared to language used in news media reporting on TST’s advocacy initiatives. A preliminary analysis of sources using this method would help to identify key differences in what words and phrases are used by each side and provide a basis upon which to further consider how language is used and to what end.

The Philo et al. example (Seale & Tonkiss, 2018, p. 410) suggests that it could be difficult to use content analysis to draw a direct connection between the content of news media reporting and public perception. Philo et al. used focus group analysis to the further explore the understanding and perception of focus group participants and where they obtained their information. It may not be feasible to use focus groups in the case of my research question given my lack of proximity to the issue and given the resources that would be involved. I’m interested in considering other multi-method approaches; however, since it may also be difficult establish a direct connection between public perception and political responses, it may be preferable to refocus my research topic more specifically on language use by TST and/or news media.

I also want to be mindful of the different factors that may affect the content of news media reports on The Satanic Temple as was raised in the discussion forum by my classmate, Sarah.  I would like to learn more about this and consider more carefully how or if news media reports can provide evidence of public perception.  What other influencing factors may be at play in news media reporting?  

Discourse Analysis

Of the other well-established research methods discussed in Unit 3, discourse analysis is the most applicable to my research topic and question.  Discourse analysis could be used to build upon the key differences in what is said by The Satanic Temple and in news media reports identified through content analysis to further study and develop insights into how this language is used and the meaning that is produced in a social context.

In Chapter 27 of Researching Society and Culture, Tonkiss (2018) discusses critical discourse analysis which is “concerned with the social and political context of discourse, based on the view that language is not only conditioned by these contexts but itself helps to constitute them,” going on to say that critical discourse analysts “see language as crucial to the ways that power is reproduced, legitimated and exercised within social relations and institutions.” (p. 480).  I would like to explore how the use of this methodology could shape my research question and focus.  I believe language use is at play in the way that institutional power is legitimated and exercised in relation to TST’s advocacy initiatives.  Using discourse analysis, I may be able to study more deeply the way language use contributes to different views of the issues at the centre of TST’s campaigns.

Research Topic/Question 

In light of the Unit 3 readings and discussion, I am becoming concerned with the scope of my research topic and question and think it may be necessary to focus in on one of TST’s advocacy initiatives in particular.  I was originally interested in TST because of the movie Hail Satan? and its coverage of TST’s Satanic Representation Campaign (The Satanic Temple n.d.); however, due to the recent overturn of the Roe v. Wade decision by the Supreme Court of the United States, TST’s Religious Reproductive Rights (RRR) campaign has gained attention (The Satanic Temple n.d.).  It would be more interesting and timely to focus on the discourse of bodily autonomy, reproductive rights, and abortion access, and how language use shapes knowledge around this social issue. 

My Revised Research Question:

How is The Satanic Temple (TST)’s Religious Reproductive Rights campaign represented by TST and in news media reports in the context of the abortion debate in the United States?

---

References

Seale, Clive, and Fran Tonkiss. 2018. “Chapter 24: Content and Text Analysis.” In Researching Society and Culture, edited by Clive Seale, 4th ed., 403–27. California: SAGE.

The Satanic Temple. n.d. “Religious Reproductive Rights Campaign.” The Satanic Temple. Accessed June 27, 2022a.

———. n.d. “Satanic Representation Campaign.” The Satanic Temple. Accessed June 27, 2022b.

Tonkiss, Fran. 2018. “Chapter 27: Discourse Analysis.” In Researching Society and Culture, edited by Clive Seale, 4th ed., 447–92. California: SAGE.

Webster, Fiona, Kathleen Rice, and Abhimanyu Sud. 2020. “A Critical Content Analysis of Media Reporting on Opioids: The Social Construction of an Epidemic.” Social Science & Medicine 244 (January): 112642.

Notes

This post is a revised version of the Summary and Reflection assignment I submitted in Week 8 of MAIS 602 Doing Interdisciplinary Research, Athabasca University (Dr. Nisha Nath, May 2022).

Featured Image: Created using the Word Cloud Generator at jasondavies.com using text from TST’s Religious Reproductive Rights Campaign webpage.