The Arts Engagement Project at
The University of Michigan

Dictionaries of Language Patterns

Measuring linguistic patterns within texts

Topics help to identify WHAT the students said in response to different questions, but we are just as interested in HOWthey expressed these topics. The linguistic and structural features of responses can provide insights into the ways that respondents frame and organize their thoughts. Patterns in these features can represent more or less complex reasoning and reflection, and well as other signals of social and psychological dynamics around the topics discussed. We can also use these features to explore relationships with demographic factors as a way of investigating potential structural drivers that shape student experiences.

In order to measure different patterns among the student responses, we used three separate linguistic analysis tools to characterize the language used. Each tool has a different focus, utilizing domain-specific dictionaries to search strings of words and classify matching words or patterns of words among a series of categories.

Docuscope Language Action Types (LATs) analyzes rhetorical features such as persuasiveness or first-person reporting; covers 40 million linguistic patterns of English classified into over 100 categories of rhetorical effects[KIBC04][IK12].
Automated Integrative Complexity (AIC) measures the cognitive structure implied in a speaker’s verbal content including how well they identify different dimensions of an issue and integrate different ideas [CCGH14][HCIG14].
Linguistic Inquiry and Word Count (LIWC) calculates the degree to which linguistic, cognitive, and affective categories of words are used in a text [TP10][PBBF].

Each tool provides a different set of measures for its categories, and these are output as csv files. The measures are then integrated back into the source data table and used as co-variates in statistical analyses of patterns present among the responses. This work draws on the examples of Michael Witmore and Jonathan Hope’s analyses of early modern texts and Shakespeare’s plays [HW10][Wit15a][Wit15b][Wit16][WHG16][HW19][WH19]. In contrast, our analysis relies on web-collected interview survey responses as the unit of analysis while introducing additional analytical measures through the combined use of AIC, LIWC, Docuscope and STM.

Docuscope Language Action Types (LATs)

Docuscope [KIBC04][IK12] is a corpus-based rhetorical analysis tool and dictionary developed by to analyze rhetorical patterns in text, student writing, and interview transcripts. Docuscope searches texts for strings of words (including words, phrases and punctuation marks) that can be classified according to a series of rhetorical categories contained in the program’s dictionaries, called ‘Language Action Types’ (LATs), and counts the frequency of those LATs for each text.

LATs range from the basic, with LATs such as ‘First Person’ (first-person singular pronouns: ‘I’, ‘me’ etc.), to the more conceptual — for example, Uncertainty uses words indicating uncertainty (e.g., maybe, perhaps); Sensory Language includes words that stir the senses (e.g., blue, textured), concrete things (e.g., table, chair), spatial relations (e.g., near to, seated with), motions (e.g., run, jump), dialogue cues (e.g., ,” he barked) and oral speech features (e.g., uh huh); Abstract Concepts is a category that includes a very large set of latin (tion, sion, ment, ogy) or greek (logy) suffixes and other patterns indicating abstract general concepts, like sociological or anthropological; Future refers to when the future is invoked; and Comparison includes words that indicate conceptual similarity and difference, like “more” or “fewer” [IKa].

The Docuscope version 3.21 (6/26/2012) dictionary is publicly available on the web through Github [IKa], and as an application for classifying text through the Visualizing English Print’s (VEP) web-based interface, Ubiquity [ubi]. More recent Docuscope versions provide a greater number of more detailed LATs, however the dictionaries and LAT descriptions are not publicly available making interpretation of the results difficult. LAT dictionary category descriptions for version 3.21 can be found in the Github repository [IKb]. There are 115 LATs in version 3.21 of Docuscope. When the document corpus is processed through Docuscope/Ubiquity, counts for each response text are tallied and exported as csv frequency counts for the LAT categories for further analysis.

Automated Integrative Complexity (AIC)

Automated Integrative Complexity (AIC) is designed to measure the Integrative Complexity (IC) of statements [CCGH14][HCIG14][Fei94]. With origins in social and political psychology research [SR76][STS92], IC measures the structural complexity of thinking and reasoning evident in a given statement in order to assess the level of cognitive differentiation and subsequent integration of idea elements pertaining to an issue [CIST18]. Automated Integrative Complexity is a dictionary-based tool, available online through a service provided by Distill Analytics {cite} AIC. Although the dictionary and algorithms used by the service are not in the public domain, the methods for coding passages of text are described elsewhere [CCGH14][HCIG14][BBBB+92].

Integrative Complexity measures two elements: differentiation and integration. The AIC tool also includes measures for the type of complexity: dialectical and elaborative [CITA+08][CCGH14]. Dialectical complexity occurs when an author deals with opposing viewpoints or some other kind of tension; elaborative complexity occurs when complexity builds on a single dominant theme or perspective [CI]. For each type (IC, Dial, or Elab), the AIC tool provides an overall measure, as well as separate measure of integration and differentiation within each type. Statements that distinguish different dimensions, topics, themes, perspectives, objects, and practices from each other score higher on differentiation. Statements score higher on integration if those differentiated elements are somehow brought together or conceptually integrated to produce a new outcome. Scores greater than 3 indicate that a statement has gone beyond differentiation towards active integration of the concepts, perspectives, possibilities, or other dimensions [BBBB+92]. For example, consider the following responses to question “How do you define ‘the arts’?” along with their observed IC measures:

“Creative Expression” (IC=1)

This example above is a simple label with no differentiation between different perspectives whatsoever.

“To me the arts include pretty much a little of everything. This is because ‘the arts’ aren’t exactly a thing, more of a method of how something is perceived or altered.” (IC=1.5)

In this example, the respondent seems slightly more open, but the respondent uses a dominant rule that situates the arts as “a little of everything” without considering a possible alternative, for example, that the arts could be “exactly a thing.”

“’The arts’” are a way to let out your creativity. They can encompass anything from drawing, painting, photography, jewelry making, sculpting, singing, dancing, theater, museums, playing an instrument, and many other ways in which you can use your creativity. They are not limited to what we may think of as ‘art’ like a painting or a drawing, but encompass music and many other outlets as well.” (IC=2)

In this example above, a list predominates and primarily reinforces the statement that “The arts are a way to let out your creativity.” Later in the statement, the respondent recognizes that the arts may not be limited to the list and could include “many other outlets as well.” This demonstrates in increased level of differentiation, with the respondent recognizing the potential for alternatives which are not elaborated on.

“I think the arts includes all forms of creative works, be it written, painted, or theater; although I also think that there is some distinction between amateur ‘arts’ and professional. The distinction between the two is more noticeable in urban areas, where both forms are around.” (IC=3)

While this statement lays out a general categorization (“all forms of creative works”), the respondent makes an additional allowance (“I also think”) of a distinction between amateur and professional. The statement goes further to describe how that distinction is more noticeable in urban areas (presumably compared to rural ones) and why that distinction is more noticeable (“both forms abound”).

“The arts include anything that someone considers to be a beautiful contribution to society. Therefore, I would even consider some scientific fields artistic because of the exploratory and innovative aspects that exist in everyone’s work. I do think that most people consider only music, art, and writing to be part of the arts, and I do believe those are more expressive arts aimed at being recognized by the public as artistic. But I believe physics demands an artistic eye.” (IC=3.5)

In the final example above, the statement takes a more integrative stance. A superordinate statement provides the lead. The author then considers some other fields and provides reasons for this, and at the same time, acknowledges what most other people consider to be true about the arts. The contrast among alternatives sets up a dialectic tension; there are reasons from both perspectives about what should the definition should encompass.

In the above examples, the responses progressively increase in complexity from a simple label with no differentiation between different perspectives toward greater integration and complexity. While the survey asked “How do YOU define ‘the arts’?” [emphasis added] the latitude that a respondent takes to include alternatives and potentially integrate those alternatives into that definition is, in part, what AIC seeks to measure.

Linguistic Inquiry and Word Count (LIWC)

Linguistic Inquiry and Word Count (LIWC) is a software application and dictionary that looks for and counts words in psychology-relevant categories across multiple written or transcribed verbal text files [TP10][pennebakerLIWC pennebaker2015development]. LIWC individually analyses each word in a text sequentially for matches with its internal dictionaries across approximately 94 separate variables. Dictionary categories include a variety of word and punctuation analytics; summary language variables (analytical thinking, clout, authenticity, and emotional tone); standard linguistic dimensions such as the percentage of pronouns or articles; psychological constructs such as anxiety, tentativeness, perceptual processes, achievement, positive emotion, and temporal orientation; and personal concern categories such as work, home, money, religion. After analyzing and classifying the words in the texts, LIWC calculates the percentage of each LIWC category in a text, and the resulting csv output lists all LIWC categories along with the rates for each category used in the given text [TP10].

Citations

ubi: Ubiqu+Ity. URL: http://vep.cs.wisc.edu/ubiq/.
BBBB+92(1,2): Gloria Baker-Brown, Elizabeth J Ballard, Susan Bluck, Brian de Vries, Peter Suedfeld, and Philip E Tetlock. The conceptual/integrative complexity scoring manual. In Motivation and personality: Handbook of thematic content analysis, pages 401–418. Cambridge University Press, 1992.
CCGH14(1,2,3,4): Lucian Gideon Conway, Kathrene R Conway, Laura Janelle Gornick, and Shannon C Houck. Automated integrative complexity. Political Psychology, 35(5):603–624, 2014.
CI: Lucian Gideon Conway III. Coding Dialectical-Complexity and Elaborative-Complexity: A Supplement to the Integrative Complexity Coding Manual. URL: http://hs.umt.edu/politicalcognition/documents/dialectical-elaborativemanual.doc.
CIST18: Lucian Gideon Conway III, Peter Suedfeld, and Philip E Tetlock. Integrative complexity in politics. In The Oxford Handbook of Behavioral Political Science. 2018.
CITA+08: Lucian Gideon Conway III, Felix Thoemmes, Amy M Allison, Kirsten Hands Towgood, Michael J Wagner, Kathleen Davey, Amanda Salcido, Amanda N Stovall, Daniel P Dodds, Kate Bongard, and others. Two ways to be complex and why they matter: implications for attitude strength and lying. Journal of Personality and Social Psychology, 95(5):1029, 2008.
Fei94: Gregory J Feist. Personality and working style predictors of integrative complexity: a study of scientists’ thinking about research and teaching. Journal of Personality and Social Psychology, 67(3):474, 1994.
HW19: Alan James Hogarth and Michael Witmore. Reflexive witnessing: boyle, the royal society and scientific style. Notes and Records: the Royal Society journal of the history of science, pages 20180051, 2019.
HW10: Jonathan Hope and Michael Witmore. The hundredth psalm to the tune of” green sleeves”: digital approaches to shakespeare’s language of genre. Shakespeare Quarterly, 61(3):357–390, 2010.
HCIG14(1,2,3): Shannon C Houck, Lucian Gideon Conway III, and Laura Janelle Gornick. Automated integrative complexity: current challenges and future directions. Political Psychology, 35(5):647–659, 2014.
IKa(1,2): Suguru Ishizaki and David Kaufer. Docuscope. URL: http://github.com/docuscope/DocuScope-Dictionary-June-26-2012.
IKb: Suguru Ishizaki and David Kaufer. Docuscope LAT descriptions. URL: http://github.com/docuscope/DocuScope-Dictionary-June-26-2012/blob/master/src/_help.txt.
IK12(1,2): Suguru Ishizaki and David Kaufer. Computer-aided rhetorical analysis. In Applied Natural Language Processing: Identification, Investigation and Resolution, pages 276–296. IGI Global, 2012.
KIBC04(1,2): David S Kaufer, Suguru Ishizaki, Brian S Butler, and Jeff Collins. The power of words: Unveiling the speaker and writer’s hidden craft. Routledge, 2004.
PBBF: JW Pennebaker, RJ Booth, RL Boyd, and ME Francis. Linguistic Inquiry and Word Count: LIWC2015. URL: http://www.LIWC.net.
SR76: Peter Suedfeld and A Dennis Rank. Revolutionary leaders: long-term success as a function of changes in conceptual complexity. Journal of personality and social psychology, 34(2):169, 1976.
STS92: Peter Suedfeld, Philip E Tetlock, and Seigfried. Streufert. Conceptual/integrative complexity. In Motivation and personality: Handbook of thematic content analysis, pages 393–400. Cambridge University Press, 1992.
TP10(1,2,3): Yla R Tausczik and James W Pennebaker. The psychological meaning of words: liwc and computerized text analysis methods. Journal of language and social psychology, 29(1):24–54, 2010.
Wit15a: Michael Witmore. Finding “distances” between shakespeare’s plays 1. Jun 2015. URL: http://winedarksea.org/?p=2225.
Wit15b: Michael Witmore. Finding “distances” between shakespeare’s plays 2: projecting distances onto new bases with pca. Jul 2015. URL: http://winedarksea.org/?p=2271.
Wit16: Michael Witmore. Latour, the digital humanities, and the divided kingdom of knowledge. New Literary History, 47(2):353–375, 2016.
WH19: Michael Witmore and Jonathan Hope. A map of early english print. Apr 2019. URL: http://winedarksea.org/?p=2703.
WHG16: Michael Witmore, Jonathan Hope, and Michael Gleicher. Digital approaches to the language of shakespearean tragedy. In The Oxford Handbook of Shakespearean Tragedy, pages 316–335. 2016.