An introduction to using computers to analyze large amounts of text, or distant reading.
Learning objectives:
Student will be able to:
-
Enter URLs or text into Voyant.
-
Demonstrate an understanding of the Voyant tool through counting words and drawing conclusions from those counts.
-
Demonstrated in their paragraphs the use of stopwords.
-
Demonstrate that they can draw conclusions based on their use of Voyant, including word counts and word patterns.
-
Clearly link secondary source material to their primary source Voyant analysis.
1. Distant vs. close reading. A lesson using historical religious and popular culture texts.
Most reading we do is "close reading." We read each word, place each word in the sentence or context, and then create meaning out of the words all strung together. For example, "today, I ate cake." You must read those words in context and in an order to understand those words.
Sometimes, we read in ways that aren’t so "close." For example, if you go to a weather website, and look up the forecast, you don’t read all the words in their context. You scan for the information you need, and ignore the rest. This is the first step to distant reading: recognizing that not all information included in a text is relevant and looking only for the material (or data) that is important.
For part of the Words module, we’re going to use distant reading websites to analyze large amounts of text. For example, below I’ve place a URL https://github.com/jackhistorynorton/history_1101/blob/master/readings/Mahabharata_Gutenberg.txt of the entire text of the Mahabarata, the other great Indian epic (along with the Ramayana) into Voyant. This tool counts words and looks for patterns. It is almost impossible to count words in large numbers for multiple books as a human, but computers can do it for us. This is what "distant reading" means: humans are away from the texts and computers "hold" and manipulate the texts.
You can access Voyant at https://voyant-tools.org/
or at the mirrors Voyant lists on the bottom of their site. If these sites are not responding, you can also download and install a local version of Voyant on your own computer.
2. The entire Mahabarata, by its words and numbers.
I clicked "Reveal" and Voyant has now analyzed the entire text, and counted every word, generating what we call a word cloud. A word cloud shows the words used most often in a text. More popular words or symbols are bigger. in the word cloud below, "said," "great," "continued," and "like" are the most popular, which is useless to us. So, we need to tell Voyant to edit out those common words. We call common words we don’t want "stop words."
3. Editing stopwords
I clicked on the switch icon below the word cloud and it will give me the option to add Stopwords.
4. Select Edit List and make sure "Apply Globally" is checked
You can also manually add words for Voyant to ignore by clicking on "Edit List."
5. Add any words to the list that you don’t want.
Here I’ve added "said," "unto," "hath," and "continued," and several other common words, hitting the return key after each word and then hitting "Save" and then "Confirm."
6. New word cloud revealed.
New words emerge as clicking on any slider icon in the upper right gives you more options.
7. Clicking on icons gives you access to more options.
Clicking on slider icon reveals more tools.
Clicking on arrow in a box icon reveals more of that section of Voyant.
8. What matters more: son, daughters, wife, or husband.
I added the words daughter, son, husband, and wife to the trends tool on the right (1).
Voyant split the Mahabarata into 10 segments, and shows the number of times each word is used in each part of the ancient Indian epic.*
Looking at this graph, I might be tempted to argue that this text is most concerned with sons. Still, I would need to know more before I offered that argument. Voyant just counts words and shows those counts. It doesn’t offer explanations for anything.
9. How can distant reading help us understand the past?
As humans we can only read a limited number of words at a time. Computers in the form of software, however, can "read" huge numbers of words, entire libraries in fact, fairly easily. More importantly, software can count, compare, and display patterns in ways we can’t.
Computers also help by revealing patterns that ore contrary to our assumptions, which is especially important when studying religion.
We make certain assumptions about word counts in historical texts. That is, we assume that a word with a higher frequency mattered more to the writer than a word with lower frequency.
Assignment
-
Enter two of the following texts into Voyant:
-
Enter stop words to eliminate irrevelant words.
-
Use word counts in your respective accounts to suggest what objects or ideas were important to the authors of these texts.
-
Connect your analysis of word counts to one of the secondary source background readings we did for this semester.
You will prouduce two short paragraphs (one for each text your chose) of fewer than 200 words.
- Voyant tools site 1 and alternative site 2 and setting up your own version of Voyant (https://www.voyant-tools.org/docs/#!/guide/server) on your own computer.
Grading Criteria
Student
-
Entered URLs or text into Voyant.
-
Demonstrated an understanding of the Voyant tool through counting words and drawing conclusions from those counts.
-
Demonstrated in their paragraphs the use of stopwords.
-
Demonstrated that they can draw conclusions based on their use of Voyant, including word counts and word patterns.
-
Linked secondary sources to primary source analysis.