Library Blog

Topic Modeling output for a Zotero collection using Paper Machines

  Share

Paper Machines, the add-on that integrates a range of text analysis tools into Zotero, has generated quite a buzz in the short period of time since its release. For those of us that store notes, citation information, PDFs, and article links in huge Zotero libraries, Paper Machines has the potential to be a game-changer in terms of how we visualize our research.

Because Paper Machines is so new, it's being updated with added functionality every few days. I'll provide step-by step documentation for how to use specific components of Paper Machines in Part II of this post. For now, I'll discuss whether or not Paper Machines might be a good fit for your research, the tools that it offers, and how it might help your work.

Paper Machines provides a broad range of text analysis tools, but it's not meant for everyone's research. You'll probably benefit most from Paper Machines if you:

1.  Already use Zotero to manage your sources. Paper Machines draws on a number of open source tools available elsewhere on the web. If you want to visualize your data but aren't already comfortable using Zotero, you might want to look elsewhere.
2. Have a relatively large or robust Zotero library. At the time of this posting, Paper Machines incorporates the full text of Web snapshots and OCR'd PDF files into its text analysis, as well as the title, place, date, and subcollection of a source. The option to include notes, tags, and links to live websites will be available shortly.
3.  Are collaborating on a Zotero library with a group. Paper Machines is very good at helping you figure out the contents of a collection. If you're working on a collection with multiple group members, it's a quick way to visualize what kinds of material your collaborators are adding.

What kinds of analysis tools does Paper Machines employ?

• A word cloud with the option to filter out commonly used words.
• Phase nets, which allow you to visualize relationships between common words in your text (for example, x and y; x is y)
• A Geoparser, which uses location information to produce beautiful visualizations of the places mentioned in your texts.
• DBpedia Annotation, which produces a visualization of what people, places, and things are mentioned in your texts.
• MALLET-based topic modeling, which generates visualizations based on commonly occurring topics in your texts. The author offers some additional information about information about Paper Machines' use of topic modeling here.

What can Paper Machines help you do?

• Assess the contents of a collection. Looking through the Paper Machines results is a helpful way to get to know the contents of a group library or to get reacquainted with a collection that you haven't used for a while.
• Identify gaps in your material. Reviewing the MALLET output for a specific collection in my Zotero library (canonical works in US history) I noticed a surge in books about women's labor history (which MALLET identified using the terms women, labor, work, and activism) during the 1980s. I also noticed a lack of items in my library about these topics since 2000.
• Compare collections. Analyzing two collections with Paper Machines makes similarities and differences evident. Using topic modeling, for example, I could see what subjects came up most frequently in the two collections and if they coincided. The word cloud function is the easiest way to identify concurrent terms and subjects at a glance.
• Find patterns in your collections. Using the "Phrase Net" function, I conducted an "x is y" analysis on one of my collections. I was surprised to see that "democracy is necessary" and "Cold War is necessary" were recurring phrases a number of sources.

The Geoparser links texts in a Zotero collection to the places that they mention.

Additional examples of Paper Machines visualizations are available on the developer's site. The add-on is available for Firefox and Zotero Standalone, and visualizations  can also be saved as html files. While the occasional error or puzzling result is inevitable early on, the creator of Paper Machines is constantly tweaking the interface in response to feedback from users.

Reading room in the Georgia State Archives in Morrow, GA,
image courtesy of the Georgia State Archives

  Share

Late on the afternoon of September 13, Georgia Secretary of State Brian Kemp announced that on November 1 he would be eliminating all public access hours and several staff positions at the Georgia State Archives.  His decision was in response to a mandate from Governor Nathan Deal that all state agencies reduce their budgets by 3%.  The original press release from the Secretary of State promised that the Archives would remain open by appointment, but that those appointments would be limited based on the schedule of remaining staff.  On Friday, September 14, Kemp announced that seven of the ten current Archives employees would lose their jobs effective October 31.  This will leave the Archives with two professional archivists and one facilities manager on staff, and as a result the Georgia public can expect little to no access to its public record.

The Coalition to Save the Georgia Archives, which includes a number of organizations such as the Friends of Georgia Archives and History, the Society of American Archivists, the Association County Commissioners of Georgia, and the Georgia Historical Records Advisory Board, has launched an aggressive campaign to keep the Georgia State Archives open.  The State Archives has already been decimated by consistent budget cuts over the last several years.  At stake is not only access to these records for historical scholarship and family history research, but the civil rights of Georgia citizens and the transparency and effectiveness of the state government of Georgia.  Unfortunately, two archivists are not capable of offering an appointment schedule that will satisfy the legal mandate to make these records available to the public.  

Governor Deal has promised that the archives will remain open; however, he has yet to offer a detailed plan and has stated that he has no ability to overturn Secretary Kemp's decision.  If Kemp's decision stands, the Archives will suffer a loss of talent and reputation it will take years to recover.  Please stand up for the Archives and contact the Secretary of State's office and your local legislators.  They need to know that you support access to Georgia's public record.  You can find your legislators and their contact information at votesmart.org.

  Share

Beck Center Grad Fellow Sara Palmer tells us how MALLET works and what you can (and cannot) do with Topic Modeling

This past summer I have enjoyed working with a tool called MALLET (MAchine Learning for LanguagE Toolkit) that can generate a series of potential topics in a corpus of texts.  Created by Andrew McCallum at University of Massachussetts, it is an open source toolkit and that is fairly easy to download and install.  While the setup instructions on the website are minimal, Shawn Graham’s has written a helpful overview of how to get started.  

Topic modeling is based on the idea that within a set of related texts, certain words will occur near each other with statistically significant frequency. MALLET works on documents like its acronymic namesake by shattering texts into an array of words.  It then applies a statistical method called Latent Dirichlet Allocation to put related words into clusters. 

These related groups of words, or ‘word bags’ as they are often called, can be interpreted as making up a topic.  For example, the group of words ‘needle,’ ‘stitching,’ and ‘fabric’ could reasonably be labeled as the topic ‘sewing.’  Of course, scholars familiar with a corpus can readily name a list of topics but what if you’re working with a large set of texts and don’t have time to read all of them carefully?  MALLET allows you to quickly obtain a general idea of what topics may be present.  

Perhaps the most prominent example of topic modeling with MALLET is Cameron Blevins’s work on the diary of Martha Ballard, an American midwife who wrote daily entries for 27 years.  Blevins’s topic model identified key themes in the diary such as ‘death,’ ‘shopping’ and ‘gardening’ and graphed their prominence over the course of Ballard’s life.  These charts often corresponded with known biographical details.  For example, the topic ‘emotion’ peaked between 1803 and 1804 when her husband was imprisoned for debt and her son was indicted for fraud.   

The field of DH itself has also been put under the MALLET “macroscope.”  Elijah Meeks’s work visualizes the network of associated ideas in the self-definitions of digital humanists while Matt Jockers analyzes the work performed in the field by culling themes from the blog posts of 117 digital humanists on a single day (March 18, 2010).  The Maryland Institute of Technology in the Humanities (MITH) has an excellent overview of how topic modeling has been used in the humanities.  Other prominent topic modeling projects featured on the MITH blog include Travis Brown’s work on Austen and Byron and Jeff Drouin’s work on Proust. 

Interest in topic modeling has grown at DiSC since it hosted a talk by Robert Nelson from the University of Richmond this past January.  In his research on the American Civil War, Nelson ran MALLET on the issues of Richmond Daily Dispatch newspaper from 1860-1865.  He was able to graph and contextualize topics such as fugitive slave ads and military recruitment, the results of which are illustrated beautifully on the project’s website.  For each topic Nelson has a separate page displaying the graphs of its prevalence over time, the most prominent key words, as well as the articles, ranked by compositional percentage, that most clearly exemplify the topic.  

These two components, keywords and compositional percentage, can be respectively derived from the MALLET output files ‘topic-keys’ and ‘doc-topics.’  At the Beck Center we wanted to see how this output might be used for understanding the journal Southern Changes, published monthly by the Atlanta-based Southern Regional Council from 1979 through 2003.  Comprised of 110 issues containing 978 articles, it is considerably smaller than the data set Nelson worked with but still large enough to make it a good fit for MALLET’s method.  We hoped to identify key terms with which to catalogue the collection and also to generate topical groups of articles that could be featured on the website.  The results were encouraging as I found the themes suggested by the topic keyword lists to roughly correspond with the subjects provided by Allen Tullos, editor of Southern Changes from 1982 through 2003.  Tullos writes:

 “The articles in Southern Changes range across many subjects: racial justice and the freedom struggle, voting rights, educational opportunity, economic democracy, social equality and inclusion, women's rights, environmental justice, critical regional studies, regional-global issues, and popular culture.”

After discovering a ten-topic run of MALLET to be a bit broad, I opted for 20 topics.  In THIS TABLE I have labeled the topic keyword lists with my overall impressions in black.  The generally correspondent Tullos subjects—labeled in green—are included for most of the MALLET topics.  This is, of course, a very rough sketch but it does appear to suggest correlation between MALLET data and expert knowledge.

As a model built on sampling and probability, MALLET naturally does not generate the same exact output each time it is run.  The topics and keywords do vary but are generally similar from one run to the next.  Below is a table comparing the keywords from the ‘Voting Rights’ topic in the previous example to those produced in five additional runs.  The original run was based on 10,000 iterations but for the sake of time, I used 1,000 iterations on the additional trials.  VIEW THOSE RESULTS HERE

While the topic-keys outline the word composition of the topics, the doc-topics indicates the topic composition of each document.  It simply lists, in descending order, the ID number of each topic and the percentage of the document that it makes up.  CLICK HERE for a table of the documents in which ‘Voting Rights’ comprises over 40% of the composition.

From this one can get a clearer picture of what kind of content is picked up by MALLET for the ‘Voting Rights’ topic and see that the topic is most prevalent in the journal around the 1980 and 2000 election cycles. 

In sum, MALLET did what we wanted it to do: we obtained key terms and a general overview of topics present.  My own attempts to graph the topics over time did not illustrate trends with nearly as much clarity as I would have hoped.  However, my objective is not to prove that MALLET is some kind of magic bullet, ripping through a corpus to the heart of its intelligibility.  Rather, I found that the output, when interpreted cautiously, can offer an excellent starting point for more detailed content analysis.  

  Share

Valentine's Day, 1989: the day when everything changed for Salman Rushdie. It was the day that Ayatollah Khomeini of Iran issued a fatwa ordering Rushdie's execution because of his novel The Satanic Verses, a book that many (despite never reading the novel itself) claimed was blasphemous to Islam. That spring, violent protests against the book broke out across the world, while Rushdie went into hiding under the protection of the British Royal Police. Eventually, the protests died down, but for Rushdie, that day marked the beginning of almost a decade of living in the shadows, worrying about the safety of his loved ones, and moving from place to place when a new threat appeared. Rushdie's newly released memoir tells the story of this decade in his life. The book title Joseph Anton: A Memoir, references the pseudonym he used while living in hiding, itself a literary reference to Joseph Conrad and Anton Chekov.

Rushdie's source material for his memoir was not only drawn from his memory; he researched his own archives which are held in MARBL. Emory's archives "actually allowed me to write the memoir," says Rushdie, speaking during a March 2, 2012 discussion in the Research Commons of the Robert W. Woodruff Library during his recent visit as University Distinguished Professor. When Emory acquired Rushdie's archive, it was a massive collection of cardboard boxes, the contents of which were filled with a chaotic jumble of drafts, grocery lists, letters, scribbles, newsclippings and computers. Simply put, it was "a complete mess," Rushdie admits. "There was no organization — 100 boxes of everything and I didn't even know what was there."

MARBL spent several years processing and organizing Rushdie's collection, which also made it possible for the celebrated author to tackle in-depth research for his memoir. As a researcher dedicated to preserving fact, Rushdie knows firsthand that relying upon memory alone has its dangers, making original documentation essential. While it might seem a bit odd to the average person—going to an archive library in order to research one's life—MARBL's intensive processing of the collection allowed Rushdie to access materials and details that might have otherwise been buried in his memory or hidden in the recesses of a computer hard drive. Rushdie's research process is also mirrored in the style of the memoir itself—calling his attempt to write a memoir in the first person "too narcissistic" he eventually decided to use the third-person and treat Joseph Anton, his pseudonymic self, as a character.

For more information on visiting MARBL to view the Salman Rushdie collection in our reading room, please go to our website or email marbl@emory.edu.

Even if data visualization isn't the primary goal of a project, adding an animated or interactive map can be an effective way to enrich a presentation, article, or lecture--and it doesn't have to take up huge swaths of time. As part of the Tweeting Occupy Wall Street project, I tested web-based mapping tools that would allow us to plot some of the 10 million tweets related to Occupy Wall Street.

Dozens of geographic data visualization tools, many of them open-source, are available on the web, but for this particular project I investigated tools that are 1) powerful enough to handle large data sets; 2) relatively easy to learn and share; and 3) free. Here's a rundown of the two tools that I found to be most effective in the Tweeting #OWS project: OpenHeatMap and Geocommons.

OpenHeatMap allows users to create static or animated heat maps (also called intensity or chloropleth maps) based on data uploaded through a Google Doc or Excel spreadsheet. Heatmaps plot values in a range of colors that indicate intensity, similarly to a meteorological radar map. One of the most user-friendly tools that I tested, all OpenHeatMap requires to generate a map is 1) location information, in the form of latitude and longitude or state/country abbreviations; 2) a column of values (used to plot intensity); and 3) if you want an animated heatmap, an optional column marked "time."

Customization options in OpenHeatMap allow creators to control features like color and size of the data and map. After customizing, a user can simply use an autogenerated code to embed the map into an website. Alternatively, Open HeatMap offers an option to host the map on your own site and fully customize it using the Heatmap API. My attempt to do so was unsuccessful, however, and and a number of sources suggest simply using the embed code to store and share a map.

More than a site for creating maps, Geocommons is a robust data analysis, management, and visualization platform. Like its name suggests, Geocommons embraces the open-source model and strongly encourages users to make their maps and data public (20 megabytes of private data storage is also available with a free membership). This means that, along with uploading data, users can access hundreds of data sets including census data, zip code and county maps, and much more.

It's possible to run analyses on data from within Geocommons, but I found it to be much faster to do the process in Excel or Google Docs first and upload the finished dataset that contained the values I wanted to plot. Geocommons makes it easy to aggregate data into non-map geographic visualizations, like this chart I make of the top states with Twitter activity related to Occupy Wall Street.

Geocommons serves up simple embed codes for its maps, as well as a customizable Javascript API geared toward site developers. Like OpenHeatMap, embedding maps is either straightforward and noncustomizable or tricky and flexible. I'd suggest using the standard embed code unless you have some experience with Javascript.

  Share

  Share

A new project from the Digital Scholarship Commons provides analysis of an archive of 10 million tweets from the Occupy Wall Street movement.

Since 13 October 2011, Emory University Library has collected over ten million tweets from the Occupy Wall Street Movement. For the anniversary of the first day of the Fall 2011 demonstrations, DiSC created a site that provides some visualizations of the tweets—heat maps, word clouds, and charts—as well some analysis.

We encourage you to take a look, share with friends, and let us know what you think.