The Open Syllabus Project is happy to to have Overview, an open-source document analysis and visualization system originally developed at the Associated Press for investigative journalists, as our newest project partner. The Overview team recently welcomed its own new addition, David McClure, who will be helping the OSP put 2 million scraped syllabi online, do natural language processing to extract citations from each syllabus, and build visualizations to do citation analysis on our massive corpus.
To extend our welcome to David and the rest of our new partners, we wanted to briefly introduce the Overview project and how its goals wonderfully correspond with the OSP’s. As an open-source tool originally designed to help journalists find stories in large numbers of documents, Overview automatically sorts documents according to topic and provides a fast visualization and reading interface. Since right now the OSP has accumulated more than 2 million syllabi, we’re glad to have Overview’s expertise on search and UI to help make all our data user-friendly and accessible.
Syllabi are increasingly messy documents, lacking standardization in both structure and, of course, content. That’s why Overview will be especially helpful for our task: Overview is designed specifically for text documents where “the interesting content is all in narrative form.” It’s been used to analyze emails, declassified document dumps, material from Wikileaks releases, social media posts, online comments, and now syllabi.
As its goal is to make advanced document mining capability available to anyone who needs it, Overview will critically support OSP’s citation analysis and back-end infrastructure. Ultimately, Overview will help us rigorously examine the millions of syllabi we currently have, their implications, and their impact. We’re looking forward to making the results of this analysis public for further discussion.
Overview is a project of The Associated Press, supported by the John S. and James L. Knight Foundation as part of its Knight News Challenge. Learn more about Overview and what the project has accomplished here.