By Aaron Foote ’24
This post describes the project I did last summer as part of the QAC Summer Apprenticeship. The project was done in support of the work of the Wesleyan Media Project and the DeltaLab. I developed a method for efficiently extracting text from videos and, in the process, got some insights on how sharp human faces are when examined by computer vision algorithms.
In case you have not noticed from the multiple TV ads, for a few years now IBM has been positioning itself as a Big Data company, with its Watson platform and cloud-based services. One of them is the Alchemy Language API, which packs together functions for text analysis and information retrieval. As part of learning how to handle this API from R, I tried it on a news story about a sci-fi book publishing business. Overall, the results were strong, although not without some amusing quirks…
Sundays seem natural for large TV events. Why wouldn’t they? NFL’s Super Bowl has been on Sundays forever. It feels like the proper order of things that the Academy Awards ceremony is also on a Sunday. Every year, somewhere near the end of February, start of March. Yet, a simple dataset of telecast dates points out that this practice is a relatively recent phenomenon and for a long while things were quite different. For a quick summary of the data, look at the chart below: it shows the progression of the ceremony dates from the most distant (1953) to the closest (2014). For more details on why the changes occurred, keep reading on.
Twitter has emerged as a convenient source of data for those who want to explore social media. The company provides several access endpoints through APIs. There is a REST API for collecting past tweets and a streaming API for collecting tweets in real time. R has libraries for working with both. As is usual in data collection, the catchphrase is “more” – we want more tweets, ideally all that are relevant to our research question. While REST API is rate-limited (a user can submit 180 requests per 15 minutes, with each request returning 100 tweets), the streaming API holds a promise of delivering much more. The nagging question, though, is “how much?”
