Bookworm data hack lets you map every word in 100,000 screenplays
Hat tip to longtime reader Daniel Cossu for sending along this Guardian article:
If you’ve ever wanted to find out that Steven Soderbergh never put the word ‘sex’ in his films in the late 90s, or that ‘fuck’ has overtaken ‘shit’ as South Park’s curse of choice, you’re in luck: an ingenious data visualisation by US professor Benjamin Schmidt is mapping all the words used in 100,000 films and TV show episodes.
It uses the Bookworm tool, which has already visualised the words used in science publications, newspapers and Congress bills, and scans a database of captions to draw down its data.
You can tailor your enquiry to an impressive degree, searching the words used in individual shows like Friends and Family Guy, or by individual film directors including John Ford, Claude Chabrol, and Takashi Miike. There’s also scope for searching by genre, language, film studio and even filming location. Once you’ve got your search parameters, a graph is created with frequency of the words used plotted over the last few decades.
Some search examples:
* Across all film and TV, “sex” became more and more popular as the 20th century went along, reaching an apex in 2008 — since when it has markedly dropped off. Maybe we’re not all so single-minded after all.
* Doctor Who mentioned “time” a faintly ridiculous 4,400 times per million words for its 1983 episodes, later settling back down to its usual level of 1,500.
* For Steven Spielberg, “love” always conquers “dead”, with Saving Private Ryan notching a big 1,205 mentions of love. Only Indiana Jones and the Temple of Doom sees more “dead” mentions (see below).
I did my own search for the word “beer” in the TV series “The Simpsons”. The results are here. Evidently the threat of Homer becoming an alcoholic is decreasing over the years.
Unfortunately Professor Schmidt has yet to go through the TV series “True Detective” as I wanted to check the use of the words “time” and “circle”. However there’s this:
* The cast of Lost, much like the audience, used the word “confused” more and more as the seasons passed.
Sounds about right!
For the rest of the article, go here.