Word Frequency in Shakespeare's Plays

What do geeks do over Holiday break? One option is devoting some time to hobbies and interests that they often can't, because of work getting in the way? In my case, that will mean seeing whether interesting bits can be learned my combining Shakespeare and data analysis.

I'm still thinking a lot about what directions I can take this analysis. An obvious first step is doing some kind of word frequency analysis to see what that might indicate about the play. Given a decent database containing Shakespeare's work and a little programming ability...let's see what we can come up with!

I've had a small C# app for a while that can request string searches and full downloads from a Shakespeare data source online. Recently, I added code that breaks down the lines into individual words and then counts how often each appears in a play, sorting those totals from most common to least common. Let's see if that will provide any interesting results...comparing one of the Bard's most-loved comedies (As You Like It) with the harsh, Machiavellian play Richard III.

As You Like It

A pastoral comedy, As You Like It was probably written in 1599; it was not published until 1623's First Folio. It contains many ingredients that Shakespeare had been incorporating into his plays:
- young lovers on the run from forces that would destroy them
- the wilderness of a forest that, ironically, has fewer and more direct dangers than the "civilized world"
- a woman disguising herself as a man so that she can wander the world more safely
- a fool (Touchstone)
- a cynic (Jacques)

It's main character, Rosalind, is widely regarded as one of Shakespeare's finest creations. Though she herself is showing the slightly daft behavior typically shown by young people in love, she has the self awareness to know that she is acting that way. In her disguise as a boy, she can thoroughly sound out her intended Orlando's thoughts and feelings about her, knowing that any artifice he might show in wooing her as a girl will not be present. In this natural world, she also gets to see him capably facing danger (snakes, lions) but also showing nurturing behavior for an aging retainer who had earlier helped him avoid a threat to his life.

Though there are some serious threats from man and beast in this play, this is truly one of Shakespeare's happiest comedies.

Richard III
Fact or fiction, true chronicle or Tudor propaganda...wherever it play falls it is a deeply engrossing and popular play. Coming at the end of Shakespeare's History plays about the last Plantagenets, Richard III is the story of the Duke of Gloucester, a noble son of Henry VI's Duke of York who aims to win the crown for himself at any cost. He eventually succeeds but only through means that eventually doom him.

Early in the play, his arc ascendant, Gloucester successfully woos a woman whose husband and father-in-law he had earlier killed in the play 3 Henry VI. Later, having ordered the murder of two young nephews, he tries to duplicate this feat and fails...without even realizing that he failed. A consummate actor, he begins the play emotionally seducing watchers of the play. But his crimes become too rank and he ends the play so utterly alone that even he does not love himself.

Lets take a look at the frequency with which words appear in this play and see whether they correspond to the plots:

Richard III is approximately 30,000 words long. There are about 4100 distinct words in this play.

The most common word in it:
the         989 times

In any play, many of the most common words are going to be names, pronouns and articles such as "the", so leaving them in serves no value. When I removed those words, these were some of the words that jumped out as useful to note, along with their frequency:

king        262
lord         240
queen      176
death         72
murderer    67
love           64
prince        48
blood         39
dead          38
noble         37
die            36
tower        30
royal         28
bloody       28
ghost         19
horse         17
   :
angry          6
leisure         6
slaughter     6





It's probably not surprising that these words overwhelmingly pertain to royal titles, violence and death. And even where the word "love" appears, it does not have the happy connotations with which it is usually endowed. Rather, it has a sense of irony and contempt: Gloucester wooing the Lady Anne over the corpse of Henry VI, his claims to love the brother Clarence who he is conspiring to kill or the forced amity of a dying King Edward IV trying to quell family hostility before he dies.

This is not a cheerful play.

Now let's compare this to the word frequencies in As You Like It, which has about 22000 words in total (3280 distinct words).

Again, "the" is the most frequently appearing word, occurring 694 times.The most significant words include:


good     115
love       111
duke       93
forest      39
brother    37
father      37
fool         36
heart        31
shepherd  25
marry      22
daughter  18
kill            9

Again unsurprisingly, the words in this play are usually positive words about relationships. "Good" and "love" are two of the most frequently appearing words. And in contrast to Richard III, "love" is not a cynical affectation. There is humor about the behavior of lovers but it is not a cynical humor.

The word duke appears pretty frequently, as there are two "dukes" in the play. The forest (Arden) appears frequently, as it is the main setting of the play. There are words pertaining to danger but they are a minor point.

This IS a cheerful play about finding love and happiness in life. This sense is reflected well with the words Rosalind uses to close the play:

    It is not the fashion to see the lady the epilogue;
    but it is no more unhandsome than to see the lord
    the prologue. If it be true that good wine needs
    no bush, 'tis true that a good play needs no
    epilogue; yet to good wine they do use good bushes,
    and good plays prove the better by the help of good
    epilogues. What a case am I in then, that am
    neither a good epilogue nor cannot insinuate with
    you in the behalf of a good play! I am not
    furnished like a beggar, therefore to beg will not
    become me: my way is to conjure you; and I'll begin
    with the women. I charge you, O women, for the love
    you bear to men, to like as much of this play as
    please you: and I charge you, O men, for the love
    you bear to women--as I perceive by your simpering,
    none of you hates them--that between you and the
    women the play may please. If I were a woman I
    would kiss as many of you as had beards that pleased
    me, complexions that liked me and breaths that I
    defied not: and, I am sure, as many as have good
    beards or good faces or sweet breaths will, for my
    kind offer, when I make curtsy, bid me farewell.