Word Frequency in Shakespeare's Plays

What do geeks do over Holiday break? One option is devoting some time to hobbies and interests that they often can't, because of work getting in the way? In my case, that will mean seeing whether interesting bits can be learned my combining Shakespeare and data analysis.

I'm still thinking a lot about what directions I can take this analysis. An obvious first step is doing some kind of word frequency analysis to see what that might indicate about the play. Given a decent database containing Shakespeare's work and a little programming ability...let's see what we can come up with!

I've had a small C# app for a while that can request string searches and full downloads from a Shakespeare data source online. Recently, I added code that breaks down the lines into individual words and then counts how often each appears in a play, sorting those totals from most common to least common. Let's see if that will provide any interesting results...comparing one of the Bard's most-loved comedies (As You Like It) with the harsh, Machiavellian play Richard III.

As You Like It

A pastoral comedy, As You Like It was probably written in 1599; it was not published until 1623's First Folio. It contains many ingredients that Shakespeare had been incorporating into his plays:
- young lovers on the run from forces that would destroy them
- the wilderness of a forest that, ironically, has fewer and more direct dangers than the "civilized world"
- a woman disguising herself as a man so that she can wander the world more safely
- a fool (Touchstone)
- a cynic (Jacques)

It's main character, Rosalind, is widely regarded as one of Shakespeare's finest creations. Though she herself is showing the slightly daft behavior typically shown by young people in love, she has the self awareness to know that she is acting that way. In her disguise as a boy, she can thoroughly sound out her intended Orlando's thoughts and feelings about her, knowing that any artifice he might show in wooing her as a girl will not be present. In this natural world, she also gets to see him capably facing danger (snakes, lions) but also showing nurturing behavior for an aging retainer who had earlier helped him avoid a threat to his life.

Though there are some serious threats from man and beast in this play, this is truly one of Shakespeare's happiest comedies.

Richard III
Fact or fiction, true chronicle or Tudor propaganda...wherever it play falls it is a deeply engrossing and popular play. Coming at the end of Shakespeare's History plays about the last Plantagenets, Richard III is the story of the Duke of Gloucester, a noble son of Henry VI's Duke of York who aims to win the crown for himself at any cost. He eventually succeeds but only through means that eventually doom him.

Early in the play, his arc ascendant, Gloucester successfully woos a woman whose husband and father-in-law he had earlier killed in the play 3 Henry VI. Later, having ordered the murder of two young nephews, he tries to duplicate this feat and fails...without even realizing that he failed. A consummate actor, he begins the play emotionally seducing watchers of the play. But his crimes become too rank and he ends the play so utterly alone that even he does not love himself.

Lets take a look at the frequency with which words appear in this play and see whether they correspond to the plots:

Richard III is approximately 30,000 words long. There are about 4100 distinct words in this play.

The most common word in it:
the         989 times

In any play, many of the most common words are going to be names, pronouns and articles such as "the", so leaving them in serves no value. When I removed those words, these were some of the words that jumped out as useful to note, along with their frequency:

king        262
lord         240
queen      176
death         72
murderer    67
love           64
prince        48
blood         39
dead          38
noble         37
die            36
tower        30
royal         28
bloody       28
ghost         19
horse         17
   :
angry          6
leisure         6
slaughter     6





It's probably not surprising that these words overwhelmingly pertain to royal titles, violence and death. And even where the word "love" appears, it does not have the happy connotations with which it is usually endowed. Rather, it has a sense of irony and contempt: Gloucester wooing the Lady Anne over the corpse of Henry VI, his claims to love the brother Clarence who he is conspiring to kill or the forced amity of a dying King Edward IV trying to quell family hostility before he dies.

This is not a cheerful play.

Now let's compare this to the word frequencies in As You Like It, which has about 22000 words in total (3280 distinct words).

Again, "the" is the most frequently appearing word, occurring 694 times.The most significant words include:


good     115
love       111
duke       93
forest      39
brother    37
father      37
fool         36
heart        31
shepherd  25
marry      22
daughter  18
kill            9

Again unsurprisingly, the words in this play are usually positive words about relationships. "Good" and "love" are two of the most frequently appearing words. And in contrast to Richard III, "love" is not a cynical affectation. There is humor about the behavior of lovers but it is not a cynical humor.

The word duke appears pretty frequently, as there are two "dukes" in the play. The forest (Arden) appears frequently, as it is the main setting of the play. There are words pertaining to danger but they are a minor point.

This IS a cheerful play about finding love and happiness in life. This sense is reflected well with the words Rosalind uses to close the play:

    It is not the fashion to see the lady the epilogue;
    but it is no more unhandsome than to see the lord
    the prologue. If it be true that good wine needs
    no bush, 'tis true that a good play needs no
    epilogue; yet to good wine they do use good bushes,
    and good plays prove the better by the help of good
    epilogues. What a case am I in then, that am
    neither a good epilogue nor cannot insinuate with
    you in the behalf of a good play! I am not
    furnished like a beggar, therefore to beg will not
    become me: my way is to conjure you; and I'll begin
    with the women. I charge you, O women, for the love
    you bear to men, to like as much of this play as
    please you: and I charge you, O men, for the love
    you bear to women--as I perceive by your simpering,
    none of you hates them--that between you and the
    women the play may please. If I were a woman I
    would kiss as many of you as had beards that pleased
    me, complexions that liked me and breaths that I
    defied not: and, I am sure, as many as have good
    beards or good faces or sweet breaths will, for my
    kind offer, when I make curtsy, bid me farewell.

Some Favourite Shakespeare Quotes

The Merchant of Venice


Quote #1
Portia

The quality of mercy is not strain'd,
It droppeth as the gentle rain from heaven
Upon the place beneath: it is twice blest;
It blesseth him that gives and him that takes:
'Tis mightiest in the mightiest: it becomes
The throned monarch better than his crown;
His sceptre shows the force of temporal power,
The attribute to awe and majesty,
Wherein doth sit the dread and fear of kings;
But mercy is above this sceptred sway;
It is enthroned in the hearts of kings,
It is an attribute to God himself;
And earthly power doth then show likest God's
When mercy seasons justice.

Quote #2

Bassanio

(Who has just solved the puzzle left by Portia's
father and is entitled to marry her...gives the
final choice about whether he is worthy to her)

Reads:
You that choose not by the view,
Chance as fair and choose as true!
Since this fortune falls to you,
Be content and seek no new,
If you be well pleased with this
And hold your fortune for your bliss,
Turn you where your lady is
And claim her with a loving kiss.

Speaks to Portia:
A gentle scroll. Fair lady, by your leave;
I come by note, to give and to receive.
Like one of two contending in a prize,
That thinks he hath done well in people's eyes,
Hearing applause and universal shout,
Giddy in spirit, still gazing in a doubt
Whether these pearls of praise be his or no;
So, thrice fair lady, stand I, even so;
As doubtful whether what I see be true,
Until confirm'd, sign'd, ratified by you.

Hamlet

Quote #1

Hamlet

'Tis well: I'll have thee speak out the rest soon.
Good my lord, will you see the players well
bestowed? Do you hear, let them be well used; for
they are the abstract and brief chronicles of the
time: after your death you were better have a bad
epitaph than their ill report while you live.

Polonius

My lord, I will use them according to their desert.

Hamlet

God's bodykins, man, much better: use every man
after his desert, and who should 'scape whipping?
Use them after your own honour and dignity: the less
they deserve, the more merit is in your bounty.

Quote #2

Hamlet

I do not well understand that. Will you play upon
this pipe?

Guildenstern

My lord, I cannot.

Hamlet

I pray you.

Guildenstern

Believe me, I cannot.

Hamlet

I do beseech you.

Guildenstern

I know no touch of it, my lord.

Hamlet

'Tis as easy as lying: govern these ventages with
your lingers and thumb, give it breath with your
mouth, and it will discourse most eloquent music.
Look you, these are the stops.

Guildenstern

But these cannot I command to any utterance of
harmony; I have not the skill.

Hamlet

Why, look you now, how unworthy a thing you make of
me! You would play upon me; you would seem to know
my stops; you would pluck out the heart of my
mystery; you would sound me from my lowest note to
the top of my compass: and there is much music,
excellent voice, in this little organ; yet cannot
you make it speak. 'Sblood, do you think I am
easier to be played on than a pipe? Call me what
instrument you will, though you can fret me, yet you
cannot play upon me.

As You Like it

Rosalind

I am not
furnished like a beggar, therefore to beg will not
become me: my way is to conjure you; and I'll begin
with the women. I charge you, O women, for the love
you bear to men, to like as much of this play as
please you: and I charge you, O men, for the love
you bear to women--as I perceive by your simpering,
none of you hates them--that between you and the
women the play may please. If I were a woman I
would kiss as many of you as had beards that pleased
me, complexions that liked me and breaths that I
defied not: and, I am sure, as many as have good
beards or good faces or sweet breaths will, for my
kind offer, when I make curtsy, bid me farewell.

Richard III

Queen Margaret

And leave out thee? stay, dog, for thou shalt hear me.
If heaven have any grievous plague in store
Exceeding those that I can wish upon thee,
O, let them keep it till thy sins be ripe,
And then hurl down their indignation
On thee, the troubler of the poor world's peace!
The worm of conscience still begnaw thy soul!
Thy friends suspect for traitors while thou livest,
And take deep traitors for thy dearest friends!
No sleep close up that deadly eye of thine,
Unless it be whilst some tormenting dream
Affrights thee with a hell of ugly devils!
Thou elvish-mark'd, abortive, rooting hog!
Thou that wast seal'd in thy nativity
The slave of nature and the son of hell!
Thou slander of thy mother's heavy womb!
Thou loathed issue of thy father's loins!
Thou rag of honour! thou detested--

The Barbican Hamlet

Cumberbatch played a serviceable Hamlet last night but for me, whatever fine qualities it had were overshadowed by the revisionist script.

He must have forgotten his "Now is the winter of our discontent" speech altogether. He had NO humpback or limp. And the climax was the oddest staging of the battle of Bosworth field I've ever seen...

Just kidding. It was a fine production. Cumberbatch gave a fine performance. I thought Kobna Holdbrook-Smith was excellent as  Laertes. Ophelia's part was very well done by Sian Brooke, brittle from the start and painful to watch as the play shattered her into a million pieces. 

Jim Norton's Polonius I thought was a bit vague, especially compared to Richard Briers in Branagh's production. Leo Bill's Horatio was weak, I thought, disappointing as he's Hamlet's lifeline in the play.

The staging was creative, a dark palace gradually being filled with some sort of black ash. Kind of a Macbeth quality, really. And the near stop motion, partial lighting effects during soliloquies were cleverly done.

Well done!

How Joss Whedon Prepared Me To Love Shakespeare

If you love Joss Whedon, chances are really good that you might love Shakespeare too.

You don’t usually think of a contemporary TV show as something that helps boost your understanding of serious culture. The two are supposed to be antithetical and God knows that most television is rubbish. But Joss Whedon’s shows are different. I was enjoying them for a while before I started thinking about WHY I enjoyed them so much. Given that I have never been that much of a regular TV viewer, why was I drawn to Joss’s work, like a moth to a flame?

It turns out that a lot of the reasons he strike such a chord are similar to the reasons why Shakespeare is so revered, studied and beloved after over 400 years. The reasons why universities around the world can discuss 25 years of one man’s work for centuries and STILL not run out of things to say.

These artists share abundant creativity. The have authenticity and universality. Their protagonists (I’m not sure Jayne counts) have a basic goodness. They are stimulating mentally.

For example:

Creative use of language – Whedon’s work uses phrasing and word choices that are not typical or commonplace. The language is lively, flows, and engages. This is reminiscent of Shakespeare’s plays, which use prose and blank verse with various meters to make listening to the text more stimulating. Think about it: Shakespeare was writing for a theater which had no options for artificial lighting or other special effects, where the artists’ voices and actions were the only stimulation for the audience over plays that typically ran 2-3 hours or more. He had to hold the attention of highly educated and the undereducated groundlings. Creative word play has served both artists well.

Genre-bending story lines  - Whedon’s stories often alternate or even veer crazily between comedy and drama or horror. At the end of Buffy season one, Willow and Cordelia are laughing about their friends camped out watching cartoons and suddenly find that they are all dead, killed by vampires. Later in the series, Buffy is trying to solve the mystery of a girl robot and in the last seconds of the show, walks into her home to find her mother suddenly dead. Bitter fight scenes are often leavened with lines of offbeat banter.

Shakespeare’s plays also defy simple genre descriptions. His “comedies” include a man nearly having a pound of flesh cut from his chest after defaulting on a loan, drunken sword fights and a man framed for madness and confined in a dark place. The tragedy Hamlet includes endless off color jokes. Macbeth’s night porter jokes about alcohol-induced impotence. Shakespeare’s history plays include comedy and horror and frequently distort historical timelines tremendously for dramatic effect.

Strong women – Buffy the Vampire Slayer starts off with a very strong female character but Whedon’s casts are rife with strong characters of all demographics…and weak ones. Basically, humans. And sometimes it isn’t the obvious ones who act incredibly bravely, like Xander talking down Willow when she’s about to destroy the world. 

Shakespeare also showed himself capable of making any of his strong/capable characters male or female, of any race, of any religion. Queen Margaret in the Henry VI/Richard III plays is strong, though ruthless in a way that exacerbates familial tensions to make the War of the Roses even more bloody. King Lear’s daughter Cordelia chose the hard path of honesty in answering her father’s foolish “love test”, is banished for her troubles, yet still comes back to rescue him when her hard-hearted sisters inevitably turn on Lear.

A tendency for the heroes to be good, decent people while not necessarily religious – Whedon’s work gradually engaged my admiration through the years, as I watched more of it. But when I started to LOVE his work was in Firefly. In the Train Job, there is a moment when Mal discovers that the crate his crew just stole contained medication that the locals desperately needed to control a degenerative disease. When he returns it, the sheriff says that “A man might not look too hard at a job he needs but when he finds the details of a situation like ours, he has a choice.” And Mal answers “I don’t believe he does.”

Holy shit, that got me in the feels.

Distrust of distant, overreaching, ruthless authority – For a guy who claims liberal views over libertarian ones, Joss certainly demonstrates a great deal of unease with overreaching, unaccountable government forces. There’s the Initiative in Buffy, the Alliance in Firefly and the whatever-they-ares in Cabin in the Woods. Shakespeare has similar unsettling figures. In “Measure for Measure”, a puritanical judge is planning to execute a man for fornication at the same time he propositions the man’s sister…a nun. Even rulers who are better than the alternative, such as Malcolm in Macbeth, are frequently revealed as cold manipulators.

Aside from plot points and style, I’m starting to look at Whedon and Shakespeare characters which may have substantial similarities:

For example, there are similarities between Buffy and Prince Hal/Henry V. Both frittered away the early years they were meant to spend in training for their roles but then used resources acquired during their early years (for Buffy, friendship; for Hal, a connection with common folks) to become excellent leaders.

Xander is a bit like Shakespeare’s fools, in that he sometimes says crazy things…which are right on the mark. And he always makes a point of coming through for his friends.

Adam, the creature created by the Initiative, is a bit like Edmund in King Lear. Amoral and calculating, he destroys his creator, just as Edmund destroys Gloucester. His end game is to get two large groups to fight to the death for his benefit, just as Edmund seduces both Goneril and Regan and would probably have killed at least one of them.

Sunnydale’s Mayor, AKA the Giant Snake Demon, reminds me of Claudius from Hamlet, who plots Hamlet's death (and the death of his father) whiles smiling amiably.

There’s a lot of food for thought here and I intend to digest every bit of it that I can.

 Resources for the interested:

Books:

"Shakespeare: The Invention of the Human", Harold Bloom.

"Shakespeare After All", Marjorie Garber

"Shakespeare", John Middleton Murray

"Shakespeare Is Hard But So Is Life", Fintan O'Toole

The Great Courses

"Shakespeare: Comedies, Histories, and Tragedies"
Professor Peter Saccio, Ph.D.

"Shakespeare: The Word and the Action"
Professor Peter Saccio, Ph.D.

"Shakespeare's Tragedies"
Professor Clare R. Kinney, Ph.D.

Political Figures Take Facebook "Which Shakespeare Character Are You?" test...

  

Donald Trump shared “Which Shakespeare Character Are You?”
You got Jack Cade from 2 Henry VI!

The tool of someone else with political ambitions but actual discretion, your goal seems to be to run wild at the head of a populist mob, lynching the literate. This may work for you for a while and you may actually succeed…in doing some damage. But ultimately, you will end up on the run, alone.

 

Hillary Clinton shared:

“Which Shakespeare Character Are You?”

You got Lady Macbeth!

You started strong and spurred your mate onto greatness. Despite this, you didn’t actually succeed in killing a king. Although you certainly played a big part in killing the Democratic stranglehold on Congress in 1994! And right now you're looking pretty fragile...

Bernie Sanders shared:

“Which Shakespeare Character Are You?”

You got the Duke of Navarre from Loves Labours Lost!

While you’ve got a few good ideas, their execution is, well, pretty much doomed.

Dude: Navarre nagged members of his kingdom into pledging to avoid the company of women for 3 years…just before the Princess of France was dude to arrive for some crucial negotiations. And then fell in love with the Princess.

Seriously, do you plan ahead AT ALL???

 

Mass Media Hairdo Collective shared:

“Which Shakespeare Character Are You?”

You got Rumour from 2 Henry IV!

Your sole function in life seems to be to mislead people and promulgate incorrect information. Kind of ironic since conveying information is what you’re supposed to be about. Sucks to be awful at your one job, eh?

The Comedy of Errors, by the Seattle Shakespeare Company

I saw Seattle Shakespeare’s production of Shakespeare’s The Comedy of Errors last Sunday and, as always, it was delightful.  This is a fantastic group and I always look forward to their performances.

The Comedy of Errors was Shakespeare’s first comedic play, probably performed first in 1592, after the Henry VI trilogy and Richard III. It is almost certainly derived from Menachmi of Plautus, another play based on identical twins and mistaken identities. Shakespeare doubled down by creating two pairs of twins who could not only confuse other people but encounter alternate versions of each other. As one expects from the genre-bending Shakespeare, there are serious threatening elements incorporated in the story, including a man seeking lost family members who is under threat of death throughout the play.

That tells you a bit about the plot but there’s a lot more going on than simply a few plot elements acted out onstage. This is a VERY lively and witty play. There is a lot of rhyme play, banter between characters (particularly the Antipholuses and Dromios) and in many performances, physical slapstick. 

Seattle Shakespeare upped the ante by having aerialist and performance artist Lara Paxton set a trapeze disguised as a clock with a pendulum into motion. And the two Dromios were played by Kevin Kelly and Arjun Pande, two gifted slapstick comics. Not mentioning the cast is not meant to slight them because they were uniformly excellent and kept this pot boiling throughout the show. I do need to comment on Linda Morris’s performance as well…I thought that I recognized her as Goneril from last year’s King Lear and my wife and I found ourselves sitting next to her parents, who proudly confirmed that. Seeing her brilliantly carry off two parts that were so different was a real treat!

If you haven’t seen performances by the Seattle Shakespeare Company, do yourself a favor. They are definitely an under-appreciated cultural treasure!

What American Votes Are Saying

There are a lot of wacky Republican and Democrat candidates jockeying for a chance to be our ruler leader after Obama's second term. Since I don't think many of their fans have thought through what a vote for them actually means, I am providing a handy translator:

A vote for Donald Trump means:
"Screw that 'Land of the Free', Home of the Brave crap, I'm scared! Scared enough to vote for a guy who's been an idiot and a pathetic bully since kindergarten.Because I want to make the world nervous about an unstable wanker having nuclear capability."

A vote for Hillary Clinton means:
"Lie to me."

A vote for Rand Paul means:
"I like to call myself a Libertarian and I am (aside from wanting government to be the lifestyle police)."

A vote for Bernie Sanders means:
"Only the government can make me safe and they can only do it by controlling every aspect of my life. If you don't need an illusion of safety or want to be controlled, tough."

A vote for Mike Huckabee means:
"What the heck! (Honey, let's vote and then take our salaries to the casino and run them up!)"

A vote for (generic Republican) means:
"I want an amoral government to make everyone moral by legislation and screw the Constitution!"

A vote for (generic Democrat) means:
"I want a dangerous government to make everyone live according to my ideas of safety and screw the Constitution!"