Action Nugget: Wolves, Bears, Dogs and Bull. Using video speech analysis to debunk 'reality'.

Wednesday, April 18, 2012

Wolves, Bears, Dogs and Bull. Using video speech analysis to debunk 'reality'.

A Bear of a tool

There have been thousands, if not millions, of words written about the Republican presidential primary debates. Social media and the web are bristling with commentary and analysis; the mainstream news services manage to create exciting stories out of the most mundane and unimportant details.

We often focus heavily on personalities and debating styles. 

In the immortal words of Jules in Pulp Fiction,

’I wouldn't go so far as to call a dog filthy, but it's definitely dirty. 
But dogs got personality, personality goes a long way.’

When you move beyond the personality and try to use an analytics-driven approach to understand the debates, you start to see some differences between what is being said, how the media reports on the debates, and the public’s resulting perception. It also helps to apply some nifty information design to help understand the textual analysis.

Thanks to some personal connections*, we were able to use Cisco’s Pulse Video Analytics Tool, a feature of the CiscoMedia Experience Engine (MXE) 3500 to analyze what was actually said during one of the debates and then compared what we found with the media reports. The MXE 5000 lets you see a breakdown of who spoke and when as well as a summary of topics without having to transcribe a video first. So if you want to know who had more speaking time or who spoke most about ‘education’, you just use pull the data out of the Analytics Tool. 

Here's what the user interface looks like. 

Debate data being Pauled
After some configuration, it’s very user-friendly. You process a super-set of videos which produces a data dictionary. The Tool then shows on the timeline, who spoke when, using a color code. In the example above, the little red scrubber marks show when Ron Paul was speaking. The data is quite fascinating and you can see how it would be useful in a corporate setting if, for example, you posted meetings to this tool and then allowed people to search for keywords that were relevant to them rather than sitting through a whole boring meeting [yawwwn].

We love the idea of using technology to make our lives less boring.

But back to the debates Wolf

The kind people at Cisco also provided a big data dump for us to work our analysis magic with raw data.

Fairly big data
We took all of the terms and speakers from the Arizona debate on 22nd February and ran them through our patented Information Design ProcessMonkey tool. We then thought about how to present the information in a way that lets you see the story without eating 10,000 rows of Excel. As part of the process, Cisco excluded some irrelevant terms such as ‘question’ so you start to get a pretty interesting picture. 

Click to get an interactive demo of debate terms by speaker.

You can immediately see that the most-uttered terms were Program and Programs. You'd expect that for a GOP debate when all candidates' focus has been on deficit reduction. Where it starts to get really interesting though is
that one of the other most used phrases was ‘Governor Romney’. The hypothesis would be that, since Romney is the front-runner, the other candidates were probably attacking him. Using the Pulse tool, we were able to go back and test the theory and figure out what each candidate was saying when he used that phrase. 

Here’s a breakdown of how many times each candidate said ‘Governor Romney’.

John King
Governor Romney
Rick Santorum
Governor Romney
Newt Gingrich
Governor Romney

John King was the moderator but you can see that Rick Santorum was all over Governor Romney. Let’s dig into the tool and get some actual quotes. 
  • Senator Santorum’s first mention of ‘Governor Romney’ is at Minute 13:32. He says ‘…Governor Romney voted to raise taxes. Even today….’
  • Next up is at 25:48. ‘…Governor Romney asked for that earmark…’
  • At 34:49, it’s ‘…destructive capital…limited government’
  • At 35:08, Governor Romney ‘…supported the folks on Wall Street...’

You have to sort of love the evolution of Gov. Romney’s expression as the debate goes on.
  • And Sen. Santorum continues to attack at 56:07 ‘…the bill you drafted in Massachusetts, Romneycare…’ 
  • He rounds out an enjoyable evening at by describing Gov. Romney as ‘…the model for Obamacare…’ at 57:10.
Phew. You get a pretty clear flavor of what a large portion of the debate was about from this one small piece of analysis. So how does this compare to the media analysis on 23rd Feb?

The view from the media
The key message from ABC’s post-debate report was
‘It generated a lot of light, but very little heat. And, it did produce one sure loser: Rick Santorum.’
Based on the analysis of that one debate, that's quite a stretch.

Even if you ignore the weird grammar, the following quote does not seem to be in line with the reality of the text: 
‘WINNERS: Mitt Romney: As he did in the Florida debates when he relentless attacked Newt Gingrich, Romney came to Mesa loaded for bear. Just a few minutes into the first question, Romney was already turning his sites on Rick Santorum. And, it worked as he kept Santorum on his heels for the entire evening…LOSERS:  Rick Santorum: Whatever momentum Santorum had came to a screeching halt in tonight’s debate [our italics]’ 
Not sure we were watching the same debate here!
We could definitely dig deeper into the perceptions (or biases) that the press analysis showed but even with this brief look, it’s pretty clear that some of the more subjective analysis is not always based on reality.

Of course, in the end, Santorum did drop out but February 22nd was a high point, not a low.

What does all of this mean for brands?

From a marketing perspective, you can see where this is going. 

Using automated video content analysis we’re able to very quickly figure out both the message contained in the content and the sentiment. 

The automation point here is the key. A tool like this digests gigabytes of video content and allows an analyst to quickly gain a sense of the meaning, tone and content of the speakers.

Tune in for next week’s analysis to hear the really interesting news – how this approach will affect web, social and sentiment analytics for Marketing.

- Robert Sandler

*Full disclosure: Irene Sandler, Senior Manager, Enterprise Video Solutions Marketing at Cisco Systems is married to Robert Sandler, Managing Partner at Action Nugget.


  1. More political commentary. Never mind this marketing hoohaa.

  2. Thanks for the comment Deanna. We have to be careful about political positioning. Based, as we are, in Portland OR, we have clients of all political stripes. They run from fairly left of center to far, far, very far left of center. so we need to make sure we're not offending anybody who is say, off-the-charts left of center.

    But I leave you with this deep thought: perhaps politics IS marketing. Oooohhhh.....


Web Statistics