By Thomas M. O’Toole, Ph.D.
The weird house at the end of the street…everyone had one as a child. It was usually occupied by some odd person who kept to him or herself. It was always ripe for gossip amongst the neighborhood kids and even some of the adults. Didn’t someone die in that house? Was it murder? I heard the owner performs odd rituals in the basement every Tuesday night.
The unknown always produces some form of hysteria born out of intrigue, fear, hatred, or other emotions. It’s one of those defining characteristics of human psychology. It takes control out of our hands, which we generally do not like. Consequently, we are desperate to find certainty where it does not exist, with the strategies for doing so ranging from reasonable to ridiculous…anything that gives one a sense of control over things.
Nowhere is the ridiculous side of this desperate need for control more apparent than in the jury trial. The black box that is jury deliberations constitutes the ultimate loss of control for attorneys. So much time and so much money is put on the line with some random group of folks who are not well-equipped for a litigation setting and who’s “inferior” minds might decide the case based on something as silly as the tie the attorney wore on the fourth day of trial (insert sarcastic tone here).
Fortunately (or unfortunately), there’s a whole segment of the jury consulting industry out there that is willing to sell these attorneys the illusion of control. In this blog, I’ll focus on one that has become very popular in recent years: the use of perceptions analyzers or dials in jury research.
Perception analyzers or dials, the little dial boxes you often see on CNN come election time, are all the rage. These boxes purport to gather moment-to-moment data that reflects an audience’s reaction to anything and everything. An entire book could be written on the problems associated with these devices. They are a colossal waste of time and resources…and this comes from someone who has used them. But I try my hardest to steer clients away from them. They are a distraction at best. Yet, they are intriguing. They provide more data and imply more opportunities for control. This is where a fundamental mistake is often made in jury research. Quality research should be judged by the ability to collect meaningful data and, more important, the ability to interpret that data and translate it into practical recommendations for improving the case. Attorneys often mistake the amount of data collected for the quality of the analysis. Here are just a few of the common problems associated with perception analyzers.
a. Moment-to-moment reactions do not correspond with actual leanings. Some research shows, despite movement throughout a presentation, perception analyzer research does not predict actual leanings. In one study, researchers used two infomercials, which registered similar data points on the perception analyzers. However, when tested for persuasiveness, one infomercial achieved success while the other failed, despite the similar dial results. One of the most famous anecdotes about perception analyzers is that the data collected from them in the pilot testing of the sitcom Seinfeld indicated the show would fail.
b. Interpretation of the data collected by perception analyzers is speculative at best and meaningless at worst. The only certainty we can have about perception analyzer data is that a “40” means “40.” Dial defenders would argue that this problems is resolved by interviewing jurors afterwards, but interviewing jurors afterwards provides the same feedback regardless of whether or not dials are used, so what unique value does the dial add? In short, dials add detail without insight, which brings no value to litigation prep efforts.
c. Attempts to interpret data introduce the bias of the interpreter who applies his or her own interpretation of what “40” means with little contextual data to anchor this interpretation, making the interpretation particularly unreliable.
d. Participants don’t understand how to use dials even with extensive instruction. Participants experience the same confusion with providing the data as researchers experience in trying to interpret the data. The dials ask participants to figure out a way to translate their thoughts, opinions, and emotions to a single number, often on a scale of 1 to 100. Participants try, but most will readily tell you that they don’t understand what the numbers mean.
e. Some research shows one participant’s dial responses are influenced by other participants in the room. In other words, a participant may be cued by other respondents to either respond a certain way or pay attention to the dial during a particular segment.
f. The dials introduce a confounding variable. In other words, they can actually skew the data. Normal perceptions and opinion formation do not involve attention devoted to a device that forces one to remain conscious of their perceptions and opinions. For example, one common problem dials have is that some presentations draw participants in, causing them to completely forget about the dial. Reminding participants to use their dials disrupts this part of the opinion formation process.
Clients should critically evaluate whether or not there is actual need and value to using perception analyzers in jury research. A common, after-the-fact complaint by clients is that they received a thick report with endless tables of data-reporting from the dials, but little to no insight into the meaning of the data or what the clients should do with it.
