One of the first concepts I learned in graduate school, from Dr. Dale Berger, is “The data are what the data are.”
What this means is that the results of your study are neutral and objective. If they support your hypotheses, they aren’t “good,” and if they don’t support your hypotheses, they aren’t “bad.” The data simply are essentially an objective snapshot of the world and, as such, they are value neutral.
That doesn’t mean data can’t be wrong. The sample in your study could be “off” (although this is a rare event) and there is always some systematic (question wording) and random (person didn’t understand the question) error in every study. So, one of the advantages of meta-analyses (and other methods which combine study results) is their ability to combine the results of similar studies allowing the possible sampling, systemic, and random errors in individual studies to cancel each other out giving a very, very good estimate of the truth of the relationship one is examining.
Which brings me to this year’s presidential election. There are several sites available that are combining poll data in ways that ought to account for individual poll errors and should give us a good idea of what the current “truth” about voting is out there. I prefer the Princeton Election Consortium because it uses nothing but recent poll data (either the last 3 state polls or polls from the last week). Other people like FiveThirtyEight, although I am less enthusiastic about that data because it includes economic data which are probably antecedents (“causes”) along with the polls, which I consider to be the outcomes (“effects”), i.e., the dependent behavior we’re watching. (NB: Yes, I KNOW those are not really causes and effects, but not everyone got an A in Research Methods.)
Now, I better understand why the 24 hour news sites don’t want to refer to the meta-analysis of the polls. The latest poll result (singular) allows them to work their readers or viewers up into an emotional state (Despair! Joy! Romentum! Anger! FEAR!!!!!), which simply isn’t the case if you look at the meta-analyses–a more stable and valid assessment of *all* the state polls.
And now I also think I understand why some folks who don’t know stats that well pay more attention to individual polls than the meta-analyses. From their perspective, I am recommending that they should try and prefer this “chocolate ice cream” as opposed the “vanilla ice cream” they’ve been eating all along. But that’s the wrong food analogy. The meta-analyses are more like a vegetable soup (all cooked and combined together) and I want to say “You’re eating a raw onion. Spit it out and have a sip of vegetable soup.”
What is frustrating to me, though, is people who ought to know better, who have had advanced statistics, and who are still “crossing their fingers” that a standardized combination of polls is more accurate than the last poll they just saw, and which has also been incorporated into the meta-analyses. I don’t get that.
However, I do understand that people do not like the results of the data analyses. As someone who has voted for the losing candidate over 50% of the time (yes, I did just calculate that!), I get being frustrated and angry that the data aren’t going my way. And good cow, I can’t even possibly calculate how many times that has happened in my research! HA! No, I mean. Wait. ((cough, cough)) My data always supports my hypotheses. ((Ducking the lightening from the research gods and goddesses))
But remember: the data are what the data are. Could these analyses be completely wrong? Yes, of course! But the problem would not be in their statistical calculations. The problem would be that the polls are completely inaccurate assessments of people’s statement of what their behavior will be on voting day. And that would mean we have NO IDEA what is going to happen on November 6. And more importantly, that the trends up for Obama after the DNC and sharply down after the first debate are simply flukes of data collection and no reflection at all about people’s opinions about, preferences for or potential voting behavior for the president. If that’s the case, pollsters are going to have to seriously revise how they collect data. Hello, President Dewey!
At this point, I think it’s more accurate to believe that the data are what the data are. That’s a hard thing to do. It involves a controlled emotional response to ignore the hype, the interpretation, and the spin and to form your own opinion. The data are what the data are. And we’ll know if they are representative of the actual behavioral “truth” in one week.