Why Surveys? A Critique of the Tools Used to Judge Us All

About the author: Christopher Hutton is a freelance journalist from Bloomington who writes on technology, religion, and the ideas of the day. He currently writes for Christ and Pop Culture and Paste Magazine. He is also the Social Media Manager/Intern for Rivendell Sanctuary, a new education program designed to provide a truly thorough education. Follow Christopher’s blog at http://liter8.net.

Every day I seem to see a new study or statistic being used to prove a point about something.  Beginning in the 1980’s, the religious survey organization known as the Barna Group provided data point after data point in order to reveal truths of our culture and how to interact with it.

Most recently, Generations Radio host Kevin Swanson teamed up with Brian Ray in order to record the social and spiritual conditions of the millennial generation in a survey. (You can find the survey at gen2survey.com.)

Now, to gather research is not bad.  In fact, it’s the very first thing that self-proclaimed culture-reclaimers should do.

But are they doing it effectively?

The problem with surveys is that they rely on the human language to present and record human behavior. Number concepts are different than word concepts because their definitions are clearer. If you say “five”, then people understand you are discussing a quantity. However, if you say the word “religion”, then the topic becomes fuzzier. Cultural and personal circumstances often cause words to be understood in different ways.

This causes companies to create a lot of ideologically inaccurate surveys. Consider a survey by the Barna Group which seemed to state that “only 4 percent of Christians have a biblical worldview”.  This is a big claim, and can seem scary to those fighting for a biblical worldview.

But what does Barna mean by a biblical worldview?  This isn’t obvious at first glance.  Historically, the concept of a Biblical Worldview has historically had only a few tenets which almost all people who argue for it would agree with (Inerrancy of the Bible, existence of God, Personhood of Christ, etc.), but everything else is flexible. So, was Barna adding political elements to their definition?  Were they adding debatable theological ideas into the biblical worldview?  It’s hard to know from how a group like Focus on the Family used the Barna Group’s statistics.

Kevin Swanson’s latest study is another clear example of this.  If you look at the language used in the study, it does focus on his particular form of Christianity, which emphasizes extreme forms of Conservative Christian theology, as well as an instinctual anti-government bent, an emphasis on the family relationship, and limited options for explaining one’s relationship with God, family, and the Church.

This kind of ideological shifting is dangerous, because it causes the otherwise objective data to be skewed and misbalanced.  It will misrepresent its survey-takers. It will also skew the facts.

So, be wary of surveys. Mark Twain once famously stated that “There are three kinds of lies; lies, damned lies and statistics.”  While quantitative data is helpful, it is dangerous if it is not true. So track statistics, check the facts, and never let them skew your take on a topic.

7 thoughts on “Why Surveys? A Critique of the Tools Used to Judge Us All

  1. Lana July 23, 2013 / 3:56 pm

    I feel like they are trying to say that Biblical principles, Biblical patriarchy, etc, leads to a life of morality and happiness. But I would have them to know that many of us are very happy and moral without those things.

    Like

  2. Christopher Hutton (@liter8media) July 23, 2013 / 4:42 pm

    @Lana, would these Biblical Principles be something different than Moral principles, especially considering that a majority of these biblical principles still overlap with the moral principles?

    Like

  3. Lois Manning (@lmanningok) July 24, 2013 / 4:27 pm

    Here are a couple of traps: The definitions of “biblical principles” and “moral principles.” Which Bible…The Torah (Old Testament) or New Testament. Moral principles about sex or helping the poor? If it’s the poor, which poor? Only poor Jews, as in the Torah, or including poor goyim (gentiles) as in the New Testament. If it’s sex, does that mean lesbianism (not mentioned in either Old or New) or just male homosexuality? Does it include the Torah’s condoning rape a woman/girl and then demand he be given her in marriage because she’s now “damaged goods.” (Remember that creepy phrase?) It couldn’t mean abortion because that isn’t MENTIONED in either the Old or New Testament. Why? Because according to Job 10:18-19, people don’t even exist until after we’re born, so abortion isn’t relevant.

    Like

  4. Stuart Blessman (@ST_U2) July 26, 2013 / 7:41 am

    Inerrancy is a historical Christian tenet? So right off the bat the survey is flawed and a little biased.

    Like

Leave a comment