National Home Education Research Institute

Stop Using My Homeschool Success Story to Erase Others’ Educational Neglect

HA Editorial TeamLeave a comment

CC image courtesy of Flickr, CollegeDegrees360.

Editorial note: The following is reprinted with permission from Libby Anne’s blog, Love, Joy, Feminism. It was originally published on February 22, 2016.

In a commentary piece in the Chicago Tribune, David McGrath, a college professor, explains his transition from believing that homeschooling deprives children of their right to an education to believing that homeschooling is superior to other forms of education. Here’s the bit that stopped this homeschool graduate up short:

All that changed when I started teaching at the college level, on an evening when I came home from work, slipped off my shoes, collapsed into the recliner and announced to my wife that the best student in my college composition class had been home-schooled.

An 18-year-old only child, who had been educated by her parents for all 12 grades, chose a seat in the front row on the first day of class.

The following 16 weeks, she maintained eye contact throughout lectures and discussions, listened intently to me and her classmates, raised her hand to offer an observation, an answer or to ask a question when no one else would, followed instructions to the letter, communicated verbally and in writing more clearly than everyone else and received the highest grade on every assignment.

She was the first student to arrive, had perfect attendance the entire semester and was a catalyst for every lesson I ventured.

McGrath could be describing me as an undergraduate a decade ago. I, too, had perfect attendance, sat in the front, listened carefully, followed instructions perfectly, raised my hand constantly, and got the highest grades on every assignment. I was every professor’s dream student. I graduated college with a stellar GPA and went on to graduate school at a research university. But you know what? I am not at all okay with the way McGrath is using my story and that of other homeschool graduates like me.

Take a look at this bit, for example:

In the past 15 years, I’ve known of over a dozen home-schooled students in my college freshman and sophomore classes. All were competent in social interaction, and all had already developed their own methods of inquiry for independent learning.

Do you know who McGrath didn’t meet? Homeschool graduates so severely educationally neglected that college was completely out of the question.

According to the Coalition for Responsible Home Education, there are actually a number of reasons to believe that homeschooling depresses college attendance rates—potentially by a lot. The number of homeschooled students who take the SAT and ACT is surprisingly low, and the only extant random-sample study of homeschool graduates found that having been homeschooled decreased the amount of higher education respondents went on to receive. But McGrath wouldn’t have any way to know about the educational wellbeing of those other students, because, as a college instructor, he’s only seeing the ones who attend college.

Let me put it more personally. McGrath didn’t met the kids I grew up with who were not educated, and for whom college was simply not an option. McGrath isn’t meeting educationally neglected homeschooled children because they’re not going to college. In a study published in 2010, researcher Michael Cogan found that the homeschool graduates at the private university he studied had higher GPAs than their public or private school graduate peers, but you know what he left unexamined? The question of why only 1% of the students at that university were homeschool graduates when a full 3.4% of students were homeschooled in 2011. In other words, Cogan was looking at the cream of the crop, and the other students were simply missing.

I’m also wondering how McGrath knows that every homeschooled student he has encountered was a good student. I’m a college instructor too, and you know what? I don’t usually know whether my students were homeschooled, public schooled, or private schooled. That’s because I don’t generally have any reason to ask that. I’ve taught roughly 250 students over the past year and a half, and I’m sure at least some of them were homeschooled, but I wouldn’t know because I’ve never asked. I suspect that McGrath has also met homeschool graduates who were underprepared for college—and I know plenty such individuals personally—but doesn’t realize it because he assumed they weren’t homeschooled because they didn’t meet his stereotype.

I’m also put off by McGrath’s insistence that homeschooled students are automatically independent critical thinkers who love learning and drink up knowledge. Sure, that describes me and others like me, but what about the homeschool graduates I know whose homeschooling consisted of nothing more than being made to fill out worksheets at the kitchen table for years on end? I know situations where homeschooling killed students’ love for learning. McGrath talks about the benefits of receiving one-on-one instruction, but what of homeschooled children who were one of six, eight, or ten children, who clamored for attention but got lost in the mix because there were too many diapers to change and meals to fix? What of them?

Anyway, McGrath goes on as follows:

While my experiences are anecdotal, clinical studies have arrived at similar conclusions, such as the one conducted by Dr. Brian Ray of the National Home Education Research Institute. His study of 11,000 home-schooled students found they scored higher, on average, than public school students on national standardized tests by a whopping 37 percentile points.

McGrath is a college professor. He should know better than to fall for shitty statistics. The study he cites used a volunteer sample of students from highly motivated highly educated non-poor families. To match the effect of homeschooling you need to compare these students with demographically matched peers, not the public school average. The results of studies that use from a more comprehensive data set (see the data covered here) or pair students with demographically matched peers (see Martin-Chang here) look far different from those released by Ray, whose National Home Education Research Institute is for all intents and purposes an arm of the Home School Legal Defense Association.

There’s another point worth noting here. McGrath is an English professor. Why does that matter? Because homeschooling appears to decrease students’ math scores while either having no effect or a modest positive effect on their reading scores. And it’s not just me saying that, either. Allow me to quote from an exhaustive research review published by professors Milton Gaither and Robert Kunzman:

Given this persistent corroboration across two decades we might conclude, tentatively, that there may be at least a modest homeschooling effect on academic achievement—namely that it tends to improve students’ verbal and weaken their math capacities.

In other words, McGrath’s experience would likely have been very different had he been a math professor rather than an English professor. The Coalition for Responsible Home Education draws on a variety of different data sources to outline this discrepancy in their post, The Homeschool Math Gap. In fact, there is research to suggest that having been homeschooled even affects students’ choice of major, making them less likely to major in STEM fields. McGrath probably doesn’t know this, but then, has he ever thought to even ask, or to look into it? It sounds as though he did a quick google search, fell for the first statistic that confirmed his anecdotal experience, and determined that there was no need to research further.

McGrath began his essay talking about his doubts about homeschooling and his concern about there being “little oversight of home-schooled students in half of all states” including his own. He finishes his essay with this statement:

An estimated 1.8 million students are home-schooled in the United States, often for religious reasons, or for insulation from schoolyard problems such as bullying. But the best reason may be that they get a better education.

Yes, that’s right, he flat-out states that homeschooled students “get a better education.” Don’t get me wrong, I’m glad McGrath has learned that homeschooled students can receive a good education! I am just as unhappy with stereotypes that posit all homeschoolers as backward and uneducated as I am with stereotypes that posit all homeschoolers as innovative and well educated. Both stereotypes are wrong. But while McGrath may have decided that there’s nothing at all to be concerned about with regards to homeschooling, I know that this is not the case.

The lack of oversight for homeschooling in most states is a very serious problem, and leaves too many children without an education. I saw it growing up, and I see it today in networks of homeschool alumni such as Homeschoolers Anonymous. Some children thrive being homeschooled while others fall on their faces with no way to pick themselves up. I know homeschool graduates whose parents gave up teaching them algebra because it was too difficult. I know homeschool graduates who had to teach themselves to read at 16. I know homeschool graduates whose education was so spotty that they can’t pull together a high school transcript. And don’t even get me started on child labor law violations, because what I’ve seen is completely egregious. We desperately need accountability for homeschooling parents.

I am not okay with McGrath using homeschool graduates like me as an excuse to display an utter lack of regard for my less-fortunate friends. I am not okay watching my friends and their pain erased in a paean to an educational method that is only as good as the parents who facilitate it. I am not okay with being part of an argument for maintaining a status quo that deprives children of their right to an education.

Count me out.

When Homeschoolers Turn Violent: Conclusion

HA Editorial Team7 Comments
Screen Shot 2014-03-02 at 5.36.11 PM

HA note: The series conclusion of “When Homeschoolers Turn Violent” is written by Rachel Coleman, Rachel Lazerus, and Dr. Chelsea McCracken from Homeschooling’s Invisible Children.

*****

Homeschooling communities assume that homeschooled kids escape the pernicious influences of the wider world — if violent behavior is something that you catch, like a disease, homeschooling families assume that they can protect their children from exposure. Some homeschooling advocates also claim that those who have been homeschooled properly will never turn out to be violent criminals or have mental health issues.

The assumption that homeschoolers are essentially good kids is also held by members of the general public. Joshua Komisarjevsky was sentenced to only nine years for his role in burglarizing 18 homes, despite the maximum statute being ten years for each offense. When he came before the parole board, he was seen as “young, white, bright, home-schooled, remorseful, never identified as a person with high mental health needs.”[1] He was paroled in April 2007, and invaded the Petit family home on July 23rd, 2013.

Homeschool advocate Brian Ray implicitly endorses this view, writing:

Discussion with leaders in the homeschool movement of the past several years of news and careful Internet searches reveal 2 and possibly 3 homeschooled teens (ages 14 to 17) who were accused or convicted of homicide during the past 8 years. …If home-based education of teens of ages 14 to 17 comprises 3% of that age-group population and calculations are adjusted for number of homicide offenders and number of years (with a liberal estimate of 3 during 8 years), then homicide offenders of homeschool students ages 14 to 17 would be about 0.004 per 100,000. If true, this would mean that the general-population teen of this age is 2,500 times more likely to commit homicide than a home-educated teen.

In this conclusion post to our series on homeschoolers who have turned to violence, we would like to explain why it is both unfair and unscholarly to compare homicide rates in the way Ray does — and how these flawed assumptions make homeschooling families and communities less safe.

First, we would like to note that it is unscientific to conclude from anecdotes that there were only two or three homeschool homicides during the previous eight years. For one thing, our list includes more than twice this number, and we do not assume it is a complete list. Not every youth homicide makes the news, not every news story on a youth homicide discusses the offender’s educational background, and not every story that makes the news is easily accessible on the internet. To accurately compare the rate of homeschool homicides to the national rate, one would need to access the criminal records for all youth homicides within a given time period and look at the educational background of each. It is irresponsible to speculate on youth homicide rates without this data.

Even more importantly, comparing youth homicide rates between the homeschool population and the general population would not actually tell us anything about whether homeschooling itself affects the youth homicide rate. When comparing homicide rates between two populations, it is crucial to consider all of the ways the populations differ. In other words, if the homicide rate is indeed lower among homeschooled youth, which it may be, this might be because homeschooled youth differ from youth in the general population in various ways that are unrelated to school choice.

When comparing crime rates in two populations, it is important to take into account various factors that might impact the crime rate. For example, one cannot compare Alabama’s high crime rate with New Hampshire’s low crime rate without taking into account the difference in poverty rates: Alabama has one of the highest poverty rates in the country while New Hampshire has one of the lowest. According to the National Center for Child Death Review, “Major contributing factors [to youth homicide] in addition to poverty include easy access to handguns, involvement in drug and gang activity, family disruption and school failure.” Other factors which may also contribute to crime rates include education level, community involvement, availability of mental health services, condition of law enforcement, etc. When comparing two populations, all of these factors must be taken into account.

The homeschool population varies from the general population in a number of ways. The homeschool population tends to be more rural than urban, and homeschooled children are more likely to live in households with two parents than are children in the general public. It has traditionally been thought that parents who homeschool tend to be wealthier and better educated than average (although this has recently been called into question). All of these factors correlate with lower crime and homicide rates. Without correcting for these various factors, it is impossible to know whether homeschooling might play any role in lowering homicide rates.

Many homeschool families also have high levels of parental involvement, which is not easily quantifiable. Parental involvement leads to higher educational attainment in any educational setting and also correlates with lower crime rates and lower engagement in risky behaviors. As a result, lower homicide rates among homeschooled youth could be a result of high parental involvement rather than of homeschooling per se — and highly involved parents would affect their children’s lives in this way regardless of the educational method they chose.

Could these background factors be corrected for? Yes. To do so, one would first have to collect accurate data on homeschooled students and their families. Then, one could compare the homicide rate among homeschooled youth with the homicide rate among youth in the general population with the same background factors. In other words, if homeschooled youth are more likely to be rural, more likely to have two parents, more likely to live in families with slightly higher incomes and educational attainment, and more likely to have high parental involvement, the homicide rate among homeschooled youth would need to be compared to the homicide rate among a sample of the general population that is also more likely to be rural, more likely to have two parents, more likely to live in families with slightly higher incomes and educational attainment, and more likely to have high parental involvement. That way one could isolate any effect educational method might have.

As the director of the National Home Education Research Institute, Ray has conducted a number of studies on homeschoolers; however, he has consistently failed to correct for these background factors. In several of his studies, for example, he gathers testing data from a nonrepresentative sample of highly educated, high income, highly involved homeschool families and then deceptively attributes their predictably high test scores to the fact that they were homeschooled. (For more, see reviews of Ray’s work here.)

When a youth homicide occurs, the student’s educational background, whether public, private, or homeschool, is not generally the causal factor. Attending public school does not cause an adolescent to commit a homicide any more than being homeschooled causes an adolescent to commit a homicide. Rather than focusing on which educational method is correlated with the lowest homicide rate, we should instead study how various factors may contribute to adolescent homicides in any educational setting. We prefer to focus on keeping youth safe, not on making an ideological point.

We do not have the data to know whether homicide rates are higher or lower among homeschooled youth, or what role homeschooling plays. However, there are some factors we may observe as common themes in homeschool homicides. In many of the cases we have collected, homeschoolers who commit homicide come from families with extreme religious or ideological beliefs (patriarchy, white supremacy, anti-government views, etc.). Many of these youths have easy access to firearms in their homes. They may have untreated mental health issues or developmental disabilities. Several youths come from disrupted homes, either through adoption or divorce. Parental abuse and neglect, as well as drug and alcohol abuse, also play a role. Though in some cases these factors may correlate with homeschooling, this does not imply that homeschooling causes homicides — the same factors have been implicated in homicides committed by youths who attend school.

It is a common and understandable impulse to try to keep ourselves and our community safe by believing that violence could never happen to us. Unfortunately, this reaction is not productive. The best way to keep homeschoolers safe is not to deny that there could ever be any problems, but rather to learn about the factors that contribute to problems and to be on the lookout for mental illness and abuse among homeschooled students, homeschool grads, and homeschool parents.

[1] Rep. Mark Lawlor, Chairman, CT Judicial Committee in The Cheshire Murders, HBO documentary, 2013.

View the case index here.

Crosspost: Methodological Problems with Kevin Swanson and Brian Ray’s Gen 2 Survey

HA Editorial Team7 Comments

Crosspost: Methodological Problems with Kevin Swanson and Brian Ray’s Gen 2 Survey

HA note: The following is reprinted with permission from Libby Anne’s blog Love Joy Feminism. It was written by a guest writer, Apodosis, and was originally published on Patheos on July 17, 2013. 

Libby Anne posted recently about a new survey conducted by Brian Ray of the National Home Education Research Institute, a non-profit which conducts studies of questionable scientific validity on homeschooling. As a Ph.D. social scientist myself, I looked over the new survey with a critical eye and I’m sorry to say there won’t be much useable data gleaned from it because it is rife with methodological problems.

The survey has over 100 questions and nearly every question needs revisions, so I’m just going to summarize my main critiques of the study.

1. Institutional Review Board

When you are conducting a survey that collects personal information from the participants, every IRB in the country requires that you have signed informed consent forms for each of your participants. In online surveys, these tend to include a cover page describing the purpose of the study and what is asked of the participants, as well as a signature box to show you have understood your rights.These forms often include the phrases “You may stop the survey at any time without giving a reason” and “You are not obligated to answer any question you don’t want to”. They provide contact information for the primary investigator (PI) and other associated researchers, as well as for the IRB that approved it. “If you feel your rights have been violated, call this number” etc.

In the Gen 2 survey, there is no such information. It is not clear who is being asked to participate (see #2), the goals of the study are not honestly stated (see #3), and there is no contact information for the PI or an IRB to contact if you feel your rights have been violated. I had to dig on NHERI’s website to find the email address of the PI, Dr. Brian D. Ray (it’s bray@nheri.org, btw, in case you feel your rights have been violated), and he actively discourages you from contacting him. Red flag.

A peer-reviewed publication will not publish any results from research that was not overseen by an IRB.

2. Target Population

It is not clear who is being asked to participate in the survey. In the “About” section, Ray says the participants are “those between the ages of 18-38 years old that grew up in religious homes”, but in the FAQ he says “Anyone between the ages of 18-38″ may participate. I am between the ages of 18 and 38; I was raised in a moderate mainline Protestant family, and I am now a progressive mainline Protestant. I honestly cannot tell if he wants my data or not. Though both of those statements about who should participate apply to me, the questions on the survey indicate otherwise.

3. Goals

Which brings me to my most serious methodological critique of the study, which is that the goals are not honestly stated. There are two different studies conflated here which have entirely different goals. Dr. Ray even alludes to this in the “About” section: one goal is “to come up with data points of key influences that either encouraged or deterred the participants from practicing the same faith as their parents”; the other goal is to “use the statistics from this survey to help equip parents to make more informed decisions in the education and spiritual guidance of their children.” That is, in simpler terms, the goals are (A) to find out how young people’s religious views change as they reach adulthood, and (B) to figure out how to make sure young fundamentalists/evangelicals stay in the fold.

In fact, the main goal of the study seems to be (B) with a shallow veneer of (A) superimposed on it. Now, (A) is an interesting study whose results I would look forward to reading. (B) is not a scientific study. You do not perform a scientific study with the goal of achieving a certain result. Social science is about trying to describe and explain human behavior, not about trying to change it or attach value judgments.

Here are some examples of the problems that arise when the two studies are conflated. Dr. Ray pays lip service to the idea that young people may belong to a variety of faiths, as evidenced by his questions:

Generally, what kind of religious service did you attend as a child?

What kind of church or religion do you currently associate yourself with?

Available answers include Atheist, Agnostic, Buddhist, Hindu, Jewish, Muslim, and a variety of mainline Protestant denominations in addition to the exhaustive list of evangelical/fundamentalist groups. However, on the next pages he asks questions like

My current church is serious about applying the biblical principles of eldership, shepherding, mentoring, and church discipline.

Did you ever get any “worldview” training?

How often did your Father/Mother read the Bible to you?

These questions indicate that the primary audience of this study is people raised in a fundamentalist/evangelical/Christian patriarchy home. How are Buddhists and Hindus and Atheists supposed to answer these questions? Note that there is no “Not applicable” option available. This brings me to #4.

4. Not Applicable

It is important in surveys to allow your participants the freedom to answer the questions honestly, and not be forced to pick the “closest” answer. Many of the questions follow the “Have you stopped beating your wife” pattern, where e.g. a person who has never beaten their wife has no honest way to answer the question. “Not applicable” is only an available answer for one of 100 questions in the survey. This becomes a problem with questions like the following:

How distant or close do you feel to God most of the time?

What was the status of your parents’ marriage during your raising?

My father was very involved in our family home life.

How would an atheist answer the first question without “Not applicable”? How would someone answer the second question if they were raised by two parents in a committed relationship who were not married? And in the third case, what if you didn’t have a father?

“Other” followed by a fill-in-the-blank should also occur much more frequently as an available answer in questions such as:

Have you, or your significant other, ever chosen to have an abortion? Yes/ No

Do you think that people should wait to have sex until they are married, or not necessarily? Yes/ Not necessarily

How would you describe yourself when you were a child? I was very rebellious/I struggled with rebellion, but overcame it/I was always fairly obedient and honoring as a child

In the first question, it would be more appropriate to ask “Have you ever had an unwanted pregnancy?” and then to ask how it was dealt with. As it stands, a person who has never been pregnant and a person who has carried a rapist’s child to term would give the same answer. In the second and third questions, there are clearly more than just the possible answers given. My answers would be “No, unless they want to” and “My parents were supportive of what I chose to do and who I chose to be”. Participants in the study need a way to answer these questions honestly, so even if Dr. Ray wants to limit his variables it would be more methodologically sound to provide an “Other” response with a fill-in-the-blank to such questions.

5. Leading/Biased Questions

This brings me to another point: almost every question is leading or has other problems caused, for the most part, by faulty assumptions and a lack of imagination on the part of Dr. Ray.

“What kind of church or religion do you currently associate yourself with?” Some people consider themselves to be a member of a faith without having a specific faith community. Some people consider themselves to be members of multiple faith traditions. A better question would be “How would you describe your religion or faith?” with multiple check-boxes allowed. And he should list all the religions, not just the ones he could think of off the top of his head! He forgot at least (off the top of my head) Baha’i, Sikh, Wicca, Christian Science, Animism, Deist, Shinto, Unitarian Universalist, Not applicable, Other____, and he did not provide any different denominations of Judaism or Islam. In addition, he should have grouped the religions together by type rather than alphabetically—this list makes it very hard to see if your faith is represented. And groups like “Mormons” and “Congregationalists” should be called by their actual names—LDS and UCC, respectively. This list of religions, while it pays lip service to religious diversity, is actually offensive in its exclusions and ignorance. Plus, there is no acknowledgment in this survey of mixed-faith households. What if your parents were of different faiths, or what if you and your significant other do not share a faith?

Questions such as

What is your sex/gender? Male/ Female

How was your relationship with your Mother / Father when you were 16-17 years old?

How often did your Mother / Father explain biblical principles to you?

presume an oversimplified, heteronormative view of gender and family. Sex is not the same as gender; there are quite a few other genders than just male and female; you might have parents of multiple genders or the same gender (and therefore not have a “Mother” and a “Father”); and you might have been raised by other family members, or primarily by friends, or in foster homes. If this was really a survey about changes in young people’s religious views, it would try to get an accurate picture of their lives without limiting them to these binaries. Ray should also ask about people’s sexual orientations if he’s that interested in the status of their romantic lives; however, his reference to “homosexual” “encounters” in one of the questions indicates that he probably does not believe in sexual orientation as a concept (see #6).

6. Limited Mindset

Many of these questions portray a mindset that is isolated to the evangelical/fundamentalist/Christian patriarchy/Quiverfull/Purity movements–it’s possible that Dr. Ray does not even realize there are other ideologies out there. For instance, there is this question/answer set:

What statement most aligns with how many children would you like to have? I don’t want to have children / I want no more than a few children / As many children as God will provide / I don’t know

This really should be divided into multiple questions. First, he should ask “Do you plan your pregnancies?” If no, he should ask about “how many children you hope God will give you”, but if yes, he should ask how many you plan to have. I don’t think Dr. Ray realizes that some people plan their pregnancies. Then there is the following question/answer pair:

If you have—or were to have—children, what form of education do you plan to use for them? Christian school / Christian school and homeschool / Christian school and non-Christian private school / Christian school and public school / Homeschool / Homeschool and non-Christian private school / Homeschool and public school / Private school, non-Christian / Public school / Charter school/virtual charter / Other

At least this one has “Other” as an option, though it doesn’t let you write in your response. First of all, it’s a badly designed question—he should just list the types of school and let you check as many boxes as you want. Second, the question relies on the assumption that parents would choose their children’s manner of schooling before the children even exist. What if the child is gifted or disabled? Would that change the parents’ plans? And Dr. Ray does not even realize that people who would answer like me are out there—”It depends on the child’s needs and wants, as well as other considerations such as expense, distance, quality of education, etc.”

Then there’s this question/answer pair:

Did your parents use corporal discipline (spanking) with you?No / Yes, consistently and they were generally under loving control / Yes, consistently and often they were not under loving control / Yes, inconsistently and they were generally under loving control / Yes, inconsistently, and often they were not under loving control

These answers presuppose a worldview where the value of spanking is not open to debate (spoiler: we don’t live in that world). It would be more accurate to ask “What was your parents’ position on spanking?” “How often were you spanked?” “Are your attitudes toward it positive, neutral, or negative?” “Would you spank your own children?” etc.

And then there’s the part where he puts these two questions next to each other, clearly conflating them (shades of Libby Anne’s two boxes): “Have you had a sexual encounter or physical relationship with someone to whom you are not married?” and “Were you ever sexually abused before age 18?” Note that a girl who was repeatedly raped by her Christian patriarchy father for ten years and a girl who was raped once by an acquaintance at a party as a teenager would probably have different reactions to their faith traditions, though they would give the same “Yes” response. No effort is made to make this distinction in the survey. And nowhere does the survey ask about physical abuse, since Dr. Ray probably doesn’t think it exists.

Or how about the question “If you were unsure of what was right or wrong in a particular situation, how would you decide what to do?” with the available answers: Do what would make you feel happy / Do what would help you to get ahead / Follow the advice of a parent or teacher, or other adult you respect / Do what you think God or the scripture tells you is right / Something else. These answers reveal Dr. Ray’s belief that morality does not exist outside (his brand of) Christianity.

7. Exclusionary Language

The language Dr. Ray uses is exclusionary and often confusing. For instance, he persists in using terms like church, pastor, scripture, prayer, Bible, youth group, Sunday School when these terms are uniquely Christian and do not apply to people of other faiths–he ought to say faith community, religious leader, holy book, prayer/meditation, religious education if he wanted to get accurate data. He also fails to define several terms which I don’t understand because I was not raised in a fundamentalist environment: family-integrated, homeschooling-friendly, shepherding, church discipline, worldview training. In his questions about belief in “God” and “heaven”, he ought to ask separate questions about each property he wants to assign to these words’ meanings (e.g. “Do you believe in a deity or deities?” “If yes, do you believe the deity/deities is/are omniscient? Omnipresent? Omnipotent? Does it/do they have a gender? Does it/do they affect everyday events?” etc.). If Ray wants accurate data, he should define confusing and ambiguous terms.

You should not be able to tell someone’s political and religious beliefs from survey questions designed to elicit yours. You should not be asked to give dishonest answers to survey questions because your honest answers are unavailable as options. You should not have to infer how exclusionary language in the questions would apply to your situation. You should not be asked to give out your personal information without giving your informed consent.

An IRB would not approve this study. I would not rely on any of the conclusions that come out of it.