Studies

When I was a lad, the YouTube atheist community took up a surprising amount of my viewing time. Whether the appeal was on watching slovenly, hyperaged fat people argue with each other and beg for gibs or on tormenting my oddly cynical and miserable 10-year-old self with news about how everything was shit, I don’t know. Naturally, I was there when the thing morphed into Atheism+ and then into GamerGate, but it all started with free thought and rationality and skepticism. Supposedly.

If there’s one thing I’ve taken from maturity, it’s that authorities are never truly so. No watchman is empirical, and past your observations, no fact is absolutely so. We all have biases, and biases are a valuable part of life that postmodernist losers don’t realize are there for a reason.

I’m about to start saying things that’ll make me sound anti-science, so let me clarify terms. Science as a method is fantastic. Skepticism is an important part of not getting fooled by marketing and conmen. I’m referring to science as an institution, the idea of studies and research institutes playing with numbers and data to paint some kind of picture and maybe even influence policy decision. That’s not infallible, and people, even otherwise intelligent people people, treat it like it is. Drives me up a wall.

People get a lot wrong about science. People’s general understanding of it is that Science Is The Truth, Science Proves Things Right, and Smart People Do Science, essentially. It’s all bullshit, really. We’ve built up this junk “science” image of big-brained people going into a lab and coming out with The Truth, when the reality is that science as an institution is just as susceptible to human error as any other pursuit. Science doesn’t prove anything, it merely discredits what’s wrong. Researchers could be valiant and curious people, or they could be taking bribes on the side. We don’t know! You have to be vigilant.

Science has essentially enabled our internal Dunning-Kruger to sing alto in the choir of horseshit.

How many times have you seen someone, in a heated discussion, just go “that’s not what the studies say”, and then with a big ol’ shit-eating grin, act like they just figured it all the fuck out? Maybe, if you get a real high IQ individual, they’ll even link you to a study, one I guarantee you they haven’t read, or at least anything more than the abstract of. I recently ran into this when I got banned from a Discord for an off-topic discussion on the troon question. It’s such a heated shitshow of a question that I’m not gonna get into my feelings on it here, but I faced the usual retorts from the educated and informed crowd: “That’s backwards.” “You’re using anecdotes.” “That’s not what the studies say.” *baby seal noises*

In an informal discussion. On a Discord server. With randos online.

There’s an old saying that has no originator, though people usually give it to Mark Twain. “There are three kinds of lies: lies, damned lies, and statistics.” Data is…just that, really, data. How you gather it makes a difference. How you interpret it makes a difference. The motives behind the people doing the study make a difference. If you give me 1,000 hours in MS Paint, I can make the data say whatever I goddamn want it to. Science is constantly proving itself wrong because data is not empirical, it’s mere sampling and observation, prone to noise and error like anything else humans touch.

Let me count the ways a study can get botched. Just taking the most simplistic kind of study I can think of at the moment, consumer preference between phone brands, here’s a probably non-exhaustive list of ways I can make the thing skew in whatever direction I want:

  • I can use too few people.
  • Less specifically, I can pick a non-representative sample of people (say, people who frequent tech forums, or people who skew younger than the population).
  • Too few people responding can throw the data too, if they’re more inclined to respond or have more skin in the game than the average consumer.
  • I can widen the net on the demographics sampled if I’m trying to inflate numbers, say, to anyone who owns another product from my preferred brand.
  • I can count people who intend to switch to my side’s brand along with the current adopters.
  • I can use loaded questions and unclear wording to confuse people into picking whichever answer I want.
  • I can assume neutral answers as being in favor of the side I want to win. Alternatively, I can assume neutral answers for the other side as disapproval.
  • I can cheese the margin of error to make my side’s numbers a few percentage points higher and the other side’s a few lower.

These are all very real examples of how a study can be fudged. If we’re talking about more serious things, for example a study on rape statistics, I can cast the net as wide as possible and include things like drunk sex and sexual harassment to make it look like rape is a fucking epidemic. It’s unhelpful for policy decisions, it’s alarmist, and it shoves political persuasion into the public discourse like it absolutely shouldn’t be.

And therein lies the problem with people talking to me about what “the data” says.

Now, does this mean studies aren’t useful? Definitely not, but they need to be carefully controlled for. You need to watch for methodology. You need to watch for biases on the part of the study author and maybe even who’s financing it. You need to realize that even the good studies are nothing more than a data point, and data can only suggest or disprove, it cannot prove.

And if you don’t have time for all that, maybe just don’t take your internet discussions about your pet political topics all too seriously? Opinions are fine and dandy, you know, and people generally aren’t assholes. Let them be. You don’t have to be right about everything, smarty pants.

Comments are closed.