Shitty Psychological Tests and Pseudo-Science

posted
Total Posts
15
Topic Starter
Railey2
People love seeing themselves on scales. We all want to get to know ourselves a little better, want to understand whats going on in our heads. Testing offers an easy way to achieve that, just answer some questions and an algorithm will give you what you want to know.

Lets take the autism test from the other thread for example: http://archive.wired.com/wired/archive/9.12/aqtest.html
( t/102245&start=0 )




ASD is a disorder, that primarily affects information-processing, which results in more or less severe impairments in all sorts of social settings.
Now, since the test was made to identify autistic traits, one thing it checks for is competence in social situations. When you take a look at the test, you notice that many questions are targeted at this specific field, namely:
1), 10), 11), 13), 15), 17), 18), 22), 26), 27), 31), 33), 35), 36), 38), 39), 44), 45), 47), 48), 50)

thats 21 questions.



As it happens to be, there are other things than autism that can cause incompetence in social settings, the most relevant example being social anxiety with its various causes.
Seeing how many questions were targeted at social competence, I strongly suspected that anyone with social anxiety will automatically score a lot higher, regardless of whether he/she has autism or not.
A quick trial-run where I simulated social anxiety (answering all aforementioned social questions with either strongly agree/disagree, depending on their wording), while neglecting other questions (answering with slightly agree/disagree in random manner), gave me a score of 30.

Take a look at this:
here it tells me how the questions were weighted and how the final score was computed. As it turns out, 17 of the social competence-questions were counted and added to my final score (surprise). I gained 17 points by simply pretending to be socially anxious.




This test is extremely confounded. One thing that can be and most of the times is completely unrelated to autism, and in addition to that happens to occur quite often, distorts the result by an entire order of magnitude.

This is bad, very very bad. Even worse because it wouldn't be even that hard to design the test in a way that it circumvents this problem.
It's a shitty test.

And thats not the only example of shitty tests making their way to the public and to this forum. The INTJ-Whatever test-thread comes to mind.
( t/216342&start=0 )

The worst thing is, that people take tests like this seriously, because they don't know how to tell when a test is badly designed. They then identify themselves with the results, which can have a grave impact on their lives (self-fulfilling prophecies).
Don't fall for that, and don't support shitty tests. Don't support pseudo-science by spreading it to unassuming people.


Discuss
Green Platinum

Railey2 wrote:

The worst thing is, that people take tests like this seriously, because they don't know how to tell when a test is badly designed. They then identify themselves with the results, which can have a grave impact on their lives.
Do they?
Topic Starter
Railey2

Green Platinum wrote:

Railey2 wrote:

The worst thing is, that people take tests like this seriously, because they don't know how to tell when a test is badly designed. They then identify themselves with the results, which can have a grave impact on their lives.
Do they?
Yes they do. These people even built and entire business-mode around their nonsense-theory: http://www.16personalities.com/free-personality-test . This is only possible because people do care, otherwise it wouldn't be worth the effort.
Particularly MBTI and its various scam-derivatives are overwhelmingly popular, which is also the reason why you see threads about them popping up everywhere.
Professional, expensive tests are widespread.

Generally, people love to categorize themselves and others. We are wired to do that, and tests give us perfect backing in doing so.


If you read the autism thread, you even see people making an effort to compare the score to their perceived autism-level, aka "IRL I would be .. " and so on. When people start to think about a topic, its a really big sign that they do take it seriously.
Also, it takes time to answer a test. The fact that they were willing to answer 50 questions, tells you that they don't think that it is worthless (although I admit that many people will just do the autism-test for funsies. MBTI is a better example for people that really care about the results).
mashedpopoto
I get what you're saying buddy.

People honestly shouldn't be self diagnosing themselves with mental disorders anyway; especially using online tests.
Topic Starter
Railey2

Philantropist wrote:

I get what you're saying buddy.

People honestly shouldn't be self diagnosing themselves with mental disorders anyway; especially using online tests.
this is particularly what I meant with self-fulfilling prophecies. It is also why diagnosing yourself is an absolute no-go. Its actually dangerous to self-diagnose.
Vuelo Eluko
i knew the test was wrong when it gave me a 14.

the questions were so shit. often noticing license plates? even actual autistic people would have a hard time answering that.
agu
Few online tests are to be trusted, I don't see a direct problem with self-diagnosing but if you believe that you have an actual diagnosis it would be best to get help from someone who can actually verify this.

I'll just say if you believe that you're autistic for instance without verification, you shouldn't let this change you. If you feel the need to, you should probably stick to saying something like you experience similiar symptoms to whatever disorder.

I'd also like to mention that you shouldn't judge people based on you getting the impression that they don't have a disorder they claim they do, seriously, don't do this. You don't know anything about their life, no sense in bashing them because they think they have a disorder.


to end it off the most important advice I can give is that if you think you have a serious disorder or whatever, get help, chances are you're being paranoid but if you feel like this disorder is crippling, you honestly need help. (I know I'm kind of repeating myself but I'd like to underline this)

EDIT: also in case it wasn't obvious self-diagnosing isn't good, but you need to evaluate whether it is possible that you have a problem so you can get help with said problem
Vuelo Eluko
im scared to get tested for autism because I don't want to find out that I don't have it.
that would break me
ive built myself around having it
Green Platinum

agu wrote:

Few online tests are to be trusted, I don't see a direct problem with self-diagnosing but if you believe that you have an actual diagnosis it would be best to get help from someone who can actually verify this.

I'll just say if you believe that you're autistic for instance without verification, you shouldn't let this change you. If you feel the need to, you should probably stick to saying something like you experience similiar symptoms to whatever disorder.

I'd also like to mention that you shouldn't judge people based on you getting the impression that they don't have a disorder they claim they do, seriously, don't do this. You don't know anything about their life, no sense in bashing them because they think they have a disorder.


to end it off the most important advice I can give is that if you think you have a serious disorder or whatever, get help, chances are you're being paranoid but if you feel like this disorder is crippling, you honestly need help. (I know I'm kind of repeating myself but I'd like to underline this)
Exactly these online tests like everything online should be taken with a grain of salt and at most should be used as a litmus test. I don't know any cases of people making huge life changing decisions on them tho.
Nameless

Green Platinum wrote:

Exactly these online tests like everything online should be taken with a grain of salt and at most should be used as a litmus test. I don't know any cases of people making huge life changing decisions on them tho.
I agree.
Cephre
wait people actually take these things seriously
Zymaz
people still play this game ?
Sanxu

Railey2 wrote:

We all want to get to know ourselves a little better, want to understand whats going on in our heads.
Dear god no. Being an introvert with shitloads of time to analyse just why I do the things I do and think the things I think and doing some honest self assessment I am absolutely disgusted by myself whenever I do. I'll pass.
Jordan
Anyone that isn't retarded takes these tests with a grain of salt. It's just fun to compare results. As the social animals that we are, we will always try to fit within a stereotype, which is kind of funny, because we often do it without even realizing it, or rather, without thinking about it. The average person will tell you they hate being stereotyped yet they will try hard to fit into one. Anyone who diagnoses mental disorders based on online tests does it for the same reason, too - some people build themselves around having some mental disorders (le bipolar tumblr girl), they will answer the questions that will obviously give out that result and then post said result as "proof".

tl;dr nobody should take these tests seriously
sottovoce
MBTI properly administered can be a powerful and useful tool. Almost all of the controversy around can be attributed to poor understanding of Jungian typology and simplistic online tests with leading questions. Some people say that MBTI has been discredited by the academic research into it, and they might even quote some surface level study. It suffices to say the topic is far more complicated than that, and MBTI is certainly not an academic 'black hole'. I don't have the time (or the inclination) to mount a proper defence here - and I suspect no one here really cares anyway - but no need to dismiss it out of hand.
Please sign in to reply.

New reply