I’ve never known much about IQ tests or how useful it is as a measurement, though I’ve had the vague impression that the whole concept was outdated and problematic. So I was quite interested to learn about it in my data collection/research methods class this week. I won’t subject you to the real topic of the lessons (factor analysis—only interesting to psychometricians and the wonkiest of wonks), but here are some highlights of the questions/current research on IQ that I thought were pretty nifty to think about.
What do the tests really measure?
Intelligence? What does that even mean? Verbal or math skills? Problem-solving? Mental agility/reaction time? (Western) cultural literacy? Then there’s Gardner with his 7 kinds of intelligence, and Sternberg with analytical, creative, and practical. It’s a big old hairy mess just defining terms.
The truth is that no one really knows what IQ tests measure, so they give it a letter. Whatever it is those questions are testing for, let’s call it “g.”
How do you write a test to measure g when you don’t even know what it is?
Easy, as it turns out. Over the years, you just throw out questions on the test that aren’t correlated with each other. If most people who score well on the test get #17 wrong, you throw out #17—it’s not measuring the same thing as all your other questions. Eventually, you end up with hundreds of questions that seem to test for the same indefinable skill (g). Ding! An IQ test.
What the hell good is g?
Even though no one is quite sure what kind of “intelligence” g is, it does seem to be strongly correlated with all sorts of things we’re interested in. There are plenty of crappy studies on IQ, but some have held up to a lot of scrutiny. One example: IQ is one of the most powerful predictors of job performance—more powerful than several other measures of ability, resume quality, and almost three times more powerful than interviews (a stat I would like to share with my future potential employers). Ignoring such a useful measure entirely doesn’t make a lot of sense.
Are IQ tests biased against African Americans?
Yeah, here’s the juicy stuff. As a good liberal, I totally bought that tests were biased, but damned if I knew why or how, so I was quite interested to hear a more informed perspective. The short answer is yes, probably, but not nearly as badly as they used to be, and not in the ways you might think. In the past, the tests were extremely class-biased, requiring such cultural literacy as knowing something about Goethe and Schubert. Such obvious problems are gone in today’s questions. Yet the group IQ average of African Americans remains significantly lower than the white average. Do white people really have more g? Unlikely. What seems to drive the differences is self-handicapping during test taking, in response to the “Stereotype Threat.” Essentially, people do worse on tests when they think the results will be used to evaluate their group’s performance against another group that’s expected to do better. This phenomenon is not unique to IQ tests: many fun experiment-based studies document its effects in all kinds of situations. Women do worse on math tests; white men do worse when they believe Asian men will do better; senior citizens do worse on memory tests when they think the results are used to evaluate memory loss among the elderly. One of my favorites: black men do worse at golf when told it measures “sports intelligence,” and white men do worse when told it measures “natural ability.”
In regards to IQ tests in particular, there are more interesting results. First, there is a lot of evidence that African Americans have a much higher stress level before and during the tests—less sleep the night before, lower ability to focus, and a higher sense of stress and belief that the test is unfair. They do significantly better on the tests when they are administered by African Americans. They also do better following a writing exercise at the beginning that reaffirms a sense of personal adequacy. Finally, in experiments where people were told the IQ test was a puzzle, not an intelligence test, the racial gap disappeared completely.
So, IQ. There it is. Other potential topics for this space were the litigation history of school segregation (conclusion: not good, my friends, not good) or a brief evaluation of the different ways to calculate high school dropout rates.
I love school.