Friday, August 24, 2012

All in All, I’d Rather Be Errol

A podcast on the Scientific American site caught my ear the other day. It was by James Flynn, after whom the Flynn Effect is named. I had read here and there about this effect before, but it was interesting to hear from the fellow himself. The Flynn Effect is the apparent and puzzling rise in IQs since IQ tests were invented a century ago. He documented the rise back in the 80s and has kept up with his research since then.

The rise hadn’t received attention prior to the 80s because of the way IQ tests are written and scored. The tests – first devised prior to World War 1 – never were intended to measure IQ changes over time. They were and are designed to measure relative intelligence at a particular moment in time. Assume for the moment that the tests remain unchanged for two years. (Actually, new questions are added and old ones dropped frequently, but a test might remain the same for a few years running.) In any given year, the average number of correct answers by test-takers is arbitrarily given a score of 100, which, by definition, is “average IQ.” The most common tests have a standard deviation of 15, meaning 50% of the population falls in the “normal” range of 85-115. 25% fall above and 25% below. 95% score between 70 and 130, with only 2.5% above 130 and 2.5% below 70. But what if the average number of correct answers is higher among test-takers this year than last year? That higher number is called 100 this year. So, the identical answers to the identical test can produce two different IQ scores depending on when the test was taken.

This is exactly what has happened, and the effect over decades is not a small one. If you don’t “renormalize” the scores (i.e. reset the average to 100), the raw data indicate that IQs consistently have risen a full 3 points per decade in the US since the tests started to be given. Most of the world has seen a similar rise, though there is variation nation by nation. So, if you give students in 2012 the exact same intelligence test given to students in 1952 and score the test exactly the same way it was scored in 1952, today’s students have an average score of 118. Another way of saying this is that the average student in 1952 was of below normal intelligence by 2012 standards. The average person of 1910 would have been below 70 IQ by 2012 standards, and so not legally competent. Einstein by 2012 scoring was a fairly ordinary guy. (There were no IQ tests in the time of Isaac Newton, but, if we extrapolate back to then, the average person would have had no brains at all.)

Clearly, something is wrong here, and Flynn himself expresses doubt about what the numbers mean. “Why,” he asks, “did teachers of 30 years experience not express amazement at finding their classes filling up with gifted students?” They don’t express any such thing, of course. Quite the contrary, at least in mid and upper grades. Thanks mostly to pre-school (almost nonexistent a half century ago), very young kids actually do outperform their peers of 50 years ago in reading and arithmetic. Kids often enter kindergarten already reading; no one did in my kindergarten, and that was the norm. The head start of today’s young kids doesn’t confer any lasting advantage, however. By 5th grade it has faded away. Despite heavier homework loads and vastly more expensive schools, kids in high school perform worse than their older peers – so much so that SATs had to be made easier to keep the nominal scores from tanking too far. If you took SATs in the 1970s you should add 70 points to your verbal scores and 30 points to your math in order to adjust them to 2012 standards. (Note that SATs test general knowledge, not abstract reasoning.) 12th graders have a smaller active vocabulary than 12th graders of 50 years ago. They have no better understanding of algebra or geometry, and, if you take away their calculators (which, admittedly, no one does anymore), their basic math skills are worse.

Some analysts go so far as to argue that modern high school graduates are intellectually impaired. See The Dumbest Generation: How the Digital Age Stupefies Young Americans and Jeopardizes Our Future (Or, Don't Trust Anyone Under 30) by Mark Bauerlein. I think his concerns are overblown, but not all of Bauerlein’s arguments are fanciful.

There is another point. I’m old enough to remember average people who were born in the 19th century, including two of my grandparents. (The other two were 1900, which technically was also 19th century, but, by convention, we choose to call it the first year of the 20th.) None of them was retarded. Anything but. In practical matters (mechanics, construction, animal husbandry, etc.), all of them were cleverer and more competent than I am; if you thought you could outsmart them you probably were making a mistake. However, none went beyond the 8th grade (only 5% of Americans did in 1900) and I can see how they might have been baffled by abstract questions on IQ tests – e.g. this very typical example: (picture of squiggly line) is most similar to A. (picture of cube) B. (picture of square) C. (picture of straight line) or D. (picture of circle). I know all of my grandparents would have considered this a damn fool question: “They aren’t similar!” If forced to answer, they might have said circle because of the curved line. The answer, of course, is C because it is one-dimensional (other than the thickness of the line itself) while the other images display area or volume. My grandparents understood dimensions. The charming novel Flatland: A Romance of Many Dimensions was written in 1884, after all. They would have “gotten” why C was the right answer when you pointed it out to them. But they just weren’t used to thinking this way on written exams, and still would have thought it a damn fool question. Modern students would pick the correct answer right away, because this is exactly the sort of question they encounter regularly in school and on aptitude tests. They are taught, sometimes privately tutored besides, to answer questions like this. Abstract questions, unsurprisingly, are the ones on which modern test-takers have gotten better; they haven’t improved on the basic knowledge side of the IQ tests.

Beyond this enhanced performance on certain types of standardized tests, is there any indication that people have gotten smarter? Not in school, apparently. Elsewhere? If so, it’s not obvious. There is no noticeable increase, for example, in “critical thinking,” the ability to evaluate hypotheses skeptically. Outlandish conspiracy theories are as popular as ever, and belief in the paranormal actually increases with education. According to a 2006 study by Bryan Farha and Gary Steward Jr. (source: LiveScience), 23% of college freshman believe in the general gamut of the paranormal (including astrology, clairvoyance, and ghosts), while 31% of seniors and 34% of graduate students do. The numbers are far higher for individual beliefs: among college freshman, for example, 40% believe in haunted houses, with 25% unsure.

Has the intellectual quality of political discourse risen? Flynn thinks so, pointing to some boneheaded comments in the House of Representatives a hundred years ago. I think he is overly generous to the current crop of politicos. I can link to plenty of boneheaded comments and speeches today, including a video of one sitting member of Congress expressing concern that the island of Guam might tip over. On balance I think the trend is the other way. Consider these very characteristic remarks before a crowd of farmers and local merchants by the barely schooled Abraham Lincoln in 1838. He praises the Founders of the nation and then warns about this:

“The field of glory is already harvested and the crop is already appropriated. But new reapers will arise and they too will seek a field. It is to deny what the history of the world tells us is true to suppose that men of ambitions and talents will not continue to spring up among us. And when they do they will as naturally seek the gratification of their ruling passions as others have done before them. The question, then, is can that gratification be found in supporting and maintaining the edifice that has been erected by others? Most certainly it cannot.”

No politician talks to a crowd this way today, and few would be capable of it. None would offer a theory of history or philosophize about the nature of man. Modern politicians assume (probably correctly) that such things are way over the heads of most voters, and they dumb down their messages accordingly. I also guarantee that folks in 1865 would have seen the irony in these earlier remarks, even if many in 2012 do not.

Yet, the rise in raw scores on IQ tests is real. The tests are measuring a change in something, but what? People today of all ages face abstract questions and puzzles more often than in years past. Many of these puzzles are encountered in non-school environments, such as when navigating the artificial realities of video games or when figuring out the arbitrary but (usually) self-consistent rules of some computer program. We develop a knack for solving them. Flynn uses the example of crossword puzzles. When he first became interested in them, he was bad at them. He had to learn to think in terms of unusual uses of words and to anticipate puns in the clues. Eventually, he became good at them. He thinks something analogous might be going on in IQ tests, but believes this to be only a partial answer for the rise in scores. While acknowledging that he is the expert, I nonetheless suspect it is the whole answer, or at least something very close to the whole. It would explain why these higher scores haven’t translated into better academic performance. I, too, have developed a modest skill at crossword puzzles in the past few years, but learning that “wapiti” is the word for the North American elk (among other trivia) hasn’t enhanced my performance at anything other than more crossword puzzles. It is a niche skill. So, it seems, is whatever is being measured in IQ tests.

Perhaps there is something valuable in these ever-more-finely honed test-taking skills, even if it is just a greater facility at playing Call of Duty on the latest Xbox. Let us hope so. Even if the only transference is to an aptitude for being foolish in a more clever fashion, that at least would have entertainment value, to the observers anyway.

Sam Was Ahead of His Time Academically

No comments:

Post a Comment