Strengthening Constitutional Self-Government

No Left Turns

Education

Standard Deviations

How low are the standards in New York City's schools?  Would you believe that one can pass the test, and move up to the next grade, by guessing the answers?

Categories > Education

Discussions - 1 Comment

Technically no matter how high the standard is on multiple choice tests it will always be "possible" to guess the answers and get a passing score, since guessing perfectly implies no contradiction even if it is extremely unlikely.

In this sense the back of the envelope test is sort of interesting, but not that damning,

"Her discussion with annonymous is interesting: annonymous writes: In both cases, that pattern-based approach happened to produce considerably higher scores than would be expected for random guessing: 12/26 in Grade 6 ELA, versus an expected outcome of 6.5. And 11/30 on Grade 7 Math, versus an expected outcome of 7.5.

But if you flip the subjects by grade, and apply that same “guessing algorithm” to Grade 6 Math and Grade 7 ELA, you get 4/25 (which results in the lowest possible scale score) and 6/26 (level 1) respectively. Those scores are both much more in line with what one would expect from random guessing, although, of course, if lots of students make random guesses then their resulting scores would be distributed in a normal curve. I.e. some kids who just fill in random patterns will indeed get lucky, and end up with higher scores than appropriate.

Of course, more meaningful results are produced when students are not guessing randomly. A really low-performing student is reasonably likely to score worse than a student picking randomly, because the items feature distracters that are designed to appeal to students who have common misconceptions."

What is still ridiculous in making the strongest case for the test is what the author points out in reply, namely that the sixth grade test only requires 27% on the multiple choice....this does mean that you have to be somewhat lucky (25% is your positive expectations). This means that her examples were somewhat cherry picked and non random, but actually her point is solid.

If you knew just one answer and guessed the rest randomly you would stand close to 50/50 chance of passing. If you knew 2 or 3 with various levels of certainty and could make some educated guesses that beat random walk you would have to be unlucky not to pass.

The bottom line is that 27% is unconscionably low, but more likely than not to fail by perfectly random guessing. If the test allowed 24% then there would be no exageration in the dire blog posts, as a slight majority of guessers would then stand to pass. But as it stands pure guessing should give a 56% fail rate, which itself would be seen with alarm... those who failed and scored 1's knew nothing and didn't get lucky... For a large enough school to get a high 95% pass rate...its students must do more than simply guess.

Also it seems possible that those who score 2's might be flagged, such that a judgement is made on a case by case basis to determine if they move on. Perhaps then by not writting the essay or doing other parts of the test a child who scores a two will be fast tracked into being held back.

Also perhaps these tests aren't necessarily determinative for moving on to the next grade, but the assumption that they are is employed to encourage an honest appraisal and effort.

If the real purpose of these tests are to compare school districts then they might still serve to do so meaningfully, regardless of if a score of 2 is a pitiful standard.

Leave a Comment

* denotes a required field
 

No TrackBacks
TrackBack URL: http://nlt.ashbrook.org/movabletype/mt-tb.cgi/14313