Item Response Theory

Behind this technical name sits a plain question every fair test must answer: how do you compare two people who answered different questions?

The insight is to grade the questions, not just the students. A hard question answered correctly tells you more than an easy one; a question that strong students miss and weak ones pass is a broken question. By measuring the items — their difficulty, their power to separate the prepared from the unprepared — the test can place anyone on one scale, whatever items they saw.

The everyday version of this wisdom is worth keeping: not all evidence is equal. A man's behaviour under provocation tells you more than his behaviour at a festival. The easy circumstances prove little; the difficult ones discriminate.

So when you weigh people — employees, friends, yourself — grade the items. Anyone is kind on a good day. The informative data points are the hard questions: the tired evening, the expensive mistake, the moment with nothing to gain. That is where the scale lives.