Creating, Reviewing and Maintaining Good Test Content.
When we refer to "test content" or "items", we mean questions and answers and any associated parameters such as question reference, point value, randomization, etc.
Depending on the type of testing being done and the audience, rescoring is laden with controversy. In a New York Times article, "State Ends a Practice of Rescoring Regents Tests" on May 25, 2011, the State of New York Regents exam officials have '...ordered schools to end the longstanding but controversial practice of rescoring tests that fall just below the passing grade.'
Rescoring is a big deal if it means the difference between graduating--or not, getting a promotion--or not or keeping your job--or not. Depending on the test and the organization involved, there are formal petition processes that test takers go through to have their test re-scored.
Rescoring at any level can get messy. "We have a situation here..." seems to be a line in many a disaster flick. You know something bad has happened and the outlook is usually grim. A less ominous "situation" is one we hear from test administrators who discover that the test they've published contains one or more bad items. By "bad" we mean the question is poorly written, or technically inaccurate or the answer the test author listed as correct is actually incorrect. What to do? The answer is, it depends.
Online (LAN, intra- or internet) Testing
It depends on...
1- The organization's policy re rescoring tests
2- Whether or not an organization's tests will be audited
3- At what point in the testing process the test administrator recognizes the problem: whether or not it's too late in the testing process to unpublish the test, create a new test, require those who have already taken the test to retake the corrected assessment.
4- Whether or not the test administrator recognizes that modifying a test, that has already been given, by correcting or deleting a bad item, or changing one of several key test parameters, will corrupt the test data.
Items 1,2 and 3 above may be factors especially if answer sheets are used and scanned.
If tests are manually graded, it's easier for the test administrator to make allowances for bad test items. If the test will be given to more than one group, the test author can correct the bad item and reprint the test. If you are currently doing paper-and-pencil testing, we welcome your comments/feedback regarding how you handle the rescoring of tests containing bad items.
In the case of online tests that were created by drawing questions from one or multiple question banks, it's important that the test author take steps to correct the bad item in the question bank and include a journal entry under the question's Notes field describing the correction they've made.
We recognize that in some organizations, modifying a test items may involve some sort of psychometric review by one or more trained test authors.
For those of you interested the topic, read more.
We welcome your comments and feedback regarding your policies and practices regarding rescoring.