McKinzey, R. K., & Campagna, V. (2002, April 27). The Rorschach, Exners Comprehensive System, Interscorer Agreement, and Death. WebPsychEmpiricist. Retrieved April 27, 2002, from http://www.wpe.info/papers_table.html
Interscorer agreement of Exner's Comprehensive System for the Rorschach was tested. Thirty qualified psychologists volunteered to score a single protocol, which had been used as part of a Death Penalty appeal in a case of rape and murder. 63% of the psychologists made scoring errors. After the errors were corrected, the resulting scorings showed wide variation. Only 10% agreed with the original psychologist's conclusions that both SCZI and DEPI were elevated. The range of scorings and amount of disagreement make Rorschach interpretations easily rebutted in forensic cases.
McKinzey, R. K. (2002, April 30). Prior peer review of Rorschach interscorer agreement. WebPsychEmpiricist. Retrieved April 30, 2002, from http://wpe.info/papers_table.html
The prior peer reviews of previous versions of The Rorschach, Exners Comprehensive System, Interscorer Agreement, and Death are presented verbatim, together with the journals editors rejection letters. Alternate explanations for the results are offered; poor protocol administration, demand characteristics, patient characteristics, and unstandardized format. The generalization of the results is questioned, as well as the nature of Interscorer agreement, the CS coding system, the use of single data points, the rhetorical style, and possible bias.
McKinzey, R. K., & Campagna, V. (2002, April 30). Authors' Reply to prior review of "Rorschach interscorer agreement". WebPsychEmpiricist. Retrieved April 30, 2002, from http://www.wpe.info/papers_table.html
The authors of The Rorschach, Exners Comprehensive System, Interscorer Agreement, and Death included six blind, anonymous reviews offered during previous submissions. The papers authors reply, arguing that the paper was wrongly rejected, the sole conclusion remains valid, and another study has already partially replicated the results.
McKinzey, R. K. (2005, November 28). Authors Update to Rorschach Interscorer Agreement. WebPsychEmpiricist. Retrieved November 28, 2005, from http://wpe.info/papers_table.html
Since the 5/1/02 publication of The Rorschach, Exners Comprehensive System, Interscorer Agreement, and Death, other relevant articles have become available and are reviewed. In the first update (2/22/03) an article admitted as true several criticisms made by the CS critics. Another, by Exner and colleagues, presented an interscorer reliability study using the same paired design as called for by the critics and as done by three previous studies. The results are so completely different that they must be viewed as suspect. A third article is supportive, but only through apparently selective reporting. In the second update (4/14/03), a new book presents the case against the CS in detail, citing our article. In the third update (6/16/05), two more articles are reviewed, both of which have cited the paper. One agrees with the misscoring problems, but insists they are due to human error, rather than the test itself. The second challenged the use of a dissertation (Mittman, 1983) in supporting a citation of the Rorschachs false positive rate at 81%. The fourth update (8/10/05) notes a Rorschach apologist has written about how easily a given Rorschach can be rebutted.