A Rosenhan Experiment For The PTO

Cross posted at my work blog.

How accurate is the Patent and Trademark Office? Can its examiners tell good patents from bad? Many doubt that they can and the PTO is often criticized for allowing bad patents. A famous experiment from psychology – the Rosenhan Experiment – suggests a way to cheaply test whether the PTO’s examining core are doing their job effectively: send them imposter patent applications.

The original Rosenhan Experiment was an amazing, and brave, study designed to test the accuracy of psychiatric diagnoses. The study had two parts. First, Professor David Rosenhan and seven associates had themselves admitted to a variety of mental hospitals. They described auditory hallucinations and were diagnosed as paranoid schizophrenics. After they were admitted, they acted normally and waited for the hospital staff to notice. While many of the real patients noticed that the imposters lacked symptoms of mental illness, the hospital staff did not. In some cases, Rosenhan’s associates were not released for months.

In a second part of Rosenhan’s study, a hospital was told that one or more imposters would attempt to be admitted as psychiatric patients. After hearing about Rosenhan’s initial test, this institution had claimed that similar errors would not occur there. Staff at the hospital subsequently determined that 41 out of 193 patients were highly likely to be imposters. But, in reality, Rosenhan had sent no imposters to the hospital. Rosenhan ultimately concluded that “one thing is certain: any diagnostic process that lends itself too readily to massive errors of this sort cannot be a very reliable one.”

Rosenhan’s famous experiment suggests that imposter patent applications could be used to test the accuracy of the PTO’s diagnostic process. I can think of three useful imposters: 1) applications for patents on existing technology (which test the examiner’s independent literature review); 2) applications for impossible technology (which test the examiner’s technical expertise); and 3) applications that cite to invalidating prior art (which tests whether the examiner is actually reviewing the cited prior art). By sending in imposters, the PTO can evaluate examiner thoroughness and competence in all of these areas. This can be compared to the TSA’s covert testing of airport security measures. In both cases, the testing keeps the examiners on their toes.

In fact, as Rosenhan’s second study showed, even the possibility of imposter patent applications will likely make the examiners more cautious. This is a good thing. Part of the problem at the PTO is that the incentives encourage the examiners to allow patent applications. Some may object that increased caution will worsen the backlog in patent applications. But the backlog should be dealt with by increasing application fees and PTO resources, not sloppy work. Errors at the PTO are particularly costly because juries must defer to the PTO when deciding validity questions. See Microsoft Corp. v. i4i Ltd. Partnership, 131 S.Ct. 2238 (2011) (unanimously upholding the “clear and convincing” standard for establishing patent invalidity in court).

Unlike the original Rosenhan Experiment, which required Rosenhan and his associates to endure weeks in psychiatric institutions, a Rosenhan Experiment for the PTO would only require preparing a handful of fake patent applications. This would be a cheap and effective way to test PTO competence and accuracy. The results could show us where the PTO succeeds and where it fails. And if the results show consistent failure, then it would provide strong reasons for abandoning the “clear and convincing” standard for proving patent invalidity in court.

Sadly, Professor Rosenhan passed away last month. As Stanford Law School mourns his passing, we can reflect on how his pioneering work in law and psychology could be applied in other areas.

Notes:

Professor Rosenhan’s original 1973 paper, published in Science, is available here (gated link).

For some of the academic commentary on the effectiveness of the PTO read Rethinking Patent Law's Presumption of Validity by Douglas Lichtman and Mark Lemley or As Many as Six Impossible Patents Before Breakfast: Property Rights for Business Concepts and Patent System Reform by Robert Merges.