iPathology

Board Recertification Should be All or None

American specialty Boards play an essential role in setting standards for basic medical competence for graduates of residency programs. Historically, passing a Board exam was the final milestone to beginning a lifelong career in a medical specialty. In the last 10 to 20 years, however, Boards began to issue certificates which were time-limited, generally to 10 years. Under this scheme, a two-tier system of physician certification is established, namely that candidates who have passed the board exam more recently hold a time-limited certificate whereas physicians who took and passed the boards years or even decades earlier enjoyed the privilege of lifelong board certification. This dichotomy is not only anomalous but clearly unfair to more recent graduates of residency programs. There is no reason to believe, for example, that more recent graduates of residency programs are any less up to date with current medical practice than are their more senior colleagues. In fact, one could well believe that the opposite is true. For many physicians, there is a tendency toward specialization and therefore an increasing distance from the knowledge base of some areas of practice.

Given the rapid advances in biomedical research, it is probably a good idea to require periodic recertification of all physicians in their specialty area, and 10 years seems to be an appropriate interval. Ensuring that physicians are up to date with their field is consistent with the current trends towards emphasis on quality in the delivery of healthcare. This means that for a typical physician, he or she would be required to re-certify about three or four times in the course of a career. For those physicians who hold lifelong certificates, recertification in pathology is currently optional. Whether it remains that way is uncertain. It is entirely possible that hospital staffing committees or payers will require that all practicing physicians undergo periodic recertification if it is available. One point seems clear: if physician recertification is going to be required, it should be required for all physicians.

The Mislabeling Problem

One of the most common problems affecting patient safety in the clinical laboratory is the receipt of samples that are mislabeled. Despite all the advances in analytical performance, informatics and so on, it seems that getting the right name on the sample in a consistent fashion still eludes the healthcare enterprise. Why is this? Frequently the problem begins at the time of specimen collection when the phlebotomist is required to match a sample with the patient's name. This kind of task invariably poses the risk of human error. So many problems can arise in this situation including carelessness, haste, language difficulties between the patient and the phlebotomist etc. Even the use of bar coding of patient wristbands for automated reading by label printers does not seem to totally eliminate the risk. In the laboratory itself, there are ample opportunities for specimens and patient names to become decoupled. Surgical pathology slides can be labeled with the wrong patient identifier with potentially serious results for patient safety. Perhaps the area that poses the greatest immediate risk is the blood bank. There are few procedures that the clinical laboratory can perform that have greater potential to immediately jeopardize patient health than administration of an incorrect blood unit. It seems that if a choice of labs is to be made for implementing newer methods to eliminate the mislabeling problem, the blood bank should be among the highest priorities.

The Effect of Disease Prevalence on the Predictive Value of Diagnostic Tests

This is an updated version of a note I first wrote in 1999. Because it has been cited in several publications, I am posting it again at the ipathology.com website. 

Standard Metrics of Diagnostic Tests 

When considering the diagnostic utility of tests to distinguish persons with disease from normal persons, reference is frequently made to common metrics, namely, sensitivity, specificity, positive predictive value, and negative predictive value of a test. Traditionally, these concepts are explained with a simple matrix to define the terms true positive (TP), false positive (FP), true negative (TN), and false negative (FN) as follows. 

 

	Test Positive	Test Negative
Disease	TP	FN
Normal	FP	TN

 

Using these terms, the following metrics can be defined. 

Sensitivity = TP/(TP+FN) 

Specificity = TN/(TN+FP) 

Positive predictive value (PPV) = TP/(TP+FP) 

Negative predictive value = TN/(TN+FN). 

 

What is not so clear from these definitions is the effect of disease prevalence on the predictive values of test results, particularly the more commonly-used positive predictive value. Although standard textbooks of clinical pathology make reference to the fact that disease prevalence influences the positive predictive value, this relationship is seldom explained. 

Effect of Overlapping Values 

Given the definition of PPV above, it is apparent that this metric is a function of the TP and FP values. If there were no false positives, the PPV would be TP/TP or 100%. This would occur if the test could reliably distinguish normal persons from persons with disease without any overlap of values. However, this is rarely the situation with laboratory results that fall on a continuous scale. Almost always there is some degree of overlap between results from normal and diseased subjects. For example, low levels of blood markers of myocardial infarction can be detected in healthy subjects. These may overlap the lower range of values found in patients with MI. When there is such overlap, some cut-off point must be established to distinguish a positive from a negative result. It is worth noting that even results which are reported as "postivie" or "negative" by analytical instruments are generated from continuous scales by using a cut-off point. An example of this is the reporting of HIV-1 antibody screening tests. In a popular assay system, these are detected by ELISA reactions which result in changes in absorbance as measured by a spectrophotometer. The lower end of absorbance readings in HIV-1 positive patients overlap those of normal subjects. The computer in the analytical instrument defines a cut-off point of absorbance readings and then reports the results as positive or negative based on the cut-off value. Given that most assays show some overlap of values between normal and diseased subjects, it is apparent that it is very unusual for assays to have no FP results. 

Prevalence of Disease Influences Positive Predictive Value 

There is another situation in which there are no false positive results, namely, if everyone in the population had the disease. In this situation, every positive result would be a true positive. There could be no false positive results, and the PPV would be 100%. Conversely, if no one in the population had the disease, every positive result would be a false positive. There could be no true positives, and the PPV would be 0%. This leads us to conclude that the disease prevalence influences the PPV by influencing the true positive and false positive rates. 

Bayesian Analysis to Determine Positive Predictive Value 

The effect of disease prevalence may be determined by performing a Bayesian analysis. Consider a disease with a prevalence of 10% in a given population. A diagnostic test for the disease has both a sensitivity and a specificity of 95%. What is the positive predictive value of the test? 

Using Bayesian analysis we can determine the likelihood that a person with a positive test result has the disease as follows. For those not familiar with Bayesian calculations, the steps are outlined here and shown in the following tables. 

The two mutually exclusive possibilities are that the person has disease or is normal (row A). The a priori probability of having disease is 10% (0.1) and of being normal is 90% (0.9) (row B). If a person has the disease, the probability of having a positive test result is 0.95. If a person is normal, the probability of having a positive test result is 0.05 (row C). These probabilities are obtained from the sensitivity and specificity values above. If the sensitivity is 95%, then FN is 5%. If the specificity is 95%, then FP is 5%. 

The joint probabilities are the products of multiplying values in rows B and C (row D). The final probability for each of the two mutually exclusive states is obtained by dividing each value in row D by the sum of the two values in row D. This is shown in row E. 

 

A		Disease	Normal
B	A priori probability	0.1	0.9
C	Probability of a positive test result	0.95	0.05
D	Joint probability	0.1 × 0.95 = a	0.9 × 0.05 = b
E	Final probability	a/(a+b) = 0.68	b/(a+b) = 0.32

 

In this case, with a disease prevalence of 10%, a person with a positive test result has a 68% probability of having disease, and a 32% probability of being normal. 

In a population in which the disease prevalence is 0.1% (0.001), the calculations are as follows: 

 

A		Disease	Normal
B	A priori probability	0.001	0.999
C	Probability of a positive test result	0.95	0.05
D	Joint probability	0.001 × 0.95 = a	0.999 × 0.05 = b
E	Final probability	a/(a+b) = 0.019	b/(a+b) = 0.981

 

In this case, with a disease prevalence of 0.1%, a person with a positive test result has a 1.9% probability of having disease and a 98.1% probability of being normal. 

The effect of disease prevalence on the positive predictive value for a test with sensitivity and specificity of 95% is shown in the following table. 

 

Prevalence	PPV
0.1%	1.9%
1%	16%
10%	68%
20%	83%
50%	95%

Conclusion

The prevalence of a disease in the population has an important influence on the positive predictive value of a diagnostic test. With increasing disease prevalence, the more likely it becomes that a person with a positive test result has the disease, and the less likely it becomes that a positive result is a false positive. The effect of disease prevalence can be easily determined by Bayesian analysis.

 

Copyright, © Anthony A. Killeen, 1999-2007. All rights reserved.

Molecular Testing for Cardiac Rejection

Believe it or not it's been 40 years since the first human to human heart transplant was performed by Christiaan Barnard in South Africa.  The success of this procedure is now attested to by the fact that over 2000 patients in the United States receive a heart transplant each year with an estimated 50% 10 year survival.  In 1967 the first patient to receive a heart transplant survived only 18 days.

 

One of the main medical concerns following heart transplantation is rejection of the organ by the recipient immune system.  A new test, AlloMap, is now available that can detect the presence of cardiac rejection noninvasively [Starling RC et al., J Heat Lung Transplant, 25:1389, 2006].  The test is based on the measurement of expression of certain genes in peripheral blood mononuclear cells.  A panel of 11 genes is reliably informative for the identification of patients who have a very low risk of organ rejection.  While the negative predictive value of the test is extremely high the positive predictive value is quite low, in part because of the fact that with modern immunosuppressive therapies, relatively few patients develop acute cellular rejection particularly after more than a year post-transplantation. It is possible that the new test will reduce the number of cardiac biopsies that are performed posttransplantation.  If so, this will be probably be welcomed by patients.

Presumably the AlloMap test will be applied to other organ transplants such as liver, kidney and lung.  At the time of the first human heart transplant the procedure seemed extraordinary.  In many ways, the ability to detect organ rejection by gene expression profiling in a blood sample seems equally extraordinary.