PET Performance Measurements Using the NEMA NU 2-2001 Standard

Margaret E. Daube-Witherspoon; Joel S. Karp; Michael E. Casey; Frank P. DiFilippo; Horace Hines; Gerd Muehllehner; Vilim Simcic; Charles W. Stearns; Lars-Eric Adam; Steve Kohlmyer; Vesna Sossi

Abstract

The NU 2-1994 standard document for PET performance measurements has recently been updated. The updated document, NU 2-2001, includes revised measurements for spatial resolution, intrinsic scatter fraction, sensitivity, counting rate performance, and accuracy of count loss and randoms corrections. The revised measurements are designed to allow testing of dedicated PET systems in both 2-dimensional and 3-dimensional modes as well as coincidence gamma cameras, conditions not considered in the original NU 2-1994 standard. In addition, the updated measurements strive toward being more representative of clinical studies, in particular, whole-body imaging. Methods: Performance measurements following the NU 2-1994 and NU 2-2001 standards were performed on several different PET scanners. Differences between the procedures and resulting performance characteristics, as well as the rationale for these changes, were noted. Results: Spatial resolution is measured with a point source in all 3 directions, rather than a line source, as specified previously. For the measurements of intrinsic scatter fraction, sensitivity, and counting rate performance, a 70-cm line source is now specified, instead of a 19-cm-long cylindric phantom. The longer configuration permits measurement of these performance characteristics over the entire axial field of view of all current PET scanners and incorporates the effects of activity outside the scanner. A measurement of image quality has been added in an effort to measure overall image quality under clinically realistic conditions. This measurement replaces the individual measurements of uniformity and of the accuracy of corrections for attenuation and scatter. Conclusion: The changes from the NU 2-1994 standard to the NU 2-2001 standard strive toward establishing relevance with clinical studies. The tests in the updated standard also are, in general, simpler and less time-consuming to perform than those in the NU 2-1994 standard.

For more than 10 y a concerted effort has been made to standardize performance measurements of PET scanners. Initially, a task group from the Society of Nuclear Medicine (SNM) defined a set of measurements, which was published in 1991 (1). Shortly after this task group began its work, the National Electrical Manufacturers Association (NEMA) formed a committee with members from all of the PET manufacturers, including 2 members from the SNM task group. The SNM tests were adopted and refined by NEMA, resulting in the NU 2-1994 document of performance standards for PET scanners (2). In parallel with the SNM task group, the European Economic Community underwent a concerted action to develop measurements for PET scanners. This work resulted in the International Electrotechnical Commission (IEC) Standard (3). Although there is some overlap between the NEMA and IEC standards, many of the specific details of the measurements (i.e., phantoms, acquisition procedures, or data processing methods) are different. Therefore, one does need to make a choice between these 2 standards when testing and specifying the performance of a PET scanner. Recently, the NEMA standard has been updated (4) by a committee, comprising members from manufacturers of dedicated PET scanners and coincidence imaging systems, as well as 2 members from the original SNM task group. During the development of the standard over the past few years, input was also sought from experienced PET users and experts in PET instrumentation. Although the new NEMA NU 2-2001 standard supersedes the original NU 2-1994, it does not resolve the differences between the NEMA and IEC standards.

Since the publication of the NU 2-1994 standard, there have been several developments in PET scanner technology. The NU 2-2001 standard is an attempt to respond to this changing technology. In particular, 3-dimensional (3D) imaging has become an important acquisition and reconstruction mode, whereas the previous standard assumed only 2-dimensional (2D) imaging. It became necessary to standardize how oblique lines of response (LORs) would be handled during measurement of scanner performance in 3D mode.

In addition, all coincidence gamma cameras have a very large axial field of view (FOV), typically 30–40 cm. Yet, the standard NEMA phantom defined in NU 2-1994 is only 19-cm long, and the tests using this phantom assumed that the axial FOV of the scanner was ≤17 cm. Before 1991 all commercial PET scanners did, in fact, have an axial FOV shorter than 17 cm. Today there are also several dedicated PET systems with an axial FOV as large as 25 cm, so the NU 2-1994 tests cannot adequately accommodate these scanners or the coincidence gamma-camera systems.

An issue related to the axial FOV is that the 20 × 19 cm NEMA phantom was designed to represent a compromise between brain imaging and body imaging. In the last 10 y, whole-body ¹⁸F-FDG studies have become the predominant type of PET study performed by most centers, driven partly by the available reimbursement through Medicare and local insurance carriers. Therefore, it is more relevant than before to measure the performance of PET scanners under conditions that better represent whole-body studies. In fact, a brain is much smaller than the NEMA phantom (although the head volume is comparable to the phantom volume), whereas a body is much larger. However, because most PET systems before 1991 operated in 2D mode with septa and with an axial FOV of <17 cm, the counting rate data acquired with the NEMA phantom served as a reasonable test of the scanner performance for both conditions. This is no longer true for a modern 3D PET system without septa, particularly if the axial FOV is >17 cm. Therefore, the most significant change in the NU 2-2001 standard, compared with the NU 2-1994 standard, is the change from a 19-cm-long phantom to a 70-cm-long phantom (albeit with a line source) for several of the tests, including sensitivity, scatter fraction, counting rate performance, and count loss correction. The 70-cm-long phantom is a better approximation to the activity distribution in whole-body studies, because the effects of out-of-field activity (OFA) are included in the performance measurement. The 20 × 19 cm NEMA phantom, on the other hand, continues to be used to test the performance of scanners used primarily for brain imaging, particularly for scanners that are dedicated brain imaging instruments.

Another major change in the updated standard is the addition of an image quality test. This measurement attempts to reproduce a clinical situation (an FDG whole-body study) and to take all aspects of camera performance and clinical imaging into consideration, including image reconstruction. The phantom used for this measurement is the IEC body phantom (3) with hot and cold spheres with an additional cylindric insert to simulate lung, as well as a second phantom to provide activity outside the scanner to simulate body uptake. This test was designed to mimic an FDG tumor study, with typical background activity and hot sphere contrasts encountered in those studies. The test is performed with a standard acquisition protocol that allows comparison between different types of PET scanners.

This article presents a description of each test and discusses the changes from the NU 2-1994 standard, along with illustrative data using several different PET systems. The NEMA Standards Publication NU 2-2001 (4), which provides more detail than presented here, is available from NEMA (1300 N. 17th St., Suite 1847, Rosslyn, VA 22209, www.nema.org). Readers wishing to perform the measurements should consult this document for a complete description of the procedures.

MATERIALS AND METHODS

The performance tests are divided into 2 groups. The first group includes the basic intrinsic measurements of (a) spatial resolution, (b) sensitivity, (c) scatter fraction, and (d) count losses and random coincidences. The second group includes measurements of the accuracy of corrections for physical effects, specifically (e) count losses and random coincidences, and (f) overall image quality. The only radioisotope that is used is ¹⁸F. It is our experience that other radioisotopes (e.g., ⁶⁸Ga or ²²Na) are not as simple to calibrate for the sensitivity and count losses measurements, and their longer positron ranges preclude their use for measuring spatial resolution. Each test requires that the operating parameters of the scanner be adjusted, as they would be for a typical patient study, including the energy window, axial acceptance angle, coincidence time window, and slice thickness. The only exception to this requirement is for the measurement of spatial resolution, where finer sampling than is used clinically is allowed. The data processing and reconstruction algorithms should also be the same as those used for a typical patient study, with the exception that some tests require the use of filtered backprojection with a ramp reconstruction filter or single-slice rebinning (5) for standardization among systems. Système Internationale d’Unites (SI) units are used for all reports of performance measurements. For all studies, fully tomographic data are acquired (i.e., rotating scanners must rotate to provide complete angular sampling).

Spatial Resolution

The spatial resolution of a system represents its ability to distinguish between 2 points of radioactivity in an image. The purpose of the measurement of spatial resolution is to characterize the widths of the point spread function (PSF) in the reconstructed image of compact radioactive sources. The width of the PSF is reported as the full width at half maximum (FWHM) and full width at tenth maximum (FWTM). The spatial resolution is measured in the transverse slice in 2 directions, radially and tangentially, and in the axial direction.

For all scanners, point sources of ¹⁸F (<1-mm extent in any direction) are imaged in air. The point source consists of a small quantity of concentrated radioactivity inside a glass capillary (wall thickness, <0.5 mm). The resolution is measured with the sources at 6 locations. Two axial positions are selected—namely, the center of the axial FOV and a position one fourth of the axial FOV from the center. For both axial locations, the source is imaged at 3 positions, (a) x = 0 and y = 1 cm (to avoid the exact center of the scanner where the sampling density of lines of response may be very high), (b) x = 0 and y = 10 cm, and (c) x = 10 and y = 0 cm.

The data are reconstructed with filtered backprojection using an unapodized filter (ramp filter with a cutoff at the Nyquist frequency); the image pixel size should be smaller than one third of the expected FWHM. No smoothing of the data is performed. The FWHM and FWTM of the PSFs are determined in all 3 directions by forming 1-dimensional (1D) response functions through the peak of the distribution in the 3 orthogonal directions. The width of these profiles at right angles to the direction of measurement is ∼2 times the FWHM in those directions, rather than a single pixel, to reduce measurement variability. The FWHM and FWTM are calculated by linear interpolation between adjacent pixels at one half or one tenth of an estimate of the maximum value of the response function. This maximum value is determined by a parabolic fit using the peak value and its 2 nearest neighboring points. The radial, tangential, and axial resolutions for each radius, averaged over both axial positions, are reported.

Intrinsic Scatter Fraction

The scattering of annihilation photons leads to falsely positioned coincidence events. Variations in design cause PET scanners to have different sensitivities to scattered radiation. The intrinsic scatter fraction is a measure of the relative system sensitivity to scatter. For a given source distribution, a lower scatter fraction is more desirable, regardless of the accuracy of the method for scatter correction, because correction techniques cannot compensate for the noise introduced by the unwanted events and can, potentially, add bias to the image. Here, we define the scatter fraction to be the ratio of scattered events to total events, which are measured at a sufficiently low counting rate that random coincidences, dead-time effects, and pileup are negligible. Total events, therefore, are the sum of unscattered events (trues) and scattered events.

The phantom in this measurement, as well as that of count losses and randoms, comprises a 20-cm-diameter solid polyethylene cylinder with an overall length of 70 cm. The data for the measurement of intrinsic scatter fraction are taken from the low-activity scans of the measurement of counting rate performance, when the count loss and randoms rates are both <1% of the true rate. Activity is placed in a line source that is threaded through a hole in the cylinder at a radius of 4.5 cm and parallel to the central axis. It should be noted that the line source (∼2.3-mm inner diameter, ∼3-mL fillable volume) is sufficiently large to preclude problems with bubbles, despite its length, provided reasonable care is taken during filling. For consistency, the phantom is rotated such that the line source is at the lowest position (i.e., nearest to the patient bed), because the measured result will depend on the relative orientation of the line source and the bed.

Data are sorted into a sinogram, which is the 2D projection space representation (projection ray vs. angle) of a transverse plane. Oblique LORs are assigned to the slice where the LOR crosses the scanner axis using single-slice rebinning (5). Because the transaxial physical FOV of scanners varies, a fixed diameter of 24 cm (4 cm larger than the phantom diameter) is used for the calculation of scatter, as was done in the NU 2-1994 standard. The sinogram profile is used to calculate the number of scatter events within the FOV and the number of trues within a 2-cm radius of the source. The scatter within the peak is estimated by assuming a constant background under the peak, the level of which is determined by the average of the intensities near the edge of the peak (at ±2 cm). The sinogram profile is analyzed as a function of angle and the results are averaged. The scatter fraction for each slice and the average of the slice scatter fractions are reported.

Sensitivity

The sensitivity of a scanner represents its ability to detect annihilation radiation. In the NU 2-2001 standard, the absolute sensitivity of a scanner (expressed as the rate of detected coincidence events in counts per second [cps] for a given source strength, expressed in MBq) is measured. The measurement technique is based on the method of Bailey et al. (6). Because the emitted positrons annihilate with electrons to create a pair of γ-rays, a significant amount of material must surround the source to ensure annihilation. This surrounding material also attenuates the created γ-rays, prohibiting a measurement without interfering attenuation. To arrive at an attenuation-free value of the sensitivity, successive measurements are made with a uniform line source surrounded by known absorbers. The sensitivity with no absorber can be extrapolated from these measurements.

In the NU 2-1994 standard, the sensitivity was measured using a standard distributed source and reported as the rate of coincidence events for a given activity concentration in that phantom (20-cm diameter, 19-cm long). For scanners with an axial FOV longer than 19 cm, however, this measurement could underrepresent the efficiency of the scanner. For other measurements (e.g., scatter), a 70-cm-long phantom is used, as a way to standardize the phantom. A 70-cm-long uniform cylinder, however, would be impractical to fill and to handle. For those reasons, it was decided to move from the historical but arbitrary method of measuring sensitivity with a cylindric phantom to one less affected by the self-attenuation of a uniform cylindric source.

A 70-cm-long plastic tube is filled with a known amount of radioactivity, sufficiently low that count losses and randoms are negligible. The tubing is encased in metal sleeves of varying thickness and imaged, suspended in the center of the transverse FOV. Alternatively, the radioactivity can be placed in the smallest metal sleeve, as long as it is well sealed. Consideration was given to the sleeve material; it was agreed that the metal does not need to be specified, because the measurement is self-consistent. Single-slice rebinning is used to assign oblique LORs to the slice where the LOR crosses the scanner axis. The counting rate in the absence of attenuation R_o is determined by fitting the measured counting rates R_j as a function of sleeve thickness X_j: $\text{[math]}$ Eq. 1 The attenuation coefficient μ may be unknown, because the composition of the metal sleeves is not specified; in addition, small amounts of scattered radiation present will cause μ to deviate from the narrow-beam value for the metal used. For these reasons, both R_o and μ are determined by the fitting process. The total sensitivity S_tot is given by R_o, divided by the measured activity (in MBq). The slice sensitivity S_i is determined by scaling S_tot by the fraction of total counts contained in slice i for the thinnest sleeve (R_1,i): $\text{[math]}$ Eq. 2 where $\text{[math]}$ The measurement of total sensitivity is repeated with the source offset radially 10 cm from the center of the transverse FOV. The total sensitivity and slice sensitivities for both radial positions are reported.

Counting Rate Performance

Most patient PET studies are not performed under conditions of low counting rate losses or negligible randoms rates. At higher activity levels, coincidence events are lost because of system dead time, whereas the rate of random coincidences rises. It is necessary to measure the counting rate performance (both dead-time losses and randoms) as a function of activity to understand the scanner’s behavior for a wide range of scanning conditions.

The 70-cm polyethylene cylinder with a line source described for the measurement of the intrinsic scatter fraction is used in this measurement. For this procedure, however, the line source is filled with a known initial amount of activity sufficiently high that both the peak trues rate and the peak noise equivalent count (NEC) rate (7) can be measured. This starting activity is determined empirically for each scanner and acquisition mode; the manufacturer will generally provide a recommended initial activity. Data are taken until the randoms and dead-time losses are negligible. To avoid potential problems with nonuniform dead time associated with the off-center line source, completely tomographic data are acquired. Sinograms are generated, and oblique sinograms are collapsed into a single sinogram for each slice while preserving counts by using single-slice rebinning.

The total counting rate within a 24-cm transverse FOV is determined as the activity decays. The background, resulting from randoms and scatter, is estimated as was done for the intrinsic scatter fraction measurement. The true event rate R_trues is then determined by subtracting the randoms + scatter background from the total rate R_total. The randoms rate R_randoms is calculated as: $\text{[math]}$ Eq. 3 where SF is the intrinsic scatter fraction. The scatter rate R_scatter is given by: $\text{[math]}$ Eq. 4 Finally, the NEC rate is computed as a function of activity: $\text{[math]}$ Eq. 5 where k = 2 if the scanner clinically does on-line randoms subtraction (i.e., the estimate of randoms is noisy) and k = 1 otherwise (i.e., the estimate of randoms is noise-free). The total, true, randoms, scatter, and NEC rates are plotted against an effective activity concentration, computed as the total activity in the line source divided by the total volume of the cylindric phantom (22,000 mL). This effective activity concentration, though an arbitrary scaling of the line source activity, corresponds closely to clinical activity concentrations and the counting rates seen at those clinical concentrations. From the counting rate plot, the peak true counting rate and peak NEC rate are determined and reported, along with the activity concentrations where these peak rates occur.

Accuracy of Corrections for Count Losses and Randoms

To achieve quantitative measurements of source activity distributions under widely varying conditions, PET scanners must have the capability to compensate for dead-time losses and randoms. The accuracy of these corrections, particularly at the highest counting rates encountered in clinical imaging, is reflected by the bias with which the scanner reports counts.

The accuracy of corrections for dead-time losses and randoms is determined from the counting rate performance data, described above. The data are reconstructed with all count-rate–dependent corrections (i.e., for dead-time losses and randoms) applied. A large (18-cm diameter) region of interest (ROI) is defined, centered on the phantom, and the residual error ΔR as a function of effective activity concentration is given by: $\text{[math]}$ Eq. 6 where R_extrap is determined from the low counting rate data, where there are negligible dead-time losses or randoms, and is assumed to be a linear function of activity. A large ROI is used, as opposed to a small region centered on the line source, to reflect any inaccuracies in randoms subtraction over the entire phantom. The errors are calculated for each slice, and the largest and smallest errors over all slices are plotted as a function of effective activity concentration. The maximum absolute value of the bias is also reported for activity values at or below the activity level of the peak NEC rate.

Image Quality Measurement

Because of the complex interplay of different aspects of system performance, it is desirable to be able to compare the image quality of different imaging systems for a standardized imaging situation that simulates a clinical imaging condition. The purpose of this measurement is to produce images simulating those obtained in a whole-body study with both hot and cold lesions. Spheres of different diameters are imaged in a simulated body phantom with nonuniform attenuation. Activity is also present outside the scanner.

The phantom consists of a torso phantom (3,8), containing hot and cold spheres (wall thickness, ≤1 mm) in a warm background. The hot spheres have inner diameters of 1.0, 1.3, 1.7, and 2.2 cm; the cold spheres have inner diameters of 2.8 and 3.7 cm. A 5-cm-diameter insert with an attenuation coefficient approximately equal to the average value in lung (density, 0.30 g/mL) is also placed in the center of the phantom, and measured attenuation correction is performed. The background is filled with ¹⁸F at an activity concentration typical of what is seen in patient FDG studies (370 MBq/70-kg patient, or 5.3 kBq/mL). The hot spheres are sequentially filled with activity concentrations of 8 and 4 times that of the background (2 sequential acquisitions). This process is efficiently accomplished by first putting the background activity in one eighth of the background’s volume, withdrawing sufficient volume to fill the spheres, and then filling the rest of the background with nonradioactive water. After 1 half-life has elapsed, another sample of background activity (measured at the same time as the initial aliquot) is used to increase the background activity back to the starting level and reduce the hot sphere activity ratio to 4:1. The line source of the 70-cm-long phantom is filled with sufficient activity to yield an effective activity concentration (as defined above) equal to that of the background in the torso phantom; the 70-cm-long phantom abuts the body phantom to approximate the clinical situation of having activity outside the scanner FOV.

The acquisition protocol being simulated is to scan an axial distance of 100 cm in 60 min. The data acquisition time is determined by considering the axial distance Δz that the bed is translated between positions in a whole-body study (typically less than the axial FOV). The acquisition time T_acq is calculated by: $\text{[math]}$ Eq. 7 T_acq includes both emission and transmission scan durations and any other nonimaging transition times (e.g., for bed movement or transmission source loading or unloading). This normalization of the scan duration is essential because of the wide range of axial FOVs in current-generation PET systems. Three replicate measurements should be acquired to improve the reliability of the results.

The data are reconstructed with all available corrections applied, using the standard reconstruction algorithm and usual parameters for whole-body studies. To quantitate the image quality, ROIs with diameters equal to the physical inner diameters of the spheres are drawn on the spheres and throughout the background. Twelve background ROIs are drawn in the central slice, as well as in slices ±10 mm and ±20 mm away, for a total of 60 background ROIs of each size. The coefficient of variation of the means in these background ROI values for each sphere size is taken as a measure of the background variability. The hot sphere contrast recovery coefficient (CRC_hot) is calculated as: $\text{[math]}$ Eq. 8 where C_hot and C_bkgd are the average of the counts measured in the hot sphere ROI and the average of the counts in all background ROIs, respectively, and a_hot /a_bkgd is the ratio of the activities in the hot sphere and background. The cold sphere CRC (CRC_cold) is calculated as: $\text{[math]}$ Eq. 9 where C_cold is the average of the counts measured in the cold sphere ROI. The hot and cold sphere CRCs for the 6 spheres, as well as their corresponding values of background variability, are reported.

In addition, the accuracy of the corrections for attenuation and scatter is assessed by drawing an ROI in the region of the lung insert. The residual error (ΔC_lung) is calculated for each slice as: $\text{[math]}$ Eq. 10 where C_lung is the average of the counts in the lung insert ROI. Transverse and coronal images are also reported because of the difficulty in devising appropriate quantitative measures to express all aspects of image quality.