Appropriate application of spatially variant system models can correct for degraded resolution response and mispositioning errors. This paper explores the detector blurring component of the system model for a whole body positron emission tomography (PET) system and extends this factor into a more general system response function to account for other system effects including the influence of Fourier rebinning (FORE). We model the system response function as a three-dimensional (3-D) function that blurs in the radial and axial dimension and is spatially variant in radial location. This function is derived from Monte Carlo simulations and incorporates inter-crystal scatter, crystal penetration, and the blurring due to the FORE algorithm. The improved system model is applied in a modified ordered subsets expectation maximization (OSEM) algorithm to reconstruct images from rebinned, fully 3-D PET data. The proposed method effectively removes the spatial variance in the resolution response, as shown in simulations of point sources. Furthermore, simulation and measured studies show the proposed method improves quantitative accuracy with a reduction in tumor bias compared to conventional OSEM on the order of 10%-30% depending on tumor size and smoothing parameter.