We introduce a new type of multi-resolution image pyramid for high-resolution images called <i>sparse pdf maps</i> (sPDF-maps). Each pyramid level consists of a sparse encoding of continuous probability density functions (pdfs) of pixel neighborhoods in the original image. The encoded pdfs enable the accurate computation of non-linear image operations directly in any pyramid level with proper pre-filtering for anti-aliasing, without accessing higher or lower resolutions. The sparsity of sPDF-maps makes them feasible for gigapixel images, while enabling direct evaluation of a variety of non-linear operators from the same representation. We illustrate this versatility for antialiased color mapping, <i>O</i>(<i>n</i>) local Laplacian filters, smoothed local histogram filters (e.g., median or mode filters), and bilateral filters.