Spatial pyramid matching (SPM) has achieved impressive successes in analyzing and classifying images across several domains. SPM computes a similarity measure over images by using bag of words similarity score over different levels of coarseness of the images. In this paper we propose a novel, simple approach based on SPM, differential SPM (DSPM) that incorporates finer differences among images while determining image similarity. The approach propagates the differences seen at fine levels to dampen the similarity observed at the coarser levels, thereby highlighting differences among images at small, localized regions. The resulting similarity scores among images can better separate images that match at coarse levels, but have subtle differences. DSPM integrated with K-nearest neighbor classification approaches was used to identify and analyze retinal Optical Coherence Tomography (OCT) images containing normal retinal scans as well as those from subjects with AMD (age-related macular degeneration) and DME (diabetic macular edema). The proposed approach achieved higher classification accuracy with smaller training overheads in comparison to SPM in all cases in our experiments.