We present a novel approach to the problem of detection of visual similarity between a template image, and patches in a given image. The method is based on the computation of a local kernel from the template, which measures the likeness of a pixel to its surroundings. This kernel is then used as a descriptor from which features are extracted and compared against analogous features from the target image. Comparison of the features extracted is carried out using canonical correlations analysis. The overall algorithm yields a scalar resemblance map (RM) which indicates the statistical likelihood of similarity between a given template and all target patches in an image being examined. Performing a statistical test on the resulting RM identifies similar objects with high accuracy and is robust to various challenging conditions such as partial occlusion, and illumination change.