Protein families involved in chromatin-templated events are emerging as novel target classes in oncology and other disease areas. The ability to discover selective inhibitors against chromatin factors depends on the presence of structural features that are unique to the targeted sites. To evaluate challenges and opportunities toward the development of selective inhibitors, we calculated all pair wise structural distances between 575 structures from the protein databank representing 163 unique binding pockets found in protein domains that write, read or erase post-translational modifications on histones, DNA, and RNA. We find that the structural similarity of binding sites does not always follow the sequence similarity of protein domains. Our analysis reveals increased risks of activity across target-class for compounds competing with the cofactor of protein arginine methyltransferases, lysine acetyltransferases, and sirtuins, while exploiting the conformational plasticity of a protein target is a path toward selective inhibition. The structural diversity landscape of the epigenetics pocketome can be explored via an open-access graphic user interface at thesgc.org/epigenetics_pocketome.