Extending the work of Fröhlich, Lue and Furtado-Coelho, we consider the theory of Baer invariants in the context of semi-abelian categories. Several exact sequences, relative to a subfunctor of the identity functor, are obtained. We consider a notion of commutator which, in the case of abelianization, corresponds to Smith’s. The resulting notion of centrality fits into Janelidze and Kelly’s theory of central extensions. Finally we propose a notion of nilpotency, relative to a Birkhoff subcategory of a semiabelian category.