In this paper we study a monolithic Newton-Krylov solver with exact Jacobian for the solution of incompressible FSI problems. A main focus of this work is on the use of geometric multigrid preconditioners with modified Richardson smoothers preconditioned by an additive Schwarz algorithm. The definition of the subdomains in the Schwarz smoother is driven by the natural splitting between fluid and solid. The monolithic approach guarantees the automatic satisfaction of the stress balance and the kinematic conditions across the fluid-solid interface. The enforcement of the incompressibility conditions both for the fluid and for the solid parts is taken care of by using inf-sup stable finite element pairs without stabilization terms. A suitable Arbitrary Lagrangian Eulerian (ALE) operator is chosen in order to avoid mesh entanglement while solving for large displacements of the moving fluid domain. Numerical results of two and three-dimensional benchmark tests with Newtonian fluids and nonlinear hyperelastic solids show a robust performance of our fully incompressible solver especially for the more challenging direct-to-steady-state problems.