Synopsis This paper is focused on the theoretical modeling of the rheological properties of the magnetic suspensions in shear flows under an external magnetic field aligned with the streamlines. The conventional theory postulates that the field-induced aggregates of magnetic particles are highly anisotropic and aligned with the flow direction. Therefore, no substantial variation in suspension viscosity would be expected in the presence of field. However, experiments reveal a strong Bingham rheological behavior of the suspensions with a dynamic yield stress of the same order of magnitude that the one measured in the magnetic fields perpendicular to the flow. We explain the high level of shear stress, generated in longitudinal magnetic fields, by stochastic rotary oscillations of the aggregates caused by many-body magnetic interactions with neighboring aggregates. The inter-aggregate interactions are accounted for by an effective rotational diffusion process with a diffusion constant proportional to the mean square interaction torque – a net magnetic torque exerted to a given aggregate by all the neighboring aggregates. Using the equations for the orientation tensors coupled with the balance of forces acting on aggregates, we find the orientation distribution and size of the aggregates as function of the magnetic field intensity and shear rate. Our theory, developed for semi-dilute regime, reproduces the Bingham behavior observed experimentally and fits the 2 experimental data reasonably well in a wide range of particle concentrations. We find that the yield stress increases quadratically with the magnetic field strength and exhibits a cubic growth with respect to the particle volume fraction. A part from resolving a particular rheological problem, the new concept of magnetically induced diffusion reveals the importance of long-range non-hydrodynamic interactions in the rotary diffusion process and could probably contribute to the understanding of this process in other concentrated systems subject to non-hydrodynamic interactions.