A finite element modeling approach of ultrasonic propagation combined with a description of the microstructure at the scale of the grains has been implemented. The simulations seek to determine the contribution of scattering to the ultrasonic attenuation in polycrystalline materials. The approach is applied to anisotropic microstructures exhibiting both elongated grains and transversely isotropic crystallographic texture which can be found in multipass welds. The ultrasonic propagation is computed with the 2D finite element code ATHENA 2D. The description of the propagation media accounts for the geometric, elastic and crystallographic properties of anisotropic welds. The study is focused on two types of welds made of austenitic 316L stainless steel and Ni-based alloy (182). The attenuation was computed from the decay of multiple backwall echoes. The contribution of the microstructure scattering was isolated by a correction of the attenuation data with the attenuation obtained in an equivalent homogenized material. The simulation investigates the attenuation as a function of several parameters: grain orientation and size, ultrasonic frequency, or anisotropy level. The attenuation level of elastic pressure waves was specifically examined as a function of the angle between the propagation direction and the grain orientation. The evolution of the attenuation is consistent with the theoretical models. Moreover the simulation results were compared to experimental data available from the literature in 316L stainless steel welds. The simulated and experimental values are in very good agreement.