Traditionally, the frequency response function has been estimated directly by dividing the discrete Fourier transforms of the output and the input of the system. This approach suffers from leakage errors and noise sensitivity. Lately these errors have been studied in detail. The main observation is that the error has a smooth frequency characteristic that is highly structured. The recently proposed local polynomial method uses this smoothness, and tries to estimate the frequency response function along with a smooth approximation of the error term. In this paper we propose a method, closely related to the local polynomial method, but instead of using the smoothness of the error we explore the structure even further. The proposed approach to estimate the frequency response function seems promising, as illustrated by simulations and comparison with current state of the art methods.