Vibration caused by mechanical resonance and time delay caused by signal detection and transmission degrade the control performance of a servo controller for a multimass mechanical system. A precise numerical model that represents resonance characteristics and time delay is necessary to design a desired control system. This paper presents an identification method using the iterative process of the linearized and weighted total least-squares method. The proposed method derives a transfer function without any prior knowledge of resonance characteristics and time delay. The order of the transfer function is determined with a coefficient diagram that shows coefficients of the denominator of the transfer function. Identification results with an experimental setup are shown to demonstrate the performance of the proposed method. A velocity servo controller with vibration-suppression control is designed with the transfer function, and control performance is verified with the experimental setup to validate the transfer function.